Campfire: Compressible, Regularization-Free, Structured Sparse Training for Hardware Accelerators

Authors: Noah Gamboa, Kais Kudrolli, Anand Dhoot, Ardavan Pedram
Published: 2020-01-09 | Updated: 2020-01-13

Source: https://arxiv.org/abs/2001.03253

PDF: https://arxiv.org/pdf/2001.03253

AIにより推定されたラベル

プルーニング手法 深層学習手法 ロバスト性に関する評価

※ こちらのラベルはAIによって自動的に追加されました。そのため、正確でないことがあります。
詳細は文献データベースについてをご覧ください。

Abstract

This paper studies structured sparse training of CNNs with a gradual pruning technique that leads to fixed, sparse weight matrices after a set number of epochs. We simplify the structure of the enforced sparsity so that it reduces overhead caused by regularization. The proposed training methodology Campfire explores pruning at granularities within a convolutional kernel and filter. We study various tradeoffs with respect to pruning duration, level of sparsity, and learning rate configuration. We show that our method creates a sparse version of ResNet-50 and ResNet-50 v1.5 on full ImageNet while remaining within a negligible <1 sparse training does not harm the robustness of the network, we also demonstrate how the network behaves in the presence of adversarial attacks. Our results show that with 70 achievable.