论文标题
ShiftNAS：自动生成高级无误神经网络
ShiftNAS: Towards Automatic Generation of Advanced Mulitplication-Less Neural Networks
论文作者
论文摘要
随着计算密集型乘积被轻巧的位移动操作取代，无乘法神经网络可大大降低硬件平台上的时间和能源成本。但是，现有的位移位网络均直接从最新的卷积神经网络（CNN）转移，这导致了不可忽略的准确性下降甚至模型收敛的失败。为了应对这一点，我们提出了ShiftNA，这是第一个框架调整神经体系结构搜索（NAS），以实质上减少位移位神经网络与其实际价值对应物之间的准确差距。具体而言，我们开拓NAS将NAS拖入面向移位的搜索空间，并赋予其与拓扑相关的搜索策略以及自定义的正则化和稳定化。结果，我们的ShiftNA破坏了传统的NAS方法对位移位神经网络的不兼容，并在准确性和收敛方面取得了更理想的性能。广泛的实验表明，ShiftNA为位移位神经网络设置了一个新的最先进的实验，其中CIFAR10上的精度增加（1.69-8.07）％（CIFAR100（5.71-18.09）％（5.71-18.09）％的CIFAR100和（4.36-67.07.07.07）％（4.36-67.07）％的Imagenet上的刻度cnns cornift cornift cornift cornift cornift cornift cornift cornift cornift cornift cornift inforce cornift cornift infimt crespege cornift infte cornift。
Multiplication-less neural networks significantly reduce the time and energy cost on the hardware platform, as the compute-intensive multiplications are replaced with lightweight bit-shift operations. However, existing bit-shift networks are all directly transferred from state-of-the-art convolutional neural networks (CNNs), which lead to non-negligible accuracy drop or even failure of model convergence. To combat this, we propose ShiftNAS, the first framework tailoring Neural Architecture Search (NAS) to substantially reduce the accuracy gap between bit-shift neural networks and their real-valued counterparts. Specifically, we pioneer dragging NAS into a shift-oriented search space and endow it with the robust topology-related search strategy and custom regularization and stabilization. As a result, our ShiftNAS breaks through the incompatibility of traditional NAS methods for bit-shift neural networks and achieves more desirable performance in terms of accuracy and convergence. Extensive experiments demonstrate that ShiftNAS sets a new state-of-the-art for bit-shift neural networks, where the accuracy increases (1.69-8.07)% on CIFAR10, (5.71-18.09)% on CIFAR100 and (4.36-67.07)% on ImageNet, especially when many conventional CNNs fail to converge on ImageNet with bit-shift weights.
