论文标题
关键字发现系统以及低功率边缘微控制器上修剪和量化方法的评估
Keyword Spotting System and Evaluation of Pruning and Quantization Methods on Low-power Edge Microcontrollers
论文作者
论文摘要
关键字斑点（KWS）对基于语音的用户与边缘的低功耗设备的交互是有益的。边缘设备通常始终在线，因此Edge Computing带来带宽节省和隐私保护。这些设备通常具有有限的内存空间，计算性能，功率和成本，例如基于皮质的微控制器。面临的挑战是满足这些设备深度学习的高计算和低延迟要求。本文首先显示了我们在STM32F7微控制器上使用Cortex-M7 Core @216MHz和512KB静态RAM上运行的小型kWs系统。我们选择的卷积神经网络（CNN）体系结构具有简化KW的操作数量，以满足边缘设备的约束。我们的基线系统为每个37ms生成分类结果，包括实时音频特征提取部分。本文进一步评估了微控制器上不同修剪和量化方法的实际性能，包括稀疏性的不同粒度，跳过零重量，重量优先的环路订单和SIMD指令。结果表明，对于微控制器，加速非结构化的修剪模型面临着巨大的挑战，并且结构化的修剪比非组织修剪更友好。结果还验证了量化和SIMD指令的性能改进。
Keyword spotting (KWS) is beneficial for voice-based user interactions with low-power devices at the edge. The edge devices are usually always-on, so edge computing brings bandwidth savings and privacy protection. The devices typically have limited memory spaces, computational performances, power and costs, for example, Cortex-M based microcontrollers. The challenge is to meet the high computation and low-latency requirements of deep learning on these devices. This paper firstly shows our small-footprint KWS system running on STM32F7 microcontroller with Cortex-M7 core @216MHz and 512KB static RAM. Our selected convolutional neural network (CNN) architecture has simplified number of operations for KWS to meet the constraint of edge devices. Our baseline system generates classification results for each 37ms including real-time audio feature extraction part. This paper further evaluates the actual performance for different pruning and quantization methods on microcontroller, including different granularity of sparsity, skipping zero weights, weight-prioritized loop order, and SIMD instruction. The result shows that for microcontrollers, there are considerable challenges for accelerate unstructured pruned models, and the structured pruning is more friendly than unstructured pruning. The result also verified that the performance improvement for quantization and SIMD instruction.
