论文标题
算术电路，结构化矩阵和（不是）深度学习
Arithmetic Circuits, Structured Matrices and (not so) Deep Learning
论文作者
论文摘要
这项调查表明，在算术电路复杂性，结构化矩阵和深度学习的交集中，结果一定是不完整的（和偏见）的概述。最近，通过结构化替换神经网络中非结构化的体重矩阵的研究活动（目的是减少相应的深度学习模型的大小）。这项工作的大部分都是实验性的，在这项调查中，我们将研究问题正式化，并展示了最新的工作结合了算术电路复杂性，结构化矩阵和深度学习的方法，基本上回答了这个问题。 这项调查的目标是复杂的理论家，他们可能喜欢阅读有关算术电路复杂性中开发的工具如何帮助设计（据我们所知）一个新的结构化矩阵家族，这反过来又非常适合深度学习。但是，我们希望主要对深度学习感兴趣的人们也会欣赏与复杂性理论的联系。
This survey presents a necessarily incomplete (and biased) overview of results at the intersection of arithmetic circuit complexity, structured matrices and deep learning. Recently there has been some research activity in replacing unstructured weight matrices in neural networks by structured ones (with the aim of reducing the size of the corresponding deep learning models). Most of this work has been experimental and in this survey, we formalize the research question and show how a recent work that combines arithmetic circuit complexity, structured matrices and deep learning essentially answers this question. This survey is targeted at complexity theorists who might enjoy reading about how tools developed in arithmetic circuit complexity helped design (to the best of our knowledge) a new family of structured matrices, which in turn seem well-suited for applications in deep learning. However, we hope that folks primarily interested in deep learning would also appreciate the connections to complexity theory.
