刘嘉， 公茂果^*

（  西安电子科技大学 智能感知与图像理解教育部重点实验室,陕西西安　710071 ；  ）

摘要： 深度神经网络是一种具有复杂结构和较深层次的神经网络模型,其强大的数据表示和建模能力使其在各类应用中发挥了巨大的作用.稀疏理论的引入进一步提高了深度神经网络的性能.本文从两个方面对深度神经网络中的稀疏模型进行介绍.首先是表示稀疏,深度神经网络的一个重要的特性就是无监督特征学习,将稀疏约束加入到无监督特征学习模型中,可以使网络学习到更多的结构特征,同时增强对噪声及干扰的鲁棒性,进而提高深度网络的数据抽象及判别能力.另一方面是连接结构稀疏,总所周知,大脑的神经元不计其数,但每个神经元只与少量的神经元相互连接,因此,很多工作针对稀疏连接结构的研究.通过去除冗余的连接,可以有效提高深度神经网络的泛化性能,特别是网络规模较大,参数较多的情况下.本文将对这两种稀疏模型及优化方法进行介绍并设计实验分析了几种模型的性能.
关键词： 深度神经网络;稀疏理论;无监督特征学习;结构优化.

LIU Jia， GONG Maoguo^*

（  Key Laboratory of Intelligent Perception and Image Understanding, Xidian University, Xi’an 710071 ；  ）

Abstract： Deep neural networks are a type of artificial neural network models with complex architecture and deep layers. They have been widely applied to many applications with excellent performance due to its powerful data representing and modeling capbabilities. The performance of deep neural networks can be further improved by introducing sparse theory. This paper introduces two types of sparse models in deep neural networks. The first one are sparse feature learning models. Unsupervised feature learning is one of the properties of deep neural networks. By introducing sparsity, the network can learn more structured features and the robustness to noise can be improved. This can increase the abstraction and discrimination abilities of the network. The second one are sparse connecting structures. As we all know, the structure of brain is sparse with large number of neurons but less connections. There are many works focusing on the sparse structure research. By removing redundant connections, the generalization of the network can be improved, especially under the case of large scale network with numberous parameters. In this paper, the two sparse models and their optimization methods are introduced and the performance of them are compared by several experiments.

Tag：

点此返回栏目查看更多>>>参考论文