Abstract：This paper proposes a convolution kernel initialization method based on sample image local pattern clustering, which can be used to initialize the convolution kernel in Convolutional Neural Network training. In convolutional neural networks, the main role of convolution kernels can be seen as the use of matched filtering to extract local patterns in an image and use it as a feature for subsequent image object recognition. To this end, a part of typical sample images are selected in the image training set, and subgraphs of the same size as the convolution kernel are extracted from these images as image local pattern vector sets. Firstly, the topological characteristics of the local pattern subgraph set are applied to the rough classification. Then, for each subclass after the rough classification, the potential local pattern subgraph is obtained by using the potential function clustering method to form the candidate subgraph pattern set. They are trained as the initial convolution kernel of CNN. The experimental results show that the proposed method can significantly accelerate the convergence speed of the initial CNN network training, and also improve the network recognition accuracy after the final training.
Glorot X, Bengio Y. Understanding the difficulty of training deep feedforward neural networks[J]. Journal of Machine Learning Research, 2010: 249-256.
[2]
He K, Zhang X, Ren S, et al. Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification[J]. international conference on computer vision, 2015: 1026-1034.
[3]
Luan S, Chen C, Zhang B, et al. Gabor Convolutional Networks[J]. IEEE Transactions on Image Processing, 2018, 27(9): 4357-4366.
[4]
Ioffe S, Szegedy C. Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift[J]. international conference on machine learning, 2015: 448-456.
[5]
Mishkin D, Matas J. All you need is a good init[J]. international conference on learning representations, 2016: 1-13.
[6]
Xie D, Xiong J, Pu S, et al. All You Need is Beyond a Good Init: Exploring Better Solution for Training Extremely Deep Convolutional Neural Networks with Orthonormality and Modulation[J]. computer vision and pattern recognition, 2017: 5075-5084.
[7]
Ozay M, Okatani T. Optimization on Submanifolds of Convolution Kernels in CNNs[J]. arXiv: Computer Vision and Pattern Recognition, 2016: 1-9. http://arxiv.org/abs/1610.07008.
[8]
Kumar S K. On weight initialization in deep neural networks.[J]. arXiv: Learning, 2017: 1-9 http://arxiv.org/abs/1704.08863.
[9]
Hendrycks D, Gimpel K. Adjusting for Dropout Variance in Batch Normalization and Weight Initialization.[J]. arXiv: Learning, 2017: 1-10 http://arxiv.org/abs/1607.02488.
[10]
Huang G, Liu Z, Der Maaten L V, et al. Densely Connected Convolutional Networks[J]. computer vision and pattern recognition, 2017: 2261-2269.
[11]
He K, Zhang X, Ren S, et al. Deep Residual Learning for Image Recognition[J]. computer vision and pattern recognition, 2016: 770-778.
[12]
Saxe A M, Mcclelland J L, Ganguli S, et al. Exact solutions to the nonlinear dynamics of learning in deep linear neural networks[J]. international conference on learning representations, 2014: 1-22. http://arxiv.org/abs/1312.6120.
[13]
Seyfioglu M S, Gurbuz S Z. Deep Neural Network Initialization Methods for Micro-Doppler Classification With Low Training Sample Support[J]. IEEE Geoscience and Remote Sensing Letters, 2017, 14(12): 2462-2466.
[14]
Krizhevsky A, Sutskever I, Hinton G E, et al. ImageNet Classification with Deep Convolutional Neural Networks[C]. neural information processing systems, 2012: 1097-1105.
[15]
Ba, J. L., Kiros, J. R., & Hinton, G. E. Layer Normalization[J]. 2016: 1-14. arXiv:1607.06450v1.
[16]
Wen, H., Xie, W., Pei, J., & Guan, L. An incremental learning algorithm for the hybrid RBF-BP network classifier. EURASIP Journal on Advances in Signal Processing[J]. 2016: 1-15. https://doi.org/10.1186/s13634-016-0357-8.
2019, Vol. 35 
