Document Type
Article
Publication Date
2024
DOI
10.1109/TIT.2024.3439136
Publication Title
IEEE Transactions on Information Theory
Volume
Article in Press
Pages
1-18
Abstract
We consider deep neural networks (DNNs) with a Lipschitz continuous activation function and with weight matrices of variable widths. We establish a uniform convergence analysis framework in which sufficient conditions on weight matrices and bias vectors together with the Lipschitz constant are provided to ensure uniform convergence of DNNs to a meaningful function as the number of their layers tends to infinity. In the framework, special results on uniform convergence of DNNs with a fixed width, bounded widths and unbounded widths are presented. In particular, as convolutional neural networks are special DNNs with weight matrices of increasing widths, we put forward conditions on the mask sequence which lead to uniform convergence of the resulting convolutional neural networks. The Lipschitz continuity assumption on the activation functions allows us to include in our theory most of commonly used activation functions in applications.
Rights
© 2024 The Authors.
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 License (CC BY-NC-ND 4.0).
Original Publication Citation
Xu, Y., & Zhang, H. (2024). Uniform convergence of deep neural networks with Lipschitz continuous activation functions and variable widths. IEEE Transactions on Information Theory. Advance online publication. https://doi.org/10.1109/TIT.2024.3439136
Repository Citation
Xu, Yuesheng and Zhang, Haizhang, "Uniform Convergence of Deep Neural Networks With Lipschitz Continuous Activation Functions and Variable Widths" (2024). Mathematics & Statistics Faculty Publications. 262.
https://digitalcommons.odu.edu/mathstat_fac_pubs/262