Document Type

Article

Publication Date

2024

DOI

10.1109/TIT.2024.3439136

Publication Title

IEEE Transactions on Information Theory

Volume

Article in Press

Pages

1-18

Abstract

We consider deep neural networks (DNNs) with a Lipschitz continuous activation function and with weight matrices of variable widths. We establish a uniform convergence analysis framework in which sufficient conditions on weight matrices and bias vectors together with the Lipschitz constant are provided to ensure uniform convergence of DNNs to a meaningful function as the number of their layers tends to infinity. In the framework, special results on uniform convergence of DNNs with a fixed width, bounded widths and unbounded widths are presented. In particular, as convolutional neural networks are special DNNs with weight matrices of increasing widths, we put forward conditions on the mask sequence which lead to uniform convergence of the resulting convolutional neural networks. The Lipschitz continuity assumption on the activation functions allows us to include in our theory most of commonly used activation functions in applications.

Rights

© 2024 The Authors.

This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 License (CC BY-NC-ND 4.0).

Original Publication Citation

Xu, Y., & Zhang, H. (2024). Uniform convergence of deep neural networks with Lipschitz continuous activation functions and variable widths. IEEE Transactions on Information Theory. Advance online publication. https://doi.org/10.1109/TIT.2024.3439136

Share

COinS