Date of Award

Summer 2024

Document Type

Dissertation

Degree Name

Doctor of Philosophy (PhD)

Department

Mathematics & Statistics

Program/Concentration

Computational and Applied Mathematics

Committee Director

Yuesheng Xu

Committee Member

Raymond Cheng

Committee Member

Guohui Song

Committee Member

Lucia Tabacu

Committee Member

Jian Wu

Abstract

Deep learning has achieved immense success in the past decade. The goal of this dissertation is to understand deep learning through the framework of reproducing kernel Banach spaces (RKBSs), which were originally proposed for promoting sparse solutions. We begin by considering learning problems in a general functional setting, and establishing explicit and data-dependent representer theorems for both minimal norm interpolation (MNI) problems and regularization problems. These theorems provide a crucial foundation for the subsequent results derived for both sparse learning and deep learning. Next, we investigate the essential properties of RKBSs capable of encouraging sparsity in learning solutions. With the proposed sufficient conditions on RKBSs, we develop sparse representer theorems by leveraging previously established representer theorems. Illustrative examples include the sequence space ℓ1(N) and a function space constructed by measures, both of which can enjoy sparse representer theorems. Subsequently, we introduce a hypothesis space for deep learning that employs deep neural networks (DNNs). By treating the DNN as a function of two variables, the physical variable and the parameter variable, we construct a vector-valued RKBS with reproducing kernel formed by the product of the DNN with a weight function. We explore MNI problems and regularization problems considering in this vector-valued RKBS, and develop the corresponding representer theorems for deep learning, demonstrating solutions in the form of DNNs. Finally, motivated by the desire of alleviating potential overfitting and accelerating inference, we examine sparse deep learning models with ℓ1 regularization, which encourages a significant portion of parameters in DNNs to be zero. We provide an iterative algorithm for selecting regularization parameters to ensure that the weights in each layer of the resulting sparse DNN meet a specified sparsity level. Numerical experiments validate the efficiency of the proposed algorithm in acquiring appropriate regularization parameters to achieve desirable sparse DNNs.

Rights

In Copyright. URI: http://rightsstatements.org/vocab/InC/1.0/ This Item is protected by copyright and/or related rights. You are free to use this Item in any way that is permitted by the copyright and related rights legislation that applies to your use. For other uses you need to obtain permission from the rights-holder(s).

DOI

10.25777/a77h-3q15

Download

Share

COinS