Title:	Optimization Theory and Methods for Machine Learning with Applications in Neural Network Training
DNr:	Berzelius-2024-13
Project Type:	LiU Berzelius
Principal Investigator:	Alp Yurtsever <alp.yurtsever@umu.se>
Affiliation:	Umeå universitet
Duration:	2024-01-30 – 2024-08-01
Classification:	10201
Homepage:	https://www.umu.se/en/research/groups/mathematical-programming2/
Keywords:

Abstract

This project focuses on large-scale optimization problems and their applications in machine learning and data science which involve vast datasets distributed across the networks with limited access, numerous parameters, and complex models featuring non-smooth, non-convex loss functions. Our research develops optimization theory and methods, particularly for efficient neural network training. We focus on analyzing how constraints and regularization influence convergence rates and generalization in neural networks. Initial experiments on small-scale examples have shown our approach outperforming existing models; notably, constrained models with linear activation surpassing standard ReLU activation models. Consequently, we are eager to evaluate our proposed methods across a diverse range of larger-scale neural network benchmarks.