Navigating the non-convex landscape via amplifying escape directions of saddle points

Speaker:  Prof. Ziye Ma
          City University of Hong Kong

Title:   "Navigating the non-convex landscape via amplifying escape
          directions of saddle points"

Date:     Monday, 23 September 2024

Time:     4:00pm - 5:00pm

Venue:    Lecture Theater F
          (Leung Yat sing Lecture Theater)
          near lift 25/26, HKUST

Abstract:

The training process of modern machine learning models is essentially the
solving of non-convex optimization problems. One major challenge that
comes with non-convex objectives is the ubiquitous presence of degenerate
critical points such as saddle points and spurious solutions. In order to
better solve non-convex problems, we study how these degenerate points can
be escaped, under the context of matrix sensing, which is a benchmark
non-convex problem that is equivalent to the training of a two layer
quadratic neural network. In this talk, we focus on two ways of doing it:
1) through over-parametrization via lifting the problem to tensor space,
which has the ability to convert spurious solutions to strict saddles; 2)
through the use of higher-order loss function to amplify escape
directions.


****************
Biography:

The speaker Ziye Ma is currently a presidential assistant professor in the
computer science department at the City University of Hong Kong. Prior to
this, he completed his PhD in the EECS department at UC Berkeley under the
guidance of Somayeh Sojoudi and Javad Lavaei. His areas of research cover
explainable and efficient machine learning systems, emphasizing the
exploration of new theoretical tools and perspectives, especially via the
use of optimization theory.