Randomized Kaczmarz, Geometric Smoothing, and Momentum
Randomized Kaczmarz, Geometric Smoothing, and Momentum
Start:
Wednesday, July 17, 2024 3:00 pm
Location:
STAG 163
Nick Marshall
Oregon State University
Abstract. This talk discusses the effect of adding geometrically smoothed momentum to the randomized Kaczmarz algorithm, which is an instance of stochastic gradient descent on a linear least squares loss function. We prove a result about the expected error in the direction of singular vectors of the matrix defining the least squares loss. We present several numerical examples illustrating the utility of our result and pose several questions.
Contact:
Holly Swisher