Dr. Linglong Kong

University of Alberta, Department of Mathematical and Statistical Science

M-estimation in Low-rank Matrix Factorization: a General Framework

Many problems in science and engineering can be reduced to the recovery of an unknown large matrix from a small number of random linear measurements. Matrix factorization arguably is the most popular approach for low-rank matrix recovery. Many methods have been proposed using different loss functions, such as the most widely used L2 loss, more robust choices L1 and Huber loss, and quantile and expectile loss for skewed data. All of them can be unified into the framework of M-estimation. In this paper, we present a general framework of low-rank matrix factorization based on M-estimation in statistics. The framework mainly involves two steps: we first apply Nesterov’s smoothing technique to obtain an optimal smooth approximation for non-smooth loss functions, such as L1 and quantile loss; secondly, we exploit an alternative updating scheme along with Nesterov’s momentum method at each step to minimize the smoothed loss function. Strong theoretical convergence guarantee has been developed for the general framework, and extensive numerical experiments have been conducted to illustrate the performance of the proposed algorithm.