Date: Wednesday, June 22

Time: 1:00 PM CDT

Presenter: Ju-Heung (Jun) Kim, PhD candidate in Statistics

Title: LSSVR+PSO=MFG

Abstract: The least square support vector regression (LSSVR) with the Gaussian kernel has two hyperparameters, sigma, $\sigma$, a bandwidth showing the spread of the kernel, and C, a regularization gauging the level of errors. In this paper, we want to systematically study how the hyperparameters are related to the LSSVR learning, thanks to the observation that the ergodic and stationary mean field game (MFG) implied hyperparameters are estimated based on the closed form solution given arbitrarily fixed bounded hyperparameters. Inversely, this closed form solution enables us to estimate MFG implied hyperparameters. The LSSVR with particle swarm optimization (LSSVR + PSO) allows a new way to estimate hyperparameters that doesn't involve cross-validations. The exact solvable MFG equivalent to our model has a stationary and ergodic solution of nonlinear Schr\"{o}dinger equation (NLS). The equilibrium solution of MFG is a pair of optimal action and the interaction density and this pair in NLS is the ergodic and stationary chemical potential and the interacting residual particle density, or $|\psi|^2$ in NLS. The ergodic solution of NLS ergodicises and stationarizes the LSSVR residual particles via the local interaction measure and we call this \emph{'the ergodic learning.'}  As long as the non-zero volatility sigma and the non-zero mass ($\mu=1/C$) are bounded and the input and output data are required to have a non-degenerate support vector kernel, the input and output processes themselves don't need to be stationary, while all local interaction measures of the residual particles are ergodic. Depending on the (ergodic) learning classifications into condensed learning, intermediate learning and classical learning, based on theoretical temperatures, many candidate models for prediction are ergodically generated and the region of ergodicized and stationalized residuals via local interaction measure can be found for each hyperparameter pairs. As the C increases (or as we decrease the mass) or as the sigma increases, the volatility of learning increases. The learning tends to be more classical and less condensed. For each ergodic path generated as the ergodic solution to the MFG system, we profile the system response with respect to hyperparameters. Also, even in ergodic learning setting, phase transition behavior has been observed when PSO correctly keep the stationary energy conserved ergodically and stationarily. Lastly, it is shown that the zero crossing number of LSSVR $+$ PSO $=$ MFG is given by $E(N_0(T))=\frac{T}{2 \pi},$ given the data $\{(x_i, y_i)\}_{i=1}^T,$ x input, y output.