Ph.D. Seminar: Charlie Labuzzetta, "Conformal prediction under covariate shift: Subsampling to achieve approximate exchangeability"
Title: Conformal prediction under covariate shift: Subsampling to achieve approximate exchangeability
Abstract: This paper describes a novel method for applying conformal prediction to data that are not exchangeable. Specifically, we show a nearest neighbor subsampling method under inductive conformal prediction can account for covariate shift between the calibration and test set data, achieving approximate exchangeability between subsampled calibration observations and the test set. Our method produces provably approximately valid prediction sets under minimal assumptions. We empirically compare our method to weighted conformal prediction for covariate shift through classification experiments and conclude our method significantly outperforms many existing approaches in the literature. The experiments show subsample conformal prediction produces prediction sets with coverage approximately centered at the specified nominal rate across many trials, whereas this is not guaranteed under covariate shift with any other method. Finally, we discuss the promising potential and future developments of this method, especially in the context of classification, which is an underrepresented but important application of conformal prediction. The paper presented in this defense will also be connected to the additional chapters of the dissertation, specifically in regard to uncertainty estimation for land cover classification.