In the upcoming Bayes group meeting we will be discussing a paper recommended by one of the regular meeting attendees. Here is a link to the paper. The title and abstract are below. Please take some time to skim the paper before joining on Monday, and bring any discussion points you come up with!

Title: Evaluating Bayesian Models with Posterior Dispersion Indices

Authors: Alp Kucukelbir, Yixin Wang, David M. Blei

Abstract: Probabilistic modeling is cyclical: we specify a model, infer its posterior, and evaluate its performance. Evaluation drives the cycle, as we revise our model based on how it performs. This requires a metric. Traditionally, predictive accuracy prevails. Yet, predictive accuracy does not tell the whole story. We propose to evaluate a model through posterior dispersion. The idea is to analyze how each datapoint fares in relation to posterior uncertainty around the hidden structure. This highlights datapoints the model struggles to explain and provides complimentary insight to datapoints with low predictive accuracy. We present a family of posterior dispersion indices (PDI) that capture this idea. We show how a PDI identifies patterns of model mismatch in three real data examples: voting preferences, supermarket shopping, and population genetics.