Variational Inference - A Review for Statisticians is perhaps the go to paper in order to learn variational inference (VI). After all, the paper has over 2800 citations indicating its popularity in the community. I recently decided to reread the paper while trying to closely follow the derivations. In this blogpost, I'll extend the derivations of the Gaussian Mixture model of the paper in the hope to elucidate some of the steps over which the authors went quickly.

1

In this Blogpost we will derive the equations required to fit Poisson mixture from scratch and implement the model using Python.

2

Pyro is a powerful probabilistic programming language, allowing to define and perform inference with complex statistical models. The usage of the library has become widespread in our lab, as the library enables to perform stochastic variational inference, which enables to scale statistical models to large data. In this post, I’ll take a closer look on Pyro’s enumeration strategy for discrete latent variables and illustrate this feature in a simple model.

3