Variational Inference for Large-scale and Complex Models

Variational methods have long been used in Machine Learning as a faster alternative to approaches such as Markov chain Monte Carlo sampling for performing inference in intractable probabilistic models. However, traditional variational approaches require conjugacy between prior and posterior distributions and do not scale well for large models. Recent work on variational methods has addressed these limitations, making it possible to use probabilistic approaches in large-scale and complex domains, such as deep learning. In this tutorial, we give an introduction to variational methods and review the recent literature on their extension to large-scale and complex models.