This chapter is about uncertainty and probabilistic approaches. State-of-the-art machine learning systems have two significant shortcomings.

First of all, they can be overconfident (or sometimes underconfident) in their prediction. In practice, given noisy data, even if we observe the best practice of cross-validating with unseen datasets, this confidence might not be warranted. Especially in regulated or sensitive environments, such as in financial services, healthcare, security, and intelligence, we need to be very careful about our predictions and how accurate they are.

Secondly, the more complex a machine learning system is, the more data we need to fit our model, and the more severe the risk of overfitting.

Probabilistic models are models that produce probabilistic inferences using stochastic sampling techniques. By parametrizing distributions and inherent...