We have previously established that the goal of supervised learning is always to predict labels (or target values) for data. However, depending on the nature of these labels, supervised learning can come in two distinct forms:

• Classification: Supervised learning is called classification whenever we use the data to predict categories. A good example of this is when we try to predict whether an image contains a cat or a dog. Here, the labels of the data are categorical, either one or the other, but never a mixture of categories. For example, a picture contains either a cat or a dog, never 50% cat and 50% dog (before you ask, no, here we do not consider pictures of the cartoon character, CatDog), and our job is simply to tell which one it is. When there are only two choices, it is called two-class or binary classification. When there are more than...