Given the different kinds of machine learning techniques, it is highly important to know the challenges in modeling. We will use the previous digit classification method. We previously tried to model it using all pixels as the available input. The dimensions for the input are the image size, that is, h x w. It ranges from several hundreds to a few thousand. This size is considered as the input dimension, and as it increases, the computation as well as uncertainty in estimation increase. We need a bigger model to perform better estimation if the input dimension increases. This is termed curse of dimensionality.

In order to resolve this curse, it is highly recommended to reduce the input dimensions. For example, instead of using pixel values as input, we can extract strong features and use them as input to the model. This will reduce the input dimensions...