First, we will explore the concept of mathematical optimization. Optimization is the central component of machine learning problem. It turns out that the learning process is nothing more than a mere mathematical optimization problem. The trick is to define it properly. To come up with a good definition, we first need to understand how mathematical optimization works and which problems it can solve.
If you work in the business sector, I bet that you hear the word optimization several times a day. To optimize something means to make it more efficient, cut costs, increase revenues, and minimize risks. Optimization involves taking a number of actions, measuring results, and deciding whether you have ended up in a better place.
For example, to optimize your daily route to work, you can minimize the total time you spend driving from home to the...
