
Scala for Machine Learning, Second Edition
By :

As an open source platform, Apache Spark MLlib can be easily customized, and extended to address specific problems related to requiring the implementation of new statistics, mathematical, or machine learning solutions.
The purpose of this section is to create a Kullback-Leibler divergence as a Spark evaluator. The implementation follows two steps:
The Kullback-Leibler has been briefly introduced in Chapter 5, Dimension Reduction – Divergences in the context of evaluating the similarity of the frequencies distribution between two datasets. The Kullback-Leibler divergence is a concept borrowed from information theory and is commonly associated with Information Gain. The Kullback-Leibler divergence is also known as the relative entropy or information divergence between two distributions. It measures the dissimilarity of the distribution...