Browse Library
Sign In Start Free Trial
Close icon Search icon
Account
Browse Library Advanced Search Sign In Start Free Trial

Add to playlist

Create a Playlist

Modal Close icon
You need to login to use this feature.
  • Book Overview & Buying Clojure for Data Science
  • Table Of Contents Toc
  • Feedback & Rating feedback
Clojure for Data Science

Clojure for Data Science

By : Garner
5 (4)
Buy this Book
close
close
Clojure for Data Science

Clojure for Data Science

5 (4)
By: Garner
Buy this Book

Overview of this book

The term “data science” has been widely used to define this new profession that is expected to interpret vast datasets and translate them to improved decision-making and performance. Clojure is a powerful language that combines the interactivity of a scripting language with the speed of a compiled language. Together with its rich ecosystem of native libraries and an extremely simple and consistent functional approach to data manipulation, which maps closely to mathematical formula, it is an ideal, practical, and flexible language to meet a data scientist’s diverse needs. Taking you on a journey from simple summary statistics to sophisticated machine learning algorithms, this book shows how the Clojure programming language can be used to derive insights from data. Data scientists often forge a novel path, and you’ll see how to make use of Clojure’s Java interoperability capabilities to access libraries such as Mahout and Mllib for which Clojure wrappers don’t yet exist. Even seasoned Clojure developers will develop a deeper appreciation for their language’s flexibility! You’ll learn how to apply statistical thinking to your own data and use Clojure to explore, analyze, and visualize it in a technically and statistically robust way. You can also use Incanter for local data processing and ClojureScript to present interactive visualisations and understand how distributed platforms such as Hadoop sand Spark’s MapReduce and GraphX’s BSP solve the challenges of data analysis at scale, and how to explain algorithms using those programming models. Above all, by following the explanations in this book, you’ll learn not just how to be effective using the current state-of-the-art methods in data science, but why such methods work so that you can continue to be productive as the field evolves into the future.
Table of Contents (12 chapters)
close
close
close
Preface
Icon
Preface
Icon
What this book covers
Icon
What you need for this book
Icon
Who this book is for
Icon
Conventions
Icon
Reader feedback
Icon
Customer support
Free Chapter
1
1. Statistics
Icon
1. Statistics
Icon
Downloading the sample code
Icon
Running the examples
Icon
Downloading the data
Icon
Inspecting the data
Icon
Data scrubbing
Icon
Descriptive statistics
Icon
Variance
Icon
Quantiles
Icon
Binning data
Icon
Histograms
Icon
The normal distribution
Icon
Poincaré's baker
Icon
Skewness
Icon
Comparative visualizations
Icon
The importance of visualizations
Icon
Adding columns
Icon
Comparative visualizations of electorate data
Icon
Visualizing the Russian election data
Icon
Comparative visualizations
Icon
Summary
2
2. Inference
Icon
2. Inference
Icon
Introducing AcmeContent
Icon
Download the sample code
Icon
Load and inspect the data
Icon
Visualizing the dwell times
Icon
The exponential distribution
Icon
The central limit theorem
Icon
Standard error
Icon
Samples and populations
Icon
Confidence intervals
Icon
Visualizing different populations
Icon
Hypothesis testing
Icon
Testing a new site design
Icon
The t-statistic
Icon
Performing the t-test
Icon
One-sample t-test
Icon
Resampling
Icon
Testing multiple designs
Icon
Multiple comparisons
Icon
The browser simulation
Icon
jStat
Icon
B1
Icon
Plotting probability densities
Icon
State and Reagent
Icon
Simulating multiple tests
Icon
The Bonferroni correction
Icon
Analysis of variance
Icon
The F-distribution
Icon
The F-statistic
Icon
The F-test
Icon
Effect size
Icon
Summary
3
3. Correlation
Icon
3. Correlation
Icon
About the data
Icon
Inspecting the data
Icon
Visualizing the data
Icon
The log-normal distribution
Icon
Covariance
Icon
Pearson's correlation
Icon
Hypothesis testing
Icon
Confidence intervals
Icon
Regression
Icon
Ordinary least squares
Icon
Goodness-of-fit and R-square
Icon
Multiple linear regression
Icon
Matrices
Icon
The normal equation
Icon
Multiple R-squared
Icon
Adjusted R-squared
Icon
Collinearity
Icon
Prediction
Icon
Summary
4
4. Classification
Icon
4. Classification
Icon
About the data
Icon
Inspecting the data
Icon
Comparisons with relative risk and odds
Icon
The standard error of a proportion
Icon
The binomial distribution
Icon
Significance testing proportions
Icon
Chi-squared multiple significance testing
Icon
Classification with logistic regression
Icon
Implementing logistic regression with Incanter
Icon
Probability
Icon
Naive Bayes classification
Icon
Decision trees
Icon
Classification with clj-ml
Icon
Bias and variance
Icon
Ensemble learning and random forests
Icon
Saving the classifier to a file
Icon
Summary
5
5. Big Data
Icon
5. Big Data
Icon
Downloading the code and data
Icon
The reducers library
Icon
Mathematical folds with Tesser
Icon
Multiple regression with gradient descent
Icon
Scaling gradient descent with Hadoop
Icon
Stochastic gradient descent
Icon
Summary
6
6. Clustering
chevron up
Icon
6. Clustering
Icon
Downloading the data
Icon
Extracting the data
Icon
Inspecting the data
Icon
Clustering text
Icon
Creating term frequency vectors
Icon
Clustering with k-means and Incanter
Icon
Better clustering with TF-IDF
Icon
Large-scale clustering with Mahout
Icon
Running k-means clustering with Mahout
Icon
Cluster evaluation measures
Icon
The drawbacks of k-means
Icon
The curse of dimensionality
Icon
Summary
7
7. Recommender Systems
Icon
7. Recommender Systems
Icon
Download the code and data
Icon
Inspect the data
Icon
Parse the data
Icon
Types of recommender systems
Icon
Item-based and user-based recommenders
Icon
Slope One recommenders
Icon
Building a user-based recommender with Mahout
Icon
k-nearest neighbors
Icon
Recommender evaluation with Mahout
Icon
Probabilistic methods for large sets
Icon
Jaccard similarity for large sets with MinHash
Icon
Dimensionality reduction
Icon
Large-scale machine learning with Apache Spark and MLlib
Icon
Machine learning on Spark with MLlib
Icon
Summary
8
8. Network Analysis
Icon
8. Network Analysis
Icon
Download the data
Icon
Graph traversal with Loom
Icon
Breadth-first and depth-first search
Icon
Finding the shortest path
Icon
Whole-graph analysis
Icon
Scale-free networks
Icon
Distributed graph computation with GraphX
Icon
Summary
9
9. Time Series
Icon
9. Time Series
Icon
About the data
Icon
Fitting curves with a linear model
Icon
Time series decomposition
Icon
Discrete time models
Icon
Maximum likelihood estimation
Icon
Time series forecasting
Icon
Summary
10
10. Visualization
Icon
10. Visualization
Icon
Download the code and data
Icon
Exploratory data visualization
Icon
Using Quil for visualization
Icon
Visualization for communication
Icon
Summary
11
Index
Icon
Index

Running k-means clustering with Mahout

Now that we have a sequence file of vectors suitable for consumption by Mahout, it's time to actually run k-means clustering on the whole dataset. Unlike our local Incanter version, Mahout won't have any trouble dealing with the full Reuters corpus.

As with the SequenceFilesFromDirectory class, we've created a wrapper around another of Mahout's command-line programs, KMeansDriver. The Clojure variable names make it easier to see what each command-line argument is for.

(defn run-kmeans [in-path clusters-path out-path k]
  (let [distance-measure  "org.apache.mahout.common.distance.CosineDistanceMeasure"
        max-iterations    100
        convergence-delta 0.001]
    (KMeansDriver/main
     (->> (vector "-i"  in-path
                  "-c"  clusters-path
                  "-o"  out-path
                  "-dm" distance-measure
                  "-x"  max-iterations...

Unlock full access

Continue reading for free

A Packt free trial gives you instant online access to our library of over 7000 practical eBooks and videos, constantly updated with the latest in tech
Start a 7-day FREE trial
End of Section 10
Next Section

Create a Note

Modal Close icon
You need to login to use this feature.
notes
bookmark search playlist font-size

Change the font size

margin-width

Change margin width

day-mode

Change background colour

Close icon Search
Country selected
Close icon Your notes and bookmarks

Delete Bookmark

Modal Close icon
Are you sure you want to delete it?
Cancel
Yes, Delete

Delete Note

Modal Close icon
Are you sure you want to delete it?
Cancel
Yes, Delete

Confirmation

Modal Close icon
claim successful
Order Page Read Now

Buy this book with your credits?

Modal Close icon
Are you sure you want to buy this book with one of your credits?
Close
YES, BUY