Browse Library
Sign In Start Free Trial
Close icon Search icon
Account
Browse Library Advanced Search Sign In Start Free Trial

Add to playlist

Create a Playlist

Modal Close icon
You need to login to use this feature.
  • Learning Data Mining with Python
  • Toc
  • feedback
Learning Data Mining with Python

Learning Data Mining with Python

By : Robert Layton
3.7 (7)
close
Learning Data Mining with Python

Learning Data Mining with Python

3.7 (7)
By: Robert Layton

Overview of this book

If you are a programmer who wants to get started with data mining, then this book is for you.
Table of Contents (15 chapters)
close
close
Preface
Preface
What this book covers
What you need for this book
Who this book is for
Conventions
Reader feedback
Customer support
Free Chapter
1
1. Getting Started with Data Mining
1. Getting Started with Data Mining
Introducing data mining
Using Python and the IPython Notebook
A simple affinity analysis example
A simple classification example
What is classification?
Summary
2
2. Classifying with scikit-learn Estimators
2. Classifying with scikit-learn Estimators
scikit-learn estimators
Preprocessing using pipelines
Pipelines
Summary
3
3. Predicting Sports Winners with Decision Trees
3. Predicting Sports Winners with Decision Trees
Loading the dataset
Decision trees
Sports outcome prediction
Random forests
Summary
4
4. Recommending Movies Using Affinity Analysis
4. Recommending Movies Using Affinity Analysis
Affinity analysis
The movie recommendation problem
The Apriori implementation
Extracting association rules
Summary
5
5. Extracting Features with Transformers
5. Extracting Features with Transformers
Feature extraction
Feature selection
Feature creation
Creating your own transformer
Summary
6
6. Social Media Insight Using Naive Bayes
chevron up
6. Social Media Insight Using Naive Bayes
Disambiguation
Text transformers
Naive Bayes
Application
Summary
7
7. Discovering Accounts to Follow Using Graph Mining
7. Discovering Accounts to Follow Using Graph Mining
Loading the dataset
Finding subgraphs
Summary
8
8. Beating CAPTCHAs with Neural Networks
8. Beating CAPTCHAs with Neural Networks
Artificial neural networks
Creating the dataset
Training and classifying
Improving accuracy using a dictionary
Summary
9
9. Authorship Attribution
9. Authorship Attribution
Attributing documents to authors
Function words
Support vector machines
Character n-grams
Using the Enron dataset
Summary
10
10. Clustering News Articles
10. Clustering News Articles
Obtaining news articles
Extracting text from arbitrary websites
Grouping news articles
Clustering ensembles
Online learning
Summary
11
11. Classifying Objects in Images Using Deep Learning
11. Classifying Objects in Images Using Deep Learning
Object classification
Application scenario and goals
Deep neural networks
GPU optimization
Setting up the environment
Application
Summary
12
12. Working with Big Data
12. Working with Big Data
Big data
Application scenario and goals
MapReduce
Application
Summary
13
A. Next Steps…
A. Next Steps…
Chapter 1 – Getting Started with Data Mining
Chapter 2 – Classifying with scikit-learn Estimators
Chapter 3: Predicting Sports Winners with Decision Trees
Chapter 4 – Recommending Movies Using Affinity Analysis
Chapter 5 – Extracting Features with Transformers
Chapter 6 – Social Media Insight Using Naive Bayes
Chapter 7 – Discovering Accounts to Follow Using Graph Mining
Chapter 8 – Beating CAPTCHAs with Neural Networks
Chapter 9 – Authorship Attribution
Chapter 10 – Clustering News Articles
Chapter 11: Classifying Objects in Images Using Deep Learning
Chapter 12 – Working with Big Data
More resources
14
Index
Index

Text transformers

Now that we have our dataset, how are we going to perform data mining on it?

Text-based datasets include books, essays, websites, manuscripts, programming code, and other forms of written expression. All of the algorithms we have seen so far deal with numerical or categorical features, so how do we convert our text into a format that the algorithm can deal with?

There are a number of measurements that could be taken. For instance, average word and average sentence length are used to predict the readability of a document. However, there are lots of feature types such as word occurrence which we will now investigate.

Bag-of-words

One of the simplest but highly effective models is to simply count each word in the dataset. We create a matrix, where each row represents a document in our dataset and each column represents a word. The value of the cell is the frequency of that word in the document.

Here's an excerpt from The Lord of the Rings, J.R.R. Tolkien:

 

Three Rings for the Elven...

End of Section 4
Next Section
bookmark search playlist download
font-size

Change the font size

margin-width

Change margin width

day-mode

Change background colour

Close icon Search
Country selected
Close icon Your notes and bookmarks

Delete Bookmark

Modal Close icon
Are you sure you want to delete it?
Cancel
Yes, Delete