Sign In Start Free Trial

Book Overview & Buying
Table Of Contents
Feedback & Rating

Hands-On Data Analysis with Pandas

By : Stefanie Molin

4.7 (11)

Hands-On Data Analysis with Pandas

4.7 (11)

By: Stefanie Molin

Overview of this book

Data analysis has become a necessary skill in a variety of domains where knowing how to work with data and extract insights can generate significant value. Hands-On Data Analysis with Pandas will show you how to analyze your data, get started with machine learning, and work effectively with Python libraries often used for data science, such as pandas, NumPy, matplotlib, seaborn, and scikit-learn. Using real-world datasets, you will learn how to use the powerful pandas library to perform data wrangling to reshape, clean, and aggregate your data. Then, you will be able to conduct exploratory data analysis by calculating summary statistics and visualizing the data to find patterns. In the concluding chapters, you will explore some applications of anomaly detection, regression, clustering, and classification using scikit-learn to make predictions based on past data. By the end of this book, you will be equipped with the skills you need to use pandas to ensure the veracity of your data, visualize it for effective decision-making, and reliably reproduce analyses across multiple datasets.

Preface

Preface

Who this book is for

What this book covers

To get the most out of this book

Conventions used

Get in touch

Free Chapter

Section 1: Getting Started with Pandas

Section 1: Getting Started with Pandas

Introduction to Data Analysis

Introduction to Data Analysis

Chapter materials

Fundamentals of data analysis

Statistical foundations

Setting up a virtual environment

Summary

Exercises

Further reading

Working with Pandas DataFrames

Working with Pandas DataFrames

Chapter materials

Pandas data structures

Bringing data into a pandas DataFrame

Inspecting a DataFrame object

Grabbing subsets of the data

Adding and removing data

Summary

Exercises

Further reading

Section 2: Using Pandas for Data Analysis

Section 2: Using Pandas for Data Analysis

Data Wrangling with Pandas

Data Wrangling with Pandas

Chapter materials

What is data wrangling?

Collecting temperature data

Cleaning up the data

Restructuring the data

Handling duplicate, missing, or invalid data

Summary

Exercises

Further reading

Aggregating Pandas DataFrames

Aggregating Pandas DataFrames

Chapter materials

Database-style operations on DataFrames

DataFrame operations

Aggregations with pandas and numpy

Time series

Summary

Exercises

Further reading

Visualizing Data with Pandas and Matplotlib

Visualizing Data with Pandas and Matplotlib

Chapter materials

An introduction to matplotlib

Plotting with pandas

The pandas.plotting subpackage

Summary

Exercises

Further reading

Plotting with Seaborn and Customization Techniques

Plotting with Seaborn and Customization Techniques

Chapter materials

Utilizing seaborn for advanced plotting

Formatting

Customizing visualizations

Summary

Exercises

Further reading

Section 3: Applications - Real-World Analyses Using Pandas

Section 3: Applications - Real-World Analyses Using Pandas

Financial Analysis - Bitcoin and the Stock Market

Financial Analysis - Bitcoin and the Stock Market

Chapter materials

Building a Python package

Data extraction with pandas

Exploratory data analysis

Technical analysis of financial instruments

Modeling performance

Summary

Exercises

Further reading

Rule-Based Anomaly Detection

Rule-Based Anomaly Detection

Chapter materials

Simulating login attempts

Exploratory data analysis

Rule-based anomaly detection

Summary

Exercises

Further reading

Section 4: Introduction to Machine Learning with Scikit-Learn

Section 4: Introduction to Machine Learning with Scikit-Learn

Getting Started with Machine Learning in Python

Getting Started with Machine Learning in Python

Chapter materials

Learning the lingo

Exploratory data analysis

Preprocessing data

Clustering

Regression

Classification

Summary

Exercises

Further reading

Making Better Predictions - Optimizing Models

Making Better Predictions - Optimizing Models

Chapter materials

Hyperparameter tuning with grid search

Feature engineering

Ensemble methods

Inspecting classification prediction confidence

Addressing class imbalance

Regularization

Summary

Exercises

Further reading

Machine Learning Anomaly Detection

Machine Learning Anomaly Detection

Chapter materials

Exploring the data

Unsupervised methods

Supervised methods

Online learning

Summary

Exercises

Further reading

Section 5: Additional Resources

Section 5: Additional Resources

The Road Ahead

The Road Ahead

Data resources

Practicing working with data

Python practice

Summary

Exercises

Further reading

Solutions

Solutions

Other Books You May Enjoy

Other Books You May Enjoy

Leave a review - let other readers know what you think

Appendix

Appendix

Data analysis workflow

Choosing the appropriate visualization

Machine learning workflow

Customer Reviews

4.7 (11)

5 star

90.9%

4 star

0

3 star

0

2 star

9.1%

1 star

0

Cleaning up the data

Let's move on to the 3-cleaning_data.ipynb notebook for our discussion on data cleaning. We will begin by importing pandas and reading in the data/nyc_temperatures.csv file, which contains the maximum daily temperature (TMAX), minimum daily temperature (TMIN), and the average daily temperature (TAVG) from the LaGuardia airport station in New York City for October 2018:

>>> import pandas as pd

>>> df = pd.read_csv('data/nyc_temperatures.csv')
>>> df.head()

The data we retrieved from the API is in the long format; for our analysis, we want it in the wide format, but we will address that in the Pivoting DataFrames section later this chapter:

	`attributes`	`datatype`	`date`	`station`	`value`
`0`	`H,,S,`	`TAVG`	`2018-10-01T00:00:00`	`GHCND:USW00014732`	`21.2`
`1`	`,,W,2400`	`TMAX`	`2018-10-01T00:00:00`	`GHCND:USW00014732`	`25.6`
`2`	`,,W,2400`	`TMIN...`

Search

Your notes and bookmarks