Sign In Start Free Trial

Pandas Cookbook

By : Theodore Petrou

4.3 (32)

Pandas Cookbook

4.3 (32)

By: Theodore Petrou

Overview of this book

This book will provide you with unique, idiomatic, and fun recipes for both fundamental and advanced data manipulation tasks with pandas 0.20. Some recipes focus on achieving a deeper understanding of basic principles, or comparing and contrasting two similar operations. Other recipes will dive deep into a particular dataset, uncovering new and unexpected insights along the way. The pandas library is massive, and it's common for frequent users to be unaware of many of its more impressive features. The official pandas documentation, while thorough, does not contain many useful examples of how to piece together multiple commands like one would do during an actual analysis. This book guides you, as if you were looking over the shoulder of an expert, through practical situations that you are highly likely to encounter. Many advanced recipes combine several different features across the pandas 0.20 library to generate results.

Preface

Preface

What this book covers

What you need for this book

Who this book is for

Conventions

Assumptions for every recipe

Dataset Descriptions

Sections

Reader feedback

Customer support

Free Chapter

Pandas Foundations

Pandas Foundations

Introduction

Dissecting the anatomy of a DataFrame

Accessing the main DataFrame components

Understanding data types

Selecting a single column of data as a Series

Calling Series methods

Working with operators on a Series

Chaining Series methods together

Making the index meaningful

Renaming row and column names

Creating and deleting columns

Essential DataFrame Operations

Essential DataFrame Operations

Introduction

Selecting multiple DataFrame columns

Selecting columns with methods

Ordering column names sensibly

Operating on the entire DataFrame

Chaining DataFrame methods together

Working with operators on a DataFrame

Comparing missing values

Transposing the direction of a DataFrame operation

Determining college campus diversity

Beginning Data Analysis

Beginning Data Analysis

Introduction

Developing a data analysis routine

Reducing memory by changing data types

Selecting the smallest of the largest

Selecting the largest of each group by sorting

Replicating nlargest with sort_values

Calculating a trailing stop order price

Selecting Subsets of Data

Selecting Subsets of Data

Introduction

Selecting Series data

Selecting DataFrame rows

Selecting DataFrame rows and columns simultaneously

Selecting data with both integers and labels

Speeding up scalar selection

Slicing rows lazily

Slicing lexicographically

Boolean Indexing

Boolean Indexing

Introduction

Calculating boolean statistics

Constructing multiple boolean conditions

Filtering with boolean indexing

Replicating boolean indexing with index selection

Selecting with unique and sorted indexes

Gaining perspective on stock prices

Translating SQL WHERE clauses

Determining the normality of stock market returns

Improving readability of boolean indexing with the query method

Preserving Series with the where method

Masking DataFrame rows

Selecting with booleans, integer location, and labels

Index Alignment

Index Alignment

Introduction

Examining the Index object

Producing Cartesian products

Exploding indexes

Filling values with unequal indexes

Appending columns from different DataFrames

Highlighting the maximum value from each column

Replicating idxmax with method chaining

Finding the most common maximum

Grouping for Aggregation, Filtration, and Transformation

Grouping for Aggregation, Filtration, and Transformation

Introduction

Defining an aggregation

Grouping and aggregating with multiple columns and functions

Removing the MultiIndex after grouping

Customizing an aggregation function

Customizing aggregating functions with *args and **kwargs

Examining the groupby object

Filtering for states with a minority majority

Transforming through a weight loss bet

Calculating weighted mean SAT scores per state with apply

Grouping by continuous variables

Counting the total number of flights between cities

Finding the longest streak of on-time flights

Restructuring Data into a Tidy Form

Restructuring Data into a Tidy Form

Introduction

Tidying variable values as column names with stack

Tidying variable values as column names with melt

Stacking multiple groups of variables simultaneously

Inverting stacked data

Unstacking after a groupby aggregation

Replicating pivot_table with a groupby aggregation

Renaming axis levels for easy reshaping

Tidying when multiple variables are stored as column names

Tidying when multiple variables are stored as column values

Tidying when two or more values are stored in the same cell

Tidying when variables are stored in column names and values

Tidying when multiple observational units are stored in the same table

Combining Pandas Objects

Combining Pandas Objects

Introduction

Appending new rows to DataFrames

Concatenating multiple DataFrames together

Comparing President Trump's and Obama's approval ratings

Understanding the differences between concat, join, and merge

Connecting to SQL databases

Time Series Analysis

Time Series Analysis

Introduction

Understanding the difference between Python and pandas date tools

Slicing time series intelligently

Using methods that only work with a DatetimeIndex

Counting the number of weekly crimes

Aggregating weekly crime and traffic accidents separately

Measuring crime by weekday and year

Grouping with anonymous functions with a DatetimeIndex

Grouping by a Timestamp and another column

Finding the last time crime was 20% lower with merge_asof

Visualization with Matplotlib, Pandas, and Seaborn

Visualization with Matplotlib, Pandas, and Seaborn

Introduction

Getting started with matplotlib

Visualizing data with matplotlib

Plotting basics with pandas

Visualizing the flights dataset

Stacking area charts to discover emerging trends

Understanding the differences between seaborn and pandas

Doing multivariate analysis with seaborn Grids

Uncovering Simpson's paradox in the diamonds dataset with seaborn

Customer Reviews

4.3 (32)

5 star

75%

4 star

3.1%

3 star

6.3%

2 star

9.4%

1 star

6.3%

Introduction

It is important to consider the steps that you, as an analyst, take when you first encounter a dataset after importing it into your workspace as a DataFrame. Is there a set of tasks that you usually undertake to first examine the data? Are you aware of all the possible data types? This chapter begins by covering the tasks you might want to undertake when first encountering a new dataseset. The chapter proceeds by answering common questions that are not that trivial to do in pandas.

Search

Your notes and bookmarks