Browse Library
Sign In Start Free Trial
Close icon Search icon
Account
Browse Library Advanced Search Sign In Start Free Trial

Add to playlist

Create a Playlist

Modal Close icon
You need to login to use this feature.
  • IPython Interactive Computing and Visualization Cookbook
  • Toc
  • feedback
IPython Interactive Computing and Visualization Cookbook

IPython Interactive Computing and Visualization Cookbook

By : Cyrille Rossant
4.4 (7)
close
IPython Interactive Computing and Visualization Cookbook

IPython Interactive Computing and Visualization Cookbook

4.4 (7)
By: Cyrille Rossant

Overview of this book

Python is one of the leading open source platforms for data science and numerical computing. IPython and the associated Jupyter Notebook offer efficient interfaces to Python for data analysis and interactive visualization, and they constitute an ideal gateway to the platform. IPython Interactive Computing and Visualization Cookbook, Second Edition contains many ready-to-use, focused recipes for high-performance scientific computing and data analysis, from the latest IPython/Jupyter features to the most advanced tricks, to help you write better and faster code. You will apply these state-of-the-art methods to various real-world examples, illustrating topics in applied mathematics, scientific modeling, and machine learning. The first part of the book covers programming techniques: code quality and reproducibility, code optimization, high-performance computing through just-in-time compilation, parallel computing, and graphics card programming. The second part tackles data science, statistics, machine learning, signal and image processing, dynamical systems, and pure and applied mathematics.
Table of Contents (17 chapters)
close
close
Preface
Preface
Who this book is for
What this book covers
To get the most out of this book
Sections
Get in touch
Free Chapter
1
1. A Tour of Interactive Computing with Jupyter and IPython
1. A Tour of Interactive Computing with Jupyter and IPython
Introduction
Introducing IPython and the Jupyter Notebook
Getting started with exploratory data analysis in the Jupyter Notebook
Introducing the multidimensional array in NumPy for fast array computations
Creating an IPython extension with custom magic commands
Mastering IPython's configuration system
Creating a simple kernel for Jupyter
2
2. Best Practices in Interactive Computing
2. Best Practices in Interactive Computing
Introduction
Learning the basics of the Unix shell
Using the latest features of Python 3
Learning the basics of the distributed version control system Git
A typical workflow with Git branching
Efficient interactive computing workflows with IPython
Ten tips for conducting reproducible interactive computing experiments
Writing high-quality Python code
Writing unit tests with pytest
Debugging code with IPython
3
3. Mastering the Jupyter Notebook
3. Mastering the Jupyter Notebook
Introduction
Teaching programming in the Notebook with IPython Blocks
Converting a Jupyter notebook to other formats with nbconvert
Mastering widgets in the Jupyter Notebook
Creating custom Jupyter Notebook widgets in Python, HTML, and JavaScript
Configuring the Jupyter Notebook
Introducing JupyterLab
4
4. Profiling and Optimization
chevron up
4. Profiling and Optimization
Introduction
Evaluating the time taken by a command in IPython
Profiling your code easily with cProfile and IPython
Profiling your code line-by-line with line_profiler
Profiling the memory usage of your code with memory_profiler
Understanding the internals of NumPy to avoid unnecessary array copying
Using stride tricks with NumPy
Implementing an efficient rolling average algorithm with stride tricks
Processing large NumPy arrays with memory mapping
Manipulating large arrays with HDF5
5
5. High-Performance Computing
5. High-Performance Computing
Introduction
Using Python to write faster code
Accelerating pure Python code with Numba and Just-In-Time compilation
Accelerating array computations with NumExpr
Wrapping a C library in Python with ctypes
Accelerating Python code with Cython
Optimizing Cython code by writing less Python and more C
Releasing the GIL to take advantage of multi-core processors with Cython and OpenMP
Writing massively parallel code for NVIDIA graphics cards (GPUs) with CUDA
Distributing Python code across multiple cores with IPython
Interacting with asynchronous parallel tasks in IPython
Performing out-of-core computations on large arrays with Dask
Trying the Julia programming language in the Jupyter Notebook
6
6. Data Visualization
6. Data Visualization
Introduction
Using Matplotlib styles
Creating statistical plots easily with seaborn
Creating interactive web visualizations with Bokeh and HoloViews
Visualizing a NetworkX graph in the Notebook with D3.js
Discovering interactive visualization libraries in the Notebook
Creating plots with Altair and the Vega-Lite specification
7
7. Statistical Data Analysis
7. Statistical Data Analysis
Introduction
Exploring a dataset with pandas and Matplotlib
Getting started with statistical hypothesis testing — a simple z-test
Getting started with Bayesian methods
Estimating the correlation between two variables with a contingency table and a chi-squared test
Fitting a probability distribution to data with the maximum likelihood method
Estimating a probability distribution nonparametrically with a kernel density estimation
Fitting a Bayesian model by sampling from a posterior distribution with a Markov chain Monte Carlo method
Analyzing data with the R programming language in the Jupyter Notebook
8
8. Machine Learning
8. Machine Learning
Introduction
Getting started with scikit-learn
Predicting who will survive on the Titanic with logistic regression
Learning to recognize handwritten digits with a K-nearest neighbors classifier
Learning from text – Naive Bayes for Natural Language Processing
Using support vector machines for classification tasks
Using a random forest to select important features for regression
Reducing the dimensionality of a dataset with a principal component analysis
Detecting hidden structures in a dataset with clustering
9
9. Numerical Optimization
9. Numerical Optimization
Introduction
Finding the root of a mathematical function
Minimizing a mathematical function
Fitting a function to data with nonlinear least squares
Finding the equilibrium state of a physical system by minimizing its potential energy
10
10. Signal Processing
10. Signal Processing
Introduction
Analyzing the frequency components of a signal with a Fast Fourier Transform
Applying a linear filter to a digital signal
Computing the autocorrelation of a time series
11
11. Image and Audio Processing
11. Image and Audio Processing
Introduction
Manipulating the exposure of an image
Applying filters on an image
Segmenting an image
Finding points of interest in an image
Detecting faces in an image with OpenCV
Applying digital filters to speech sounds
Creating a sound synthesizer in the Notebook
12
12. Deterministic Dynamical Systems
12. Deterministic Dynamical Systems
Introduction
Plotting the bifurcation diagram of a chaotic dynamical system
Simulating an elementary cellular automaton
Simulating an ordinary differential equation with SciPy
Simulating a partial differential equation — reaction-diffusion systems and Turing patterns
13
13. Stochastic Dynamical Systems
13. Stochastic Dynamical Systems
Introduction
Simulating a discrete-time Markov chain
Simulating a Poisson process
Simulating a Brownian motion
Simulating a stochastic differential equation
14
14. Graphs, Geometry, and Geographic Information Systems
14. Graphs, Geometry, and Geographic Information Systems
Introduction
Manipulating and visualizing graphs with NetworkX
Drawing flight routes with NetworkX
Resolving dependencies in a directed acyclic graph with a topological sort
Computing connected components in an image
Computing the Voronoi diagram of a set of points
Manipulating geospatial data with Cartopy
Creating a route planner for a road network
15
15. Symbolic and Numerical Mathematics
15. Symbolic and Numerical Mathematics
Introduction
Diving into symbolic computing with SymPy
Solving equations and inequalities
Analyzing real-valued functions
Computing exact probabilities and manipulating random variables
A bit of number theory with SymPy
Finding a Boolean propositional formula from a truth table
Analyzing a nonlinear differential system — Lotka-Volterra (predator-prey) equations
Getting started with Sage
16
Index
Index

Profiling the memory usage of your code with memory_profiler

The methods described in the previous recipe were about CPU time profiling. That may be the most obvious factor when it comes to code profiling. However, memory is also a critical factor. Writing memory-optimized code is not trivial and can really make your program faster. This is particularly important when dealing with large NumPy arrays, as we will see later in this chapter.

In this recipe, we will look at a simple memory profiler unsurprisingly named memory_profiler. Its usage is very similar to line_profiler, and it can be conveniently used from IPython.

Getting ready

You can install memory_profiler with conda install memory_profiler.

How to do it...

  1. We load the memory_profiler IPython extension:
    >>> %load_ext memory_profiler
  2. We define a function that allocates big objects:
    >>> %%writefile memscript.py
    def my_func():
        a = [1] * 1000000
        b = [2] * 9000000
        del b
        return a
  3. Now, let&apos...

Unlock full access

Continue reading for free

A Packt free trial gives you instant online access to our library of over 7000 practical eBooks and videos, constantly updated with the latest in tech
Start a 7-day FREE trial
End of Section 6
Next Section
bookmark search playlist font-size

Change the font size

margin-width

Change margin width

day-mode

Change background colour

Close icon Search
Country selected
Close icon Your notes and bookmarks

Delete Bookmark

Modal Close icon
Are you sure you want to delete it?
Cancel
Yes, Delete