Sign In Start Free Trial

Book Overview & Buying
Table Of Contents
Feedback & Rating

PyTorch 1.x Reinforcement Learning Cookbook

By : Yuxi (Hayden) Liu

4.3 (3)

PyTorch 1.x Reinforcement Learning Cookbook

4.3 (3)

By: Yuxi (Hayden) Liu

Overview of this book

Reinforcement learning (RL) is a branch of machine learning that has gained popularity in recent times. It allows you to train AI models that learn from their own actions and optimize their behavior. PyTorch has also emerged as the preferred tool for training RL models because of its efficiency and ease of use. With this book, you'll explore the important RL concepts and the implementation of algorithms in PyTorch 1.x. The recipes in the book, along with real-world examples, will help you master various RL techniques, such as dynamic programming, Monte Carlo simulations, temporal difference, and Q-learning. You'll also gain insights into industry-specific applications of these techniques. Later chapters will guide you through solving problems such as the multi-armed bandit problem and the cartpole problem using the multi-armed bandit algorithm and function approximation. You'll also learn how to use Deep Q-Networks to complete Atari games, along with how to effectively implement policy gradients. Finally, you'll discover how RL techniques are applied to Blackjack, Gridworld environments, internet advertising, and the Flappy Bird game. By the end of this book, you'll have developed the skills you need to implement popular RL algorithms and use RL techniques to solve real-world problems.

Preface

Preface

Who this book is for

What this book covers

To get the most out of this book

Sections

Get in touch

Free Chapter

Getting Started with Reinforcement Learning and PyTorch

Getting Started with Reinforcement Learning and PyTorch

Setting up the working environment

Installing OpenAI Gym

Simulating Atari environments

Simulating the CartPole environment

Reviewing the fundamentals of PyTorch

Implementing and evaluating a random search policy

Developing the hill-climbing algorithm

Developing a policy gradient algorithm

Markov Decision Processes and Dynamic Programming

Markov Decision Processes and Dynamic Programming

Technical requirements

Creating a Markov chain

Creating an MDP

Performing policy evaluation

Simulating the FrozenLake environment

Solving an MDP with a value iteration algorithm

Solving an MDP with a policy iteration algorithm

Solving the coin-flipping gamble problem

Monte Carlo Methods for Making Numerical Estimations

Monte Carlo Methods for Making Numerical Estimations

Calculating Pi using the Monte Carlo method

Performing Monte Carlo policy evaluation

Playing Blackjack with Monte Carlo prediction

Performing on-policy Monte Carlo control

Developing MC control with epsilon-greedy policy

Performing off-policy Monte Carlo control

Developing MC control with weighted importance sampling

Temporal Difference and Q-Learning

Temporal Difference and Q-Learning

Setting up the Cliff Walking environment playground

Developing the Q-learning algorithm

Setting up the Windy Gridworld environment playground

Developing the SARSA algorithm

Solving the Taxi problem with Q-learning

Solving the Taxi problem with SARSA

Developing the Double Q-learning algorithm

Solving Multi-armed Bandit Problems

Solving Multi-armed Bandit Problems

Creating a multi-armed bandit environment

Solving multi-armed bandit problems with the epsilon-greedy policy

Solving multi-armed bandit problems with the softmax exploration

Solving multi-armed bandit problems with the upper confidence bound algorithm

Solving internet advertising problems with a multi-armed bandit

Solving multi-armed bandit problems with the Thompson sampling algorithm

Solving internet advertising problems with contextual bandits

Scaling Up Learning with Function Approximation

Scaling Up Learning with Function Approximation

Setting up the Mountain Car environment playground

Estimating Q-functions with gradient descent approximation

Developing Q-learning with linear function approximation

Developing SARSA with linear function approximation

Incorporating batching using experience replay

Developing Q-learning with neural network function approximation

Solving the CartPole problem with function approximation

Deep Q-Networks in Action

Deep Q-Networks in Action

Developing deep Q-networks

Improving DQNs with experience replay

Developing double deep Q-Networks

Tuning double DQN hyperparameters for CartPole

Developing Dueling deep Q-Networks

Applying Deep Q-Networks to Atari games

Using convolutional neural networks for Atari games

Implementing Policy Gradients and Policy Optimization

Implementing Policy Gradients and Policy Optimization

Implementing the REINFORCE algorithm

Developing the REINFORCE algorithm with baseline

Implementing the actor-critic algorithm

Solving Cliff Walking with the actor-critic algorithm

Setting up the continuous Mountain Car environment

Solving the continuous Mountain Car environment with the advantage actor-critic network

Playing CartPole through the cross-entropy method

Capstone Project – Playing Flappy Bird with DQN

Capstone Project – Playing Flappy Bird with DQN

Setting up the game environment

Building a Deep Q-Network to play Flappy Bird

Training and tuning the network

Deploying the model and playing the game

Other Books You May Enjoy

Other Books You May Enjoy

Leave a review - let other readers know what you think

Customer Reviews

4.3 (3)

5 star

33.3%

4 star

66.7%

3 star

0

2 star

0

1 star

0

Preface

The surge in interest in reinforcement learning is due to the fact that it revolutionizes automation by learning the optimal actions to take in an environment in order to maximize the notion of cumulative reward.

PyTorch 1.x Reinforcement Learning Cookbook introduces you to important reinforcement learning concepts and implementations of algorithms in PyTorch. Each chapter of the book walks you through a different type of reinforcement learning method and its industry-adopted applications. With the help of recipes that contain real-world examples, you will find it intriguing to enhance your knowledge and proficiency of reinforcement learning techniques in areas such as dynamic programming, Monte Carlo methods, temporal difference and Q-learning, multi-armed bandit, function approximation, deep Q-Networks, and policy gradients—they are no more obscure than you thought. Interesting and easy-to-follow examples, such as Atari games, Blackjack, Gridworld environments, internet advertising, Mountain Car, and Flappy Bird, will keep you interested until you reach your goal.

By the end of this book, you will have mastered the implementation of popular reinforcement learning algorithms and learned the best practices of applying reinforcement learning techniques to solve other real-world problems.

Search

Your notes and bookmarks