Book Overview & Buying
Table Of Contents
Feedback & Rating

TensorFlow Reinforcement Learning Quick Start Guide

By : Balakrishnan

5 (2)

Buy this Book

TensorFlow Reinforcement Learning Quick Start Guide

5 (2)

By: Balakrishnan

Buy this Book

Overview of this book

Advances in reinforcement learning algorithms have made it possible to use them for optimal control in several different industrial applications. With this book, you will apply Reinforcement Learning to a range of problems, from computer games to autonomous driving. The book starts by introducing you to essential Reinforcement Learning concepts such as agents, environments, rewards, and advantage functions. You will also master the distinctions between on-policy and off-policy algorithms, as well as model-free and model-based algorithms. You will also learn about several Reinforcement Learning algorithms, such as SARSA, Deep Q-Networks (DQN), Deep Deterministic Policy Gradients (DDPG), Asynchronous Advantage Actor-Critic (A3C), Trust Region Policy Optimization (TRPO), and Proximal Policy Optimization (PPO). The book will also show you how to code these algorithms in TensorFlow and Python and apply them to solve computer games from OpenAI Gym. Finally, you will also learn how to train a car to drive autonomously in the Torcs racing car simulator. By the end of the book, you will be able to design, build, train, and evaluate feed-forward neural networks and convolutional neural networks. You will also have mastered coding state-of-the-art algorithms and also training agents for various control problems.

Preface

Who this book is for

What this book covers

To get the most out of this book

Get in touch

Free Chapter

Up and Running with Reinforcement Learning

Why RL?

The relationship between an agent and its environment

Identifying episodes

Identifying reward functions and the concept of discounted rewards

Learning the Markov decision process

Defining the Bellman equation

On-policy versus off-policy learning

Model-free and model-based training

Algorithms covered in this book

Summary

Questions

Further reading

Temporal Difference, SARSA, and Q-Learning

Technical requirements

Understanding TD learning

Understanding SARSA and Q-Learning

Cliff walking and grid world problems

Summary

Further reading

Deep Q-Network

Technical requirements

Learning the theory behind a DQN

Understanding target networks

Learning about replay buffer

Getting introduced to the Atari environment

Summary

Questions

Further reading

Double DQN, Dueling Architectures, and Rainbow

Technical requirements

Understanding Double DQN

Understanding dueling network architectures

Understanding Rainbow networks

Running a Rainbow network on Dopamine

Summary

Questions

Further reading

Deep Deterministic Policy Gradient

Technical requirements

Actor-Critic algorithms and policy gradients

Deep Deterministic Policy Gradient

Training and testing the DDPG on Pendulum-v0

Summary

Questions

Further reading

Asynchronous Methods - A3C and A2C

Technical requirements

The A3C algorithm

The A3C algorithm applied to CartPole

The A3C algorithm applied to LunarLander

The A2C algorithm

Summary

Questions

Further reading

Trust Region Policy Optimization and Proximal Policy Optimization

Technical requirements

Learning TRPO

Learning PPO

Using PPO to solve the MountainCar problem

Evaluating the performance

Summary

Questions

Further reading

Deep RL Applied to Autonomous Driving

Technical requirements

Car driving simulators

Learning to use TORCS

Training a DDPG agent to learn to drive

Training a PPO agent

Summary

Questions

Further reading

Assessment

Chapter 1

Chapter 3

Chapter 4

Chapter 5

Chapter 6

Chapter 7

Chapter 8

Other Books You May Enjoy

Leave a review - let other readers know what you think

Customer Reviews

5 (2)

5 star

100%

4 star

3 star

2 star

1 star

Deep Deterministic Policy Gradient

We will now delve into the DDPG algorithm, which is a state-of-the-art RL algorithm for continuous control. It was originally published by Google DeepMind in 2016 and has gained a lot of interest in the community, with several new variants proposed thereafter. As was the case in DQN, DDPG also uses target networks for stability. It also uses a replay buffer to reuse past data, and therefore, it is an off-policy RL algorithm.

The ddpg.py file is the main file from which we start the training and testing. It will call the training or testing functions, which are present in TrainOrTest.py. The AandC.py file has the TensorFlow code for the actor and the critic networks. Finally, replay_buffer.py stores the samples in a replay buffer by using a deque data structure. We will train the DDPG to learn to hold an inverted pendulum vertically, using OpenAI...