Browse Library
Sign In Start Free Trial
Close icon Search icon
Account
Browse Library Advanced Search Sign In Start Free Trial

Add to playlist

Create a Playlist

Modal Close icon
You need to login to use this feature.
  • Reinforcement Learning Algorithms with Python
  • Toc
  • feedback
Reinforcement Learning Algorithms with Python

Reinforcement Learning Algorithms with Python

By : Lonza
3 (3)
close
Reinforcement Learning Algorithms with Python

Reinforcement Learning Algorithms with Python

3 (3)
By: Lonza

Overview of this book

Reinforcement Learning (RL) is a popular and promising branch of AI that involves making smarter models and agents that can automatically determine ideal behavior based on changing requirements. This book will help you master RL algorithms and understand their implementation as you build self-learning agents. Starting with an introduction to the tools, libraries, and setup needed to work in the RL environment, this book covers the building blocks of RL and delves into value-based methods, such as the application of Q-learning and SARSA algorithms. You'll learn how to use a combination of Q-learning and neural networks to solve complex problems. Furthermore, you'll study the policy gradient methods, TRPO, and PPO, to improve performance and stability, before moving on to the DDPG and TD3 deterministic algorithms. This book also covers how imitation learning techniques work and how Dagger can teach an agent to drive. You'll discover evolutionary strategies and black-box optimization techniques, and see how they can improve RL algorithms. Finally, you'll get to grips with exploration approaches, such as UCB and UCB1, and develop a meta-algorithm called ESBAS. By the end of the book, you'll have worked with key RL algorithms to overcome challenges in real-world applications, and be part of the RL research community.
Table of Contents (19 chapters)
close
close
Preface
Preface
Who this book is for
What this book covers
To get the most out of this book
Get in touch
Free Chapter
1
Section 1: Algorithms and Environments
Section 1: Algorithms and Environments
2
The Landscape of Reinforcement Learning
The Landscape of Reinforcement Learning
An introduction to RL
Elements of RL
Applications of RL
Summary
Questions
Further reading
3
Implementing RL Cycle and OpenAI Gym
Implementing RL Cycle and OpenAI Gym
Setting up the environment
OpenAI Gym and RL cycles
Development of ML models using TensorFlow
Introducing TensorBoard
Types of RL environments
Summary
Questions
Further reading
4
Solving Problems with Dynamic Programming
Solving Problems with Dynamic Programming
MDP
Categorizing RL algorithms
Dynamic programming
Summary
Questions
Further reading
5
Section 2: Model-Free RL Algorithms
Section 2: Model-Free RL Algorithms
6
Q-Learning and SARSA Applications
Q-Learning and SARSA Applications
Learning without a model
TD learning
SARSA
Applying SARSA to Taxi-v2
Q-learning
Applying Q-learning to Taxi-v2
Summary
Questions
7
Deep Q-Network
Deep Q-Network
Deep neural networks and Q-learning
DQN
DQN applied to Pong
DQN variations
Summary
Questions
Further reading
8
Learning Stochastic and PG Optimization
Learning Stochastic and PG Optimization
Policy gradient methods
Understanding the REINFORCE algorithm
REINFORCE with baseline
Learning the AC algorithm
Summary
Questions
Further reading
9
TRPO and PPO Implementation
TRPO and PPO Implementation
Roboschool
Natural policy gradient
Trust region policy optimization
Proximal Policy Optimization
Summary
Questions
Further reading
10
DDPG and TD3 Applications
DDPG and TD3 Applications
Combining policy gradient optimization with Q-learning
Deep deterministic policy gradient
Twin delayed deep deterministic policy gradient (TD3)
Summary
Questions
Further reading
11
Section 3: Beyond Model-Free Algorithms and Improvements
Section 3: Beyond Model-Free Algorithms and Improvements
12
Model-Based RL
Model-Based RL
Model-based methods
Combining model-based with model-free learning
ME-TRPO applied to an inverted pendulum
Summary
Questions
Further reading
13
Imitation Learning with the DAgger Algorithm
Imitation Learning with the DAgger Algorithm
Technical requirements
The imitation approach
Playing Flappy Bird
Understanding the dataset aggregation algorithm
IRL
Summary
Questions
Further reading
14
Understanding Black-Box Optimization Algorithms
chevron up
Understanding Black-Box Optimization Algorithms
Beyond RL
The core of EAs
Scalable evolution strategies
Applying scalable ES to LunarLander
Summary
Questions
Further reading
15
Developing the ESBAS Algorithm
Developing the ESBAS Algorithm
Exploration versus exploitation
Approaches to exploration
Epochal stochastic bandit algorithm selection
Summary
Questions
Further reading
16
Practical Implementation for Resolving RL Challenges
Practical Implementation for Resolving RL Challenges
Best practices of deep RL
Challenges in deep RL
Advanced techniques
RL in the real world
Future of RL and its impact on society
Summary
Questions
Further reading
17
Assessments
Assessments
18
Other Books You May Enjoy
Other Books You May Enjoy
Leave a review - let other readers know what you think

Summary

In this chapter, you learned about EAs, a new class of black-box algorithms inspired by biological evolution that can be applied to RL tasks. EAs solve these problems from a different perspective compared to reinforcement learning. You saw that many characteristics that we have to deal with when we design RL algorithms are not valid in evolutionary methods. The differences are in both the intrinsic optimization method and the underlying assumptions. For example, because EAs are black-box algorithms, we can optimize whatever function we want as we are no longer constrained to using differentiable functions, like we were with RL. EAs have many other advantages, as we saw throughout this chapter, but they also have numerous downsides.

Next, we looked at two evolutionary algorithms: genetic algorithms and evolution strategies. Genetic algorithms are more complex as they create...

End of Section 6
Next Section
bookmark search playlist download
font-size

Change the font size

margin-width

Change margin width

day-mode

Change background colour

Close icon Search
Country selected
Close icon Your notes and bookmarks

Delete Bookmark

Modal Close icon
Are you sure you want to delete it?
Cancel
Yes, Delete