Hands-On Neuroevolution with Python

By : Omelianenko

3 (1)

Buy this Book

Hands-On Neuroevolution with Python

3 (1)

By: Omelianenko

Buy this Book

Overview of this book

Neuroevolution is a form of artificial intelligence learning that uses evolutionary algorithms to simplify the process of solving complex tasks in domains such as games, robotics, and the simulation of natural processes. This book will give you comprehensive insights into essential neuroevolution concepts and equip you with the skills you need to apply neuroevolution-based algorithms to solve practical, real-world problems. You'll start with learning the key neuroevolution concepts and methods by writing code with Python. You'll also get hands-on experience with popular Python libraries and cover examples of classical reinforcement learning, path planning for autonomous agents, and developing agents to autonomously play Atari games. Next, you'll learn to solve common and not-so-common challenges in natural computing using neuroevolution-based algorithms. Later, you'll understand how to apply neuroevolution strategies to existing neural network designs to improve training and inference performance. Finally, you'll gain clear insights into the topology of neural networks and how neuroevolution allows you to develop complex networks, starting with simple ones. By the end of this book, you will not only have explored existing neuroevolution-based algorithms, but also have the skills you need to apply them in your research and work assignments.

Preface

Who this book is for

What this book covers

To get the most out of this book

Get in touch

Free Chapter

Section 1: Fundamentals of Evolutionary Computation Algorithms and Neuroevolution Methods

Overview of Neuroevolution Methods

Evolutionary algorithms and neuroevolution-based methods

NEAT algorithm overview

Hypercube-based NEAT

Evolvable-Substrate HyperNEAT

Novelty Search optimization method

Summary

Further reading

Python Libraries and Environment Setup

Suitable Python libraries for neuroevolution experiments

Environment setup

Summary

Section 2: Applying Neuroevolution Methods to Solve Classic Computer Science Problems

Using NEAT for XOR Solver Optimization

Technical requirements

XOR problem basics

The objective function for the XOR experiment

Hyperparameter selection

Running the XOR experiment

Exercises

Summary

Pole-Balancing Experiments

Technical requirements

The single-pole balancing problem

Objective function for a single-pole balancing experiment

The single-pole balancing experiment

Exercises

The double-pole balancing problem

Objective function for a double-pole balancing experiment

Double-pole balancing experiment

Exercises

Summary

Autonomous Maze Navigation

Technical requirements

Maze navigation problem

Maze simulation environment

Objective function definition using the fitness score

Running the experiment with a simple maze configuration

Exercises

Running the experiment with a hard-to-solve maze configuration

Exercises

Summary

Novelty Search Optimization Method

Technical requirements

The NS optimization method

NS implementation basics

The fitness function with the novelty score

Experimenting with a simple maze configuration

Experimenting with a hard-to-solve maze configuration

Summary

Section 3: Advanced Neuroevolution Methods

Hypercube-Based NEAT for Visual Discrimination

Technical requirements

Indirect encoding of ANNs with CPPNs

Visual discrimination experiment basics

Visual discrimination experiment setup

Visual discrimination experiment

Exercises

Summary

ES-HyperNEAT and the Retina Problem

Technical requirements

Manual versus evolution-based configuration of the topography of neural nodes

Quadtree information extraction and ES-HyperNEAT basics

Modular retina problem basics

Modular retina experiment setup

Modular retina experiment

Exercises

Summary

Co-Evolution and the SAFE Method

Technical requirements

Common co-evolution strategies

SAFE method

Modified maze experiment

Modified Novelty Search

Modified maze experiment implementation

Modified maze experiment

Exercises

Summary

Deep Neuroevolution

Technical requirements

Deep neuroevolution for deep reinforcement learning

Evolving an agent to play the Frostbite Atari game using deep neuroevolution

Training an agent to play the Frostbite game

Running the Frostbite Atari experiment

Visual inspector for neuroevolution

Exercises

Summary

Section 4: Discussion and Concluding Remarks

Best Practices, Tips, and Tricks

Starting with problem analysis

Selecting the optimal search optimization method

Advanced visualization

Tuning hyperparameters

Performance metrics

Python coding tips and tricks

Summary

Concluding Remarks

What we learned in this book

Where to go from here

Summary

Other Books You May Enjoy

Leave a review - let other readers know what you think

Customer Reviews

3 (1)

5 star

4 star

3 star

100%

2 star

1 star

Visual discrimination experiment setup

In our experiment, during the training of the discriminator ANN, we use the resolution of the visual and target fields fixed at 11 x 11. Thus, the connective CPPN must learn the correct connectivity pattern between the 121 inputs of the visual field and the 121 outputs of the target fields, which results in a total of 14,641 potential connection weights.

The following diagram shows the scheme of the substrate for the discriminator ANN:

The state-space sandwich substrate of the discriminator ANN

The discriminator ANN shown in the diagram has two layers with nodes forming one two-dimensional planar grid per layer. The connective CPPN draws the connectivity patterns by connecting nodes from one layer to another.

At each generation of the evolution, each individual in the population (the genome encoding the CPPN) is evaluated for its ability...