NEAT (NeuroEvolution of Augmenting Topologies) is a popular neuroevolution algorithm, one of the few such algorithms that evolves the architectures of its networks in addition to the weights. For more information, see this research paper: http://nn.cs.utexas.edu/downloads/papers/stanley.ec02.pdf.
HyperNEAT is an extension to NEAT that indirectly encodes the weights of the network (called the substrate) with a separate network (called a CPPN, for compositional pattern-producing network). For more information on HyperNEAT, see this website: http://eplex.cs.ucf.edu/hyperNEATpage/.
Adaptive HyperNEAT is an extension to HyperNEAT which indirectly encodes both the initial weights and an update rule for the weights such that some learning can occur during a network's "lifetime." For more information, see this research paper: http://eplex.cs.ucf.edu/papers/risi_sab10.pdf.
PyTorch NEAT builds upon NEAT-Python by providing some functions which can turn a NEAT-Python genome into either a recurrent PyTorch network or a PyTorch CPPN for use in HyperNEAT or Adaptive HyperNEAT. We also provide some environments in which to test NEAT and Adaptive HyperNEAT, and a more involved example using the CPPN infrastructure with Adaptive HyperNEAT on a T-maze.
The following snippet turns a NEAT-Python genome into a recurrent PyTorch network:
from pytorch_neat.recurrent_net import RecurrentNet net = RecurrentNet.create(genome, config, bs) outputs = net.activate(some_array)
You can also turn a NEAT-Python genome into a CPPN:
from pytorch_neat.cppn import create_cppn cppn_nodes = create_cppn(genome, config)
A CPPN is represented as a graph structure. For easy evaluation, a CPPN's input and output nodes may be named:
from pytorch_neat.cppn import create_cppn [delta_w_node] = create_cppn( genome, config, ["x_in", "y_in", "x_out", "y_out", "pre", "post", "w"], ["delta_w"], ) delta_w = delta_w_node(x_in=some_array, y_in=other_array, ...)
We also provide some infrastructure for running networks in Gym environments:
from pytorch_neat.multi_env_eval import MultiEnvEvaluator from pytorch_neat.recurrent_net import RecurrentNet def make_net(genome, config, batch_size): return RecurrentNet.create(genome, config, batch_size) def activate_net(net, states): outputs = net.activate(states).numpy() return outputs[:, 0] > 0.5 def make_env(): return gym.make("CartPole-v0") evaluator = MultiEnvEvaluator( make_net, activate_net, make_env=make_env, max_env_steps=max_env_steps, batch_size=batch_size, ) fitness = evaluator.eval_genome(genome)
This allows multiple environments to run in parallel for efficiency.
A simple example using NEAT to solve the Cartpole can be run like this:
python3 -m examples.simple.main
And a simple example using Adaptive HyperNEAT to partially solve a T-maze can be run like this:
python3 -m examples.adaptive.main
Author / Support
PyTorch NEAT is extended from Python NEAT by Alex Gajewsky.
Questions can be directed to firstname.lastname@example.org.