As described in a previous post, the simple conventional neuroevolution method doesn’t seem to work well enough to both swing upwards and maintain the position by balancing itself.  I think the problem is that the training simply goes through many generations to get the a local optimal state that allows the pendulum to swing upwards, but that the training is always getting stuck at the local optimum, and cannot get to the global optimum state of being able to do both tasks (swing, and balance).

After a night of work on a flight, I managed to use the conventional neuroevolution (CNE) trainer module I created to train a box2d based double-pendulum-cart environment to self balance the a double inverted pendulum.  This was something very difficult to accomplish with normal PID controllers.  I think if I wanted to build a deterministic controller for this system, I could have used a more complicated LQR controller which would have handled the complexity of the system, but what I really wanted to accomplish was not to balance the pendulum, but rather train a network to learn about some environment and learn to adapt in that environment to achieve some task, so that these techniques can be applied to other more challenging tasks in the future.

I tried to design an interface for the trainer that doesn’t deviate from the original trainer interface designed by karpathy‘s convnet.js library.

It wasn’t too difficult to implement a conventional neuroevolution (CNE) trainer addin for convnet.js, since the existing structure of the code was beautifully designed and I can just follow and create a similar method that looks similar to existing backprop/SGD trainers.