I too am perplexed as to what the neurons represent, the blog describes the RNN as 19 inputs connected to 7 outputs.
Examining slimevolley_pro.js (Agent.prototype.drawstate) shows only the agent states are depicted onscreen x,y,vx,vy,bx,by,bvx,bvy in the top row and actions forward, jump and backward below.
The code is well documented and builds on convnet.js by Karpathy.
The network is trained by genetic algorithm and self play - so this is a neural net trained with reinforcement learning.
The author's method seems unique and effective.
The blog post comparison of the resultant 'genomes' of network weights seem to show that the space has been searched exhaustively.
The game is played with trained networks but the code contains a training flag - it will be interesting to watch them train with self play.
The method is not unique. See some of the work of Schmidhuber's group. They are doing a lot of reinforcement learning for recurrent nets (LSTMs) and also via evolutionary algorithms. Eg see Evolino.
To be honest I haven't played with NNs, but it puzzles me as to why the non-recurrent approach is so prevalent for complex tasks. I mean, it's the basic combinatorial circuit vs sequential circuits, which we all know are much more suited for complex or large outputs. Where's everything we learned from synchronous logic synthesis?
I will check those out, thanks. This is indeed a fascinating topic. I guess every scientist wants to understand learning.
The connection comes up in david mckay's Information Theory book too, a reading I definitively recommend, although I haven't been through it properly myself.
McKays Information Theory is a brilliant read so far, many thanks.
Having learning couched in Information Theory terms brings it all right back to Claude Shannon's early work on Reinforcement Learning Chess programs and Alan Turing's ideas about evolving efficient machine code by bitmask genetic recombination.
Grave's Handwriting Net was trained using Backpropagation - whereby the error between the net's estimate and a training target is sent backward through the net - so the net's estimates gradually become closer to the targets.
Backpropagation takes longer the deeper the net - Recurrent Neural Nets are deep in time so Back Propagation can become intractable or unstable.
Otoro's Slimeball demo evolves a Recurrent Net rather than training it - this appears to be a very efficient method, less likely to get stuck in local minima.
The slimes evolve through self-play which is a trial and error method and reinforcement methods seem to do better on control tasks than passive learning.
Ah I see. But as far as training goes the difference between the two methods ("evolution" and backprop) is a matter of locality, no? The backprop modifies weights loosely based on local gradiet towards fitness, and evolution goes in sparse random directions. In this view backprop is indeed vulnerable to local maxima if your optimization method isn't very good, but isn't it just a matter of choosing good optimization methods? In other words, combining local backprop optimization with global evolutionary methods should be the role of robust optimization algos, no?
It's just an abbreviated form of the nets shown in this (http://blog.otoro.net/wp-content/uploads/sites/2/2015/03/sli...) image, the 3 outputs displayed correspond to the "forward", "jump", and "back" on the diagram, and the 8 inputs correspond to the agents' and balls' position vectors (which you can see in the Brain() code in slimevolley_pro.js).
Since RNNs are self connected, in some sense it has a lot more depth than just 2, so they can learn very non-linear features (but the weights of the connections are shared across the layers).
