node.js server session training a not-so-deep convolutional neural network (convnet) for simple image classification task.

I spent the last day or two figuring out node.js and how to write node.js modules, and also managed to get convnet.js and the neural network art generator working as node.js modules to run on the server side.

In the simple example above, a node.js server session is training a convnet to recognise CIFAR10 things and the progress is streamed (using this very elegant node canvas module) into a .png file viewable by a client browser.

Now that both server and client side can use the same consistent set of machine learning algorithms written in pure javascript, for both training the algorithm, and for getting it to perform some task, we can think of interesting ways to distribute computational intensive training tasks from a server to many different browser client sessions.  It may be possible to get browsers visiting a site to donate their browser sessions for the purpose of training a large machine learning algorithm on a small subset of a training data set, like a browser-based SETI@home.  Even though a single browser running javascript will be a lot slower than a high performance dedicated machine, tens of thousands of distributed collaborating clients, even smartphone browser clients, may be a force to be reckoned with.

Distributed training of neural networks is still an active area of research.  It’s not trivial to break up a task and combine partially trained neural networks.  For simple feedforward convnets though, I think that using the dropout technique for randomly turning on and off neurons during training, to achieve regularisation, may allow easier recombination of separately trained networks.

Distributed training will be easier for neuroevolution algorithms because in many training tasks, populations can be independently trained as long as they share a common fitness evaluation function for some task, say, driving a virtual race car along some track, or learning to play Super Mario World.  However, coevolution tasks would be more challenging to distribute to many machines.