Study presents large brain-like neural networks for AI

CWI researchers Bojian Yin and Sander Bohté show how brain-like neurons combined with novel learning methods enable training fast and energy-efficient spiking neural networks on a large scale. Potential applications range from wearable AI, to pervasive speech recognition and Augmented Reality.

While artificial neural networks are the backbone of the current AI revolution, they are only loosely inspired by networks of real, biological neurons such as our brain. The brain however is a much larger network and it can respond ultra-fast when triggered by external events. Also it is much more energy-efficient: the neurons of our nervous system communicate by exchanging electrical pulses, and they do so only sparingly. Spiking neural networks are a special type of artificial neural networks that mimic these properties of biological neurons more closely.

Implemented in chips, called neuromorphic hardware, such spiking neural networks hold the promise of bringing AI programs closer to users on their own devices. These local solutions are beneficial for privacy, robustness and responsiveness. Applications range from speech recognition in toys and appliances, health care monitoring and drone navigation to local surveillance.

Just like standard artificial neural networks, spiking neural networks need to be trained to perform their tasks well. However, the way in which such networks communicate, poses serious challenges. “The algorithms needed for this require a lot of computer memory, allowing us to only train small network models mostly for smaller tasks. This holds back many practical AI applications so far”, says Sander Bohté of CWI’s Machine Learning group.

Six million neurons

The new online learning algorithm makes it possible to learn directly from the data, enabling much larger spiking neural networks. Together with researchers from TU Eindhoven and research partner Holst Centre, Bohté and Yin demonstrated this in a system designed for recognizing and locating objects. For their study they used real time images of a busy street in Amsterdam: the underlying spiking neural network, SPYv4, has been trained in such a way that it can distinguish cyclists, pedestrians and cars and indicate exactly where they are.

“Previously, we could train neural networks with up to 10,000 neurons, now we can do the same quite easily for networks with more than six million neurons,” says Bohté. “With this, we can train highly capable spiking neural networks like our SPYv4.”

Future

And where does it all lead? Now having such powerful AI solutions based on spiking neural networks, chips are being developed that can run these AI programmes at very low power. Ultimately they will show up in many smart devices, like hearing-aides and augmented or virtual reality glasses.