An evolutionary neural network implementation, inspired by Neuroevolution of Augmenting Topologies. The Caesium library is currently available for JavaScript. Python and Node.js implementations are planned.
The neural network architecture used for Caesium deviates substantially from that of a traditional neural network.
For one, there are no "weights" used in the synapses. This is perhaps the most integral part of a neural network, and the one that most closely mimics a biological neural network. Caesium's goal is not to imitate biological neural networks, but to take advantage of the unique abilities of computational processing to create a new kind of data model.
Caesium aims to create as basic as possible a framework for the evolutionary optimization strategy to build a model with. At the highest level, this is reflected by Caesium's NEAT architecture, which uses individual nodes and connections to represent the flow of data, instead of linear stacks of "layers".
Caesium uses basic operations performed on data to represent its computations, instead of a traditional model with weights, biases, activation functions, and other properties of traditional neural networks. Operations are completed in nodes, and there are currently two types: addition and multiplication. Both nodes can have an infinite number of inputs and an infinite number of outputs. The addition node takes several scalar values as input and returns the sum of all these values as its output. The multiplication node returns the product of all of its inputs.
A more basic representation of operations performed on data allows for more flexible, versatile models.
The flow of data through neural networks in Caesium is regulated by delays, stored in the connections between nodes.
The value of each node is evaluated simultaneously based on a buffer of the stored values of each node. Similarly to a biological neural network, the entire network is constantly being updated and re-evaluated, as opposed to a traditional model, in which each layer would be evaluated sequentially, until the output layer is calculated. This would not be feasible for the organic style of Caesium's networks, in which input and output nodes are spread throughout the network. Constant evaluation also allows for the network to constantly "think" about problems that may require more than one computational iteration to solve, and allows the network to recursively apply dynamic mathematical operations and functions on data to iteratively transform it.
For example, instead of stacking several multiplication nodes together to approximate an exponent function, the network could develop a "multiplication loop" that multiplies the result of a computation by another number, then repeats this operation on the output of the function until another signal tells the network to end the loop and send the output to the global output node, theoretically allowing for more efficiency and smaller networks.
A hard limit can be placed on the number of nodes and/or connections that the neuroevolution algorithm can add to the network. This prevents the network growing indefinitely in size and reduces memory usage. It also encourages the evolution-based optimization algorithm to find better solutions to problems with fewer nodes, therefore reducing overfitting and improving generalization.
This is a technical guide to specific aspects of the Caesium library implementation in different programming languages for development reference purposes. Full documentation will be created soon.
A Python implementation of Caesium is planned.
A Node.js implementation of Caesium is planned.
This project wouldn't have been possible without lots of great open-source technology that people have been kind enough to share with the world. All sources listed here are also credited in code comments.
Smaller functions and pieces of code.
var uuids = [];
const UUID = function () {
// Generate a random string of four hexadecimal digits
function s4() {
return Math.floor((1 + Math.random()) * 0x10000)
.toString(16)
.substring(1);
}
// Generate UUID from substrings
var id;
do {
id = s4() + s4() + "-" + s4() + "-" + s4() + "-" + s4() + "-" + s4() + s4() + s4();
}
while (uuids.indexOf(id) !== -1)
uuids.push(id);
return id;
}const map = function (num, in_min, in_max, out_min, out_max) {
return (num - in_min) * (out_max - out_min) / (in_max - in_min) + out_min;
}const shade_color = function (color, percent) {
var f=parseInt(color.slice(1),16),t=percent<0?0:255,p=percent<0?percent*-1:percent,R=f>>16,G=f>>8&0x00FF,B=f&0x0000FF;
return "#"+(0x1000000+(Math.round((t-R)*p)+R)*0x10000+(Math.round((t-G)*p)+G)*0x100+(Math.round((t-B)*p)+B)).toString(16).slice(1);
}var min = Math.min.apply(Math, this.nodes.map(function(x) { return x.value; }));
var max = Math.max.apply(Math, this.nodes.map(function(x) { return x.value; }));A series of demos to show just how cool Caesium is and what it is capable of. Feel free to clone any demo and create a new project out of it.
Literally just an HTML page with the Caesium library loaded. Have fun.
Visualize all the nodes and connections in a Caesium network.
Approximate a polynomial function using neuroevolution.
Evolve a randomly generated population of neural networks.
Learn to classify 2D points based on their X and Y coordinates. WARNING: Flashing colors
Media and images used in Caesium.
Used for backgrounds in the JavaScript/web version of Caesium.
transparenttextures.com/patterns/brushed-alum.png
By Tim Ward
Used for the network visualization background.
transparenttextures.com/patterns/az-subtle.png
By Anli.
Used for the control panel/sidebar background.
cssreset.com/scripts/eric-meyer-reset-css
Used to reset the default CSS rules to make development a lot easier.
Used to create the ASCII logo art displayed in the console. See issue #20.