Sigmoid derivative in nn.js #22
Comments
In order that the train method works properly with the sigmoid activation function and the corrected dSigmoid , two lines in the train method need to be changed.
This is tricky. Your original code gave the correct answer with a sigmoid activation function, |
Some other small suggestions:
Matrix.randomize() uses a function uses a function from p5.js:
To follow the book more closely, I replaced this (in nn.js) with
and added the method nn_randomize_uniform in matrix.js:
This has the virtue of following the book's default method, and also removing the dependence of nn.js Hope you don't mind these minor nitpicks! |
This is wonderful, thank you so much for this detailed set of comments! I'm in the process of creating the video tutorials that correspond to this code so I'll work on adding these in as I go! |
As I read it, your formula for the derivative of the sigmoid function is wrong in nn.js.
You have
but it should be
Reference
The style and substance of your 'Coding Train' material is very enjoyable. 👍 Thank you.
The text was updated successfully, but these errors were encountered: