Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Create Neuron.Activation class or factory #88

Open
levithomason opened this issue Dec 21, 2015 · 4 comments
Open

Create Neuron.Activation class or factory #88

levithomason opened this issue Dec 21, 2015 · 4 comments

Comments

@levithomason
Copy link
Owner

Currently there is an activation namespace. Its members are activation function definitions. These are objects with func, prime, rangeMin, and rangeMax properties.

These objects should be created by a class or factory so they can be validated, extended, and created consistently. For better cohesion, this should be a static Neuron class or factory. Same for the namespace, it should likely be a Neuron namespace.

We should likely end up with something along the lines of:

// current namespace with its activations
Neuron.ACTIVATION.tanh

// create new activation
Neuron.ACTIVATION.create({
  name: 'newActivation',
  func: x => x,
  prime: x => x,
  rangeMin: 0,
  rangeMax: 1,
})

// => Neuron.ACTIVATION.newActivation

EDIT
Also, finish the docs in the Neuron class while at this. There are missing and incomplete doc strings here.

@ckcollab
Copy link
Contributor

From my understanding that sounds good.

Will there be an easy way to iterate over them? Neuron.ACTIVATION.get_all() or what have you? May not be helpful or needed.

Also, is there a way to "choose" one for you? Neuron.ACTIVATION.find_a_good_fit(some_parameters_i_dont_understand)

@levithomason
Copy link
Owner Author

Since it is just an object you can loop over them like:

// lodash
_.each(Neuron.Activation, (activation, name) => {...})

// vanilla js
Object.keys(Neuron.Activation).forEach(key => {
  Neuron.Activation[key]
})

// or
for (const key in Neuron.Activation) {
  if (Neuron.Activation.hasOwnProperty(key) {
    // do stuff
  }
}

There are possibly heuristics we can use to try and find an activation function for you. It might require asking some questions or something. Realistically, I think knowing what they do and what they are good at is essential to good machine learning. Also, experimenting is really fun and another great way to findTheBestFunction :)

@ckcollab
Copy link
Contributor

👍 thanks for explaining

So you'd make _.each ignore create()? Or does _.each only cover objects and not functions... but I thought functions are objects... how does that work? :)

@levithomason
Copy link
Owner Author

You're right, in these cases it would totally pick up create(). That is what I get for coding at this hour :P The implementation of this fix will be something different, but the goal is the same and remains good. Split out the activation function definition somewhere, validate it, and allow adding new ones.

Thanks for the dive in and catching that. Night night 🌝

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants