Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Small correction to 06_multicat #647

Open
andrewkchan opened this issue Apr 16, 2024 · 0 comments
Open

Small correction to 06_multicat #647

andrewkchan opened this issue Apr 16, 2024 · 0 comments

Comments

@andrewkchan
Copy link

I believe the first sentence of this is not worded correctly:

Note that because we have a one-hot-encoded dependent variable, we can't directly use nll_loss or softmax (and therefore we can't use cross_entropy):

  • softmax, as we saw, requires that all predictions sum to 1, and tends to push one activation to be much larger than the others (due to the use of exp); however, we may well have multiple objects that we're confident appear in an image, so restricting the maximum sum of activations to 1 is not a good idea. By the same reasoning, we may want the sum to be less than 1, if we don't think any of the categories appear in an image.
  • nll_loss, as we saw, returns the value of just one activation: the single activation corresponding with the single label for an item. This doesn't make sense when we have multiple labels.

One-hot encoding does not disallow the use of softmax. The reason should be that the objective is multi-label, right? As the bullet points explain.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant