-
Notifications
You must be signed in to change notification settings - Fork 103
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
logloss issue with multiclass task #197
Comments
Thanks for opening this! Sorry I don't have a more helpful update, but I just wanted to say that I'm looking into this, and I do think there's a bug here. I'm making some regression tests using the Iris dataset, with a single label-encoded target column, so I'll probably ask you to try to reproduce some of my results when I'm further along in the bug hunt. In the meantime, have you tried adjusting the Like I said, I need to investigate some more, but I'd really appreciate you commenting any of your findings here! Edit: Thanks for looking for related issues, as well! |
Thanks for your quick reply! Sure I tried setting I have to say Here is the code I used to test:
|
Thanks for posting your sample code! It's very helpful! Sorry for the delay, but I've been busy with other things lately. I'm looking at this issue again today, and I have to agree with you Do you know of any other metrics that behave similarly or might cause other problems? |
Sorry for my late reply, bit busy these days... I think there are only two metrics that accept multi-column predicted probabilities:
I know in most cases there is no need to take |
Hi HunterMcGushion,
I am doing a multi-classification task and I wanna set
sklearn.metrics.log_loss
as the experiment metric, but I have a trouble:See, the target has 4 labels, 0 to 3. When I run the code above, it triggers a value error:
If I set
labels
for logloss metric,metrics=dict(logloss=lambda y_true, y_pred: metrics.log_loss(y_true, y_pred, labels=[0,1,2,3]))
, it throws out another error:ValueError: The number of classes in labels is different from that in y_pred.
I checked the examples and previous issues like #90, and I wonder have you tested logloss for multiclass task?
The text was updated successfully, but these errors were encountered: