Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add a classification accuracy objective #294

Closed
dsherry opened this issue Dec 18, 2019 · 4 comments · Fixed by #624
Closed

Add a classification accuracy objective #294

dsherry opened this issue Dec 18, 2019 · 4 comments · Fixed by #624
Assignees
Labels
enhancement An improvement to an existing feature. good first issue Issues which would be a good starting point for new hires.

Comments

@dsherry
Copy link
Contributor

dsherry commented Dec 18, 2019

Definition (from wikipedia):
Screen Shot 2019-12-18 at 11.44.38 AM.png

This came up in the discussion around Dan Putler's initial work on a cross-platform performance comparison yesterday (notes in #293). Dan used a classification accuracy objective in Applied Modeling, H20 and R, but used recall in evalml. If we implement accuracy, that may provide a better performance comparison. Plus, the more metrics we support, the better.

It would be great to keep our implementation general enough to handle multiclass, but if we have to start with binary classification, so be it.

@dsherry dsherry added the enhancement An improvement to an existing feature. label Dec 18, 2019
@kmax12
Copy link
Contributor

kmax12 commented Feb 19, 2020

related to this, we should also implement balanced accuracy

@angela97lin
Copy link
Contributor

Got it, will update the objectives project plan w/ this!

@dsherry
Copy link
Contributor Author

dsherry commented Feb 19, 2020

@angela97lin my suggestion is to list this issue in the future work section if you want, but to not allow it to affect the current objective API project scope. I think our priority should still be to finish the project; then this would be a great first issue to work on once we have it done. That sound ok?

@dsherry dsherry added the good first issue Issues which would be a good starting point for new hires. label Feb 19, 2020
@dsherry
Copy link
Contributor Author

dsherry commented Feb 19, 2020

Ok after talking to Max and Angela, the plan is:

  1. This is still separate from the objective API project
  2. We can implement this at any point, before/during/after that project, which is why it's in the dev backlog. It's not a huge amount of work
  3. Its low priority for now
  4. This would be a good issue for a new hire

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement An improvement to an existing feature. good first issue Issues which would be a good starting point for new hires.
Projects
None yet
3 participants