Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement NLP transforms, KS alarm + detector #163

Open
wants to merge 29 commits into
base: dev
Choose a base branch
from

Conversation

Anmol-Srivastava
Copy link
Contributor

@Anmol-Srivastava Anmol-Srivastava commented Oct 17, 2023

As experimental code, this serves as an outline of the NLP framework rather than final versions. Closes #152

Added

  • transforms (Representation removed) related to alibi-detect example
  • basic Detector and KS test Alarm
  • updated wilds_datasets.ipynb example to reflect testing on civil-comments dataset

Outstanding Issues

  • design of the KS Alarm scheme -- test occurs feature-wise, but how should drift alarm?

@Anmol-Srivastava Anmol-Srivastava self-assigned this Oct 17, 2023
@tms-bananaquit
Copy link
Collaborator

Will need to add citations and update references for alibi-detect's paper and/or documentation before merging the experimental branch.

@Anmol-Srivastava
Copy link
Contributor Author

This is essentially ready to be merged in -- citations are added and the code is stable enough.

This also now closes #153 #154 and can enable proper integration into menelaus

@Anmol-Srivastava
Copy link
Contributor Author

Now also closes #164

Copy link
Collaborator

@tms-bananaquit tms-bananaquit Nov 9, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • Should probably remove output (warnings) that references local files (e.g. c:\Users\ASRIVASTAVA)
  • Should test building on readthedocs to make sure this renders correctly

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Resolving once I've done these

menelaus/experimental/alarm.py Show resolved Hide resolved
setup.cfg Show resolved Hide resolved
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Design nlp submodule and skeleton of drift detection methods
2 participants