Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Number of queries in attack_classification.py etc. #43

Open
superctj opened this issue Jun 6, 2021 · 1 comment
Open

Number of queries in attack_classification.py etc. #43

superctj opened this issue Jun 6, 2021 · 1 comment

Comments

@superctj
Copy link

superctj commented Jun 6, 2021

Thank you for open-sourcing the code! I would like to double-check the following concerns

  1. Does num_queries in attack_classification.py reflect how many queries were made to the classifier for each input text?

  2. Is the similarity threshold issue reported in the text-attack library already fixed in the current text-fooler implementation?

Another comment is that in the prerequisites part of README, the -r flag is missing in pip install requirements.txt. It should be pip install -r requirements.txt. Hope it helps :-0

@jind11
Copy link
Owner

jind11 commented Jun 7, 2021

  1. number of queries is the number of times that the classifier is queried for obtaining the output probability vectors
  2. In order to replicate the numbers in our paper, you should set the similarity threshold to 0.5
    Thanks for the comment. I will correct it soon!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants