Skip to content
This repository has been archived by the owner on Apr 11, 2023. It is now read-only.

NDCG computation #233

Open
Zamony opened this issue Feb 27, 2021 · 1 comment
Open

NDCG computation #233

Zamony opened this issue Feb 27, 2021 · 1 comment

Comments

@Zamony
Copy link

Zamony commented Feb 27, 2021

Hi there! In the function corresponding to NDCG calculation I noticed that ignore_rank_of_non_annotated_urls flag is present.

CodeSearchNet/src/relevanceeval.py

Line 76 in 76a006f

ignore_rank_of_non_annotated_urls: bool=True) -> float:

The question is: are your results in the paper calculated with ignore_rank_of_non_annotated_urls=True?
@Alex-HaochenLi
Copy link

Both are reported.
In Table 4, NDCG-Within for True, NDCG-All for False.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@Zamony @Alex-HaochenLi