Skip to content

Commit

Permalink
Evaluation Script bugfix - Initalize Annotated Corpus as list [resolves
Browse files Browse the repository at this point in the history
#70]

**Changes in `__main__.py`:**
* `load_annotated_corpus()` -> `load_annotated_corpus(return_iter=False)`.
**Reason:**
* Commit 29e3b56 adds return iteator functionality to `load_annotated_corpus`. Default is set `True`.
* Returned iterator is consumed by `for doc in tqdm(anno, unit=" doc", desc=f"Calculate n-dcg score for {name}"):`.
* After it is consumed it needs to be reset so that other summarizers can use `anno` but they did not.
* As a result only first summarizer in the list was evaluated and rest did not see a test data since the iterator was consumed.
* `anno` is now a list when the argument is set to `False`. Consume it as much as you wish.
  • Loading branch information
dafajon committed Sep 4, 2020
1 parent 716c5fd commit ad41663
Show file tree
Hide file tree
Showing 2 changed files with 2 additions and 2 deletions.
2 changes: 1 addition & 1 deletion sadedegel/about.py
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
__title__ = "sadedegel" # pragma: no cover
__version__ = "0.13.4" # pragma: no cover
__version__ = "0.13.5" # pragma: no cover
__release__ = True # pragma: no cover
__download_url__ = "https://github.com/globalmaksimum/sadedegel/releases" # pragma: no cover
__herokuapp_url__ = "https://sadedegel.herokuapp.com" # pragma: no cover
2 changes: 1 addition & 1 deletion sadedegel/summarize/__main__.py
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ def cli():
@click.option("-f", "--table-format", default="github")
def evaluate(table_format):
"""Evaluate all summarizers in sadedeGel"""
anno = load_annotated_corpus()
anno = load_annotated_corpus(return_iter=False)

scores = defaultdict(list)
for name, summarizer in tqdm(
Expand Down

0 comments on commit ad41663

Please sign in to comment.