Use the ChatNoir search engine in PyTerrier.
Powered by the chatnoir-api
package.
Install the package from PyPI:
pip install chatnoir-pyterrier
You can use the ChatNoirRetrieve
PyTerrier module in any PyTerrier pipeline, like you would do with BatchRetrieve
.
from chatnoir_pyterrier import ChatNoirRetrieve
api_key: str = "<API_KEY>"
chatnoir = ChatNoirRetrieve(api_key)
chatnoir.search("python library")
Please check out our sample notebook or open it in Google Colab.
Are you participating in Touché 2023 task 1 or task 2? We've prepared a sample notebook to show you how to retrieve from the ClueWeb22. Get started by opening it in Google Colab.
If you use this package, please cite the paper from the ChatNoir authors. You can use the following BibTeX information for citation:
@InProceedings{bevendorff:2018,
address = {Berlin Heidelberg New York},
author = {Janek Bevendorff and Benno Stein and Matthias Hagen and Martin Potthast},
booktitle = {Advances in Information Retrieval. 40th European Conference on IR Research (ECIR 2018)},
editor = {Leif Azzopardi and Allan Hanbury and Gabriella Pasi and Benjamin Piwowarski},
month = mar,
publisher = {Springer},
series = {Lecture Notes in Computer Science},
site = {Grenoble, France},
title = {{Elastic ChatNoir: Search Engine for the ClueWeb and the Common Crawl}},
year = 2018
}
To build this package and contribute to its development you need to install the build
, and setuptools
and wheel
packages:
pip install build setuptools wheel
(On most systems, these packages are already pre-installed.)
Install package and test dependencies:
pip install -e .[test]
Configure the API keys for testing:
export CHATNOIR_API_KEY="<API_KEY>"
export CHATNOIR_API_KEY_STAGING="<API_KEY>"
Verify your changes against the test suite to verify.
flake8 chatnoir_pyterrier tests examples
pylint -E chatnoir_pyterrier tests examples
pytest chatnoir_pyterrier tests examples
Please also add tests for your newly developed code.
Wheels for this package can be built with:
python -m build
If you hit any problems using this package, please file an issue. We're happy to help!
This repository is released under the MIT license.