Partial reproduction of Neural Ranking Models with Weak Supervision

Files

• filter_queries.py - keeps only important queries from dataset AOL-user-ct-collection
• diff_training_validation.py - removes duplicate queries between training and validation set
• parametrize_queries.py - creates the xml file needed by the Indri indexing and query engine to run queries
• docs_sparse.py - creates compressed sparse representation of corpus documents
• queries_sparse.py - creates compressed sparse representation of training queries
• embeddings.py - create word embeddings of documents
• query_embeddings.py - create word embeddings of training queries
• score_sparse.py - uses sparse representation as input for the score neural network
• rank_embed.py - uses embeddings as input for rank neural network
• bm25_baseline.py - runs BM25 on the queries and corpus
• score_model.py - uses dense representation on score neural network