Returns query ids. Used internally.

Compute the performance scores for the provided `qrels` and `run` for all the specified metrics. Usage examples: from ranx import evaluate

Convert Run to Numba Typed List. Used internally.

Forces type conversion to numba.types.unicode_type.

Add a (doc_id, score) pair to a query (or, change its value if it already exists). Args: q_id (str): Query ID doc_id

Add a (doc_id, score) pair to a query (or, change its value if it already exists). Args: q_id (str): Query ID doc_id

Sort. Used internally.

Convert a Python dictionary in form of {q_id: {doc_id: score}} to ranx.Run. Args: d (Dict[str, Dict[str, int]]): Run as Python di

Returns query ids. Used internally.

r"""Computes CombSUM as proposed by [Fox et al.](https://trec.nist.gov/pubs/trec2/papers/txt/23.txt). ```bibtex @inproceedings{DBLP:conf/

Convert Run to Python dictionary. Returns: Dict[str, Dict[str, int]]: Run as Python dictionary

Evaluate multiple `runs` and compute statistical tests. Usage example: ```python from ranx import compare # Compare different runs a

Parse a run file into ranx.Run. Supported formats are JSON, TREC run, gzipped TREC run, and LZ4. Correct import behavior is inferred from the file ext

Add multiple queries at once. Args: q_ids (List[str]): List of Query IDs doc_ids (List[List[str]]): List of list of D

Add multiple queries at once. Args: q_ids (List[str]): List of Query IDs doc_ids (List[List[str]]): List of list of D

Compute the hit lists (at k). **Hit list** is a binary list whose element in position p is 1 if the document retrieved in position p is relevant,

Convert a Pandas DataFrame to ranx.Run. Args: df (pd.DataFrame): Run as Pandas DataFrame q_id_col (str, optional): Qu

Forces type conversion to numba.types.unicode_type.

Extract the scores from a given results dictionary.

Convert a Python dictionary in form of {q_id: {doc_id: score}} to ranx.Qrels. Args: d (Dict[str, Dict[str, int]]): Qrels as Pytho

r"""Computes Rank-Biased Centroid (RBC) as proposed by [Bailey et al.](https://dl.acm.org/doi/10.1145/3077136.3080839). ```bibtex @inproceedi

Convert Run to Pandas DataFrame with the following columns: `q_id`, `doc_id`, and `score`. Returns: pandas.DataFrame: Run as Pand

Returns Report as LaTeX table. Returns: str: LaTeX table

r"""Computes CombGMNZ as proposed by [Joon Ho Lee](https://dl.acm.org/doi/10.1145/258525.258587). ```bibtex @inproceedings{DBLP:conf/sigi

r"""Computes CombMNZ as proposed by [Fox et al.](https://trec.nist.gov/pubs/trec2/papers/txt/23.txt).<br /> ```bibtex @inproceedings{DBLP

Apply `min_max norm` to a given run. Args: run (Run): Run to be normalized. Returns: Run: Normalized run.

r"""Compute Rank-biased Precision (RBP). It is defined as: $$ \operatorname{RBP} = (1 - p) \cdot \sum_{i=1}^{d}{r_i \cdot p^{i - 1}}

Write `run` to `path` as JSON file, TREC run, LZ4 file, or Parquet file. File type is automatically inferred form the filename extension: ".json" -> "

r"""Computes SlideFuse as proposed by [Lillis et al.](https://link.springer.com/chapter/10.1007/978-3-540-78646-7_33). ```bibtex @inproceedin

r"""Computes BordaFuse as proposed by [Aslam et al.](https://dl.acm.org/doi/10.1145/383952.384007). ```bibtex @inproceedings{DBLP:conf/sigir/

r"""Computes CondorcetFuse as proposed by [Montague et al.](https://dl.acm.org/doi/10.1145/584792.584881). ```bibtex @inproceedings{DBLP:conf

r"""Compute F1 (at k). **F1** is the harmonic mean of [**Precision**][ranx.metrics.precision] and [**Recall**][ranx.metrics.recall].<br /> If

Convert a Parquet file to ranx.Run. Args: path (str): File path. q_id_col (str, optional): Query IDs column. Defaults

r"""Computes Log_ISR as proposed by [Mourão et al.](https://www.sciencedirect.com/science/article/abs/pii/S0895611114000664).<br /> ```bibtex

Adds empty results for queries missing from the run and removes those not appearing in qrels.

r"""Computes Mixed as proposed by [Wu et al.](https://dl.acm.org/doi/10.1145/584792.584908). ```bibtex @inproceedings{DBLP:conf/cikm/WuC0

r"""Compute Precision (at k). **Precision** is the proportion of the retrieved documents that are relevant.<br /> If k > 0, only the top-k re

r"""Computes ProbFuse as proposed by [Lillis et al.](https://dl.acm.org/doi/10.1145/1148170.1148197). ```bibtex @inproceedings{DBLP:conf/sigi

Converts a nested Python Dictionary to Numba Typed List to be used with ranx's metrics with no effort. Note: Doc IDs will be hashed.

r"""Compute Recall (at k). **Recall** is the ratio between the retrieved documents that are relevant and the total number of relevant documents.<

r"""Computes Reciprocal Rank Fusion as proposed by [Cormack et al.](https://dl.acm.org/doi/10.1145/1571941.1572114). ```bibtex @inproceed

Sort. Used internally.

Convert Qrels to Python dictionary. Returns: Dict[str, Dict[str, int]]: Qrels as Python dictionary

Returns the Report data as a Python dictionary. ``` { "stat_test": "fisher" # metrics and model_names allows

r"""Computes Weighted BordaFuse as proposed by [Aslam et al.](https://dl.acm.org/doi/10.1145/383952.384007). ```bibtex @inproceedings{DBLP:co

r"""Computes Weighted CondorcetFuse as proposed by [Montague et al.](https://dl.acm.org/doi/10.1145/584792.584881). ```bibtex @inproceedings{