Splits an input PCollection of examples into train and test. This uses the product id to compute the split, so that examples from the same pr

Splits an input PCollection of examples into train and test. This uses the file id (name) to compute the split, so that examples from the sam

Splits an input PCollection of examples into train and test. This uses the thread id to compute the split, so that examples from the same thr

The BERT encoder that is loaded as a module from tensorflow hub. This class tokenizes the input text using the bert tokenization library. The

BM25 baseline, using weighted keyword matching. Adapted from https://github.com/arosh/BM25Transformer/blob/master/bm25.py see Okapi BM25: a n

Abstract class to define a baseline response selection method.

The ConveRT encoder. See https://github.com/PolyAI-LDN/polyai-models. Args: uri: (string) the tensorflow hub URI for the model.

Test ConveRTEncoder.

Test running the pipeline end-to-end.

Test individual helper functions.

A model that maps from text to dense vectors.

An encoder that is loaded as a module from tensorflow hub. The tensorflow hub module must take a vector of strings, and return a matrix of en

TF-IDF baseline. This hashes words to sparse IDs, and then computes tf-idf statistics for these hashed IDs. As a result, no words are conside

A dual encoder following the USE_QA signatures. Args: uri: (string) the tensorflow hub URI for the model.

Test USEDualEncoder.

Applies a linear mapping to the response side and ranks with similarity. This learns a [dim, dim] weights matrix, and maps the response vector `x

Ranks responses using cosine similarity of context & response vectors. Args: encoder: the `Encoder` object to use.