Add cmdline arguments.

Post process logits by N-gram blocking strategy.

Compute the BLEU-4 score Args: preds (list[str]): list of prediction refs (list[str]): list of reference

Check the cross-turn repetition. Calcuate tri-gram repetition. Args: context: Words or tokens or token_ids. pred: Words or t

Performs invalid character removal and whitespace cleanup on text.

Converts a sequence of [tokens|ids] using the vocab.

Main function of converting checkpoint.

Create Knover data format.

Create predictor.

Divide dataset into train / dev / test.

Convert parameters from dygraph mode to static mode(paddle 1.x).

Convert sentences into word pieces.

A Transformer Encoder. The encoder is composed of a stack of identical layers returned by calling encoder_layer.

A Transformer encoder block. The encoder layers that can be stacked to form a deep encoder. This module consists of a multi-head (self) atten

Compute the Entity-F1 score Args: preds (list[str]): list of prediction refs (list[str]): list of reference entity_file (

Run one evaluation step

Run evaluation. Run evaluation on dataset which is generated from a generator. Support evaluation on single GPU and multiple GPUs. Single GP

Evaluation main function.

Main evaluation function.

Main evaluation function.

Evaluation main function.

Compute Rouge-L scores Args: preds (list[str]): list of prediction refs (list[str]): list of reference avg (bool, optiona

Extract dialogue state.

Filter dialogues that contain various types of text and annotation errors. Similar to the `analyze_dialogue` in https://github.com/budzianows

Get the first key.

Get the first key.

Generation from sequence to sequence.

Generate query by dialogue context. Returns: list: Input texts for query generation

Generate system response by dialogue context and retrieved knowledge.

calculate F1 score

Get all n-gram results from an utterance.

Get candidate slot values

Return a nicely extracted title and chosen sentence. :return: pair (title, sentence)

Get the difference from previous metadata to current metadata.

Mark fuzzy matching results of locations.

Mark fuzzy matching results of entities.

Get all fuzzy matching results of locations.

Get all fuzzy matching results of entities.

Get fuzzy matching results of the utterance.

Get metrics. Args: inputs: A dict mapping keys to corresponding input data. outputs: A dict mapping keys to correspon

Get metrics for KAG training.

Get the NSP scores of a batch. Args: nsp_predictor: The NSP model predictor. predictions: A batch of prediction, contains `contex

Get previous turn sample index.

Get statistics. Args: inputs: A dict mapping keys to corresponding input data. outputs: A dict mapping keys to corres

Get tensor by name.

Gets progress for training phase.

Gumbel softmax.

Inference main function.

Main inference function.

Main inference function.

Main inference function.

Inference main function.

Run inference. Args: inputs: A dict mapping keys to corresponding input data. outputs: A dict mapping keys to corresp

Inference main function.

Interaction main function.

Retrieve relevant knowledge by generated query.

Retrieve relevant knowledge by generated query.

Convert knowledge into a flatten sequecen with special symbols.

Convert knowledge into a flatten sequecen with special symbols.

Convert a knowledge record into a flatten sequence with special symbols.

Convert a knowledge record into a flatten sequence with special symbols.

Tokenization main process.

Numericalization main process.

Main function.

Main function.

Main function.

Main function.

Main function.

Main function.

Main function.

Main function.

Main function.

Main function.

Main function.

Main function.

Main function.

Main function.

Main function.

Numericalization main process.

Main function.

Main function.

Main function.

Numericalization main process.

Main function.

Main function.

Main function.

Main function.

Main function.

Main function.