Get the value of corresponding key.

Pad the instances to the max sequence length in batch.

Add cmdline arguments.

Add a new hypothesis to the list.

Parse hyper-parameters from cmdline.

Generate self attention masking matrix. This is a helpful function to generate different types of attention masking matrix. 1. Bi-dir

Record start timestamp.

Query database for a given domain and constraints. Args: domain (str, optional): The current domain. Defaults to None.

Convert arabic numerals to English representation. Args: token (str): The token. Returns: str: The token aft

Run Transformer generation network. Args: tokens_ids: represents the token id of each token, shape is [batch_size, max_seq_len, 1

Check CUDA.

Get pooled output of the last output embedding in Transformer. Args: enc_out: the output embeddings of Transformer, shape is [bat

Run postprocessing. Args: ds_seq (str): The dialogue state of model generation. prev_ds (dict, optional): The previou

Add a new hypothesis to the list.

Tokenizes a piece of text.

Convert example to record which can be used as the model's input.

Execute program in static graph mode. Args: program: The executable program. inputs: A dict mapping variable names to

Add masking and return target's labels and indices. Add mask for batch_tokens, return out, mask_label, mask_idx; Note: mask_idx (masking inde

Helpful function for running shell command in py scripts.

Padding sequence to speedup matmul OP. According to the tensor cores documentation from NVIDIA, the matmul OP in fp16 must all be multiples of

Padding a batch of records and construct model's inputs.

Convert tokens to ids.

Run one inference step.

Get the logits of generation task. The network may share weight with token embeddings. Args: enc_out: the output embeddi

Get the output logits of the classifier network. Args: pooled_out: represents the input embedding of classifier network, shape is

Parse source sequence and return corresponding fields.

Read a data file and yield records.

Get formated knowledge.

Get the model's DataLoader. Args: model: the trained model. is_infer: whether to run model in inference mode.

Get logger.

Update N-gram blocking strategy related data.

Delete redundant slot values. Args: ds_dict (dict): The dialogue state. Returns: dict: The new dialogue stat

Get batch knowledge ids.

Get batch single item.

Get the batch size of inputs.

Parse knowledge sequence and return corresponding fields.

Get formated knowledge.

Get formated knowledge.

Get formated knowledge.

Flatten dialogue state from dict to sequence. Args: ds_dict(dcit): The dialogue state dict. schema(dict): The schema of the

Get schema for a dataset.

Initalize N-gram blocking strategy related data.

Loading parameters from both. Support both static and dygraph mode.

Parse dialogue state from sequence to dict. Args: ds_seq (str): The sequence of dialogue state. remove_nm (bool, optional): If Tr

Preprocess utterance and table value.

Preprocess utterance and table value.

The key of record. We will apply sorting before batching. It can decrease the number of padding and speedup training.

Minimal tokenize the given text.

Split input embeddings into multiply chunks. Reshape the last dimension of inpunt tensor x so that it becomes two dimensions and then

Convert arabic numerals to English representation. Args: tokens (list): The token list. Returns: list: The t

Generate tgt attention masking matrix. This is a helpful function to generation different type of attention masking matrix using in generatio

Convert inputs into model's input data format. Convert hierarchical list into LoD Tensor, and keep numpy.ndarray. Args:

Get similarity score.

Parse target sequence and return corresponding fields.

Rerank the matching result based on the database matching score of the specified slot. Args: db_res (list): Database matching res

Split English and arabic numerals. Args: token (str): The token. alpha_alone (bool, optional): If True, the letters i

Add cmdline arguments.

build training data

Convert ids to tokens.

Get metrics.

Merge subword.

Post-process the context id sequence. Truncate the <bos> token. Convert token ids to words (merge = True) or tokens (merge = False). Arg

Reset pass time.

Save model. In normal mode, only the master GPU need to save the model. In sharding mode, it need to save each part of model in GPUs.

Save model in static mode(paddle 1.x). This function will save all model parameters in a directory.

write data

Convert numeric representation of time to English representation. Args: utt_ls (list): The utterance list. token_idx

Distributed batch reader. Slice dataset and feed batches to different devices. Args: batch_reader: A batch reader.

Run Transformer encode pass. Args: emb_input: represents the input embeddings of Transformer, shape is [batch_size, max_seq_len,

Get F1 score.

Generate input embeddings of Transformer Args: token_ids: represents the token id of each token, shape is [batch_size, max_seq_le

Get model's input feed dict. Args: is_infer: If true, get inference input feed dict, otherwise get training / evaluation input fe

Get slot name for a specific token. Args: utt_ls (list): The utterance list. token_idx (int): The index of token.

Parse topic sequence and return corresponding fields.

Post processing. Args: utt (str): The utterance. Returns: str: The utterance after post processing.

Run recognition network. Args: token_ids: represents the token id of each token, shape is [batch_size, max_seq_len, 1]

Run generation.

Update max_lens.

Basic convert function.

Check the in-turn repetition. Calcuate tri-gram repetition. Args: pred: Words or tokens or token_ids. is_cn: Chinese version

Yield batches from a file or a record reader. If `reader` is set, it will yield batches from the record reader, otherwise yield batches from

calculate f1 metric

calculate f1 metric

Compute F1 scores Args: preds (list[str]): list of prediction refs (list[str]): list of reference avg (bool, optional): w

Get the first value.

Get the first value.