Converts `text` to Unicode (if it's not already), assuming utf-8 input.

Reads a tab separated value file.

Build the model from config file. :param config_file: The path to the JSON configuration file. :param training: If training, the whole model w

Calculate the number of total and warmup steps. >>> calc_train_steps(num_example=1024, batch_size=32, epochs=10, warmup_proportion=0.1) (320,

Load trained official model from checkpoint. :param model: Built keras model. :param config: Loaded configuration file. :param

Load trained official model from checkpoint. :param config_file: The path to the JSON configuration file. :param checkpoint_file: The path to

Wrap layers with residual, normalization and dropout. :param name: Prefix of names for internal layers. :param input_layer: Input lay

Parse the retention configuration provided by the user. Also, check the provided rentention config for correctness.

Get multi-head self-attention builder. :param name: Prefix of names for internal layers. :param head_num: Number of heads in multi-head self-

Searches for a good output reduction configuration on given model and data. lambda_hyperparam: See the paper for the mean

Get position-wise feed-forward layer builder. :param name: Prefix of names for internal layers. :param hidden_dim: Hidden dimension of feed f

Carries out simple fine-tuning for the data given in train_data above. Use it only if the model at checkpoint path has never

Get BERT model. :param token_num: Number of tokens. :param pos_num: Maximum position. :param seq_len: Maximum length of the in

Computes the retention config given a trained model with soft extract layers. Args: configuration_search_mode

Multi-head self-attention and feed-forward layer. :param name: Prefix of names for internal layers. :param input_layer: Input layer. :par

Get encoders. :param encoder_num: Number of encoder components. :param input_layer: Input layer. :param head_num: Number of heads in mult

Switches Soft Extract layer to a Hard Extract layer and trains on the given data. Args: configuration_search_

Makes an instance of training. Args: dev_data: The entire validation data. train_data: The en

Initialize the layer. :param return_attention: Whether to return attention weights. :param history_only: Whether to only use history data. :par

Initialize the layer. :param head_num: Number of heads. :param activation: Activations for linear mappings. :param use_bias: Whether to use b

Test code for finetuning task.

Test code for retrained model.

Test code for configurtion search.