Converts a sequence of ids in a string, using the tokenizer and vocabulary with options to remove special tokens and clean up tokeniz

Return set of symbol pairs in a word. Word is represented as tuple of symbols (symbols being variable-length strings).

copy from https://github.com/GaiZhenbiao/ChuanhuChatGPT/

Initializes model parallelism for the given model and device map. Args: raw_model_dir (str): The directory containing th

-> [num_hidden_layers x batch x num_heads x seq_length x seq_length]

Merges attn_head_size dim and num_attn_heads dim into n_ctx

Merges attn_head_size dim and num_attn_heads dim into n_ctx

Returns list of utf-8 byte and a mapping to unicode strings. We specifically avoids mapping to whitespace/control characters the bpe code bar

A simple version of `jt.repeat_interleave` for duplicating a matrix while interleaving the copy.

Generates text using the model, given the input data and generation parameters. Args: data (str): The input text for gen

Choose different methods to generate sentences. :param input_str: The input text. :param tokenizer: Tokenizer. :param method: Genera

Inference method that computes logits and past key values. Args: input_ids (torch.Tensor): The input IDs tensor.

Load sharded checkpoints directly from huggingface dir.

Removes the prefix from the generated text. Args: preds_i (str): The generated text containing the prefix. Retu

Preprocesses the raw input text by adding the prefix and tokenizing it. Args: raw_text (str): The raw input text.

Performs a streaming top-k search using the given parameters. Args: input_ids (torch.Tensor): The input IDs tensor.

Initializes the MossModel with a given model or loads a model from the specified directory. Args: model (Optional[MossFo

:param prune_configs_by: a dict of functions that are used to prune configs, fields: 'perf_model': performance model used to predicate running t

Converts an index (integer) in a token (str) using the vocab.

Converts a token (str) in an id using the vocab.

Initialize the weights.

This function is used to re-order the `past_key_values` cache if [`~PretrainedModel.beam_search`] or [`~PretrainedModel.beam_sample`]

Tokenize a string.

Decorator for auto-tuning a :code:`triton.jit`'d function. .. highlight:: python .. code-block:: python @triton.autotune(configs=[ triton.Con

Converts a sequence of tokens (string) in a single string.