Returns available APIs with boolean flags :param data: from load_dataset, assumes ['text'] is available :param tokenizer: Tokenizer to t

Generate continuations :param input_tokens: input to model :param input_logits: output from model :param labels: lab

input_texts: a list of strings, each string is a prompt generate_params: a dictionary of parameters to pass to the generate function

Takes in data dict and parses it into API continuations :param data: data, assuming it's from load_dataset and has a text field

dir: location of the triton server model_name: the name of the model to use tokenizer_name: the name of the tokenizer to use,

Add API calls here. :param candidate: which candidate is being parsed :param outputs: individual candidate outputs :

Returns the best of n samples from a model :param model: A Huggingface model :param tokenizer: A Huggingface tokenizer :param prompt: A st

Generates continuations of valid API calls :param indices: index to start :param values: values for filtering :param

Grab continuations that are valid :param input_tokens: tokenized inputs :param input_logits: input logits :param lab

Prepare inputs for triton client inference. inputs_ids: torch.IntTensor, shape [batch_size, seq_len] new_tokens: int, number of tokens to

Call triton server for inference. triton_client: tritonclient.grpc.aio.InferenceServerClient model_name: str, name of the model input

Base API Postprocesing class :param start_tokens: token representation for [ or other tokens :param end_tokens: token repre

dir: the directory of the model tokenizer_name: the name of the tokenizer to use, if None, use the model name

dir: the directory of the model tokenizer_name: the name of the tokenizer to use, if None, use the model name

input_texts: a list of strings, each string is a prompt generate_params: a dictionary of parameters to pass to the generate function

input_texts: a list of strings, each string is a prompt generate_params: a dictionary of parameters to pass to the generate function

input_texts: a list of strings, each string is a prompt generate_params: a dictionary of parameters to pass to the generate function

input_texts: a list of strings, each string is a prompt generate_params: a dictionary of parameters to pass to the generate function