Functions — github.com/Dai-shen/LAiW

hub / github.com/Dai-shen/LAiW / functions

Functions1,647 in github.com/Dai-shen/LAiW

⨍ Functions 1,647◇ Types & classes 407↳ Endpoints 3

↓ 2 callersMethodformat_answer

(cls, query, entity)

src/financial-evaluation/lm_eval/tasks/superglue.py:321

↓ 2 callersFunctionformat_value

(args, results, model, task)

src/financial-evaluation/scripts/regression.py:95

↓ 2 callersFunctionget_overlaps_dump_path

(task_name, task_set, ngrams_n_size, limit)

src/financial-evaluation/lm_eval/decontamination/decontaminate.py:49

↓ 2 callersFunctionget_train_overlap

(docs_by_task_set, ngrams_path, limit)

src/financial-evaluation/lm_eval/decontamination/decontaminate.py:36

↓ 2 callersMethodhas_test_docs

(self)

src/tasks/legal.py:638

↓ 2 callersMethodhas_validation_docs

(self)

src/tasks/legal.py:635

↓ 2 callersMethodlast_boxed_only_string

(self, string)

src/financial-evaluation/lm_eval/tasks/hendrycks_math.py:117

↓ 2 callersFunctionmatthews_corrcoef

(items)

src/financial-evaluation/lm_eval/metrics.py:32

↓ 2 callersFunctionmetric_max_over_ground_truths

Compute max metric between prediction and each ground truth.

src/financial-evaluation/lm_eval/metrics.py:85

↓ 2 callersFunctionnormalize_answer

Taken from the official evaluation script for v1.1 of the SQuAD dataset. Lower text and remove punctuation, articles and extra whitespace.

src/financial-evaluation/lm_eval/tasks/qasper.py:43

↓ 2 callersFunctionoa_completion

Query OpenAI API for completion. Retry with back-off until they respond

src/financial-evaluation/lm_eval/models/gpt3.py:38

↓ 2 callersMethodpartial_context

(cls, doc, option)

src/financial-evaluation/lm_eval/tasks/winogrande.py:66

↓ 2 callersMethodpartial_context

(cls, doc, option)

src/financial-evaluation/lm_eval/tasks/wsc273.py:106

↓ 2 callersMethodpartial_target

(cls, doc)

src/financial-evaluation/lm_eval/tasks/winogrande.py:76

↓ 2 callersMethodpartial_target

(cls, doc)

src/financial-evaluation/lm_eval/tasks/wsc273.py:115

↓ 2 callersMethodpreprocess

(cls, text)

src/financial-evaluation/lm_eval/tasks/hellaswag.py:62

↓ 2 callersMethodprocess

(self, items)

src/tasks/legal.py:584

↓ 2 callersMethodread

(self)

src/financial-evaluation/lm_eval/decontamination/archiver.py:132

↓ 2 callersMethodremove_boxed

(self, s)

src/financial-evaluation/lm_eval/tasks/hendrycks_math.py:104

↓ 2 callersFunctionrun_task_tests

Find the package root and run the tests for the given tasks

src/financial-evaluation/lm_eval/utils.py:267

↓ 2 callersFunctionsample_stddev

(arr)

src/financial-evaluation/lm_eval/metrics.py:19

↓ 2 callersFunctionsplit_indices

Splits a string on whitespaces and records the indices of each in the original string. @:return generator((word, (start_idx, end_idx)), ...)

src/financial-evaluation/lm_eval/decontamination/janitor.py:71

↓ 2 callersFunctionstop_sequences_criteria

( tokenizer: transformers.PreTrainedTokenizer, stop_sequences: List[str], initial_decoder_input_le

src/financial-evaluation/lm_eval/models/huggingface.py:1092

↓ 2 callersMethodstrip_string

(self, string)

src/financial-evaluation/lm_eval/tasks/hendrycks_math.py:219

↓ 2 callersFunctiontextsynth_completion

Query TextSynth API for completion. Retry with back-off until they respond.

src/financial-evaluation/lm_eval/models/textsynth.py:25

↓ 2 callersMethodtok_encode

(self, string: str)

src/financial-evaluation/lm_eval/models/gpt3.py:111

↓ 2 callersMethodtok_encode_batch

(self, strings: List[str])

src/financial-evaluation/lm_eval/models/huggingface.py:533

↓ 2 callersMethodtraining_docs

:return: Iterable[obj] A iterable of any object, that doc_to_text can handle

src/financial-evaluation/lm_eval/base.py:566

↓ 2 callersFunctionweighted_mean

(items)

src/financial-evaluation/lm_eval/metrics.py:98

↓ 2 callersFunctionword_ngrams_indices

Splits a string into pairs of (ngram words, their start/end indices)

src/financial-evaluation/lm_eval/decontamination/janitor.py:78

↓ 1 callersFunctionClassification

(data, task)

precision/legal_metrics.py:101

↓ 1 callersFunctionNer

(data, task)

precision/legal_metrics.py:150

↓ 1 callersFunctionTG

(data, task)

precision/legal_metrics.py:189

↓ 1 callersMethod__init__

(self)

src/financial-evaluation/lm_eval/base.py:22

↓ 1 callersMethod__init__

(self)

src/financial-evaluation/lm_eval/tasks/ceval.py:89

↓ 1 callersMethod__init__

(self)

src/financial-evaluation/lm_eval/tasks/hendrycks_test.py:98

↓ 1 callersMethod__init__

(self)

src/financial-evaluation/lm_eval/tasks/translation.py:111

↓ 1 callersMethod__init__

(self, no_metric=False)

src/financial-evaluation/lm_eval/tasks/scrolls.py:136

↓ 1 callersMethod__init__

Initializes a `vllm.LLM` for evaluation. Args: pretrained (str): The HuggingFace Hub model ID name or the path to

src/financial-evaluation/lm_eval/models/huggingface.py:634

↓ 1 callersMethod_align_bags

Takes gold and predicted answer sets and first finds the optimal 1-1 alignment between them and gets maximum metric values over all t

src/financial-evaluation/lm_eval/tasks/drop.py:196

↓ 1 callersFunction_argmax

argmax with deterministic pseudorandom tie breaking.

src/financial-evaluation/lm_eval/tasks/bigbench.py:190

↓ 1 callersMethod_compute_f1

(self, predicted_bag, gold_bag)

src/financial-evaluation/lm_eval/tasks/drop.py:215

↓ 1 callersMethod_create_auto_model_peft

( self, *, model: transformers.PreTrainedModel, peft: str, revision: s

src/financial-evaluation/lm_eval/models/huggingface.py:344

↓ 1 callersMethod_create_auto_tokenizer

Returns a pre-trained tokenizer from a pre-trained tokenizer configuration.

src/financial-evaluation/lm_eval/models/huggingface.py:362

↓ 1 callersMethod_doc_to_few_shot_context

(self, shots)

src/financial-evaluation/lm_eval/tasks/bigbench.py:139

↓ 1 callersFunction_download_metric

()

src/financial-evaluation/lm_eval/tasks/scrolls.py:71

↓ 1 callersFunction_drop_duplicates_in_input

(untokenized_dataset)

src/financial-evaluation/lm_eval/tasks/scrolls.py:99

↓ 1 callersMethod_encode_pair

(self, context, continuation)

src/financial-evaluation/lm_eval/base.py:201

↓ 1 callersMethod_extract_answer

(self, completion)

src/financial-evaluation/lm_eval/tasks/mgsm.py:105

↓ 1 callersMethod_fix_number

(self, text)

src/financial-evaluation/lm_eval/tasks/drop.py:265

↓ 1 callersMethod_format_subject

(self, subject)

src/financial-evaluation/lm_eval/tasks/hendrycks_test.py:128

↓ 1 callersFunction_get_accelerate_args

Returns the kwargs needed to apply `accelerate` in `AutoModel.from_pretrained`.

src/financial-evaluation/lm_eval/models/huggingface.py:22

↓ 1 callersFunction_get_dtype

Converts `dtype` from `str` to torch.dtype when possible. Does not use an instantiated HF AutoConfig

src/financial-evaluation/lm_eval/models/gpt2.py:7

↓ 1 callersMethod_get_prune_text

(self, sample)

src/financial-evaluation/lm_eval/tasks/scrolls.py:171

↓ 1 callersFunction_get_unique_examples

(examples)

src/financial-evaluation/lm_eval/tasks/bigbench.py:180

↓ 1 callersFunction_get_valid_answers

(scores)

src/financial-evaluation/lm_eval/tasks/bigbench.py:175

↓ 1 callersMethod_is_correct

(self, completion, answer)

src/financial-evaluation/lm_eval/tasks/mgsm.py:112

↓ 1 callersMethod_is_correct

(self, completion, answer)

src/financial-evaluation/lm_eval/tasks/gsm8k.py:94

↓ 1 callersMethod_make_compute_metrics

(self, value)

src/financial-evaluation/lm_eval/tasks/scrolls.py:210

↓ 1 callersMethod_match_numbers_if_present

(self, gold_bag, predicted_bag)

src/financial-evaluation/lm_eval/tasks/drop.py:232

↓ 1 callersMethod_model_call

( self, inputs: TokenSequence, labels: Optional[TokenSequence] = None )

src/financial-evaluation/lm_eval/models/huggingface.py:1023

↓ 1 callersMethod_model_generate

(self, context, max_length, eos_token_id)

src/financial-evaluation/lm_eval/base.py:162

↓ 1 callersMethod_normalize

(self, answer)

src/financial-evaluation/lm_eval/tasks/drop.py:271

↓ 1 callersFunction_postprocess_output

(text, max_length, stop_string, output_regex)

src/financial-evaluation/lm_eval/tasks/bigbench.py:199

↓ 1 callersMethod_process_doc

(self, doc)

src/financial-evaluation/lm_eval/tasks/scrolls.py:357

↓ 1 callersMethod_question2id

Returns an identifier for the question in the given document.

src/financial-evaluation/lm_eval/tasks/mc_taco.py:104

↓ 1 callersMethod_remove_articles

(self, text)

src/financial-evaluation/lm_eval/tasks/drop.py:252

↓ 1 callersMethod_remove_prefixes

(self, aliases)

src/financial-evaluation/lm_eval/tasks/webqs.py:71

↓ 1 callersMethod_remove_punc

(self, text)

src/financial-evaluation/lm_eval/tasks/drop.py:258

↓ 1 callersFunction_squad_metric

(predictions, references)

src/financial-evaluation/lm_eval/tasks/squad.py:35

↓ 1 callersMethod_tokenize

(self, text)

src/financial-evaluation/lm_eval/tasks/drop.py:268

↓ 1 callersMethod_white_space_fix

(self, text)

src/financial-evaluation/lm_eval/tasks/drop.py:255

↓ 1 callersMethodadd_data

(self, key, value)

src/financial-evaluation/scripts/clean_training_data/generate_13_grams.py:104

↓ 1 callersMethodadd_data

(self, data, meta={})

src/financial-evaluation/lm_eval/decontamination/archiver.py:31

↓ 1 callersFunctionadd_json_task

Add a JSON perplexity task if the given task name matches the JSON task specification. See `json.JsonPerplexity`.

src/financial-evaluation/lm_eval/tasks/__init__.py:359

↓ 1 callersFunctionadd_json_task

Add a JSON perplexity task if the given task name matches the JSON task specification. See `json.JsonPerplexity`.

src/tasks/__init__.py:63

↓ 1 callersFunctionanthropic_completion

Query Anthropic API for completion. Retry with back-off until they respond

src/financial-evaluation/lm_eval/models/anthropic_llms.py:7

↓ 1 callersMethodappend_context

(cls, ctx, partial_ctx)

src/financial-evaluation/lm_eval/tasks/winogrande.py:101

↓ 1 callersMethodappend_context

(cls, ctx, partial_ctx)

src/financial-evaluation/lm_eval/tasks/wsc273.py:140

↓ 1 callersFunctionassert_target

(name, ob)

src/financial-evaluation/tests/test_version_stable.py:15

↓ 1 callersMethodbert_score

(self, items)

src/tasks/legal.py:350

↓ 1 callersMethodbert_score

(self, items)

src/tasks/legal.py:483

↓ 1 callersMethodbleu

Returns `t5` style BLEU scores. See the related implementation: https://github.com/google-research/text-to-text-transfer-transformer/

src/financial-evaluation/lm_eval/tasks/truthfulqa.py:370

↓ 1 callersFunctionbleu_n

(generated, reference, n)

precision/legal_metrics.py:66

↓ 1 callersFunctionbootstrap_stderr

(f, xs, iters)

src/financial-evaluation/lm_eval/metrics.py:207

↓ 1 callersFunctioncal_rouge

(generated, reference, t='rouge-1')

precision/legal_metrics.py:23

↓ 1 callersFunctioncalculate_bleu

(data)

precision/legal_metrics.py:78

↓ 1 callersFunctioncalculate_rouge_all

(data)

precision/legal_metrics.py:30

↓ 1 callersFunctioncategorise_answer

(answer_blob)

src/financial-evaluation/lm_eval/tasks/qasper.py:65

↓ 1 callersMethodclean_cpp

(self, dirty_string)

src/financial-evaluation/lm_eval/decontamination/janitor.py:197

↓ 1 callersMethodcommit

(self)

src/financial-evaluation/lm_eval/decontamination/archiver.py:89

↓ 1 callersFunctioncompress_and_move

(working_directory, output_directory, process_count)

src/financial-evaluation/scripts/clean_training_data/compress_and_package.py:31

↓ 1 callersMethodcompute_scores

(gold_list, pred)

src/financial-evaluation/lm_eval/tasks/coqa.py:104

↓ 1 callersMethodconstruct_requests

Uses RequestFactory to construct Requests and returns an iterable of Requests which will be sent to the LM. :param doc: T

src/financial-evaluation/lm_eval/base.py:619

↓ 1 callersMethodconstruct_requests

Uses RequestFactory to construct Requests and returns an iterable of Requests which will be sent to the LM. :param doc: T

src/tasks/legal.py:660

↓ 1 callersMethodcount_bytes

(cls, doc)

src/financial-evaluation/lm_eval/base.py:852

↓ 1 callersMethodcount_words

Downstream tasks with custom word boundaries should override this!

src/financial-evaluation/lm_eval/base.py:856

↓ 1 callersFunctioncreate_json_task

()

src/financial-evaluation/lm_eval/tasks/__init__.py:368

↓ 1 callersFunctioncreate_json_task

()

src/tasks/__init__.py:72

↓ 1 callersFunctioncreate_task

(subject)

src/financial-evaluation/lm_eval/tasks/ceval.py:87

← previous next →101–200 of 1,647, ranked by callers