MCPcopy Create free account

hub / github.com/GPT-Fathom/GPT-Fathom / functions

Functions419 in github.com/GPT-Fathom/GPT-Fathom

⨍ Functions 419◇ Types & classes 72↳ Endpoints 2
↓ 1 callersFunction_f1_score
(prediction: str, ground_truth: str)
evals/elsuite/utils.py:94
↓ 1 callersFunction_fix_a_slash_b
(string)
evals/utils/math_util.py:144
↓ 1 callersFunction_fix_fracs
(string)
evals/utils/math_util.py:112
↓ 1 callersFunction_fix_sqrt
(string)
evals/utils/math_util.py:169
↓ 1 callersFunction_get_json_file
(path)
evals/data.py:95
↓ 1 callersFunction_get_jsonl_file
(path)
evals/data.py:88
↓ 1 callersFunction_green
(str)
evals/record.py:273
↓ 1 callersFunction_iter
()
evals/data.py:152
↓ 1 callersMethod_process_directory
(self, registry: RawRegistry, path: Path)
evals/registry.py:299
↓ 1 callersFunction_red
(str)
evals/record.py:277
↓ 1 callersFunction_remove_right_units
(string)
evals/utils/math_util.py:159
↓ 1 callersMethod_render_chat_prompt_as_text
(self, prompt: OpenAICreateChatPrompt)
evals/prompt/base.py:113
↓ 1 callersMethod_render_text_as_chat_prompt
Render a text string as a chat prompt. The default option we adopt here is to simply take the full prompt and treat it as a system me
evals/prompt/base.py:132
↓ 1 callersFunction_stream_jsonl_file
(path)
evals/data.py:101
↓ 1 callersFunction_to_py_types
(o: Any)
evals/data.py:170
↓ 1 callersMethodadd
(self, item: Task)
evals/cli/oaievalset.py:31
↓ 1 callersFunctionappend_answer_prompt
Append answer prompt to prompt.
evals/elsuite/modelgraded/classify_utils.py:133
↓ 1 callersFunctionazure_chat_completion_create_retrying
Helper function for creating a chat completion. `args` and `kwargs` match what is accepted by `openai.ChatCompletion.create`.
evals/utils/azure_utils.py:69
↓ 1 callersFunctionazure_completion_create_retrying
Helper function for creating a completion. `args` and `kwargs` match what is accepted by `openai.Completion.create`.
evals/utils/azure_utils.py:26
↓ 1 callersFunctionchdir
(root)
evals/elsuite/utils_execute.py:151
↓ 1 callersFunctioncheck_match
(sampled, ans, dataset)
evals/elsuite/basic/math_problem.py:23
↓ 1 callersFunctionchoice_to_str
Return a string of choices, e.g. '"Yes" or "No" or "Maybe"'.
evals/elsuite/modelgraded/classify_utils.py:104
↓ 1 callersFunctionclassify
( mg: ModelGradedSpec, completion_fn: CompletionFn, completion_kwargs: Optional[dict[str, Any]] =
evals/elsuite/modelgraded/classify_utils.py:51
↓ 1 callersFunctioncompare_two_numbers
(p, gt)
evals/utils/theoremqa_utils.py:134
↓ 1 callersFunctioncompute_f_score
( confusion_matrix: np.ndarray, idx: int = 0, beta: float = 1.0 )
evals/metrics.py:73
↓ 1 callersFunctioncompute_precision
(confusion_matrix: np.ndarray, idx: int = 0)
evals/metrics.py:65
↓ 1 callersFunctioncompute_recall
(confusion_matrix: np.ndarray, idx: int = 0)
evals/metrics.py:69
↓ 1 callersFunctionconcat_n_completions
Concatenate n completions into a single text string.
evals/elsuite/modelgraded/classify_utils.py:185
↓ 1 callersFunctioncreate_chat_prompt
(text)
evals/build_data/build_realtoxic.py:7
↓ 1 callersFunctioncreate_chat_prompt
(text)
evals/build_data/build_web_questions.py:8
↓ 1 callersFunctioncreate_chat_prompt
(text)
evals/build_data/build_triviaqa.py:7
↓ 1 callersFunctioncreate_chat_prompt
(text)
evals/build_data/build_natural_questions.py:8
↓ 1 callersFunctioncreate_chat_prompt
(text)
evals/build_data/build_lambada.py:8
↓ 1 callersFunctioncreate_fewshot_prompt
(question, answer)
evals/build_data/build_web_questions.py:15
↓ 1 callersFunctioncreate_fewshot_prompt
(question, answer)
evals/build_data/build_triviaqa.py:17
↓ 1 callersFunctioncreate_fewshot_prompt
(question, answer)
evals/build_data/build_natural_questions.py:15
↓ 1 callersFunctioncreate_fewshot_prompt
(question, answer)
evals/build_data/build_lambada.py:18
↓ 1 callersFunctioncreate_tempdir
()
evals/elsuite/utils_execute.py:119
↓ 1 callersMethodcursor
(self, *args, **kwargs)
evals/utils/snowflake.py:93
↓ 1 callersFunctionestimate_pass_at_k
Estimates pass@k of each problem and returns them in an array.
evals/elsuite/utils_execute.py:323
↓ 1 callersFunctionestimator
Calculates 1 - comb(n - c, k) / comb(n, k).
evals/elsuite/utils_execute.py:326
↓ 1 callersMethodeval_sample_batch
(self, recorder, samples)
evals/elsuite/dataset_specific/ceval.py:181
↓ 1 callersMethodeval_sample_batch
(self, recorder, samples)
evals/elsuite/dataset_specific/realtoxic.py:78
↓ 1 callersMethodeval_sample_batch
(self, recorder, samples)
evals/elsuite/dataset_specific/theoremqa.py:101
↓ 1 callersMethodeval_sample_batch
(self, recorder, samples)
evals/elsuite/dataset_specific/bbh.py:112
↓ 1 callersMethodeval_sample_batch
(self, recorder, samples)
evals/elsuite/dataset_specific/agieval.py:415
↓ 1 callersMethodeval_sample_batch
(self, recorder, samples)
evals/elsuite/dataset_specific/scibench.py:168
↓ 1 callersMethodeval_sample_batch
(self, recorder, samples)
evals/elsuite/dataset_specific/mmlu_CoT.py:154
↓ 1 callersMethodeval_sample_batch
(self, recorder, samples)
evals/elsuite/dataset_specific/lambada.py:116
↓ 1 callersMethodeval_sample_batch
(self, recorder, samples)
evals/elsuite/dataset_specific/summedits.py:96
↓ 1 callersMethodeval_sample_batch
(self, recorder, samples)
evals/elsuite/basic/includes.py:92
↓ 1 callersMethodeval_sample_batch
(self, recorder, samples)
evals/elsuite/basic/long_comprehension.py:201
↓ 1 callersMethodeval_sample_batch
(self, recorder, samples)
evals/elsuite/basic/code.py:219
↓ 1 callersMethodeval_sample_batch
(self, recorder, samples)
evals/elsuite/basic/fuzzy_match.py:79
↓ 1 callersMethodeval_sample_batch
(self, recorder, samples)
evals/elsuite/basic/reading_comprehension.py:184
↓ 1 callersMethodeval_sample_batch
(self, recorder, samples)
evals/elsuite/basic/multiple_choice.py:231
↓ 1 callersMethodeval_sample_batch
(self, recorder, samples)
evals/elsuite/basic/math_problem.py:161
↓ 1 callersFunctionextract_answer
(query: str, use_azure: bool)
evals/utils/theoremqa_utils.py:14
↓ 1 callersMethodextract_answer
(self, result)
evals/elsuite/dataset_specific/theoremqa.py:47
↓ 1 callersFunctionextract_first_word
(text)
evals/elsuite/dataset_specific/lambada.py:26
↓ 1 callersFunctionfind_numbers_in_string
(s: str)
evals/utils/theoremqa_utils.py:108
↓ 1 callersFunctionfind_top_k_closest_embeddings
( embedded_prompt: list[float], embs: list[list[float]], k: int )
evals/completion_fns/retrieval.py:25
↓ 1 callersFunctionformat
Format a template string with kwargs.
scripts/modelgraded_generator.py:10
↓ 1 callersFunctionformat_prompt
Format a prompt with only necessary kwargs.
evals/elsuite/utils.py:160
↓ 1 callersFunctiongenerate_eval_examples
( num_eval_examples: int = 250, )
scripts/pattern_identification_generator.py:34
↓ 1 callersFunctiongenerate_exemplars_str
(num_exemplars: int = 8)
scripts/pattern_identification_generator.py:25
↓ 1 callersFunctionget_choice
Clean the answer string to a choice string to one of choice_strings. Return '__invalid__.' if no match.
evals/elsuite/modelgraded/classify_utils.py:109
↓ 1 callersFunctionget_choice_score
( choice: str, choice_strings: Iterable[str], choice_scores: Optional[Union[dict[str, float], str]
evals/elsuite/modelgraded/classify_utils.py:89
↓ 1 callersFunctionget_choice_strings
(choice_strings: Union[list[str], str], n: Optional[int] = None)
evals/elsuite/modelgraded/classify_utils.py:37
↓ 1 callersMethodget_class
(self, spec: EvalSpec)
evals/registry.py:171
↓ 1 callersMethodget_completion_fn
(self, name: str)
evals/registry.py:222
↓ 1 callersMethodget_completions
(self)
evals/completion_fns/retrieval.py:48
↓ 1 callersFunctionget_dataset
(local, url: str)
evals/elsuite/dataset_specific/mmlu_CoT.py:36
↓ 1 callersFunctionget_dataset
(local, dataset: str)
evals/elsuite/basic/code.py:20
↓ 1 callersFunctionget_decimal_with_wolfram
(string: str)
evals/utils/theoremqa_utils.py:88
↓ 1 callersMethodget_eval_set
(self, name: str)
evals/registry.py:230
↓ 1 callersMethodget_evals
(self, patterns: Sequence[str])
evals/registry.py:233
↓ 1 callersFunctionget_jsonl
Extract json lines from the given path. If the path is a directory, look in subpaths recursively. Return all lines from all jsonl files
evals/data.py:116
↓ 1 callersMethodget_metrics
(self)
evals/record.py:125
↓ 1 callersMethodget_modelgraded_spec
( self, name: str, **kwargs: dict )
evals/registry.py:211
↓ 1 callersFunctionget_parser
()
evals/cli/oaieval.py:26
↓ 1 callersFunctionget_parser
()
evals/cli/oaievalset.py:47
↓ 1 callersFunctiongzip_open
Wrap the given openhook in gzip.
evals/data.py:24
↓ 1 callersFunctionis_equiv
(str1, str2, verbose=False)
evals/utils/math_util.py:254
↓ 1 callersFunctionis_number
(string)
evals/utils/theoremqa_utils.py:163
↓ 1 callersMethodis_paused
(self, sample_id: str = None)
evals/record.py:115
↓ 1 callersFunctionload_embeddings
(embeddings_and_text_path: str)
evals/completion_fns/retrieval.py:17
↓ 1 callersFunctionlz4_open
( filename: str, mode: str = "rb", openhook: Any = open )
evals/data.py:32
↓ 1 callersFunctionmain
()
evals/cli/oaieval.py:217
↓ 1 callersFunctionmain
()
evals/cli/oaievalset.py:127
↓ 1 callersMethodmake_few_shot_prompt
(self, num_shot)
evals/elsuite/dataset_specific/ceval.py:101
↓ 1 callersMethodmake_few_shot_prompt
(self, num_shot)
evals/elsuite/dataset_specific/agieval.py:122
↓ 1 callersMethodmake_few_shot_prompt
(self, num_shot)
evals/elsuite/basic/multiple_choice.py:151
↓ 1 callersMethodmake_few_shot_prompt
(self, num_shot)
evals/elsuite/basic/math_problem.py:108
↓ 1 callersMethodmake_few_shot_prompts
(self, shot)
evals/elsuite/basic/code.py:105
↓ 1 callersFunctionopenai_chat_completion_create_retrying
Helper function for creating a chat completion. `args` and `kwargs` match what is accepted by `openai.ChatCompletion.create`.
evals/utils/api_utils.py:64
↓ 1 callersFunctionopenai_completion_create_retrying
Helper function for creating a completion. `args` and `kwargs` match what is accepted by `openai.Completion.create`.
evals/utils/api_utils.py:26
↓ 1 callersFunctionparse_extra_eval_params
Parse a string of the form "key1=value1,key2=value2" into a dict.
evals/cli/oaieval.py:172
↓ 1 callersMethodparse_math_answer
(self, raw_string)
evals/elsuite/dataset_specific/scibench.py:117
↓ 1 callersMethodparse_not
(self, inputs)
evals/elsuite/dataset_specific/scibench.py:49
← previousnext →101–200 of 419, ranked by callers