Functions — github.com/GPT-Fathom/GPT-Fathom

↓ 1 callersMethodpause

(self)

evals/record.py:103

↓ 1 callersMethodprompt_to_fewshot

(self, sample)

evals/elsuite/dataset_specific/lambada.py:65

↓ 1 callersMethodprompt_to_fewshot

(self, sample)

evals/elsuite/basic/includes.py:45

↓ 1 callersMethodprompt_to_fewshot

(self, sample)

evals/elsuite/basic/fuzzy_match.py:35

↓ 1 callersMethodrecord_cond_logp

(self, prompt, completion, logp, sample_id=None, **extra)

evals/record.py:234

↓ 1 callersMethodrecord_embedding

(self, prompt, embedding_type, sample_id=None, **extra)

evals/record.py:218

↓ 1 callersMethodrecord_error

(self, msg: str, error: Exception, **kwargs)

evals/record.py:258

↓ 1 callersMethodrecord_event

(self, type, data=None, sample_id=None)

evals/record.py:505

↓ 1 callersMethodrecord_extra

(self, data, sample_id=None)

evals/record.py:266

↓ 1 callersMethodrecord_final_report

(self, final_report: Any)

evals/record.py:486

↓ 1 callersMethodrecord_pick_option

(self, prompt, options, picked, sample_id=None, **extra)

evals/record.py:243

↓ 1 callersMethodrecord_raw

(self, data)

evals/record.py:252

↓ 1 callersMethodrecord_sampling

(self, prompt, sampled, sample_id=None, **extra)

evals/record.py:226

↓ 1 callersFunctionreliability_guard

This disables various destructive functions and prevents the generated code from interfering with the test (e.g. fork bomb, killing other pro

evals/elsuite/utils_execute.py:165

↓ 1 callersFunctionremove_punc

(text)

evals/elsuite/utils.py:54

↓ 1 callersFunctionrequest_with_timeout

Worker thread for making a single request within allotted time.

evals/utils/azure_utils.py:42

↓ 1 callersFunctionrequest_with_timeout

Worker thread for making a single request within allotted time.

evals/utils/api_utils.py:38

↓ 1 callersFunctionrun

(args: OaiEvalArguments, registry: Optional[Registry] = None)

evals/cli/oaieval.py:103

↓ 1 callersFunctionrun

( args: OaiEvalSetArguments, unknown_args: list[str], registry: Optional[Registry] = None, run

evals/cli/oaievalset.py:73

↓ 1 callersFunctionsample_and_concat_n_completions

( completion_fns: list[CompletionFn], prompt: OpenAICreateChatPrompt, n: int, template_i: str,

evals/elsuite/modelgraded/classify_utils.py:160

↓ 1 callersMethodsave

(self)

evals/cli/oaievalset.py:35

↓ 1 callersFunctionscrub_formatting_from_prompt

(prompt)

evals/elsuite/utils.py:128

↓ 1 callersMethodstarts_with_yes_or_no

(self, s)

evals/elsuite/dataset_specific/summedits.py:62

↓ 1 callersFunctionswallow_io

()

evals/elsuite/utils_execute.py:110

↓ 1 callersFunctiontext_prompt_to_chat_prompt

( prompt: str, role: str = "system" )

evals/prompt/base.py:72

↓ 1 callersFunctiontime_limit

(seconds)

evals/elsuite/utils_execute.py:97

↓ 1 callersFunctionto_number

(x: str)

evals/cli/oaieval.py:179

↓ 1 callersMethodunpause

(self)

evals/record.py:109

↓ 1 callersFunctionwhite_space_fix

(text)

evals/elsuite/utils.py:51

↓ 1 callersFunctionwithin_eps

(pred: float, gt: float)

evals/utils/theoremqa_utils.py:120

↓ 1 callersFunctionzstd_open

(filename: str, mode: str = "rb", openhook: Any = open)

evals/data.py:41

Method__call__

ARGS ==== `prompt`: Either a `Prompt` object or a raw prompt that will get wrapped in the appropriate `Prompt` cl

evals/api.py:27

Method__call__

( self, prompt: Union[OpenAICreatePrompt, OpenAICreateChatPrompt, Prompt], **kwargs,

evals/api.py:52

Method__call__

( self, prompt: Union[OpenAICreatePrompt, OpenAICreateChatPrompt, Prompt], **kwargs

evals/utils/test.py:19

Method__call__

(self, *args, **kwargs)

evals/utils/snowflake.py:99

Method__call__

Args: prompt: The prompt to complete, in either text string or Chat format. kwargs: Additional arguments to pass to t

evals/completion_fns/retrieval.py:90

Method__call__

( self, inputs, **kwargs, )

evals/completion_fns/llama.py:57

Method__call__

( self, inputs, **kwargs, )

evals/completion_fns/llama.py:146

Method__call__

( self, prompt: Union[str, OpenAICreateChatPrompt], **kwargs, )

evals/completion_fns/openai.py:68

Method__call__

( self, prompt: Union[str, OpenAICreateChatPrompt], **kwargs, )

evals/completion_fns/openai.py:124

Method__call__

( self, prompt: Union[str, OpenAICreateChatPrompt], **kwargs, )

evals/completion_fns/openai.py:187

Method__call__

( self, prompt: Union[str, OpenAICreateChatPrompt], **kwargs, )

evals/completion_fns/openai.py:241

Method__call__

(self, prompt, **kwargs)

evals/completion_fns/langchain_math.py:25

Method__call__

(self, prompt, **kwargs)

evals/completion_fns/cot.py:49

Method__call__

(self, prompt, **kwargs)

evals/completion_fns/langchain_llm.py:30

Method__call__

(self, **kwargs)

evals/elsuite/utils.py:209

Method__init__

( self, run_spec: evals.base.RunSpec, )

evals/record.py:75

Method__init__

(self, run_spec: RunSpec, log: bool = True)

evals/record.py:287

Method__init__

(self, log_path: Optional[str], run_spec: RunSpec)

evals/record.py:331

Method__init__

( self, completion_fns: list[CompletionFn], seed: int = 20220722, name: str =

evals/eval.py:53

Method__init__

(self, registry_paths: Sequence[Union[str, Path]] = DEFAULT_PATHS)

evals/registry.py:114

Method__init__

(self, completion: str)

evals/utils/test.py:8

Method__init__

(self, completion: str)

evals/utils/test.py:16

Method__init__

( self, autocommit=True, account: Optional[str] = None, user: Optional[str] =

evals/utils/snowflake.py:26

Method__init__

(self, response: str)

evals/completion_fns/retrieval.py:45

Method__init__

Args: retrieval_template: The template to use for the retrieval. The task prompt will be added to the end of this template.

evals/completion_fns/retrieval.py:57

Method__init__

(self, raw_data: Any, prompt: Any)

evals/completion_fns/llama.py:23

Method__init__