MCPcopy Create free account
hub / github.com/OpenSparseLLMs/MoM / fewshot_context

Method fewshot_context

lm-eval-harness/lm_eval/api/task.py:940–973  ·  view source on GitHub ↗

Returns a fewshot context string that is made up of a prepended description (if provided), the `num_fewshot` number of examples, and an appended prompt example. :param doc: str The document as returned from training_docs, validation_docs, or test_docs. :param num

(self, doc, num_fewshot)

Source from the content-addressed store, hash-verified

938
939 @utils.positional_deprecated
940 def fewshot_context(self, doc, num_fewshot):
941 """Returns a fewshot context string that is made up of a prepended description
942 (if provided), the `num_fewshot` number of examples, and an appended prompt example.
943
944 :param doc: str
945 The document as returned from training_docs, validation_docs, or test_docs.
946 :param num_fewshot: int
947 The number of fewshot examples to provide in the returned context string.
948 :returns: str
949 The fewshot context.
950 """
951
952 if num_fewshot == 0:
953 # always prepend the (possibly empty) task description
954 labeled_examples = self.config.description
955 else:
956 labeled_examples = self.config.description + self.sampler.get_context(
957 doc, num_fewshot
958 )
959
960 example = self.doc_to_text(doc)
961 if self.multiple_input:
962 return labeled_examples
963 else:
964 if isinstance(example, str):
965 return labeled_examples + example
966 elif isinstance(example, list):
967 return [labeled_examples + ex for ex in example]
968 elif isinstance(example, int):
969 if self.config.doc_to_choice is not None:
970 choices = self.doc_to_choice(doc)
971 return labeled_examples + choices[example]
972 else:
973 return labeled_examples + str(example)
974
975 def apply_filters(self):
976 """Iterates over FilterEnsembles and applies them to instances"""

Callers

nothing calls this directly

Calls 3

doc_to_textMethod · 0.95
doc_to_choiceMethod · 0.95
get_contextMethod · 0.80

Tested by

no test coverage detected