MCPcopy Create free account
hub / github.com/google-research/language / preprocess_example

Function preprocess_example

language/casper/augment/casper_formatters.py:126–138  ·  view source on GitHub ↗

Returns a preprocessed example according to the funcall format.

(example: RawExample, funcall_format: str)

Source from the content-addressed store, hash-verified

124
125
126def preprocess_example(example: RawExample, funcall_format: str) -> RawExample:
127 """Returns a preprocessed example according to the funcall format."""
128 if funcall_format == "top":
129 example = copy.deepcopy(example)
130 # For old versions of the JSONL files:
131 # Change `input_str` to match the official tokenization.
132 if "tokens" in example:
133 example["input_str"] = " ".join(example["tokens"])
134 # Change `output_str` to be the formatted logical form.
135 example["output_str"] = top_utils.format_serialized(example["output_str"])
136 return example
137 else:
138 raise ValueError(f"Unknown funcall_format: {funcall_format}")
139
140
141def rename_top_lf_labels(lfs: Sequence[top_utils.TopLF],

Callers

nothing calls this directly

Calls

no outgoing calls

Tested by

no test coverage detected

Used in the wild real call sites across dependent graphs

searching dependent graphs…