Return a path relative to repo_root when possible.

Normalize newlines for reliable substring matching.

Run a command, capture stdout/stderr, and write everything to log_handle.

Build a naming summary string from classified identifiers. Example: "func: foo bar || type: my_type || field: field1 field2 || var: i j k"

Build a naming summary string from classified identifiers. Example: "func: foo bar || type: my_type || field: field1 field2 || var: i j k"

Extract identifiers from C code using tree-sitter, classified by type.

Extract identifiers from C code using tree-sitter, classified by type.

r""" Activates the neftune as presented in this code: https://github.com/neelsjain/NEFTune and paper: https://arxiv.org/abs/2310.05914

Build a stable identifier for a case.

Deactivates the neftune method. Make sure to call `_activate_neftune` first.

Create a CaseResult with basic metadata for the given case.

Classify a tree-sitter node into identifier categories: - func: function names (definitions + calls) - var: variable names (parameters /

Classify a tree-sitter node into identifier categories: - func: function names (definitions + calls) - var: variable names (parameters /

Embed two texts in a single API call, return their embedding vectors.

Embed two texts in a single API call, return their embedding vectors.

Resolve the benchmark repo root from CLI arg, env var, or config.env.

Resolve the benchmark repo root from CLI arg, env var, or config.env.

Compute identifier-level similarity using embedding cosine similarity. Steps: 1. Extract identifiers from both texts using tree-sitter

Compute identifier-level similarity using embedding cosine similarity. Steps: 1. Extract identifiers from both texts using tree-sitter

Load config.env from the eval project root.

Load config.env from the eval project root.

Tokenize a list of strings.

Yield cases from jsonl file respecting the optional limit.

Check if the candidate C code compiles with gcc. Args: extra_info: Optional dict with 'header' key containing C header declarations.

Check if the candidate C code compiles with gcc. Args: extra_info: Optional dict with 'header' key containing C header declarations.

Aggregate statistics over all case results.

Create the per-case output directory, handling file path collisions.

Freeze all parameters except embeddings.

Generates a prompt template for chat models based on input and history. Args: input_query (str): User's current input query.

Compute word-level Jaccard similarity between two strings.

Load model checkpoint, optimizer, LR scheduler and intermedidate running states.

Load file in JSON format

Load pre-tokenized dataset. Each instance of dataset is a dictionary with `{'input_ids': List[int], 'labels': List[int], sequence: str}`