Temporarily adds a forward hook to a model module. Args: module: The PyTorch module to hook hook: The hook function to apply

Calculate accuracy for specified datasets.

Calculate accuracy for specified datasets.

Calculate accuracy for specified datasets.

Calculate 95% confidence interval for binomial proportion. Args: accuracy: The proportion (0 to 1) n: The sample size con

Calculate accuracy for specified datasets.

Calculate 95% confidence interval for accuracy data.

Calculate binomial confidence interval for accuracy.

Calculate binomial confidence interval for accuracy.

Calculate 95% confidence interval for accuracy data.

Calculate binomial confidence interval for accuracy.

Calculate 95% confidence interval for accuracy data.

Calculate 95% confidence interval for accuracy data from list of accuracies.

Check if the answer matches the ground truth, handling ambiguous cases.

Check if the answer matches the ground truth, handling ambiguous cases (for open-ended).

It's important to check that the decoder weights are normalized.

Check if yes/no answer matches ground truth. For yes/no tasks, we require: - If ground_truth is "yes": response must contain "yes" but NOT "n

Registers a forward hook on the submodule to capture the residual (or hidden) activations. We then raise an EarlyStopException to skip unneed

Create a batch of mixed sentences: target SAE test sentences + hard negatives. Args: target_sae: The SAE whose explanation we're tes

Create prompt for GPT-5-mini to identify matching sentences.

Extract the explanation text from between XML tags.

Filter bars to only include those with allowed labels. Args: names: List of LoRA names labels: List of legend labels mean

Convert SAETrainTest to a prompt including positive examples and hard negatives.

We use 'Male' and 'Female', so 'Male' is in 'Female'. So we have to be careful.

We use 'Male' and 'Female', so 'Male' is in 'Female'. So we have to be careful.

Compare gender labels carefully so that 'male' in 'female' does not cause false matches.

We use 'Male' and 'Female', so 'Male' is in 'Female'. So we have to be careful.

We use 'Male' and 'Female', so 'Male' is in 'Female'. So we have to be careful.

Return batch size based on model name.

Get a consistent color for a given label. Args: label: The human-readable label (from CUSTOM_LABELS) Returns: Color tup

HF hook with debug prints to compare against vLLM. Supports a variable number of target positions per batch element. Semantics: Fo

Get the number of layers from a HuggingFace model config.

Input: torch.Tensor of shape [batch_size, seq_length] Output: list of list of strings of len [batch_size, seq_length] Each inner list corresp

Load classification results and return dict of lr -> (iid_acc, ood_acc, iid_accuracies, ood_accuracies).

Load SAE explanations from a JSONL file.

Load gender results and return dict of lr -> (mean_accuracy, accuracies_list).

Load the max activating examples data.

Load personaqa results and return dict of lr -> (mean_accuracy, accuracies_list).

Load taboo results and return dict of lr -> (mean_accuracy, accuracies_list).

Load training loss from wandb CSV and return mean of last N steps per learning rate.

Normalize an answer to a simple whitespace-separated, ASCII lowercase form.

Normalize an answer to a simple whitespace separated, ascii lowercase form. - Decodes literal unicode-escape sequences like '\\u00ed' when presen

Normalize an answer to a simple whitespace-separated, ASCII lowercase form.

Normalize an answer to a simple whitespace separated, ascii lowercase form. - Decodes literal unicode-escape sequences like '\\u00ed' when presen

Create plot with all models as subplots. Args: all_results: List of result dictionaries for each model highlight_keywords: List o

Push the trained LoRA adapter to Hugging Face Hub. Args: model: The trained model with LoRA adapters tokenizer: The tokenize

Reorder bars: highlight first, then alphabetical by label.

Reorder bars: Full Dataset -> SPQA + Classification -> SPQA Only (Pan et al.) -> Classification -> Original Model.

items[i].steering_vectors: torch.Tensor [1, d] (bf16) items[i].ds_label: int or str Returns: X: [n, d] float32 on CPU, y: [n]

Trains the model on a single batch of data.

Format activation-bearing tokens for prompt display (V2).

Load one of the behavior JSON files and produce (data_by_label, id_tuples). - data_by_label: dict[label] -> list of 7-tuples behavior entries

Collect stats for an eval type, ordered with highlight first.

Collect stats for a model, ordered with highlight first.

Stable short hash over the full config + split. Excludes path-like fields so moving folders does not change the filename.

Materialize datasets on disk using a single process (rank 0). Each loader's `load_dataset` will create and save if missing; otherwise it simp