Return all loaded templates. Returns: List of all available templates.

Parse a YAML string into ReviewCriteria. Useful when the user provides YAML criteria directly as input text. If the YAML contains a `

Plot sensitivity and specificity across decision thresholds. Args: scores: Predicted inclusion scores for each record. labels: Gr

Screen a single record against PICO criteria. Args: record: The literature record to evaluate. criteria: Structured P

Create real OpenRouter LLM backends from config. Reads model definitions from ``configs/models.yaml`` and creates one OpenRouterAdapter per m

Truncate text to a maximum character count at a word boundary. If the text is within the limit, it is returned unchanged. Otherwise,

Assess criteria quality via LLM (Layer 2). Sends the full criteria JSON to the LLM backend using the ``validate_quality_v1`` prompt t

Call the underlying LLM API and return the raw text response. Args: prompt: The complete prompt to send. seed: Random

Format a metric dict as a Lancet-style string. Args: m: Dict with "point", "ci_lower", "ci_upper" keys. Returns: Lancet-form

Create a mock quality assessment adapter.

Create a mock adapter that returns quality scores.

Assemble a complete screening prompt. Args: record: The literature record to screen. criteria: The review criteria to

Build the article section of a screening prompt. Args: record: The literature record to format. Returns: A formatted string

Build a TRIPOD-LLM compliant audit trail entry. Args: record: The screened record. criteria: Review criteria used.

Count match/mismatch votes for a specific element across models. Inspects ``model_output.pico_assessment[element_key].match`` for each output

Fit the isotonic calibrator on validation scores. Args: scores: Raw model scores in [0, 1]. labels: Binary labels (0

Generate all paper figures and save to the output directory. Creates Figure 4 (forest plot) and Figure 5 (ablation chart). Figures 2 and 3 ar

Generate Table 1: Model Registry. Lists all LLM models from the configuration file with their versions, providers, and licenses. Args:

Generate Table 3: ASReview Benchmark Results. Args: results_dir: Path to the directory containing experiment results. Returns:

Generate Table 4: Ablation Study. Compares different HCN configurations (single model, ensemble without rules, full HCN) to quantify componen

Generate Table 5: Cost and Time Analysis. Summarises per-record processing time and API cost estimates. Args: results_dir: Path to t

Return current thresholds. Returns: Thresholds dataclass with tau_high, tau_mid, tau_low.

Plot an annotated 2x2 confusion matrix heatmap. Only considers INCLUDE and EXCLUDE decisions; HUMAN_REVIEW entries are mapped to INCLUDE for

Plot a Risk of Bias assessment heatmap. Rows represent studies (by record_id) and columns represent RoB domains. Each cell is color-coded by

Run all backends in parallel for a single record. Args: record: The literature record to screen. criteria: PICO inclu

Fit calibrators per model using Phase 3 implementations. Args: decisions: List of ScreeningDecision objects. gold_lab

Save a Plotly figure to disk. Attempts to export as a static image (PNG) via kaleido. If kaleido is not installed, falls back to an interacti

Run full validation: rules (Layer 1) + optional LLM quality (Layer 2). Args: criteria: The criteria to validate. back

Merge base layout with caller-supplied overrides. Args: **kwargs: Layout property overrides. Returns: Merged layout dict.

Run generation across all backends and merge results. Args: prompt: The prompt to send to each backend. framework: SR

Generate a benchmark results table (shared logic for Table 2 and 3). Args: data: Parsed experiment results dict. table_num: Table

RuleCheckResult with a hard violation.

Load experiment results JSON from the results directory. Args: results_dir: Path to the directory containing result JSON files. e

Create an alternative generation adapter for consensus testing.

Map raw keys to canonical field names using a mapping table. First match wins -- earlier keys in field_map have priority. Args: raw:

Merge a list field (e.g., study_design_include) as union. Args: model_outputs: All model output dicts. field: Field n

Convert a Record to a flat dict suitable for CSV/Excel. Args: record: Record object. Returns: Flat dict with string values.

Derive required_elements from the framework definition. Returns the intersection of framework-required keys and actually present elem

Execute the full assessment pipeline. Args: text: Full paper text. tool_schema: The RoB tool schema to use.

Persist session state for potential resume. Reuses the same ``session_id`` across the pipeline run so that later saves overwrite earl

Map decision to binary (1 = INCLUDE, 0 = EXCLUDE).

Build the screening instructions section. Instructions are recall-biased: when in doubt, INCLUDE the article to minimize false negatives (mis

Build the expected output JSON template. Requests ``element_assessment`` (generic key) rather than ``pico_assessment`` so prompts are framewo

Build the system-level role description for screening prompts. Returns: A string describing the LLM's role as a systematic review screene

Compute Cohen's kappa inter-rater agreement coefficient. Args: ratings_a: Ratings from rater A. ratings_b: Ratings from rater B.

Delete a session file. Args: session_id: The session to remove.

Find templates matching any of the given tags. Args: tags: List of tags to search for. Returns: Templates ma

Fit model weights from labeled validation data. Uses SLSQP to minimize binary cross-entropy between weighted scores and true labels,

Generate all 5 paper tables and write to a single markdown file. Args: results_dir: Path to the directory containing experiment result

Find a template by its ID. Args: template_id: The template identifier. Returns: Template if found, None othe

Render the criteria wizard page.

Optimize tau thresholds via Phase 3 ThresholdOptimizer. Args: decisions: List of ScreeningDecision objects. gold_labe

Parse free-text criteria into structured form (Mode A). Args: criteria_text: User-provided criteria text. framework:

Plot a bar chart of screening decisions per routing tier. Args: decisions: List of screening decisions with tier assignments. Return

Run the Cohen benchmark across all available topics. Iterates over all Cohen topics with available CSV files, runs screening for each, comput

Assess all chunks x all models in parallel. Returns: model_id -> list of per-chunk parsed dicts.

Run a single assessment call with error handling. Returns: Parsed assessment dict, or empty dict on failure.

Construct all five RoB 2 domains with their signaling questions. Returns: List of five ``DomainSchema`` instances ordered D1-D5.

Construct all four QUADAS-2 domains with their signaling questions. Returns: List of four ``DomainSchema`` instances ordered D1-D4.

Construct all seven ROBINS-I domains with their signaling questions. Returns: List of seven ``DomainSchema`` instances ordered D1-D7.

Build a framework-detection prompt. Tries to import a versioned prompt template first; if unavailable falls back to an inline minimal

Build the form generation prompt. Args: topic: Research topic description. Returns: Complete prompt string.

Call OpenRouter API with retry logic. Args: prompt: The complete prompt string. seed: Reproducibility seed (passed to

Call a single backend and parse its JSON response. Args: backend: LLM backend to call. prompt: The prompt string.

Check if a value matches the expected extraction field type. Args: value: The extracted value. expected_type: The declared field

Fallback: split text by character count with overlap. Args: text: Text without paragraph boundaries. max_chars: Maximum character

Split paragraphs into chunks respecting size limits. Args: paragraphs: List of paragraph strings. max_chars: Maximum characters p

Compute bootstrap CIs for sensitivity, specificity, precision, f1. Args: decisions: Predicted screening decisions. la

Compute majority-vote consensus per domain across models. Returns: List of RoBDomainResult with consensus judgements.

Compute majority-vote consensus across models. Args: per_model_merged: model_id -> merged extraction dict. form: Extr

Compute ensemble confidence from decision agreement. Uses normalized Shannon entropy: C_ensemble = 1 - H(p_include, p_exclude) /

Detect write format from file extension. Args: path: Output file path. Returns: Format string. Raises: Unsuppor

Estimate USD cost for a single model call. Looks up the model in the pricing table. If the model ID is not found, falls back to the most expe

Estimate the number of input tokens for a single model call. Uses the rough heuristic of 1 token per 4 characters, with a minimum of 100 toke

Estimate token count from character length. Args: text: Input text. Returns: Estimated token count.

Export full audit trail as JSON. Args: data: List of result dictionaries. output_dir: Output directory.

Export results as CSV using pandas. Args: data: List of result dictionaries. output_dir: Output directory.

Export results as Excel spreadsheet. Args: data: List of result dictionaries. output_dir: Output directory.

Export results as pretty-printed JSON. Args: data: List of result dictionaries. output_dir: Output directory.

Export results as RIS format. Args: data: List of result dictionaries. output_dir: Output directory.

Extract from all chunks x all models in parallel. Args: chunks: Text chunks. form: Extraction form. seed:

Run a single extraction call with error handling. Args: backend: LLM backend to call. prompt: Complete extraction pro

Find a suitable record ID column in a list of column names. Looks for ``record_id`` first, then any column containing ``id``. Falls back to t

Find the first recognized label column in a list of column names. Args: columns: Lowercase column names from a DataFrame. Returns:

Find the majority value (> n/2 models agree). Args: values: Non-null values from models. n_models: Total number of mo

Get calibration factor φ_i (1.0 if no calibrator).

Get weight for a model (equal if no weights set).

Walk through each element for user review and modification. Args: criteria: The criteria to refine in-place. ask_user

Load a single CSV file and build Record objects with gold labels. Args: data_path: Path to the CSV file with columns record_id, title,

Load all YAML templates from the builtin_templates directory.

Load MetaScreener config from file or defaults. Args: config_path: Optional path to config file. Returns: MetaScreenerConfig

Load a CSV file and build Record objects. Args: data_path: Path to the CSV file with columns record_id, title, abstract (and

Load a single ASReview Synergy dataset CSV and build Record objects. Column names are normalized to lowercase. The function detects the label

Load gold standard labels from CSV file. Expected columns: record_id, label (INCLUDE/EXCLUDE). Args: path: Path to the CSV file.

Load predictions from JSON file. Expected format: list of objects with record_id, decision, final_score, tier, ensemble_confidence. Args

Load a single Cohen topic CSV and build Record objects. Args: topic_name: Name of the Cohen topic (e.g., "ACEInhibitors"). data_d

Create a bootstrap-compatible function for a given metric attribute. Args: attr: Name of the ScreeningMetrics attribute to extrac