Append a new node to the journal.

General LLM query for various backends with a single system and user message. Supports function calling for some backends. Args:

Return the best solution found so far.

Execute the provided Python command in a separate process and return its output. Parameters: code (str): Python code to

Wraps code with three backticks.

Generate a natural language plan + code in the same LLM call and split them apart.

Convert node to dictionary for serialization

Create a Node from a dictionary, optionally linking to journal for relationships

Get the best implementation from a completed stage

Generate a summary of the research progress using LLM, including both successes and failures.

Parse stage name into main stage number, main stage name, sub-stage number, and sub-stage name

Generate a summary of the node's experimental findings

Get max iterations for a stage from config or default

Parse the response into name and description based on keyword prefix

Convert an image to base64 string.

Extract screenshots for figure captions ("Figure X." or "Figure X:") and also gather text blocks (anywhere in the PDF) mentioning that ex

Get response from vision-language model.

Load the experiment summaries from the base folder.

Load the idea text from the base folder.

Quickly drain all in-flight messages to prevent blocking.

Get the next available index for a log directory.

Parse the feedback from the VLM

Assigns a GPU to a process

Record a single interaction with the model.

Given raw text content, split it into lines and remove lines that are likely headers/footers or otherwise not part of the main content.

Convert a prompt into markdown format

Extract python code blocks from the text.

Look for a Python code block in triple backticks in the LLM response. Return only that code. If no code block is found, return the entire tex

Extract (presumed) natural language text up to the start of the first code block.

Format Python code using Black.

Gather citations for a paper, with ability to resume from previous progress. Args: base_folder: Path to project folder num_c

Get the appropriate AI client based on the model string. Args: model (str): string identifier for the model to use (e.g. "gpt-4-turb

Determine completed stages by checking for the existence of stage directories that contain evidence of completion (tree_data.json, tree_plot.

Calculate the size of a file (#lines for plaintext files, otherwise #bytes) Also returns a human-readable string representation of the size.

Get the mean value across all metrics and datasets

Get a node by its ID.

Load config from .yaml file and CLI args, and set up logging directory.

Releases GPU assigned to a process

Return string representation

Generate aggregated plots from multi-seed evaluation results. Args: seed_nodes: List of nodes from seed evaluation pa

Analyze experimental plots using VLM

Analyze progress and convergence in the current stage

Check if current stage is complete based on criteria

Check if the current sub-stage is complete

Create a ParallelAgent configured for the given stage

Create the initial stage configuration

Create the next sub-stage. Ask LLM to come up with the next sub-stage name and goals based on what has been done so far.

Define eval metric to be used across all experiments

Determine which datasets are successfully tested based on VLM feedback

Gather detailed metrics and analysis from the stage's nodes

Generate the next ablation idea based on what's been done

Generate the next hyperparam tuning idea based on what's been done. This is minaly for Stage 2 (baseline tuning).

Generate code for plotting experiment results