fast-rlm

A minimal implementation of Recursive Language Models (RLMs) using Deno and Pyodide.

GitHub | Documentation | PyPI

Watch the full video on YouTube RLM Tutorial

What are RLMs

RLMs are an inference technique where an LLM interacts with arbitrarily long prompts through an external REPL. The LLM can write code to explore, decompose, and transform the prompt. It can recursively invoke sub-agents to complete smaller subtasks. Crucially, sub-agent responses are not automatically loaded into the parent agent's context — they are returned as symbols or variables inside the parent's REPL.

Support

If you find this helpful, consider supporting on Patreon — it hosts all code, projects, slides, and write-ups from the YouTube channel.

---

Demo

---

Install

pip install fast-rlm

Requirements

• Python 3.10+
• Deno 2+
• macOS/Linux: curl -fsSL https://deno.land/install.sh | sh
• Windows (npm): npm install -g deno
• (Optional) Bun — only needed for the TUI log viewer

Environment Variables

Set your LLM API key before running:

export RLM_MODEL_API_KEY=sk-or-...

Variable	Description	Default
RLM_MODEL_API_KEY	API key for the OpenAI-compatible backend (falls back to OPENAI_API_KEY, then OPENROUTER_API_KEY)	—
RLM_MODEL_BASE_URL	OpenAI-compatible base URL	https://openrouter.ai/api/v1

That's all you need to get started. By default, fast-rlm uses OpenRouter; you can point it at any OpenAI-compatible API by setting RLM_MODEL_BASE_URL. fast-rlm also runs on Vertex AI, the native Anthropic API, and local ACP coding agents — see Backend setup at the end of this README.

Quick Start

import fast_rlm
from fast_rlm import RLMConfig

# primary_agent is REQUIRED — there is no default model.
config = RLMConfig(primary_agent="z-ai/glm-5")

result = fast_rlm.run("Generate 50 fruits and count number of r", config=config)
print(result["results"])
print(result["usage"])

primary_agent is required. Every run() needs a config that sets it (e.g. RLMConfig(primary_agent="...")); sub_agent is optional and defaults to primary_agent. The shorter examples below omit config= for brevity — pass the config above to run them.

From the command line

The same engine is available as a fast-rlm CLI — handy for one-off runs and shell pipelines:

# A plain prompt
fast-rlm "Generate 50 fruits and count number of r" --primary-agent z-ai/glm-5

# Feed a file as the context. Parsed by extension:
#   .json/.yaml/.yml -> dict/list   .jsonl/.ndjson -> list[dict]
#   anything else (.csv, .tsv, .xml, .toml, .txt, ...) -> raw text the model parses
#       itself (its extension is noted so it knows the format).
# The prompt becomes the instruction; for a dict input with no "instruction" key,
# it's also injected into the dict.
fast-rlm "Aggregate the reviews into a verdict" --input-file reviews.json --primary-agent z-ai/glm-5

# -q prints only the result (clean for piping); other knobs mirror RLMConfig:
fast-rlm "..." --primary-agent acp:opencode --max-depth 2 --max-global-calls 50 -q

Run fast-rlm --help for all flags (--sub-agent, --max-calls, --acp-agents, --vertex, …).

The same file loading is available from Python — run() accepts an input_file (in place of query):

fast_rlm.run(input_file="reviews.json", instruction="Aggregate into a verdict", config=config)

Model backends

The primary_agent / sub_agent string selects one of four backends:

Mode	Example primary_agent	What it is
Any OpenAI-compatible API (default)	"gpt-5-mini", "deepseek-chat", "minimax/minimax-m3"	OpenAI, DeepSeek, OpenRouter (default), or any compatible endpoint
Vertex AI	"vertex/claude-sonnet-4-6"	Google Cloud (ADC auth)
Anthropic API	"claude-haiku-4-5", "anthropic/claude-sonnet-4-6"	Native Anthropic; falls back to the OpenAI-compatible endpoint if no key
ACP coding agent	"acp:codex", "acp:claude-code", "acp:opencode"	Drives a local coding agent, read-only

Set the credential only for the backend(s) you use — see Backend setup at the end of this README. An ACP-only run needs no API key at all.

Arbitrarily Long Context

The key idea behind RLMs is that the prompt can be arbitrarily long — far beyond any model's context window. The agent explores it programmatically through the REPL rather than trying to fit it all into a single call.

import fast_rlm

transcripts = open("lex_fridman_all_transcripts.txt").read()  # millions of tokens

result = fast_rlm.run(
    "Here are the transcripts of all Lex Fridman podcasts. "
    "Summarize what the first 5 Machine Learning guests had to say about AGI.\n\n"
    + transcripts
)
print(result["results"])

The agent will write code to search, filter, and chunk the transcripts on its own — no manual splitting required.

Structured Input & Output

Instead of squeezing your data into a string, you can pass a dict as the query and ask for a typed result back via output_schema. The agent receives the dict as a real Python dict (no parsing on its first turn), and its FINAL value is validated against the schema before being returned.

import fast_rlm
from pydantic import BaseModel

class Verdict(BaseModel):
    movie: str
    average_score: float
    consensus: str

result = fast_rlm.run(
    {
        "task": "Aggregate the reviews into a single verdict.",
        "movie": "The Trail of Pixels",
        "reviews": [
            {"name": "Asha", "score": 8, "text": "Tight pacing..."},
            {"name": "Bo",   "score": 6, "text": "Beautiful but thin..."},
            {"name": "Cy",   "score": 9, "text": "Instant favorite..."},
        ],
    },
    output_schema=Verdict,
)

verdict = Verdict.model_validate(result["results"])

Structured input. When query is a dict, the agent's initial probe prints a flat top-level schema (keys + type + length + truncated preview) so it can index context["reviews"] directly instead of stringifying.

Structured output. output_schema accepts:

Form	Example
Pydantic model class	output_schema=MyModel
Pydantic generic	output_schema=list[MyModel]
Python primitive	output_schema=int (also str, float, bool, list, dict)
Raw JSON Schema dict	output_schema={"type": "array", "items": {"type": "string"}}

The schema is shown to the agent at step 0 (Required output schema for FINAL (JSON Schema):). After every FINAL(...) call the value is validated; on failure the agent receives the schema and the specific validation errors (path + message) and may retry within its remaining call budget. Pydantic is an optional dependency — only required if you pass a Pydantic class or generic.

Schemas for subagents. Inside the REPL the agent can require a subagent's output shape by passing a JSON Schema dict as the second argument to llm_query:

schema = {"type": "array", "items": {"type": "string"}}
fruits = await llm_query("Generate 25 fruit names.", schema)

The child subagent enforces the schema the same way. See examples/structured_io.py and examples/parallel_r_count.py for end-to-end demos.

Tools

Inside the REPL the agent has two built-in tools and may also receive user-defined tools as ordinary Python functions. There is no separate tool-calling API — tools are just callables in the REPL namespace.

Pass Python functions to fast_rlm.run(..., tools=[my_fn]) and they will be pre-loaded into the root agent's REPL. The RLM is shown the function name, input names, and docstring as description. They are not shown the full internal code of the tool (although they can choose to inspect it if the task requires them to). The agent calls them like any normal function inside the REPL.

def filter_short(items: list[str], max_len: int = 20) -> list[str]:
    """Return only items shorter than max_len."""
    return [x for x in items if len(x) < max_len]

result = fast_rlm.run("Pick the short titles from the list." + str(list_of_titles), tools=[filter_short])

Two rules apply to any tool that may be handed to a sub-agent:

• Sub-agents do NOT inherit tools automatically. To give a child a tool, the main agent must pass it explicitly in the REPL: await llm_query("...", tools=[filter_short]).
• Tools must be self-contained. Do imports inside the function body and don't close over REPL-level variables - the child runs in a fresh REPL where outer state does not exist.

The agent can also def new functions inside the REPL at any time and pass them down the same way.

Currently all tools are expected to be Python functions. These functions are available inside the REPL. They are NOT available when the LLM produces code or generates reasoning steps.

Passing environment variables inside the REPL

Tools often need credentials or configuration (API keys, base URLs, account IDs). Pass them through the env_variables kwarg on fast_rlm.run(...):

import os
import fast_rlm

def search_web(query: str, top_k: int = 5) -> list[dict]:
    """Search the web via Tavily and return the top results."""
    import os, urllib.request, json
    req = urllib.request.Request(
        "https://api.tavily.com/search",
        data=json.dumps({"query": query, "max_results": top_k}).encode(),
        headers={
            "Authorization": f"Bearer {os.environ['TAVILY_API_KEY']}",
            "Content-Type": "application/json",
        },
    )
    return json.loads(urllib.request.urlopen(req).read())["results"]

result = fast_rlm.run(
    "Find three recent papers on recursive language models.",
    tools=[search_web],
    env_variables={"TAVILY_API_KEY": os.environ["TAVILY_API_KEY"]},
)

Behavior:

• env_variables must be a dict[str, str].
• Each entry is injected into os.environ inside every Pyodide REPL spawned by the run — the root agent and all sub-agents.
• They are not set on the host Deno process and never appear in prompts, logs, or model context. The model only ever sees a tool's signature + docstring, so the key stays hidden as long as your tool doesn't print or return it.
• Tools read them with the normal os.environ["..."] (do the import os inside the tool body — see the self-containment rule above).

Resumable sessions

A Session lets follow-up queries reuse the work of earlier ones instead of re-exploring from scratch. After every step, the root agent's picklable REPL variables are auto-saved, REPL-defined functions/classes are saved as source, and comments the agent wrote next to assignments are attached to the variables they describe. The next query() restores everything into a fresh REPL and shows the agent the prior queries + answers and the code it ran — so it continues where it left off.

import fast_rlm

session = fast_rlm.Session(session_dir="sessions", session_id="podcasts",
                           config={"primary_agent": "z-ai/glm-5"})

r1 = session.query("Here are all transcripts... Build a guest index and "
                   "summarize what the ML guests said about AGI.\n" + transcripts)
r2 = session.query("Using the index you already built: which guests were most "
                   "optimistic about AGI timelines?")   # no re-exploration

session.variables()   # {name: {type, preview, comment, note, committed}}
session.queries()     # [{"query": ..., "final": ...}, ...]

Where state lives (both args optional):

• Session() — ephemeral: state is kept in a private temp dir and deleted when the object is closed/collected. Every Session() starts from a clean slate — the safe default for experiments.
• Session(session_dir="sessions") — persistent at sessions/state.json; re-open the same dir to resume.
• Session(session_dir="sessions", session_id="podcasts") — persistent at sessions/podcasts/state.json; session_id namespaces several sessions under one dir. (session_id without session_dir raises.)

Also available as run(..., session_dir=..., session_id=...) and fast-rlm --session-dir ... --session-id ....

Behavior and limits:

• Crash-safe. State is written after every step (atomically), not at the end — a killed run resumes from its last completed step.
• Not a 1:1 process clone. Open handles, generators, and JS proxies don't survive; they're reported to the agent as dropped on resume. Variables over 5 MB pickled are skipped.
• The conversation is not carried. A resumed query starts a fresh conversation seeded with the query/answer ledger, the code dump (comments included), and a live inventory of restored variables — so per-query context stays bounded no matter how old the session is.
• Showing earlier code makes follow-ups faster. By default the resumed agent sees the code it ran before (add_session_code_to_context=True, CLI --no-session-code to disable). In our exper