Context Engineering

"Context engineering is the delicate art and science of filling the context window with just the right information for the next step." — Andrej Karpathy

Software Is Changing (Again) Talk @YC AI Startup School

## DeepGraph

Chat with NotebookLM + Podcast Deep Dive

Comprehensive Course Under Construction

Context Engineering Survey-Review of 1400 Research Papers

Awesome Context Engineering Repo

Operationalizing the Latest Research on Context With First Principles & Visuals — July 2025 from ICML, IBM, NeurIPS, OHBM, and more

"Providing “cognitive tools” to GPT-4.1 increases its pass@1 performance on AIME2024 from 26.7% to 43.3%, bringing it very close to the performance of o1-preview." — IBM Zurich

Agent Commands

Support for Claude Code | OpenCode | Amp | Kiro | Codex | Gemini CLI

Context Engineering Survey-Review of 1400 Research Papers | Context Rot | IBM Zurich | Quantum Semantics | Emergent Symbolics ICML Princeton | MEM1 Singapore-MIT | LLM Attractors Shanghai AI | MemOS Shanghai | Latent Reasoning | Dynamic Recursive Depths

A frontier, first-principles handbook for moving beyond prompt engineering to the wider discipline of context design, orchestration, and optimization.

                    Prompt Engineering  │  Context Engineering
                       ↓                │            ↓                      
               "What you say"           │  "Everything else the model sees"
             (Single instruction)       │    (Examples, memory, retrieval,
                                        │     tools, state, control flow)

Definition of Context Engineering

Context is not just the single prompt users send to an LLM. Context is the complete information payload provided to a LLM at inference time, encompassing all structured informational components that the model needs to plausibly accomplish a given task.

— Definition of Context Engineering from A Systematic Analysis of Over 1400 Research Papers

╭─────────────────────────────────────────────────────────────╮
│              CONTEXT ENGINEERING MASTERY COURSE             │
│                    From Zero to Frontier                    │
╰─────────────────────────────────────────────────────────────╯
                          ▲
                          │
                 Mathematical Foundations
                  C = A(c₁, c₂, ..., cₙ)
                          │
                          ▼
┌─────────────┬──────────────┬──────────────┬─────────────────┐
│ FOUNDATIONS │ SYSTEM IMPL  │ INTEGRATION  │ FRONTIER        │
│ (Weeks 1-4) │ (Weeks 5-8)  │ (Weeks 9-10) │ (Weeks 11-12)   │
└─────┬───────┴──────┬───────┴──────┬───────┴─────────┬───────┘
      │              │              │                 │
      ▼              ▼              ▼                 ▼
┌─────────────┐ ┌──────────────┐ ┌──────────────┐ ┌──────────────┐
│ Math Models │ │ RAG Systems  │ │ Multi-Agent  │ │ Meta-Recurs  │
│ Components  │ │ Memory Arch  │ │ Orchestrat   │ │ Quantum Sem  │
│ Processing  │ │ Tool Integr  │ │ Field Theory │ │ Self-Improv  │
│ Management  │ │ Agent Systems│ │ Evaluation   │ │ Collaboration│
└─────────────┘ └──────────────┘ └──────────────┘ └──────────────┘

Why This Repository Exists

"Meaning is not an intrinsic, static property of a semantic expression, but rather an emergent phenomenon" — Agostino et al. — July 2025, Indiana University

Prompt engineering received all the attention, but we can now get excited for what comes next. Once you've mastered prompts, the real power comes from engineering the entire context window that surrounds those prompts. Guiding thought, if you will.

This repository provides a progressive, first-principles approach to context engineering, built around a biological metaphor:

atoms → molecules → cells → organs → neural systems → neural & semantic field theory 
  │        │         │         │             │                         │        
single    few-     memory +   multi-   cognitive tools +     context = fields +
prompt    shot     agents     agents   operating systems     persistence & resonance

"Abstraction is the cost of generalization"— Grant Sanderson (3Blue1Brown)

A Survey of Context Engineering - July 2025

On Emergence, Attractors, and Dynamical Systems Theory | Columbia DST

https://github.com/user-attachments/assets/9f046259-e5ec-4160-8ed0-41a608d8adf3

graph TD
    classDef basic fill:#e1f5fe,stroke:#01579b,stroke-width:2px,color:#01579b
    classDef intermediate fill:#e8f5e9,stroke:#2e7d32,stroke-width:2px,color:#2e7d32
    classDef advanced fill:#fff3e0,stroke:#e65100,stroke-width:2px,color:#e65100
    classDef meta fill:#f3e5f5,stroke:#6a1b9a,stroke-width:2px,color:#6a1b9a

    subgraph Basic["Level 1: Basic Context Engineering"]
        A[Atoms]
        B[Molecules]
        C[Cells]
        D[Organs]
    end

    subgraph Field["Level 2: Field Theory"]
        E[Neural Systems]
        F[Neural Fields]
    end

    subgraph Protocol["Level 3: Protocol System"]
        G[Protocol Shells]
        H[Unified System]
    end

    subgraph Meta["Level 4: Meta-Recursion"]
        I[Meta-Recursive Framework]
    end

    %% Connections
    A --> B
    B --> C
    C --> D
    D --> E
    E --> F
    F --> G
    G --> H
    H --> I

    %% Descriptions for each level
    A1["Single instructions

Simple constraints

Basic prompts"] --> A
    B1["Example pairs

Few-shot patterns

Demonstration sets"] --> B
    C1["Persistent memory

State management

Context window"] --> C
    D1["Multi-step flows

Specialists

System orchestration"] --> D
    E1["Reasoning frameworks

Verification tools

Cognitive patterns"] --> E
    F1["Continuous meaning

Attractors & resonance

Symbolic residue"] --> F
    G1["Structured templates

Field operations

Emergence protocols"] --> G
    H1["Protocol integration

System-level emergence

Self-maintenance"] --> H
    I1["Self-reflection

Recursive improvement

Interpretable evolution"] --> I

    %% Real-world parallels
    A2["Like: Basic prompt

engineering"] -.-> A
    B2["Like: Few-shot

learning"] -.-> B
    C2["Like: Conversational

chatbots"] -.-> C
    D2["Like: Multi-agent

systems"] -.-> D
    E2["Like: ReAct

Chain-of-Thought"] -.-> E
    F2["Like: Semantic

field theory"] -.-> F
    G2["Like: Protocol

orchestration"] -.-> G
    H2["Like: Self-organizing

systems"] -.-> H
    I2["Like: Self-improving

intelligence"] -.-> I

    %% Apply classes
    class A,B,C,D,A1,A2,B1,B2,C1,C2,D1,D2 basic
    class E,F,E1,E2,F1,F2 intermediate
    class G,H,G1,G2,H1,H2 advanced
    class I,I1,I2 meta

Quick Start

1. Read 00_foundations/01_atoms_prompting.md (5 min)
   Understand why prompts alone often underperform

2. Run 10_guides_zero_to_hero/01_min_prompt.py (Jupyter Notebook style) Experiment with a minimal working example

3. Explore 20_templates/minimal_context.yaml
   Copy/paste a template into your own project

4. Study 30_examples/00_toy_chatbot/
   See a complete implementation with context management

Learning Path

┌─────────────────┐     ┌──────────────────┐     ┌────────────────┐
│ 00_foundations/ │     │ 10_guides_zero_  │     │ 20_templates/  │
│                 │────▶│ to_one/          │────▶│                │
│ Theory & core   │     │ Hands-on         │     │ Copy-paste     │
│ concepts        │     │ walkthroughs     │     │ snippets       │
└─────────────────┘     └──────────────────┘     └────────────────┘
         │                                                │
         │                                                │
         ▼                                                ▼
┌─────────────────┐                             ┌────────────────┐
│ 40_reference/   │◀───────────────────────────▶│ 30_examples/   │
│                 │                             │                │
│ Deep dives &    │                             │ Real projects, │
│ eval cookbook   │                             │ progressively  │
└─────────────────┘                             │ complex        │
         ▲                                      └────────────────┘
         │                                                ▲
         │                                                │
         └────────────────────┐               ┌───────────┘
                              ▼               ▼
                         ┌─────────────────────┐
                         │ 50_contrib/         │
                         │                     │
                         │ Community           │
                         │ contributions       │
                         └─────────────────────┘

What You'll Learn

Karpathy + 3Blue1Brown Inspired Style

For learners of all experience levels

1. First principles – start with the fundamental context
2. Iterative add-on – add only what the model demonstrably lacks
3. Measure everything – token cost, latency, quality score
4. Delete ruthlessly – pruning beats padding
5. Code > slides – every concept has a runnable cell
6. Visualize everything — every concept is visualized with ASCII and symbolic diagrams

Research Evidence

Memory + Reasoning

MEM1: Learning to Synergize Memory and Reasoning for Efficient Long-Horizon Agents - Singapore-MIT June 2025

“Our results demonstrate the promise of reasoning-driven memory consolidation as a scalable alternative to existing solutions for training long-horizon interactive agents, where both efficiency and performance are optimized." — Singapore-MIT

1. MEM1 trains AI agents to keep only what matters—merging memory and reasoning at every step—so they never get overwhelmed, no matter how long the task.

2. **Instead of piling up endless context, MEM1 compresses each interaction