AI Diplomacy: LLM-Powered Strategic Gameplay

Created by Alex Duffy @Alx-Ai & Tyler Marques @Tylermarques

Overview

This repository extends the original Diplomacy project with sophisticated AI agents powered by Large Language Models (LLMs). Each power in the game is controlled by an autonomous agent that maintains state, forms relationships, conducts negotiations, and makes strategic decisions.

Key Features

🤖 Stateful AI Agents

Each power is represented by a DiplomacyAgent with:

• Dynamic Goals: Strategic objectives that evolve based on game events
• Relationship Tracking: Maintains relationships (Enemy/Unfriendly/Neutral/Friendly/Ally) with other powers
• Memory System: Dual-layer memory with structured diary entries and consolidation
• Personality: Power-specific system prompts shape each agent's diplomatic style

💬 Rich Negotiations

• Multi-round message exchanges (private and global)
• Relationship-aware communication strategies
• Message history tracking and analysis
• Detection of ignored messages and non-responsive powers

🎯 Strategic Order Generation

• BFS pathfinding for movement analysis
• Context-aware order selection with nearest threats/opportunities
• Fallback logic for robustness
• Support for multiple LLM providers (OpenAI, Claude, Gemini, DeepSeek, OpenRouter)

📊 Advanced Game Analysis

• Custom phase summaries with success/failure categorization
• Betrayal detection through order/negotiation comparison
• Strategic planning phases for high-level directives
• Comprehensive logging of all LLM interactions

🧠 Memory Management

• Private Diary: Structured, phase-prefixed entries for LLM context
• Negotiation summaries with relationship updates
• Order reasoning and strategic justifications
• Phase result analysis with betrayal detection
• Yearly Consolidation: Automatic summarization of old entries to prevent context overflow
• Smart Context Building: Only relevant history provided to LLMs

How AI Agents Work

The following diagram illustrates the complete information flow and decision-making process for each AI agent:

graph TB
    %% Game State Sources
    subgraph "Game State Information"
        GS[Game State

- Unit Positions

- Supply Centers

- Power Status]
        GH[Game History

- Past Orders

- Past Messages

- Phase Results]
        PS[Phase Summary

- Successful Moves

- Failed Moves

- Board Changes]
    end

    %% Agent Internal State
    subgraph "Agent State (DiplomacyAgent)"
        GOALS[Dynamic Goals

- Expansion targets

- Alliance priorities

- Defense needs]
        REL[Relationships

Enemy ↔ Ally Scale]

        subgraph "Memory System"
            DIARY[Private Diary

Phase-prefixed entries]

            ND[Negotiation Diary

- Message analysis

- Trust assessment

- Relationship changes]
            OD[Order Diary

- Strategic reasoning

- Risk/reward analysis]
            PRD[Phase Result Diary

- Outcome analysis

- Betrayal detection

- Success evaluation]

            CONS[Diary Consolidation

Yearly summaries

via Gemini Flash]
        end

        JOURNAL[Private Journal

Debug logs only]
    end

    %% Context Building
    subgraph "Context Construction"
        POC[Possible Order Context

- BFS pathfinding

- Nearest enemies

- Uncontrolled SCs

- Adjacent territories]

        BCP[build_context_prompt

Assembles all info]

        RECENT[Recent Context

- Last 40 diary entries

- Current relationships

- Active goals]
    end

    %% LLM Interactions
    subgraph "LLM Decision Points"
        INIT_LLM[Initialization

Set initial goals

& relationships]

        NEG_LLM[Negotiation

Generate messages

Update relationships]

        PLAN_LLM[Planning

Strategic directives]

        ORD_LLM[Order Generation

Choose moves]

        STATE_LLM[State Update

Revise goals

& relationships]
    end

    %% Prompt Templates
    subgraph "Prompt Templates"
        PROMPTS[Power-specific prompts

+ Instruction templates

+ Context templates]
    end

    %% Information Flow
    GS --> BCP
    GH --> BCP
    PS --> STATE_LLM

    GOALS --> BCP
    REL --> BCP
    DIARY --> RECENT
    RECENT --> BCP

    POC --> BCP
    BCP --> NEG_LLM
    BCP --> ORD_LLM
    BCP --> PLAN_LLM

    PROMPTS --> INIT_LLM
    PROMPTS --> NEG_LLM
    PROMPTS --> PLAN_LLM
    PROMPTS --> ORD_LLM
    PROMPTS --> STATE_LLM

    %% Diary Updates
    NEG_LLM --> ND
    ORD_LLM --> OD
    PS --> PRD

    ND --> DIARY
    OD --> DIARY
    PRD --> DIARY

    %% State Updates
    INIT_LLM --> GOALS
    INIT_LLM --> REL
    NEG_LLM --> REL
    STATE_LLM --> GOALS
    STATE_LLM --> REL

    %% Consolidation
    DIARY -->|Every 2 years| CONS
    CONS -->|Summarized| DIARY

    %% Styling
    classDef gameState fill:#e74c3c,stroke:#333,stroke-width:2px,color:#fff
    classDef agentState fill:#3498db,stroke:#333,stroke-width:2px,color:#fff
    classDef context fill:#2ecc71,stroke:#333,stroke-width:2px,color:#fff
    classDef llm fill:#f39c12,stroke:#333,stroke-width:2px,color:#fff
    classDef memory fill:#9b59b6,stroke:#333,stroke-width:2px,color:#fff

    class GS,GH,PS gameState
    class GOALS,REL,JOURNAL agentState
    class POC,BCP,RECENT context
    class INIT_LLM,NEG_LLM,PLAN_LLM,ORD_LLM,STATE_LLM llm
    class DIARY,ND,OD,PRD,CONS memory

Key Components Explained

1. Information Sources
2. Game State: Current board position, unit locations, supply center ownership
3. Game History: Complete record of past orders, messages, and results

4. Phase Summaries: Categorized analysis of what succeeded/failed each turn

5. Agent Memory Architecture

6. Private Diary: The main memory system, with structured entries for each phase
7. Diary Types: Three specialized entry types capture different aspects of gameplay
8. Consolidation: Automatic yearly summarization prevents context overflow

9. Journal: Unstructured logs for debugging (not used by LLMs)

10. Context Building

11. Strategic Analysis: BFS pathfinding identifies threats and opportunities
12. Relationship Context: Current diplomatic standings influence all decisions

13. Historical Context: Recent diary entries provide continuity

14. LLM Decision Points

15. Initialization: Sets starting personality and objectives
16. Negotiations: Generates contextual messages based on relationships
17. Planning: Creates high-level strategic directives
18. Orders: Selects specific moves with full strategic context
19. State Updates: Adjusts goals and relationships based on outcomes

Implementation Details

Core Files

1. lm_game.py - Main game orchestrator
2. Manages agent lifecycle and game phases
3. Coordinates async LLM calls for maximum performance
4. Handles error tracking and recovery

5. Saves game state with phase summaries and agent relationships

6. ai_diplomacy/agent.py - Stateful agent implementation

7. DiplomacyAgent class with goals, relationships, and memory
8. Robust JSON parsing for various LLM response formats
9. Diary entry generation for each game event

10. State update logic based on game outcomes

11. ai_diplomacy/clients.py - LLM abstraction layer

12. BaseModelClient interface for all LLM providers
13. Implementations for OpenAI, Claude, Gemini, DeepSeek, OpenRouter
14. Prompt construction and response parsing

15. Retry logic and error handling

16. ai_diplomacy/possible_order_context.py - Strategic analysis

17. BFS pathfinding on game map
18. Nearest threat/opportunity identification
19. Adjacent territory analysis

20. Rich XML context generation for orders

21. ai_diplomacy/prompt_constructor.py - Centralized prompt building

22. Assembles game state, agent state, and strategic context
23. Formats prompts for different LLM tasks

24. Integrates with template system

25. ai_diplomacy/game_history.py - Phase-by-phase game tracking

26. Stores messages, orders, and results
27. Provides historical context for agents
28. Tracks ignored messages for relationship analysis

Prompt Templates

The ai_diplomacy/prompts/ directory contains customizable templates:

• Power-specific system prompts (e.g., france_system_prompt.txt)
• Task-specific instructions (order_instructions.txt, conversation_instructions.txt)
• Diary generation prompts for different game events
• State update and planning templates

Running AI Games

# Basic game with negotiations
python lm_game.py --max_year 1910 --num_negotiation_rounds 3

# With strategic planning phase
python lm_game.py --max_year 1910 --planning_phase --num_negotiation_rounds 2

# Custom model assignment (order: AUSTRIA, ENGLAND, FRANCE, GERMANY, ITALY, RUSSIA, TURKEY)
python lm_game.py --models "claude-3-5-sonnet-20241022,gpt-4o,claude-3-5-sonnet-20241022,gpt-4o,claude-3-5-sonnet-20241022,gpt-4o,claude-3-5-sonnet-20241022"

# Run until game completion or specific year
python lm_game.py --num_negotiation_rounds 2 --planning_phase

# Write all artefacts to a chosen directory (auto-resumes if it already exists)
python lm_game.py --run_dir results/game_run_001

# Resume an interrupted game from a specific phase
python lm_game.py --run_dir results/game_run_001 --resume_from_phase S1902M

# Critical-state analysis: resume from an existing run but save new results elsewhere
python lm_game.py \
  --run_dir results/game_run_001 \
  --critical_state_analysis_dir results/critical_analysis_001 \
  --resume_from_phase F1903M

# End the simulation after a particular phase regardless of remaining years
python lm_game.py --run_dir results/game_run_002 --end_at_phase F1905M

# Set the global max_tokens generation limit
python lm_game.py --run_dir results/game_run_003 --max_tokens 8000

# Per-model token limits (AU,EN,FR,GE,IT,RU,TR)
python lm_game.py --run_dir results/game_run_004 \
  --max_tokens_per_model "8000,8000,16000,8000,8000,16000,8000"

# Use a custom prompts directory
python lm_game.py --run_dir results/game_run_005 --prompts_dir ./prompts/my_variants

Setting --models (quick guide)

• Pass one comma-separated list of up to seven model IDs in this fixed order: AUSTRIA, ENGLAND, FRANCE, GERMANY, ITALY, RUSSIA, TURKEY.

• Model-ID syntax

<client prefix:>model[@base_url][#api_key]

• prefix: – specify the client (openai, openai-requests, openai-responses, anthropic, gemini, deepseek, openrouter, together).
• @base_url – hit a proxy / alt endpoint.
• #api_key – inline key (overrides env vars).

bash # gpt-4o on openrouter for all powers: --models "openrouter:gpt-4o" # custom URL+apikey for Austria only: --models "openai:llama-3.2-3b@http://localhost:8000#myapikey,openai:gpt-4o,openai:gpt-4o,openai:gpt-4o,openai:gpt-4o,openai:gpt-4o,openai:gpt-4o"

Running Batch Experiments with experiment_runner.py

experiment_runner.py is a lightweight orchestrator: it spins up many lm_game.py runs in parallel, gathers their artefacts under one experiment directory, and then executes the analysis modules you specify. All flags that belong to lm_game.py can be passed straight through; the runner validates them and forwards them unchanged to every game instance.

Examples

# Run 10 independent games (iterations) in parallel, using a custom prompts dir
# and a single model (GPT-4o) for all seven powers.
python3 experiment_runner.py \
    --experiment_dir "results/exp001" \
    --iterations 10 \
    --parallel 10 \
    --max_year 1905 \
    --num_negotiation_rounds 0 \
    --prompts_dir "ai_diplomacy/prompts" \
    --models "gpt-4o,gpt-4o,gpt-4o,gpt-4o,gpt-4o,gpt-4o,gpt-4o"


# Critical-state analysis: resume every run from W1901A (taken from an existing
# base run) and stop after S1902M.  Two analysis modules will be executed:
#  • summary         → aggregated results & scores
#  • critical_state  → before/after snapshots around the critical phase
python3 experiment_runner.py \
    --experiment_dir "results/exp002" \
    --iterations 10 \
    --parallel 10 \
    --resume_from_phase W1901A \
    --end_at_phase S1902M \
    --num_negotiation_rounds 0 \
    --critical_state_base_run "results/test1" \
    --prompts_dir "ai_diplomacy/prompts" \
    --analysis_modules "summary,critical_state" \
    --models "gpt-4o,gpt-4o,gpt-4o,gpt-4o,gpt-4o,gpt-4o,gpt-4o"

(Any other lm_game.py flags—--planning_phase, --max_tokens, etc.—can be added exactly where you’d use them on a single-game run.)

Experiment-runner–specific arguments