Big RAG Plugin for LM Studio

A powerful RAG (Retrieval-Augmented Generation) plugin for LM Studio that can index and search through gigabytes or even terabytes (not tested) of document data. Hosted here: ari99/lm_studio_big_rag_plugin on GitHub.

Features

• Massive Scale: Designed to handle large document collections (GB to TB scale)
• Deep Directory Scanning: Recursively scans all subdirectories
• Multiple File Formats: Built-in support for PDF, EPUB, HTML, Markdown, plain text, images (OCR), and more
• User-defined extensions: Index extra plain-text types (e.g. .java, .py, .rs) via the Additional plain-text extensions setting
• OCR Support: Optional OCR for image files using Tesseract
• Vector Search: Uses Vectra with sharded indexes for efficient vector storage and retrieval (avoids single-file size limits)
• Incremental Indexing: Automatically detects and skips already-indexed files
• Concurrent Processing: Configurable concurrency for optimal performance
• Persistent Storage: Vector embeddings are stored locally and persist across sessions

Supported File Types

Built-in (no extra configuration)

Category	Extensions
Documents	.pdf, .epub, .txt, .text
Markdown	.md, .mdx, .markdown, .mkd, .mkdn, .mdown
Web	.htm, .html, .xhtml
Images (OCR)	.bmp, .jpeg, .jpg, .png

PDF, EPUB, HTML, and images use dedicated parsers. Plain-text and Markdown files use the text parser.

Additional plain-text extensions (user-configured)

Any other plain-text extension can be indexed by listing it in Additional plain-text extensions in the chat Integrations sidebar (or via BIG_RAG_ADDITIONAL_EXTENSIONS for CLI indexing).

Common examples:

.java
.cs
.py
.rs
.go
.ts
.tsx
.js
.jsx
.c
.cpp
.h
.sql
.yaml
.toml

Format rules:

• One extension per line, or comma-separated on one line
• With or without a leading dot (.java and java both work)
• Lines starting with # are comments; inline # after an extension is also stripped
• Wildcards (*, ?) are not allowed

Rejected automatically: binaries and formats that already have dedicated parsers or are unsafe to read as text — e.g. .exe, .zip, .jar, .docx, .pdf, .png. Rejections are logged as [BigRAG] Rejected additional extension … in developer logs.

After changing extensions: trigger a reindex (empty vector store + chat message, or Manual Reindex Trigger ON) so new file types are picked up. Pair with Exclude filename patterns when indexing source trees:

node_modules/**
target/**
bin/**
dist/**
.git/**

CLI / headless indexing:

BIG_RAG_ADDITIONAL_EXTENSIONS=".java;.cs;.py" \
BIG_RAG_DOCS_DIR=/path/to/repo \
BIG_RAG_DB_DIR=/path/to/vectorstore \
npm run index

Not yet supported

• RAR archives: listed in built-in types but not implemented (files are skipped)

Installation

cd big-rag-plugin
npm install
npm run build

Then choose one of the following:

Goal	Command	Plugin id for REST
Dev (hot reload, chat UI)	npm run dev	Use installed copy for REST (see below)
Local install (REST + chat)	lms dev --install -y	mindstudio/big-rag
Publish to Hub	lms login then lms push -y	mindstudio/big-rag

After code changes: npm run build then re-run npm run dev, lms dev --install -y, or lms push -y.

Hub page: lmstudio.ai/mindstudio/big-rag

Manual testing

Default chat workflow — no lms server start; the server runs inside the LM Studio app (http://localhost:1234 when enabled).

1. Open the LM Studio desktop app.
2. Load the plugin (pick one):
3. Dev: cd big-rag-plugin && npm run dev (leave running; good for UI iteration)
4. Installed (required for REST): cd big-rag-plugin && npm run build && lms dev --install -y
5. Open a Chat → right sidebar → Integrations (hammer icon) → enable Big RAG → expand the row and set Documents Directory, Vector Store Directory, etc.
6. Send a chat message. The first message auto-indexes if the vector store is empty.

Automated tests (no UI):

cd big-rag-plugin && npm test

Headless indexing (optional; LM Studio app must be open for embeddings):

cd big-rag-plugin && npm run index

Paths are set in package.json index script or via BIG_RAG_DOCS_DIR / BIG_RAG_DB_DIR env vars.

Configuration

All plugin fields appear in the chat Integrations sidebar when Big RAG is enabled (expand the plugin row). There is no separate global settings screen for document paths.

The plugin provides the following configuration options:

Required Settings

• Documents Directory: Root directory containing your documents (read access required)
• Vector Store Directory: Where the vector database will be stored (read/write access required)

Embedding model

• Embedding Model (plugin setting): String passed to LM Studio’s embedding load API. Both common forms can work for the same weights—for example mixedbread-ai/mxbai-embed-large-v1 (Hub / lms get) and text-embedding-mxbai-embed-large-v1 (as shown in lms ls). Use one spelling consistently for indexing and retrieval so it matches .big-rag-embedding.json; switching spelling without reindexing can trigger a mismatch warning. Default: nomic-ai/nomic-embed-text-v1.5-GGUF.
• After changing the embedding model, run a full reindex (toggle Manual Reindex Trigger with Skip Previously Indexed Files off, or clear the vector store and let first-run indexing rebuild). Vectors from different models are not comparable in the same index.
• .big-rag-embedding.json: Written under the vector store directory when the index has at least one chunk; records the model id and vector length used to build the index. If the configured model no longer matches this file, retrieval is blocked until you reindex or revert the setting. If the index has zero chunks, this file is removed so metadata cannot drift (including after manual shard deletion).
• Indexes built with older plugin versions may have chunks but no manifest; retrieval still works, and a full reindex will create the manifest.

Retrieval Settings

• Retrieval Limit (1-20, default: 5): Maximum number of chunks to return
• Retrieval Affinity Threshold (0.0-1.0, default: 0.5): Minimum similarity score for relevance
• Chunk Size (128-2048 tokens, default: 512): Size of text chunks for embedding
• Chunk Overlap (0-512 tokens, default: 100): Overlap between consecutive chunks

Performance Settings

• Max Concurrent Files (1-10, default: 1): Number of files to process simultaneously
• Enable OCR (default: true): Enable OCR for image files and image-based PDFs using LM Studio's built-in document parser

File indexing filters

• Exclude filename patterns: Optional globs (one per line) matched against paths relative to the Documents Directory. Example: *.png, node_modules/**, target/**. Applied after the extension gate; does not remove chunks already in the vector store.
• Additional plain-text extensions: See Additional plain-text extensions above. Configured in the same sidebar block; files matching listed extensions are read as UTF-8 plain text and chunked like .txt.

CLI equivalents: BIG_RAG_EXCLUDE_PATTERNS and BIG_RAG_ADDITIONAL_EXTENSIONS (semicolon-separated for env vars).

Reindexing Controls

• Manual Reindex Trigger (toggle): Turn this ON and submit any chat message to force indexing to run on every chat session where the plugin is enabled. Flip it OFF once you’re done to stop the automatic reindex loop.
• Skip Previously Indexed Files (default: true): If enabled while "Manual Reindex Trigger" is enabled, each manual run touches just the documents that are new or have changed since the last index; if disabled, every chat rebuilds the entire index from scratch. Combine "Skip Previously Indexed Files" and "Manual Reindex Trigger" to choose between incremental updates or repeated full refreshes.
• Automatic First-Run: If the vector store is empty, the plugin automatically indexes the configured documents the first time any chat message is processed—no manual input is required.

Usage

1. Configure the plugin (one place — the chat sidebar):
2. Open a Chat in LM Studio
3. Right sidebar → Integrations tab (hammer icon) — not Settings → Integrations
4. Enable Big RAG and expand the plugin row to show config fields
5. Set Documents Directory (e.g. /Users/user/Documents/MyLibrary)
6. Set Vector Store Directory (e.g. /Users/user/.lmstudio/big-rag-db)

Settings → Integrations (gear menu) only controls tool-call confirmation — not document paths.

1. Initial indexing:
2. The first chat message triggers a scan/index if the vector store is empty

3. Progress appears in LM Studio developer logs ([BigRAG] lines)

4. Query your documents:

5. Chat normally; the prompt preprocessor injects relevant passages automatically

Using Big RAG via REST API

Big RAG registers a prompt preprocessor (automatic RAG in chat) and a tools provider (big_rag_search, big_rag_index_status) for /api/v1/chat.

Prerequisites

1. LM Studio app open with local server enabled (http://localhost:1234).
2. Plugin installed locally or from Hub — REST requires id "mindstudio/big-rag" (owner/name from manifest.json). The dev id (dev/mindstudio/big-rag from npm run dev) works in chat UI but not in REST.
3. Install/update: npm run build && lms dev --install -y (local) or lms push -y (Hub).
4. LM Studio Server Settings: enable Allow calling servers from mcp.json (required for plugin integrations).
5. If API auth is on: create a token and pass Authorization: Bearer $LM_API_TOKEN (docs).
6. Load an LLM and embedding model in LM Studio.

REST configuration (tools)

REST tool calls have no chat session, so they do not read the sidebar directly. Paths come from (in order):

1. Chat sidebar values, merged and synced to ~/.lmstudio/big-rag-tools-config.json when the prompt preprocessor runs (send at least one chat message with Big RAG enabled after configuring paths).
2. That synced JSON file (persists until deleted).
3. Environment variables on the LM Studio process: BIG_RAG_DOCS_DIR, BIG_RAG_DB_DIR, optional BIG_RAG_EMBEDDING_MODEL, BIG_RAG_RETRIEVAL_LIMIT, BIG_RAG_RETRIEVAL_AFFINITY_THRESHOLD.

If you delete the JSON file, send a chat message again or set the env vars before calling tools via curl.

Native REST API (/api/v1/chat)

Load your API token (example: repo-root .env):

export $(grep -v '^#' .env | xargs)   # sets LM_API_TOKEN

Index status (one tool — reliable):

curl -s http://127.0.0.1:1234/api/v1/chat \
  -H "Authorization: Bearer $LM_API_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "your-model-id",
    "input": "Call big_rag_index_status and report totalChunks.",
    "integrations": [{
      "type": "plugin",
      "id": "mindstudio/big-rag",
      "allowed_tools": ["big_rag_index_status"]
    }],
    "temperature": 0
  }' | python3 -m json.tool

Search (one tool — use limit 3 in the prompt to avoid context overflow):

curl -s http://127.0.0.1:1234/api/v1/chat \
  -H "Authorization: Bearer $LM_API_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "your-model-id",
    "input": "Use big_rag_search to find content about rifle cleaning. Use limit 3.",
    "integrations": [{
      "type": "plugin",
      "id": "mindstudio/big-rag",
      "allowed_tools": ["big_rag_search"]
    }],
    "temperature": 0
  }' | python3 -m json.tool

Preprocessor-only RAG (no explicit tools — model answers using injected context; needs sidebar config + chat sync or env vars):

curl -s http://127.0.0.1:1234/api/v1/chat \
  -H "Authorization: Bearer $LM_API_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "your-model-id",
    "input": "What does the documentation say about rifling?",
    "integrations": [{
      "type": "plugin",
      "id": "mindstudio/big-rag"
    }],
    "temperature": 0
  }' | python3 -m json.tool

Plugin tools

Tool	Description
big_rag_search	Embed a query and return matching passages (JSON with scores and file names)
big_rag_index_status	Return chunk count, unique file count, and configured directory paths

REST limitations (observed behavior)

• One tool per request works reliably. Prompts like “first call X, then Y” may fail with tool_format_generation_error because smaller models (e.g. Llama 3.1 8B) emit multiple tool calls in one generation block. Run separate curl requests instead.
• Large tool outputs: big_rag_search with limit=10 returns full passage text and can exceed the model context window (e.g. 14848 tokens). Ask for limit 3 or increase context length in LM Studio.
• Permission denied: enable “Allow calling servers from mcp.json” and use a valid API token.
• “Vector store / Documents directory is not configured”: sync config via a chat message, restore ~/.lmstudio/big-rag-tools-config.json, or set BIG_RAG_DOCS_DIR / BIG_RAG_DB_DIR.
• Prefer /api/v1/chat over OpenAI-compatible /v1/chat/completions for plugin integrations.
• Use the same Embedding Model for indexing and retrieval; reindex after changing models.

Common Use Cases

Source code / repo RAG

Point Documents Directory at a project