ChatGPT Retrieval Plugin

Build Custom GPTs with a Retrieval Plugin backend to give ChatGPT access to personal documents.

Introduction

The ChatGPT Retrieval Plugin repository provides a flexible solution for semantic search and retrieval of personal or organizational documents using natural language queries. It is a standalone retrieval backend, and can be used with ChatGPT custom GPTs, function calling with the chat completions or assistants APIs, or with the ChatGPT plugins model (deprecated). ChatGPT and the Assistants API both natively support retrieval from uploaded files, so you should use the Retrieval Plugin as a backend only if you want more granular control of your retrieval system (e.g. document text chunk length, embedding model / size, etc.).

The repository is organized into several directories:

This README provides detailed information on how to set up, develop, and deploy the ChatGPT Retrieval Plugin (stand-alone retrieval backend).

Table of Contents

• Quickstart
• About
• Retrieval Plugin
• Retrieval Plugin with custom GPTs
• Retrieval Plugin with function calling
• Retrieval Plugin with the plugins model (deprecated)
• API Endpoints
• Memory Feature
• Security
• Choosing an Embeddings Model
• Development
• Setup
  • General Environment Variables
• Choosing a Vector Database
  • Pinecone
  • Elasticsearch
  • MongoDB Atlas
  • Weaviate
  • Zilliz
  • Milvus
  • Qdrant
  • Redis
  • Llama Index
  • Chroma
  • Azure Cognitive Search
  • Azure CosmosDB Mongo vCore
  • Supabase
  • Postgres
  • AnalyticDB
• Running the API Locally
• Personalization
• Authentication Methods
• Deployment
• Webhooks
• Scripts
• Limitations
• Contributors
• Future Directions

Quickstart

Follow these steps to quickly set up and run the ChatGPT Retrieval Plugin:

1. Install Python 3.10, if not already installed.
2. Clone the repository: git clone https://github.com/openai/chatgpt-retrieval-plugin.git
3. Navigate to the cloned repository directory: cd /path/to/chatgpt-retrieval-plugin
4. Install poetry: pip install poetry
5. Create a new virtual environment with Python 3.10: poetry env use python3.10
6. Activate the virtual environment: poetry shell
7. Install app dependencies: poetry install
8. Create a bearer token
9. Set the required environment variables:

``` export DATASTORE= export BEARER_TOKEN= export OPENAI_API_KEY= export EMBEDDING_DIMENSION=256 # edit this value based on the dimension of the embeddings you want to use export EMBEDDING_MODEL=text-embedding-3-large # edit this based on your model preference, e.g. text-embedding-3-small, text-embedding-ada-002

# Optional environment variables used when running Azure OpenAI export OPENAI_API_BASE=https://.openai.azure.com/ export OPENAI_API_TYPE=azure export OPENAI_EMBEDDINGMODEL_DEPLOYMENTID= export OPENAI_METADATA_EXTRACTIONMODEL_DEPLOYMENTID= export OPENAI_COMPLETIONMODEL_DEPLOYMENTID= export OPENAI_EMBEDDING_BATCH_SIZE=

# Add the environment variables for your chosen vector DB. # Some of these are optional; read the provider's setup docs in /docs/providers for more information.

# Pinecone export PINECONE_API_KEY= export PINECONE_ENVIRONMENT= export PINECONE_INDEX=

# Weaviate export WEAVIATE_URL= export WEAVIATE_API_KEY= export WEAVIATE_CLASS=

# Zilliz export ZILLIZ_COLLECTION= export ZILLIZ_URI= export ZILLIZ_USER= export ZILLIZ_PASSWORD=

# Milvus export MILVUS_COLLECTION= export MILVUS_HOST= export MILVUS_PORT= export MILVUS_USER= export MILVUS_PASSWORD=

# Qdrant export QDRANT_URL= export QDRANT_PORT= export QDRANT_GRPC_PORT= export QDRANT_API_KEY= export QDRANT_COLLECTION=

# AnalyticDB export PG_HOST= export PG_PORT= export PG_USER= export PG_PASSWORD= export PG_DATABASE= export PG_COLLECTION=

# Redis export REDIS_HOST= export REDIS_PORT= export REDIS_PASSWORD= export REDIS_INDEX_NAME= export REDIS_DOC_PREFIX= export REDIS_DISTANCE_METRIC= export REDIS_INDEX_TYPE=

# Llama export LLAMA_INDEX_TYPE= export LLAMA_INDEX_JSON_PATH= export LLAMA_QUERY_KWARGS_JSON_PATH= export LLAMA_RESPONSE_MODE=

# Chroma export CHROMA_COLLECTION= export CHROMA_IN_MEMORY= export CHROMA_PERSISTENCE_DIR= export CHROMA_HOST= export CHROMA_PORT=

# Azure Cognitive Search export AZURESEARCH_SERVICE= export AZURESEARCH_INDEX= export AZURESEARCH_API_KEY= (optional, uses key-free managed identity if not set)

# Azure CosmosDB Mongo vCore export AZCOSMOS_API = export AZCOSMOS_CONNSTR = export AZCOSMOS_DATABASE_NAME = export AZCOSMOS_CONTAINER_NAME =

# Supabase export SUPABASE_URL= export SUPABASE_ANON_KEY=

# Postgres export PG_HOST= export PG_PORT= export PG_USER= export PG_PASSWORD= export PG_DB=

# Elasticsearch export ELASTICSEARCH_URL= (either specify host or cloud_id) export ELASTICSEARCH_CLOUD_ID=

export ELASTICSEARCH_USERNAME= export ELASTICSEARCH_PASSWORD= export ELASTICSEARCH_API_KEY=

export ELASTICSEARCH_INDEX= export ELASTICSEARCH_REPLICAS= export ELASTICSEARCH_SHARDS=

# MongoDB Atlas export MONGODB_URI= export MONGODB_DATABASE= export MONGODB_COLLECTION= export MONGODB_INDEX= ```

1. Run the API locally: poetry run start
2. Access the API documentation at http://0.0.0.0:8000/docs and test the API endpoints (make sure to add your bearer token).

About

Retrieval Plugin

This is a standalone retrieval backend that can be used with ChatGPT custom GPTs, function calling with the chat completions or assistants APIs, or with the ChatGPT plugins model (deprecated).

It enables a model to carry out semantic search and retrieval of personal or organizational documents, and write answers informed by relevent retrieved context (sometimes referred to as "Retrieval-Augmented Generation" or "RAG"). It allows users to obtain the most relevant document snippets from their data sources, such as files, notes, or emails, by asking questions or expressing needs in natural language. Enterprises can make their internal documents available to their employees through ChatGPT using this plugin.

The plugin uses OpenAI's embeddings model (text-embedding-3-large 256 dimension embeddings by default) to generate embeddings of document chunks, and then stores and queries them using a vector database on the backend. As an open-source and self-hosted solution, developers can deploy their own Retrieval Plugin and register it with ChatGPT. The Retrieval Plugin supports several vector database providers, allowing developers to choose their preferred one from a list.

A FastAPI server exposes the plugin's endpoints for upserting, querying, and deleting documents. Users can refine their search results by using metadata filters by source, date, author, or other criteria. The plugin can be hosted on any cloud platform that supports Docker containers, such as Fly.io, Heroku, Render, or Azure Container Apps. To keep the vector database updated with the latest documents, the plugin can process and store documents from various data sources continuously, using incoming webhooks to the upsert and delete endpoints. Tools like Zapier or Make can help configure the webhooks based on events or schedules.

Retrieval Plugin with Custom GPTs

To create a custom GPT that can use your Retrieval Plugin for semantic search and retrieval of your documents, and even store new information back to the database, you first need to have deployed a Retrieval Plugin. For detailed instructions on how to do this, please refer to the Deployment section. Once you have your app URL (e.g., https://your-app-url.com), take the following steps:

1. Navigate to the create GPT page at https://chat.openai.com/gpts/editor.
2. Follow the standard creation flow to set up your GPT.
3. Navigate to the "Configure" tab. Here, you can manually fill in fields such as name, description, and instructions, or use the smart creator for assistance.
4. Under the "Actions" section, click on "Create new action".
5. Choose an authentication method. The Retrieval Plugin supports None, API key (Basic or Bearer) and OAuth. For more information on these methods, refer to the Authentication Methods Section.
6. Import the OpenAPI schema. You can either:
7. Import directly from the OpenAPI schema hosted in your app at https://your-app-url.com/.well-known/openapi.yaml.
8. Copy and paste the contents of this file into the Schema input area if you only want to expose the query endpoint to the GPT. Remember to c