PDFToChat

Chat with your PDFs in seconds. Powered by Together AI and Chroma.

Tech Stack · Deploy Your Own · Common Errors · Credits · Future Tasks

Tech Stack

• Next.js App Router for the framework
• Mixtral through Together AI inference for the LLM
• Chroma Cloud for vector search (hybrid dense + sparse via Qwen & SPLADE)
• LangChain.js for the RAG code
• Bytescale for the PDF storage
• Vercel for hosting and for the postgres DB
• Clerk for user authentication
• Tailwind CSS for styling

Deploy Your Own

You can deploy this template to Vercel or any other host. Note that you'll need to:

• Set up Together.ai for the LLM
• Set up Chroma Cloud for vector search
• Set up Bytescale for PDF storage
• Set up Clerk for auth
• Set up Vercel for hosting + Postgres (or use Neon)
• (Optional) Set up LangSmith for tracing

See .env.example for a list of all the required environment variables.

You will also need to prepare your database schema by running npx prisma db push.

Chroma Cloud

This project uses Chroma Cloud for hybrid vector search. Embeddings are generated automatically by Chroma Cloud using Qwen (dense) and SPLADE (sparse), combined via Reciprocal Rank Fusion (RRF) at query time. No local embedding model is needed.

1. Sign up at trychroma.com and create a database.
2. Copy your credentials and set the following environment variables:

NEXT_PUBLIC_VECTORSTORE=chroma

CHROMA_API_KEY=       # Your Chroma Cloud API key
CHROMA_TENANT=        # Your tenant ID
CHROMA_DATABASE=      # Your database name

Collections are created automatically per document — no manual index setup is required.

Common errors

• Check that you've created an .env file that contains your valid (and working) API keys.
• Check that NEXT_PUBLIC_VECTORSTORE is set to chroma and your Chroma credentials are correct.
• Check that you've run npx prisma db push to create the Document table in Postgres.
• Check that you've added a credit card on Together AI if you're hitting rate limiting issues due to the free tier.

Credits

• Youssef for the design of the app
• Mayo for the original RAG repo and inspiration
• Jacob for the LangChain help
• Together AI, Bytescale, Chroma, and Clerk for sponsoring

Future tasks

These are some future tasks that I have planned. Contributions are welcome!

• [ ] Add a trash icon for folks to delete PDFs from the dashboard and implement delete functionality
• [ ] Try different embedding models like UAE-large-v1 to see if it improves accuracy
• [ ] Explore best practices for auto scrolling based on other chat apps like chatGPT
• [ ] Do some prompt engineering for Mixtral to make replies as good as possible
• [ ] Protect API routes by making sure users are signed in before executing chats
• [ ] Run an initial benchmark on how accurate chunking / retrieval are
• [ ] Research best practices for chunking and retrieval and play around with them – ideally run benchmarks
• [ ] Try out Langsmith for more observability into how the RAG app runs
• [ ] Add demo video to the homepage to demonstrate functionality more easily
• [ ] Upgrade to Next.js 14 and fix any issues with that
• [ ] Implement sources like perplexity to be clickable with more info
• [ ] Add analytics to track the number of chats & errors
• [ ] Make some changes to the default tailwind prose to decrease padding
• [ ] Add an initial message with sample questions or just add them as bubbles on the page
• [ ] Add an option to get answers as markdown or in regular paragraphs
• [ ] Implement something like SWR to automatically revalidate data
• [ ] Save chats for each user to get back to later in the postgres DB
• [ ] Bring up a message to direct folks to compress PDFs if they're beyond 10MB
• [ ] Use a self-designed custom uploader
• [ ] Use a session tracking tool to better understand how folks are using the site
• [ ] Add better error handling overall with appropriate toasts when actions fail
• [ ] Add support for images in PDFs with something like Nougat