LlamaGPT

A self-hosted, offline, ChatGPT-like chatbot, powered by Llama 2. 100% private, with no data leaving your device.



<a href="https://umbrel.com"><strong>umbrel.com »</strong></a>






<a href="https://twitter.com/umbrel">
  <img src="https://img.shields.io/twitter/follow/umbrel?style=social" />
</a>
<a href="https://t.me/getumbrel">
  <img src="https://img.shields.io/badge/community-chat-%235351FB">
</a>
<a href="https://reddit.com/r/getumbrel">
  <img src="https://img.shields.io/reddit/subreddit-subscribers/getumbrel?style=social">
</a>
<a href="https://community.umbrel.com">
  <img src="https://img.shields.io/badge/community-forum-%235351FB">
</a>

Demo

https://github.com/getumbrel/llama-gpt/assets/10330103/71521963-6df2-4ffb-8fe1-f079e80d6a8b

How to install

Install LlamaGPT on your umbrelOS home server

Running LlamaGPT on an umbrelOS home server is one click. Simply install it from the Umbrel App Store.

Install LlamaGPT anywhere else

You can run LlamaGPT on any x86 or arm64 system. Make sure you have Docker installed.

Then, clone this repo and cd into it:

git clone https://github.com/getumbrel/llama-gpt.git
cd llama-gpt

You can now run LlamaGPT with any of the following models depending upon your hardware:

You can access LlamaGPT at http://localhost:3000.

To stop LlamaGPT, run:

docker compose down

Acknowledgements

A massive thank you to the following developers and teams for making LlamaGPT possible:

• Mckay Wrigley for building Chatbot UI.
• Georgi Gerganov for implementing llama.cpp.
• Andrei for building the Python bindings for llama.cpp.
• NousResearch for fine-tuning the Llama 2 7B and 13B models.
• Tom Jobbins for quantizing the Llama 2 models.
• Meta for releasing Llama 2 under a permissive license.

umbrel.com