Easily configure and deploy a fully self-hosted chatbot web service based on open source Large Language Models (LLMs), such as Llama 2, without the need for knowledge in machine learning or programmation.

Free and Open Source chatbot web service with UI and API. Fully self-hosted, not tied to any service, and offline capable. Forget about API keys! Models and embeddings can be pre-downloaded, and the training and inference processes can run off-line if necessary. Easy to setup, no need to program, just configure the service with a YAML file, and start it with 1 command. No need for GPU, this will work even on your laptop CPU! That said, running on CPUs can be quite slow (up to 1min to answer a documents-base question on recent laptops), so we are working on making a better use of GPU when available.

LangChain models supported:

• Llama 2 GGML
  • 7B parameters
  • 13B parameters
  • 70B parameters
• Llama 2 GPTQ
  • 7B parameters
  • 13B parameters
  • 70B parameters