LocalGPT

What is LocalGPT?

LocalGPT is an open-source, on-premises document intelligence platform that lets users ask questions, summarize files, and uncover information without sending document data away from their machine. It combines local model inference through Ollama with a retrieval system, web interface, session management, and APIs for building other RAG applications.

Its hybrid search combines semantic similarity, keyword matching, and late chunking. A smart router chooses between retrieval-augmented generation and a direct model answer for each query, while query decomposition, contextual enrichment, pruning, reranking, caching, and an independent verification pass support more focused answers.

How to Use LocalGPT

1. Install the required Python, Node.js, and Ollama components, or use the provided Docker setup.
2. Download the local inference models and embedding models needed by the deployment.
3. Start the managed services and open the local web interface.
4. Create an index and add the documents to be searched.
5. Start a chat session, ask questions in natural language, and inspect the returned document references.

Core Features

• On-premises processing: Keeps documents and model inference on the user's infrastructure.
• Hybrid retrieval: Combines semantic similarity, keyword matching, and late chunking.
• Smart routing: Chooses between RAG and direct model answering for each request.
• Answer verification: Runs an independent verification step before returning an answer.
• Session-aware chat: Maintains conversation context and organizes chats by topic.
• Developer APIs: Provides REST endpoints, processing progress, configuration, and an extension architecture.

Use Cases

• Private document Q&A: Teams can question internal files while keeping data on premises.
• Document summarization: Users can extract concise understanding from local document collections.
• RAG application development: Developers can use the APIs as a foundation for private retrieval applications.
• Controlled model experimentation: Teams can change local inference, embedding, and reranking models.

Pricing

LocalGPT is distributed under the MIT license and can be self-hosted. Users are responsible for the hardware and operation of their deployment.

Frequently Asked Questions

Does LocalGPT send documents to an external service?

The project states that document data remains on the user's machine.

Which model runtime does it use?

The documented setup uses Ollama for local model inference and supports open-source models and embeddings.

Can developers use LocalGPT without the web interface?

Yes. It provides REST APIs and can start the backend and RAG services without the frontend.