PrivateGPT

What is PrivateGPT?

PrivateGPT is an open-source API layer for turning locally hosted models into production AI applications. It provides higher-level application building blocks around a local inference server so developers can build private AI workflows without sending data to a cloud model API or recreating ingestion, retrieval, orchestration, and tool infrastructure.

The API follows the Claude messages pattern and also works with tools that expect a local OpenAI-compatible provider. PrivateGPT can sit in front of model servers such as Ollama, llama.cpp, vLLM, and LocalAI. A built-in workbench interface is available for testing, demonstrations, internal pilots, and inspecting API requests, while the API remains the primary product surface.

How to Use PrivateGPT

1. Run a supported local model server in the target environment.
2. Install and configure PrivateGPT as the application API layer.
3. Connect a developer tool, internal application, or automation platform to the local endpoint.
4. Ingest documents, artifacts, databases, CSV files, or other approved context.
5. Build message, retrieval, tool, MCP, or agent workflows against the API.
6. Use the workbench to test behavior and inspect requests before integrating the workflow into production.

Core Features

• Messages API: Supports standard messages, streaming, asynchronous processing, and token counting.
• File and artifact ingestion: Processes documents and artifacts for private AI workflows.
• Retrieval with citations: Builds source-grounded RAG answers with citation support.
• Structured data access: Works with databases, CSV files, and tabular data.
• Embeddings and orchestration: Coordinates retrieval, embeddings, tools, and local model calls.
• Custom tools and MCP: Connects local MCP services or approved online providers.

Use Cases

• Private document assistants: Answer questions from internal documents without sending them to an external model API.
• Local developer tools: Provide a private backend for coding assistants and local AI clients.
• Internal RAG applications: Combine document ingestion, retrieval, citations, and local inference behind one API.
• Structured-data workflows: Work with databases and CSV data in a private environment.
• Automation systems: Connect local AI capabilities to n8n or internal workflow platforms.

Frequently Asked Questions

Does PrivateGPT require a cloud model API?

No. It is designed to run with local model servers without depending on cloud APIs.

Which local inference servers are listed?

The website names Ollama, llama.cpp, vLLM, LocalAI, and other OpenAI-compatible servers.

What is the difference between PrivateGPT and Zylon?

PrivateGPT is the open-source local application API. Zylon is the broader enterprise infrastructure for deployment, governance, operations, identity, auditability, and support.