Jan

At a Glance:

Jan is an open-source AI product that lets users download and run local LLMs like Llama and Gemma while also connecting to cloud models from OpenAI, Anthropic, and others, with an OpenAI-compatible local API server and MCP integration for agentic capabilities.

Overview:

Jan is an open-source desktop application designed as a replacement for ChatGPT that combines local AI model execution with cloud model integration. Users can download and run open-source LLMs including Llama, Gemma, Qwen, and GPT-oss directly from HuggingFace on their own hardware. The application also supports connecting to remote AI providers such as OpenAI, Anthropic, Mistral, Groq, and MiniMax. Jan runs on macOS 13.6+, Windows 10+, and most Linux distributions, with GPU acceleration available and specific RAM recommendations for different model sizes.

Key Decision Points:

• Runs AI models locally: Downloads and executes LLMs directly on user hardware, which keeps processing local by default.

• Supports both local and cloud models: Users can choose between locally run open-source models and cloud API connections to multiple providers.

• OpenAI-compatible API server: Exposes a local server on port 1337 that other applications can use to interact with loaded models.

• Model Context Protocol support: Includes MCP integration for agentic capabilities beyond basic chat interactions.

• GPU acceleration across platforms: Supports NVIDIA, AMD, and Intel Arc GPUs on Windows, with GPU acceleration also available on macOS and Linux.

Core Features:

• Local AI model execution: Download and run LLMs including Llama, Gemma, Qwen, and GPT-oss from HuggingFace directly on your machine.

• Cloud model integration: Connect to remote AI providers including OpenAI (GPT models), Anthropic (Claude models), Mistral, Groq, and MiniMax.

• Custom assistants: Create specialized AI assistants configured for specific tasks or workflows.

• OpenAI-compatible local API: A local server at localhost:1337 that provides an API endpoint compatible with OpenAI's interface for use by other applications.

• Model Context Protocol (MCP): Integration with the Model Context Protocol to enable agentic capabilities.

• Cross-platform desktop application: Runs on macOS (13.6+), Windows (10+ with GPU support), and most Linux distributions with GPU acceleration.

Use Cases:

• Users who want to run AI models locally: Suitable for anyone who prefers running LLMs on their own hardware without sending data to external services.

• Developers integrating AI into other applications: The local OpenAI-compatible API server at port 1337 allows developers to point existing OpenAI client libraries at a locally running model.

• Users combining local and cloud AI: Those who want to use both locally downloaded open-source models and commercial cloud APIs within a single application.

Open-Source Alternative Value:

Jan positions itself as an open-source ChatGPT replacement that gives users the option to run AI models entirely on their own hardware. It supports local execution of HuggingFace models including Llama, Gemma, and Qwen, alongside cloud API connections to multiple commercial providers. The local OpenAI-compatible API server makes locally running models accessible to other tools through a standard interface, and MCP integration extends the application's capabilities toward agentic workflows. The application is built using Llama.cpp, Tauri, and other open-source components.