AI Hub

The AI Hub lets you run local language models, connect to cloud LLM providers, build AI agents with system tool access, set up a RAG knowledge base, and deploy AI-powered channels to Telegram, Slack, and email. All model management, provider configuration, and agent building are handled from a single page.

Local LLM Management

Cloud OS supports two local LLM runtimes: Ollama and LocalAI.

Ollama

Cloud OS manages Ollama as a Docker container. From the AI Hub page:

1. Open AI Hub from the dock’s AI group (available in every mode) or with Ctrl/⌘+K
2. Click Enable Ollama
3. Cloud OS pulls and starts the Ollama Docker container
4. The model library becomes available

The Models tab shows all downloaded models and a library of popular models available to pull:

Popular models include Llama, Mistral, CodeLlama, Phi, Gemma, and others.

LocalAI

LocalAI provides an OpenAI-compatible API for running models locally. Enable it from AI Hub > Runtimes alongside or as an alternative to Ollama.

GPU Detection

Cloud OS detects available GPUs and displays VRAM information on the AI Hub page. Ollama and LocalAI automatically use GPU acceleration when an NVIDIA GPU with CUDA support is detected.

LLM Gateway

The LLM Gateway routes AI requests to local or cloud models based on configurable rules, providing a unified API for all your AI interactions.

Supported Providers

Adding a Provider

1. Go to AI Hub > Providers
2. Click Add Provider
3. Select the provider type
4. Enter your API key (and base URL for custom endpoints)
5. Click Test Connection — the result appears inline on the provider card without a page reload
6. Click Save

Each provider is shown as a cosmic card with a live status badge. The card reflects connectivity in real time.

Routing Rules

Configure routing rules in AI Hub > Routing:

• Default model — which model handles requests when no specific rule matches
• Fallback model — used when the primary model is unavailable
• Task-specific routing — route certain tasks (code generation, summarization, etc.) to specific models

Chat Interface

Chat with Molly in the floating Molly window (dock button, Ctrl/⌘+M, or Ask Molly in the palette) or in the full-screen Molly workspace (the window’s expand button, or Ctrl/⌘+K → Molly workspace). Features include:

• Streaming responses — real-time token-by-token output via Server-Sent Events (SSE)
• Model and agent selector; every answer shows which model produced it
• Tool call visualization showing what the agent is inspecting, with an animated shimmer while a tool is running and a clean status transition when it completes
• Conversation history (the Conversations button in the window, a sidebar in the workspace) and Memories. Conversations are private: you only see your own, and nobody else — not even an admin — can open, continue or delete them
• Tool calls that change the system ask you to approve or deny them
• Full markdown rendering — GFM tables, fenced code blocks with a language label and one-click copy button, inline formatting, and lists render correctly (since 0.9.1)
• AI Response Integrity — every figure in a Molly answer is verified against tool results; verified answers show a Sources footer, and unverifiable figures are flagged with ⚠️ (since 0.9.1; see AI Integrity)

How Molly works through a request

Molly answers in rounds: she asks the model, runs the tools it requests (check an app, read metrics, search notes…), hands the results back, and repeats until the model can answer.

Right-sized models (model tiers)

Before answering, Molly rates your message as simple, standard or complex and picks the matching model:

Configure the models in AI Hub → Providers → Molly Agent Loop, per provider (the provider your routing rules pick). An empty field keeps the routed model, so nothing changes until you fill it in. Setting a Fast model switches the rating on for that provider — the rating itself runs on the fast model and never adds more than two seconds; if it fails or is slow, Molly treats the request as standard. Follow-up messages are never rated simple, and “are you sure”, “this is wrong”, “ты уверен”, “перепроверь” always count as complex. The model badge under an answer shows which model answered.

Rounds and finishing up

• Max rounds per reply (default 10, 2–40) is set in the same card.
• Molly plans multi-part requests up front and checks around 40% of the way whether she already has enough to answer.
• In the last three rounds only tools that save the result (create_note, memory_store, server_memory_store) still run; other lookups are skipped and Molly writes her answer from what she has.
• If she still hits the limit, you get a partial answer listing what she found so far instead of an error — ask her to continue from there.
• Stopping the reply stops the work immediately.

Daily Spend Caps

On top of the monthly budget, admins can cap AI spend per UTC day in AI Hub → Cost (pencil on Daily Spend Caps):

• Daily cap, whole server — total AI spend of the node per day.
• Daily cap per user — what one person’s Molly and AI Hub chats may spend per day.

Molly checks the caps before each reply and every couple of rounds during long answers; AI Hub chats check them before every model call. Once a cap is reached the reply stops with a message saying when it resets (midnight UTC). 0 turns a cap off. The Daily Spend Caps card shows today’s server and personal spend against each cap. Spend is calculated from the provider’s reported tokens, including prompt-cache reads and writes (billed at the provider’s discounted / surcharged rates); local models are free.

RAG Knowledge Base

The RAG (Retrieval-Augmented Generation) system lets you upload documents and query them through your AI models using semantic search.

How It Works

1. Upload documents (PDF, Markdown, text, or other supported formats)
2. Cloud OS chunks and embeds the documents using a local embedding model
3. Embeddings are stored in a Qdrant vector database (managed as a Docker container)
4. When you ask a question, relevant document chunks are retrieved via semantic search and injected into the LLM context

Setting Up RAG

1. Navigate to AI Hub > Knowledge Base
2. Click Enable Knowledge Base (this starts the Qdrant container)
3. Upload documents by dragging files or clicking Upload
4. Documents are automatically processed, chunked, and embedded

Querying the Knowledge Base

In the chat interface, toggle Use Knowledge Base to enable RAG. The AI will search your documents for relevant context before generating a response.

Agent Builder

The agent builder lets you create custom AI agents with access to system tools.

Creating an Agent

1. Go to AI Hub > Agents
2. Click New Agent
3. Configure the agent:
   • Name and description
   • System prompt — instructions for the agent behavior
   • Model — which LLM to use
   • Tools — select which system tools the agent can call

Available Tools

Agents can be granted access to system tools for interacting with your server:

Tool Execution

When an agent decides to call a tool, the request is executed server-side by the Go backend. Read-only operations execute immediately. Write operations require confirmation.

Tool Search (large tool catalogues)

Molly and AI agents can have a lot of tools: built-in server and personal tools plus every skill you install. Sending dozens of tool descriptions to the model on every message costs tokens and makes it worse at choosing the right one. Once a catalogue has more than 20 tools, Quazzar switches to deferred loading:

• A small hot set (at most 10 tools) is always loaded: a few core tools (system health, app list, alerts, notes and memory), plus the tools you use most often. The hot set learns from your usage and favours recent use.
• The tools that best match your message are loaded up front too. For example, “restart Nextcloud” loads restart_app straight away.
• Everything else stays available through a built-in tool_search tool. When the model needs something that isn’t loaded, it searches the catalogue (“create a backup”, “invite a user”). The matching tools become callable for the rest of that reply. In the chat you’ll see this as a tool_search step.

Search is keyword-based (BM25 ranking over tool names, descriptions and parameter names), runs entirely on your node and works the same with every provider: Ollama, OpenAI-compatible, Anthropic and Google. With 20 tools or fewer, nothing changes and every tool is sent as before.

Tool schema checks. Before tools are sent to a provider, and before the node serves its own MCP tools/list, every tool’s input schema is checked and repaired if needed. For example, a missing schema becomes an empty object, and an array without items gets one. Duplicate or nameless tools are dropped. A single broken skill or tool therefore can’t make the provider (or your IDE’s MCP client) reject the whole tool list. Each repair is logged once as a warning in the node log, so you can fix the source.

Fine-Tuning

Cloud OS supports fine-tuning local models with your own data:

1. Navigate to AI Hub > Fine-Tuning
2. Select a base model
3. Upload training data (JSONL format with prompt/completion pairs)
4. Configure training parameters (epochs, learning rate, batch size)
5. Start the fine-tuning job

Progress is displayed in real time. The resulting model is saved locally and becomes available in the model selector.

AI Channels

Deploy your AI agents to external communication platforms:

Telegram Bot

1. Create a bot via @BotFather  and copy the token
2. Go to AI Hub > Channels
3. Click Add Channel and select Telegram
4. Enter the bot token and select which agent handles messages
5. Users can now chat with your agent via Telegram

Slack Bot

1. Create a Slack app and configure a bot user
2. Enter the bot token and signing secret in Cloud OS
3. Select the agent and channels to monitor

Email

Configure an email inbox for the agent to monitor and respond to incoming messages via IMAP/SMTP.

Restarts and duplicate messages

Channels are built to survive a restart of the Cloud OS service (an update, a reboot, a crash) without leaving users hanging:

• Interrupted answers are announced. While the AI is answering a channel message, Cloud OS remembers which chat it came from. If the service restarts before the answer is sent, the user gets a short reply in the same place — a Telegram reply to their message, the same Slack thread, the same Discord channel, WhatsApp chat or e-mail thread — saying “I was restarted while answering your last message, so the answer was lost. Please resend your question.” If the channel is still reconnecting after the restart, Cloud OS retries for a few minutes; notices for requests older than 24 hours are dropped.
• Messages are answered once. Platforms sometimes deliver the same message twice (Telegram replays unconfirmed updates after a restart, Slack retries slow events). Cloud OS remembers the message IDs it has already handled for 72 hours and silently ignores a repeat, so the AI never answers the same message twice.

Both behaviours are on by default. Administrators can switch them off with QUAZZAR_AI_CHANNEL_RECOVERY=off and QUAZZAR_AI_CHANNEL_DEDUP=off — see AI Resilience.

AI Health Check

Cloud OS runs a self-check of its AI stack every 4 hours so a silent failure — a revoked provider API key, an MCP server that stops listing tools, a broken skill — is caught by the node instead of by a user. Each run performs four checks:

Administrators see the result under Molly > Settings > AI health check: the overall status (Healthy, Degraded, Failing), each check with its detail and duration, when the next run is due, and a Run now button. When the node is connected to a Control Panel, a failing run is reported there as an ai_canary alert (and a recovery as an info alert), and the node’s metrics include ai.canary.ok and ai.canary.failed_checks.

The interval, the provider check mode and an off-switch are set with environment variables — see AI Resilience.

MCP Server

Cloud OS includes a built-in Model Context Protocol (MCP) server that exposes your server tools and data to compatible AI clients. External AI tools and IDEs that support MCP can connect to your Cloud OS instance and interact with your server through the standardized protocol.

AI Memory

Cloud OS includes a persistent memory system for AI agents via the Model Context Protocol (MCP). Agents can store, retrieve, and search through memories across sessions, enabling context-aware responses that improve over time.

How Memory Works

The memory store is shared by the whole node (it also backs the node’s MCP memory.* tools): every user can read and search it, only admins can store or delete entries. The same goes for the patterns an agent learns from feedback.

Memory entries are key-value pairs organized by namespace. Each agent or workflow can have its own namespace, preventing cross-contamination of context.

Using Memory in Agents

When an agent is configured with memory access, it automatically:

1. Searches for relevant past interactions before generating a response
2. Stores important context from the current conversation
3. Retrieves and injects relevant memories as additional context

Memory API

MCP Memory Tools

When connected via MCP, AI clients can use these tools:

• memory.store — Save a key-value pair with optional namespace and metadata
• memory.retrieve — Get a specific entry by key, or search by query
• memory.list — List all entries in a namespace
• memory.delete — Remove a memory entry
• memory.search — Full-text search across all entries

Self-Learning Agents

Cloud OS agents can learn from interactions, building a knowledge base that improves responses over time.

Context Memory (Phase 1)

Every agent interaction is recorded. Before generating a new response, the agent retrieves similar past interactions and injects them as context. This means agents remember what worked well and avoid repeating mistakes.

Knowledge Distillation (Phase 2)

The system periodically analyzes high-rated interactions (those with positive user feedback) and extracts patterns into a knowledge base. When a new query matches a known pattern, the agent uses the learned response template as guidance.

Feedback Loop (Phase 3)

After each AI response, users can provide feedback via thumbs up/down buttons. This feedback drives the learning cycle:

• Positive feedback (score >= 4) reinforces the pattern and increases confidence
• Negative feedback flags the interaction for review and may reduce pattern confidence

Knowledge Management API

Chat Rate Limits

Free tier users have a daily limit of 50 AI chat messages. Paid plans (with the ai_chat_unlimited license feature) have unlimited messages.

The remaining message count is displayed in the chat interface. When the limit is reached, the chat shows a message suggesting an upgrade.

Check your remaining quota via the API:

GET /api/ai/chat/ratelimit

Returns {remaining: number, limit: number, resets_at: string}.

Emergency switches

Each part of Molly’s agent loop can be turned off with an environment variable on the node (set to off, then restart the service):

Troubleshooting

Ollama container fails to start

Check Docker logs for the Ollama container. Common issues include insufficient disk space for model storage or port conflicts on port 11434.

docker logs quazzar-ollama

Cloud provider returns authentication error

Verify your API key is correct and has not expired. Use the Test Connection button in the provider configuration to check connectivity.

Agent responses are slow

If using a local model, response speed depends on your hardware. Consider using a smaller model or switching to a cloud provider for complex queries. Check GPU utilization — if the GPU is not being used, verify that the NVIDIA container toolkit is installed.

The AI health check reports “Failing”

Open Molly > Settings > AI health check and read the detail of the failed check. A provider failure usually means an expired API key or an unreachable endpoint (use Test Connection on the provider); an MCP failure names the tool with the problem. Click Run now after fixing it to confirm.

Knowledge base search returns irrelevant results

Try re-uploading documents with smaller chunk sizes. Check that the embedding model is appropriate for your content type. You can also adjust the similarity threshold in Knowledge Base > Settings.