AI Orchestration
The AI orchestration system coordinates AI workloads across multiple Cloud OS instances in your fleet, automatically routing requests to the best available instance based on capacity and health.
AI orchestration requires the ai_orchestration license feature (Enterprise plan).
How It Works
When you submit an orchestrated AI task, the system:
- Evaluates capacity — checks GPU and CPU utilization across all instances with AI capabilities
- Selects instances — routes the task to the least-loaded instance (or distributes across multiple instances for team tasks)
- Handles failover — if the selected instance becomes unavailable, the task is re-routed to a healthy instance
- Aggregates results — for multi-instance tasks, collects and merges outputs from all participating instances
Submitting an Orchestrated Task
- Open Agents & chat in the sidebar (the Agents tab)
- Click Dispatch task
- Describe the task and pick the routing: Least loaded node (default) or Round robin
- Click Dispatch — the dialog shows the task status and each node’s response as it finishes
Cluster AI Capacity
The Dispatch task dialog shows the AI capacity of your fleet at the top:
- Online nodes out of the total
- Free slots for new AI tasks
- Average CPU and memory load across the online nodes
Orchestration API
| Endpoint | Method | Description |
|---|---|---|
/api/ai/orchestrate | POST | Submit an orchestrated AI task |
/api/ai/orchestrate/{id} | GET | Get task status with per-instance results |
/api/ai/capacity | GET | Cluster AI capacity overview |
Fleet Orchestrator Chat
For the chat that answers fleet questions with live lookups (and proposes bulk actions for you to confirm), see Fleet Orchestrator.