Every model
you need — one runtime.
Route between Claude, GPT-class, and self-hosted Llama with a single API. Built-in fallback, continuous evaluation, BYO keys, and cost + latency telemetry per request — without locking your codebase to any one vendor.
Routes traffic between every major model provider — your keys, our runtime
Model gallery
Pick the right brain for each step — and switch later without a rewrite
Click a card to swap the playground above. Same API, same code, different model.
One API. Every model.
Inference, streaming, and evals — without vendor lock-in
Same call shape across every provider. Built-in fallback when a model is slow or rate-limited. Continuous evaluation in CI so you only ship a model swap when the goldens still pass.
- Per-request cost, latency, and token telemetry — automatic
- BYO keys: Anthropic, OpenAI, Bedrock, Azure, Vertex, vLLM
- Eval suites block model promotion if regressions appear
- OpenTelemetry traces span model + tools + your app code
1# pip install vxcloud-ai2from vxcloud.ai import Client3 4client = Client() # picks up VX_API_KEY from env5 6# 1) Inference — same call shape across every provider.7resp = client.complete(8 model="claude-opus-4-7",9 fallback=["claude-sonnet-4-6", "gpt-reasoning"],10 messages=[{"role": "user", "content": "Refactor this function for testability."}],11 max_tokens=2048,12 metadata={"workspace": "studio", "session": "ide-42"},13)14print(resp.text)15print(resp.usage, resp.cost_usd, resp.latency_ms)16 17# 2) Streaming with cancellation.18with client.stream(model="claude-sonnet-4-6", messages=[...]) as s:19 for token in s:20 print(token, end="", flush=True)21 22# 3) Evals — score a model against your own golden set.23report = client.evals.run(24 model="claude-haiku-4-5",25 suite="support/intent-classification",26 parallelism=8,27)28print(report.summary()) # accuracy, p50/p95 latency, $/reqPick a model with a single env var
export VX_DEFAULT_MODEL=claude-sonnet-4-6 — flip every call site at once, with eval gates protecting prod.
Built for production AI
The runtime your AI features deserve
Unified inference API
Single SDK and REST surface across every provider. Swap models with a string change — no SDK migration tax.
Automatic fallback chain
Define a list. If the primary is slow, rate-limited, or errors, the runtime transparently retries on the next.
Continuous evals in CI
Run goldens on every prompt or model change. Block promotion if accuracy, latency, or cost regress.
BYO keys, BYO compute
Your Anthropic, OpenAI, Bedrock, Azure, Vertex, or self-hosted endpoint. Traffic never touches third-party.
Per-request telemetry
Cost, latency, tokens, fingerprint, and outcome — captured automatically and queryable from the dashboard.
Prompt + output filtering
PII scrubbing on the way in, output policies on the way out, with per-tenant overrides for regulated workloads.
A/B and shadow testing
Send a slice of traffic to a candidate model in shadow mode. Compare quality and cost before flipping.
Self-hosted inference
Run vLLM, TGI, or Triton in your own VPC and route to it from the same API. Air-gapped tenants supported.
Region-aware routing
Pin requests to EU, US, or APAC. Data residency for regulated workloads, lowest-latency for the rest.
Real workloads, real numbers
Production traffic — measured, not marketed
Aggregated across a 30-day window of real customer traffic on the unified runtime. Every model hits the same eval suite before reaching production.
Where this runtime earns its keep
Use cases teams are shipping today
Agent runtimes
Power the planner, retriever, and tool-calling steps in your agent stack with the right model per step — automatically.
Real-time copilots
Stream completions into IDEs, dashboards, and chat surfaces with sub-second TTFT on Sonnet or Haiku.
Document understanding
Multimodal models extract tables, line items, and signatures from PDFs, with citations back to the source page.
A/B model evaluation
Run shadow traffic on a candidate model for a week before flipping production — full quality and cost diff in one report.
Regulated workloads
Self-host inference in your own VPC. PII scrubbing, output policies, and audit trail meet HIPAA, PCI, and FedRAMP scopes.
Batch / nightly inference
Cheap-tier models on parallel batches: classification, summarization, embedding refresh — billed by the second.
One API. Every model. Zero lock-in.
Open the playground, drop your key, and route between Claude, GPT-class, and self-hosted Llama without rewriting a single function call.