Dynamically distribute inference queries across Claude Sonnet 5.5, Opus 5.5, Haiku 4.5, and specialized model clusters. Preserve warm prompt caches and slash inference expenses by up to 84%.
Test how VecRoute classifies prompt intent, context length, and tool definitions to select the optimal model tier.
Single API endpoint. Global sub-3ms evaluation. Automatic fallback & cache-affinity dispatch.
Eliminate hard-coded model selection. Let semantic classifiers and cost policies govern execution.
Native parsing of JSON-RPC MCP tool calls. Understands tool cardinality and dispatches agent loops to models with highest tool execution fidelity.
Maintains session pinning for multi-turn conversations and long system instructions, guaranteeing maximum KV-cache hits and saving 90% on input tokens.
Speculatively executes lightweight models first. If schema validation fails or confidence score drops below threshold, escalates seamlessly to Claude Opus 5.5.
Live dashboard measuring cache hit ratios, routing decisions, latency variations per provider, and total dollar savings per customer tenant.
Queries are evaluated entirely in volatile memory. Prompts, completions, and thinking traces are never persisted to disk or used for training.
Zero SDK migration required. Update your base_url to api.vecroute.xyz/v1 and start routing across providers instantly.
Use our lightweight SDKs or point any existing OpenAI/Anthropic client to the VecRoute gateway.
from vecroute import AsyncVecRouter
# Initialize router client
router = AsyncVecRouter(api_key="vr_sk_live_...")
# Route query dynamically with cache affinity
response = await router.route(
messages=[{
"role": "user",
"content": "Refactor auth session verification to use Ed25519 signatures."
}],
strategy="cost_quality_optimal",
cache_affinity=True,
mcp_servers=["github", "postgres"]
)
print(response.model_used) # → "claude-sonnet-5-5"
print(response.cache_discount) # → "90% prompt cache applied"
print(response.cost_saved) # → "82.4%"
import { VecRouteClient } from "@vecroute/sdk";
const client = new VecRouteClient({ apiKey: process.env.VECROUTE_API_KEY });
const completion = await client.chat.completions.create({
model: "vecroute/auto",
messages: [{ role: "user", content: "Analyze quarterly metrics CSV" }],
routing: {
preferred_providers: ["anthropic", "vertex"],
fallback_tier: "claude-opus-5-5"
}
});
console.log(completion.routed_model); // → claude-haiku-4-5
curl https://api.vecroute.xyz/v1/chat/completions \
-H "Authorization: Bearer vr_sk_live_..." \
-H "Content-Type: application/json" \
-H "x-vecroute-strategy: cost-optimized" \
-d '{
"model": "vecroute/auto",
"messages": [
{ "role": "user", "content": "Extract SKU numbers from shipping manifest" }
],
"routing": {
"strategy": "latency_first",
"target_models": ["claude-haiku-4-5", "claude-sonnet-5-5"]
}
}'
// claude_desktop_config.json / MCP Gateway Config
{
"mcpServers": {
"vecroute": {
"command": "npx",
"args": ["-y", "@vecroute/mcp-router"],
"env": {
"VECROUTE_API_KEY": "vr_sk_live_...",
"CACHE_AFFINITY": "true"
}
}
}
}
Pay only for routing decisions and cache optimization. Connect your own Anthropic or cloud API keys.
Deploy VecRoute into your agent loop in minutes. Cut token expenses by up to 84% starting today.