MV
ModelValve INVITE-ONLY CONTROL PLANE
LIVE GATEWAY DEMO: 7,289,926 LLM CALLS · $35,652.71 SPEND
Multi-Provider AI Inference Control Plane & Waste Firewall

Everyone else shows you what you spent.
We show you which dollars were defects—and block the next one.

Passive dashboards log your bankruptcy; bloated Python proxies (LiteLLM #30244) leak ghost spend on client disconnects and choke Postgres pools. Below is one operator's live production gateway telemetry across 11 LLM providers, search/media tool lanes, and coding agents—exposing silent provider model swaps, negative-ROI cache thrashing, and reasoning starvation in real time.

Snapshot live · auto-polls every 60s
ZERO-EGRESS LOCAL CLI AUDIT (225-LINE STDLIB NODE) Inspect Source ↗
curl -fsSL https://modelvalve.com/cli.mjs -o mv-audit.mjs && node mv-audit.mjs
SHA-256: 8b6c3a92237ece4c…03654296 Flags: --json · --no-color · --print-source
  • Runtime Network Lock: Overrides globalThis.fetch to throw on startup—0 bytes can leave your host.
  • Deduplicated Log Audit: Deduplicates ~/.claude/projects JSONL turns by message.id + requestId to calculate true Anthropic Net Cache P&L and max_tokens cutoffs.
  • Precision Prefix Linter: Pinpoints prompt-interpolated new Date() / UUID cache busters while ignoring latency timers.
Want to test a payload in-browser with 0 network requests? Open Client-Side Auditor →
Identified Defect & Prefix Waste
$4,421.51
12.4% of $35,652.71 total spend ($2,770.51 in 850 findings + $1,651 uncached prefix gap)
Settled LLM Inference Calls
7,289,926
+ 3.46M /auto router dispatches ($0) · + 786k search/media tool calls
Global Prefix Cache Hit Rate
80.3%
54.12B cache read / 67.42B prompt tok (~$46.8k per-lane savings)
Transparent Lane Failovers (30d)
73,735
2.1% of /auto dispatches recovered upstream 503/429s + 31,697 remaps
Silent Model Swaps Caught
39
Non-trivial servedAs ≠ modelId across 1,024 routes · 2,482 0ms preflight blocks
Last 30 Days: Daily Settled Spend & Prefix Cache Efficiency
Bar height = daily settled spend ($); bar color = daily prefix cache hit rate (■ ≥80% · ■ 60–79% · ■ <60%). Click or hover any bar to inspect.
Loading 30-day timeline...
$1,500 $750 $0
Settled LLM Inference Providers ($34,742.04 · 7.29M Calls)
Click any provider row to filter the Top Production Model Routes table. Reconciles live wallet headroom and prefix-cache hit rates.
LLM Provider Settled Calls Settled Spend Prompt (M) Cache Read (M) Cache Hit % Route Status & Operational Note
Gateway Auto-Router & Non-LLM Agent Tool Lanes (Separated from LLM Token Math)
Lane Role Dispatches / Calls Settled Spend Operational Note
Top 16 Production Model Routes by Spend
Internal gateway lane IDs (e.g. claude-fable-5-1, gpt-6-astra) are shown alongside their canonical frontier lane role and uncached prefix gap.
Provider / Gateway Lane ID Calls Spend Prompt (M) Uncached (M) Cache Hit Prefix / Waste Diagnosis
Quantified Defect Classes Across 850 Audited Findings ($2,770.51 Direct + $1,651 Uncached Prefix Gap)
Instead of drowning operators in raw log lines, ModelValve rolls every anomaly into 6 financial defect classes with copyable gateway remediation rules.
Defect Class (kind) Open Findings Resolved Findings Open Impact ($) Total Quantified Waste ($) Automated Control Plane Action
Provider Honesty Ledger: Silent Model Substitutions (servedAs ≠ modelId)
Live canary probes across 1,024 routes caught 39 non-trivial cases where the upstream API returned a different response.model than requested—including cross-vendor replacements.
39 Live Mismatches
Provider Requested modelId Actually Returned (servedAs) Listed Price ($/M In·Out) Probe ms
Prefix Cache Physics & Anthropic Write-Penalty Break-Even
Why claude-code achieves 95.7% cache hit rate while direct anthropic callers hit 38.8% on the exact same models.
EXPLICIT ANTHROPIC NET CACHE P&L EQUATION
Net Cache P&L = (0.90 × C_base × T_read) − (0.25 × C_base × T_write_5m) − (1.00 × C_base × T_write_1h)
Anthropic charges a +25% surcharge on 5-minute cache writes and +100% surcharge on 1-hour writes. Furthermore, look at anthropic/claude-fable-5-1 in Tab 1: while its cached writes are read 26×, 208.0M of its 344.7M prompt tokens were sent uncached because callers omitted cache_control or mutated early system tokens.
Prefix Failure Mode Root Cause in Request Payload Financial Blast Radius
Prefix Hash Bust Injecting new Date().toISOString() or request UUID in system prompt Invalidates 100% of downstream tools/docs; breaks OpenAI 256-tok shard routing
Tool Array Shuffle Non-deterministic dict/MCP tool ordering across agent turns Forces full +25% cache re-creation on every turn after the first mutated tool
TTL Expiry Thrash Background cron or batch worker polling at 6–12 min intervals (> 5m TTL) Pays 1.25× input price on every call with 0.0% cache reads
Cold-Start Stampede N parallel workers firing simultaneously before turn 1 finishes writing cache All N workers pay the +25% write penalty in parallel for the identical prefix
Proxy Header Strip Routing anthropic/claude-sonnet-5 through an OpenAI-compat proxy that drops cache_control 0.0% cache hit across 176.8M tokens ($798.46 billed at 100% full input price)
Top Open Waste & Reliability Findings (Live Production Database)
Sorted by quantified dollar impact (impactUsd). Workload names sanitized for public display.
Reasoning Starvation Cliffs & Enforced max_tokens Floors
When hidden reasoning chains consume 100% of max_tokens, the caller receives 0 visible content (or truncated JSON) while paying full output rates. ModelValve enforces safe floors inline.
Model Route Starvation Cliff Gateway Floor Starved Calls Wasted Spend Inline Containment Rule
Structured Error & Advisory Disclosure (X-Advisory / X-Error-Class)
Unlike opaque proxies that return generic 500s or silently drop parameters, ModelValve classifies every interception so autonomous agents know whether to halt or fail over.
HTTP X-Error-Class Retryable? Agent Action Mechanism
Architectural Containment: Why High-Concurrency Teams Outgrow LiteLLM
How a zero-bloat control plane eliminates proxy-induced outages and ghost spend.
Failure Vector Standard Proxy / LiteLLM Behavior ModelValve Inline Firewall Behavior
Client Disconnect / 504 Timeout Upstream LLM continues generating and billing thousands of reasoning tokens into a dead socket (Issue #30244). Fires AbortController.abort() on the upstream provider stream within 1ms of client socket close.
Invalid Param / Delisted Model Forwards request over the network; waits 400–2,000ms for upstream 400/404; triggers blind agent retries. 2,482 preflight rejections blocked locally in 0ms for $0 + 31,697 automatic model remaps.
Prepaid Wallet Exhaustion Returns generic 402/500 indistinguishable from your own project budget cap; halts production pipelines. Distinguishes 402 budget-exceeded (halt) from 402 upstream-credit (auto-bench route & fail over).
Leaked XML/Text Tool Calls Open-weight provider returns raw <tool_call> text with finish_reason="stop"; agent crashes. 338 tool-call leaks intercepted and repaired into canonical tool_calls[] arrays inline.
Evidence-Based Model Arbitrage Matrix (Deterministic Schema & Tool-Call Replay)
Evaluate your actual production prompt shapes against open-weight and flash lanes using deterministic JSON-schema validation and AST tool-call parity—zero LLM-as-judge overhead. Savings calculated per 100M total tokens (65M input / 35M output mix).
Workload Archetype Incumbent Frontier Route Incumbent Rate Verified Arbitrage Candidate Candidate Rate Schema Parity Latency Delta Savings / 100M Tok (65/35 Mix)
100% Client-Side Prompt, Session JSONL & Cache-Bust Auditor
Paste an OpenAI/Anthropic request JSON, multi-turn Claude Code .jsonl lines, or prompt template. Runs 100% in your browser's JavaScript VM—0 bytes are sent over the network.
✓ 0 Network Requests · Verify in DevTools
Load Preset:
Execution mode: Pure client-side JS (0 network bytes transmitted)
Instant Forensics & Net Cache P&L Output
Pinpoints prefix-busting lines, deduplicates multi-turn JSONL usage, and computes Anthropic write-penalty P&L.
Select a preset or paste a payload on the left and click Run Client-Side Waste Audit.