The Modelith reference

Modelith exposes a single OpenAI-compatible chat completions endpoint. Streaming is supported (SSE, stream: true). All other surfaces (Command Center, Sandbox, Billing) are dashboard routes served by the same auth layer.

All authenticated requests must send an Authorization: Bearer mlth_… header. The bearer token is the full API key, including the mlth_ prefix. The key is rate-limited per key, revocable from the dashboard, and never returned in any API response after creation. Dashboard sessions additionally use a separate JWT cookie scoped to the dashboard domain.

A leaked key should be revoked immediately from Dashboard → API keys. Revocation is effective on the next request — the gateway checks key state on every call.

The governor is governed by two headers. They are the entire way to attach budget and loop-detection semantics to a request.

Responses are OpenAI-compatible. The Modelith-specific metadata is attached under x_modelith on the root object. This is the block your dashboard and your own logging will read.

{
  "id": "chatcmpl-...",
  "model": "anthropic/claude-sonnet-4",
  "choices": [{ "index": 0, "message": { "role": "assistant", "content": "..." }, "finish_reason": "stop" }],
  "usage": { "prompt_tokens": 312, "completion_tokens": 144, "total_tokens": 456 },
  "x_modelith": {
    "session_id": "run-7a3f",
    "step": 3,
    "routing_mode": "balanced",
    "complexity_score": 6.4,
    "initial_tier": 2,
    "final_tier": 2,
    "escalated": false,
    "provider": "openrouter",
    "estimated_cost_usd": 0.00213,
    "latency_ms": 287,
    "spent_usd": 0.0071,
    "budget_limit_usd": 0.05,
    "budget_remaining_pct": 85.8
  }
}

On a halt, the response is an HTTP error with a structured body. The same x_modelith keys appear so the caller can reconcile spend and state in one place.

1. Open. First request with a new session id and a budget header creates the session. The first tier is selected from complexity scoring and your allowed tiers.
2. Run. Each request increments the step counter, records the prompt fingerprint (UUIDs and numbers stripped), and adds estimated cost to the running total.
3. Halt. The governor returns 402 (budget) or 429 (loop or steps) the moment a guardrail trips. The response carries the halt reason and the spent/limit pair.
4. Close. A session can be closed explicitly by sending the close header, or it will TTL after 24 hours of inactivity. The trace is preserved in Postgres either way.

The hold is the estimated cost of the current request, computed from input length and the initial-tier price. If the actual response cost is lower, the difference is credited back to the session on settlement.

The loop detector is the second-most-common reason customers cite for choosing Modelith. A runaway agent that re-prompts itself with the same context will burn through a monthly budget in minutes; the loop halt turns that into a 200-millisecond structured 429.

The max-steps halt is the polite refusal. Where the loop halt says "you repeated yourself", the max-steps halt says "this is taking too long; ask for a human". Both are server-side, both are non-overridable by the client.

Each prompt is scored 1–10 by six independent analyzers. The scores are weighted, summed, and floored to a tier. A confidence score is computed as 1 − (stddev / 10). If the analyzers disagree, the final score is bumped by +1 — we'd rather over-pay than under-pay for a hard prompt.

The engine is frozen. Weights sum to 1.0. There is no machine learning, no embeddings, no semantic cache. The decision to keep it simple is a product decision — it means the routing is predictable, the bills are reproducible, and a compliance team can audit it line by line.

Three fixed tiers, mapped to specific upstream models on the platform. Tiers are a constraint for the governor, not a guarantee: the actual model dispatched is recorded in x_modelith.final_tier and x_modelith.model on every response.

Allowed tiers are configured per account at Settings → Routing. If a customer has licensed only T1 and T2, the governor will never select T3, regardless of the complexity score.

Modes are a hint to the governor about how to constrain the search space. The mode you send in X-Modelith-Mode (or the per-request x_modelith_mode JSON field) limits the maximum tier the governor will pick.

• cheap — caps at T1. Use for background jobs, batch classification, anything where latency budget is loose and cost is tight.
• balanced (default) — caps at T2. The right choice for most production traffic.
• powerful — caps at T3. Use when you explicitly need the best answer; pair with a tight session budget to keep spend predictable.

The governor can escalate a request to a higher tier when the complexity score warrants it, as long as the higher tier is in your allowed set. Escalation never moves down — once a session is on T3, it stays on T3 for the rest of the run. This is by design: a flaky T1 → T2 → T3 → T2 oscillation is what runaway loops look like.

The complete tier and escalation history is in the dashboard session detail view, including the per-step complexity score, the chosen model, the latency, and the cost. This is the "execution trace" the landing page refers to — not a black box.

Plans are billed monthly and renew automatically. You can cancel from the Paddle customer portal at any time; cancellation stops the next renewal. Existing usage within the current period is not retroactively refunded.

• Free — $0. 1,000 proxy requests per month, no platform spend. Governor on. For evaluation and pre-production.
• Starter — $19/mo. 25,000 requests OR $12 platform spend, with a wallet.
• Pro — $49/mo. 150,000 requests OR $60 platform spend, with a wallet and team-wide aggregation.
• Enterprise — from $299/mo. Custom quotas, SSO, invoice billing, SOC 2 / HIPAA BAA where applicable.

The wallet is a prepaid balance applied to platform-key traffic. Min top-up $10. Each platform request is held against the wallet at the initial-tier price, then settled at the actual upstream cost on response. Unused balance rolls forward indefinitely and is refundable on account closure.

BYOK traffic is billed by your upstream provider and does not draw from the wallet. The wallet exists to make platform spend predictable for your finance team.

Full economics are in Terms §payments. The full margin equation is in the deck.

Refunds are handled by Paddle for subscriptions and by Modelith for wallet balance. We do not charge a refund fee. The full terms — including the 14-day window where applicable, the PAYG non-refundable rule, and how to file a request — are on the Refund policy page.

The session budget was exceeded (per-session), or the plan limit was hit (per-account). The response body includes x_modelith.spent_usd, x_modelith.budget_limit_usd, and x_modelith.step. To recover: either increase the limit for the next call, close the session and start a new one, or upgrade the plan. The session is not auto-closed on a 402; further calls with the same id will continue to return 402 until you choose what to do.

Three causes share the same status code, distinguished by x_modelith.halt_reason in the body:

• loop_detected — same fingerprint 4× in 10s. The fingerprint cache is per session. Closing the session clears the cache.
• max_steps — the configured step cap was reached. The cap is per account; raise it from Settings → Routing if you have a long-running agent.
• rate_limited — per-key rate limit. Wait or rotate keys.

The platform upstream (OpenRouter, or your BYOK provider) returned an error. The governor does not retry by default; the response is the upstream's status. Check the Status page, and inspect the upstream request id in the response body for support. If you have a BYOK key for the failing provider, set it to bypass the platform pool.

The Command Center is the only place to see aggregate truth. Three views matter when you debug a session:

• Session detail. One row per session, with the spent/budget pair, the step counter, the tier history, and a per-step trace. Open any session to see exactly which calls escalated and which halted.
• Logs. Every request, structured, searchable, filterable by status, model, session id, and time range.
• Costs. The ledger. Every cent, by team, by project, by model. Exports to CSV for finance.

Still stuck? Email founders@modelith.cloud with the session id and we will follow up.