1,000+
endpoints across the fleet
733 models routable today across 13 providers — frontier cloud, specialist inference, and local models on consumer-grade hardware, behind one OpenAI-compatible endpoint
the-array
One gateway. 1,000+ endpoints. Every dollar accounted for.
Most AI practices run on a pile of API keys, a spreadsheet nobody updates, and a monthly invoice that arrives as a surprise. the-array is the alternative we built for ourselves and now productize for clients: a self-hosted, LiteLLM-based gateway that routes every model call in the practice — and prices, observes, and governs each one.
Rent, or own
One premium AI subscription runs $200 to $300 a month. It is locked to a single vendor, a single set of usage caps, and it leaves no record of what you actually spent.
Last month, the-array ran every frontier model worth running (Claude, GPT, Grok, Gemini) alongside seven models on local hardware, for about $129, under a hard $400 ceiling that stops the bill before it happens.
The difference is ownership. You are not renting one company's API and calling it a strategy; the proxy, the routing, and the receipt for every call are yours. When a vendor goes down or changes its terms, the work moves to the next model in line, or to your own machine. And nothing you send leaves the building unless you decide it should.
The live numbers
68
fallback chains
every task degrades gracefully; every chain terminates at local inference, so the practice never goes dark
16
task aliases
work is requested by intent (reasoning, coding, vision, private), and the router picks the model; swapping vendors is a config edit, not a migration
Team- and key-level budgets
with hard monthly caps — spend is governed before it happens, not reconciled after
Full multimodality behind one key — text · vision · image (Imagen, DALL·E) · video (Veo) · music (Lyria) · speech (TTS + Whisper)
Privacy that fails closed
Sensitive work routes through privacy-preserving endpoints. If that route degrades, requests fall back to local hardware in the building — never silently to a third-party cloud. Fail-closed isn't a policy statement; it's the routing table.
Observability all the way down
Prometheus metrics, Grafana dashboards, Tempo traces, and Loki logs on every request: cost per call, cache savings, latency by model, spend by team. When a model misbehaves or a bill spikes, the answer is on a dashboard, not in a support ticket.
The mission brief
The stack
the-array is the gateway. This is the fleet it routes across, and the tools that sit on top.
Sovereign inference
Every device in the fleet can run models locally: seven purpose-picked models on the Windows box through LM Studio, on-device on the M1 iMac and iPhone through Apple Silicon and MLX, and inside Msty Studio's bundled engines. Fallback chains terminate at that local inference, so sensitive work answers on hardware in the building and fails closed to local when a cloud route degrades.
One stack, every device
The iPhone 17 Pro Max and the 2021 M1 iMac are clients of the-array: from either, a request runs against the whole fleet, and the Apple Watch Ultra's Action button puts a trigger on the wrist. The phone also stands on its own: Locally AI runs open models on-device and offline, with no network at all.
Metered routing
Every call is priced and drawn on a dashboard (Prometheus, Grafana, Tempo, Loki), held under a per-key, per-team budget that stops spend before it happens. Behavioral evals and an audit trail keep the routing reviewable.
Retrieval you can inspect
Retrieval over private material with the receipts in view: Msty Studio Knowledge Stacks build RAG over local files and past work, and a Chunk Console shows exactly what was retrieved and lets PII redaction be checked before a model sees it. Embeddings run locally. It is the same citation discipline the Institute publishes under.
Contained agent tooling
Tools run as Model Context Protocol servers, wired into Msty Studio's Toolbox or run as isolated, credential-scoped containers. They get tested in a console before an agent touches them, and multi-step work runs as reusable, inspectable flows in Turnstiles.
Open weights, pinned
Models come from Hugging Face, over two million open-weight checkpoints, pulled down to run locally (GGUF on the PC, MLX on Apple Silicon) or reached through its OpenAI-compatible router as one more upstream behind the-array. The model behind a client's workflow is one you can pin to a version and re-run.
Why it matters to you
If your organization is accumulating AI subscriptions and API keys faster than it can account for them, this pattern — one gateway, routed by intent, priced per request, observable end to end, sovereign by default — is buildable inside your walls. That's what Praxen Development does.
The problem, the constraint, and what shipping looks like for you: chris@cld-dev.io