AI models keep getting cheaper per token. Somehow, companies are still spending more.
That sounds contradictory until you look at how AI is actually being used.
A simple chatbot might generate a few hundred tokens. An agent can read thousands of tokens of context, call tools, reason across multiple steps, and repeat that process dozens of times before finishing a task.
Multiply that across an organization and token consumption grows very quickly.
The next challenge for enterprise AI isn’t simply getting access to better models. It’s deciding:
- which model should handle each request
- where that request should run
- how much it should cost
- what the model should be allowed to do
That is the problem Msty Nexus is designed to solve.
The Token Bill Is Starting To Show Up
For the first few years of generative AI, much of the user experience hid the underlying economics.
Pay a subscription. Send prompts. Let the provider worry about inference.
Agentic AI changes that equation.
TechCrunch reported in June 2026 that Uber had already exhausted its entire annual AI coding budget by April. Microsoft reportedly pulled back Claude Code access for developers after initially expanding it, while a Priceline employee said a routine Cursor renewal came back four to five times more expensive.
Cursor itself ran into the same economics on the provider side.
In 2025, the company changed its pricing because request-based limits no longer reflected what AI agents were actually consuming. Cursor explained that newer models performing longer-running tasks could consume dramatically more tokens, with its hardest requests costing an order of magnitude more than simple requests.
And this isn’t limited to coding.
The Wall Street Journal reported that legal AI company Harvey grew from around 1 trillion tokens per month to 14.5 trillion in just six months as it expanded its AI capabilities. One response has been adding lower-cost open-weight models alongside frontier models from providers such as Anthropic, Google, and OpenAI.
The pattern is becoming clear: cheaper tokens don’t necessarily mean a cheaper AI bill.
As AI becomes more useful, we use dramatically more of it.
We Heard The Same Thing At Ai4
We recently spent a few days at Ai4 2026 in Las Vegas demonstrating Msty Nexus and talking with people deploying AI inside large organizations.
One theme kept coming up.
Companies are starting to crack down on AI token usage and rein in unrestricted access to expensive models.
Employees we spoke with described organizations paying much closer attention to which models teams were using, how much AI they were consuming, and what that usage was costing.
It led to some of our most interesting conversations at the show.
Simply telling employees to use less AI is not a great solution.
The better question is: how do you give people more access to AI while gaining more control over how that access is used?
That’s where an AI gateway starts to become much more than a proxy.
The Flat-Rate Era Won’t Hide The Economics Forever
AI products have historically absorbed, bundled, or abstracted some of the underlying inference cost.
Cursor, for example, has described providing bonus capacity beyond guaranteed usage included with subscriptions. Its pricing has increasingly exposed the economics underneath through included usage pools, API-rate usage, and different model options.
Cursor’s pricing documentation now shows how model selection affects token output and included usage consumption.
Other AI products have moved toward usage-based credits and billing as agentic workloads consume more compute.
This shouldn’t be surprising.
A company deploying AI broadly can’t build its infrastructure strategy around the assumption that every employee, application, and autonomous agent can indefinitely send every request to the most capable frontier model available.
Eventually someone has to ask: did that request actually need the expensive model?
Most Requests Don’t Need Your Best Model
Consider an internal AI assistant:
- One employee asks: summarize this email in three bullets.
- Another asks: analyze these three contracts, identify conflicting indemnification provisions, and explain the financial exposure.
Those requests probably shouldn’t go to the same model.
Yet that’s how many AI deployments work today.
Users select a favorite model, often the most powerful one available, and too much routine work goes there by default.
The frontier model handles the difficult reasoning. It also handles simple tasks like rewriting a sentence.
That’s where routing becomes an economic control.
Dynamic Routing Changes The Equation
Msty Nexus sits between your applications and the models they use.
Applications connect to one OpenAI-compatible gateway. Behind it, Nexus can connect to cloud providers alongside models running locally or on infrastructure you control.
With Smart Routes, the application doesn’t have to decide which model should receive every request.
Instead, organizations can make multiple models available and intelligently route requests based on the task.
That creates a very different architecture:
- Simple request → smaller, efficient model
- Sensitive request → private local model
- Complex reasoning → frontier model
- Coding task → coding-optimized model
- Provider unavailable → approved fallback
The user doesn’t have to think through the routing decision. They ask the question, and Nexus determines the best place for it to go.
The Price Difference Can Be Enormous
Model pricing is no longer separated by a few percentage points.
Depending on the model and provider, the difference can be an order of magnitude or more.
That makes model selection itself an infrastructure decision.
If a large portion of an organization’s requests can be handled well by an efficient model, or by a model running on hardware the organization already owns, there is little reason to send all of that traffic to the most expensive frontier model.
The goal is not to use the cheapest model. The goal is to use the right model for the job:
- a small local model for routine work
- an efficient cloud model when hosted inference makes sense
- a frontier model when the request genuinely needs the highest capability
Smart Routes make that decision something the infrastructure can handle instead of something every employee has to understand.
Then Stop Paying To Read The Same Thing Twice
Routing controls which model handles a request. Caching controls how much repeated context the model has to process once the route is chosen.
Long-running conversations and agents repeatedly send large amounts of the same context. By turn ten, a request might look something like this:
System prompt 2,000 tokens
Tool definitions 3,000 tokens
Conversation 20,000 tokens
New question 40 tokensThe user added 40 tokens, but the model may need to process roughly 25,000. Then turn eleven sends almost all of it again.
Prompt caching helps supported providers reuse previously processed prompt prefixes instead of processing the same context at the normal input rate every time. For cloud models, that can mean lower token costs. For local inference, the benefit is less wasted GPU time, memory bandwidth, power, and latency.
The hard part is preserving cache hits once real infrastructure gets involved. Cache efficiency can drop when:
- a load balancer moves a conversation to a machine without the same local prefix cache
- a router switches models unnecessarily
- a system prompt changes between turns
- tool definitions are reordered
- security or transformation layers alter previously processed text
Nexus helps by coordinating routing decisions with prompt-caching behavior through one gateway. It can keep requests on the right model or local runtime, avoid unnecessary route changes, preserve reusable prompt prefixes where supported, and make repeated work easier to detect.
That creates another layer of cost reduction beyond model routing alone. First, Smart Routes help avoid sending simple work to expensive models. Then caching-aware gateway behavior helps avoid paying, or waiting, for the same context to be processed again and again. When routing, balancing, caching, and model selection are managed together, the infrastructure can make better decisions than any one application could make independently.
The Goal Isn’t To Use Less AI
All of this points to the same conclusion: organizations are right to worry about AI costs.
But the answer can’t simply be: stop using so much AI.
AI is becoming more capable precisely because we’re asking it to do more:
- agents use more context
- models call more tools
- employees use AI across more workflows
- applications increasingly make model calls without a human explicitly initiating every one
Trying to solve that growth entirely with usage caps will eventually limit the productivity organizations were trying to unlock in the first place.
The better goal is to get more value from every token you pay for, and every piece of compute you already own.
That requires more than a cheaper model list. It requires a shared control layer that can understand models, routes, usage, caching, credentials, and local or hosted infrastructure together.
The Future Isn’t One Model
The economics of AI are pushing the industry toward a different architecture:
- not one model
- not one provider
- not cloud-only
- not local-only
- not every request sent blindly to the biggest model available
Instead:
- many models
- many environments
- one governed gateway deciding how they’re used
As AI adoption grows, model intelligence alone isn’t enough.
Organizations also need to manage AI economics, infrastructure, access, and governance.
Msty Nexus brings those decisions into a control plane you own, with local models, cloud models, Smart Routes, load balancing, caching, guardrails, credentials, and usage visibility behind one gateway.
Your employees shouldn’t have to understand token prices.
They shouldn’t need to know which model is cheapest.
They shouldn’t have to decide whether a request belongs on a local model or a frontier model.
And every application shouldn’t need credentials for every AI provider your organization uses.
They should just ask. Nexus handles where the request goes while you stay in control of why.
Msty Nexus can run on your machines, your cloud, or both, giving you one gateway across local and hosted AI infrastructure.
Help Us Shape Nexus For The Enterprise
We’re currently looking for early pilot testers for Msty Nexus Enterprise.
If your organization is trying to get better control over AI costs, model access, routing, local and cloud infrastructure, or guardrails, we’d love to work with you.
Early pilot partners will have the opportunity to test Nexus in real-world environments and provide direct feedback as we continue building out our enterprise capabilities.
Interested in running a pilot? Email us at [email protected].