Controlling AI costs has become one of the biggest challenges for companies that have moved past the experimentation phase and placed artificial intelligence tools at the heart of their development operations.
The logic seems simple at first: you adopt an AI tool for the dev team, productivity skyrockets, everyone is happy. Then the bill arrives. 😅
At Databricks, the use of coding agents measurably improved every speed metric the company tracks, and in some teams it multiplied output by ten. But alongside that massive gain came a problem nearly every company that scaled AI has faced: costs growing exponentially. That curve is unsustainable. If nobody intervenes, at some point it surpasses the company’s own revenue.
This is the paradox that companies like Databricks, Stripe, Uber, Coinbase, and Ramp discovered firsthand. On one side, there is the desire to push AI transformation to the max and put powerful tools in employees’ hands. On the other, there is the need to reconcile that with an aggregate cost profile that threatens to undermine, or even reverse, the very efficiency gains AI delivers.
The good news is that several of these companies, among the earliest to adopt AI at scale, converged on a set of strategies that solves this puzzle. The goal became clear and even got a name: the dual mandate. It means providing broad access to AI tools with as little friction as possible while keeping spending within a roughly fixed envelope per user. 🚀
The Real Problem Behind the AI Bill
Before talking about solutions, it is worth understanding why this problem shows up so forcefully and so quickly. When a company starts using AI in a limited way, the financial impact is nearly invisible. A small team, a few calls per day, a model here and there. The cost appears as a discreet line item and nobody audits it carefully.
The real problem begins when adoption truly scales, when dozens or hundreds of developers start depending on these tools every day. When a user types a simple request into a coding agent, something like investigate and fix this bug, the agent goes out and collects a massive volume of context, invokes multiple tools, scours the codebase, and integrates information and capabilities provided by the company. By the time the expensive language model inference actually happens, the user’s original text represents a tiny fraction of everything fed into the system. In other words, the cost is dominated by context the user never intentionally included.
Receive the best innovation content in your email.
All the news, tips, trends, and resources you're looking for, delivered to your inbox.
By subscribing to the newsletter, you agree to receive communications from Método Viral. We are committed to always protecting and respecting your privacy.
The Efficiency Frontier for Code Models
The single biggest cost lever is shifting spending toward more efficient models as they are released. And there is an important nuance here. In everyday conversation, the term frontier model usually means the most intelligent model in existence, and leading labs focus on pushing that peak of intelligence forward. These models can already solve novel math and cybersecurity problems.
But when AI is deployed at scale, a different kind of frontier matters more: the efficiency frontier. It is defined by the set of models that offer the best price for a given level of intelligence. Most day-to-day coding does not require mathematical proofs or security breakthroughs. What matters in the aggregate is the cost of models that meet the quality bar for typical software engineering work. And that efficiency frontier is advancing much faster than the intelligence frontier, with new models appearing almost every week offering better intelligence per unit of price. 💡
Lever 1: Open Source and Lower-Cost Models
Quickly adopting newer, more efficient models delivers the largest cost savings of any technique. But to capture those savings, a company first needs to know which models actually outperform the ones it already uses. This is not trivial, because public benchmarks do a poor job of indicating real-world performance on coding tasks.
To evaluate new models, many companies have built automated evaluations they consider more representative of their internal development mix. Databricks recently published an example of this kind of benchmark, in which it observed highly competitive price-performance from GLM models, which led the company to make them available internally for developers.
It is worth noting that not every new model advances the efficiency frontier. Evaluations frequently return negative results. Stripe found that Opus 4.7 did not meaningfully improve quality over Opus 4.6, and it actually increased costs, so the company decided not to roll it out internally. Databricks saw similar cost regressions when comparing Opus 5.0 with 4.8.
Harness and Model Flexibility
Since the biggest gains come from swapping in new models, adopting tools that allow model flexibility has become a critical piece of keeping costs under control. The tool most tightly coupled with a specific model is called a harness. Proprietary frontier models are increasingly designed to work well with specific harnesses. If a company wants to preserve model independence, there are basically two paths:
- Ask users to switch harnesses: the company offers a set of harnesses, such as Claude Code, Codex, or Cursor, and asks developers to switch when it is time to migrate spending to cheaper models. The downside is that the switching cost for a developer can be high, and the harness effectively becomes a lock-in to a specific model.
- Use a meta-harness: a newer and increasingly popular approach is to use a meta-harness, which presents a unified experience to the developer while dispatching requests to the underlying harnesses behind the scenes. This ensures model independence and reduces switching costs. At Databricks, this is the default mode with Omnigent.
Lever 2: Dynamic Request and Task Routing
Instead of asking users to pick the most appropriate model on their own, a growing body of research suggests that automatic model and tool selection can squeeze even more efficiency out of agentic coding workflows. Routing approaches fall into three categories:
- Request-level routing: a stateful proxy sits between the client and the foundation models, trying to direct each request to the cheapest model capable of handling it. This routing also needs to account for server-side caching, since a cold cache is extremely expensive with large context payloads. Examples include Cursor Router, OpenRouter’s AutoRouter, Ramp’s Router, and Databricks’ own Smart Routing in the Unity AI Gateway.
- Task-level routing: a client-side process distributes tasks to different harnesses based on complexity. Renaming a component from X to Y is a simple task, while exploring design considerations to reduce latency is a complex one. Omnigent is an example of a meta-harness that supports this pattern.
- Escalation and delegation patterns: a single harness combines two models, one expensive and smart, one cheap and workhorse. In approaches like Claude’s Advisor Tool, the cheap model drives and escalates when it senses the task needs more firepower. In Cognition’s Devin Fusion, the pattern is reversed: the expensive model runs the main loop and outsources work to the cheaper one.
Internal results from Databricks show that the AI Gateway’s Smart Router reduces the average cost per task by more than 30% while maintaining virtually the same quality as the most expensive model in the pool. Other companies have reported similar numbers. 😎
Lever 3: Visibility, Alerts, and Budgets
It might come as a surprise that this article did not start and end with just give everyone a monthly budget and call it a day. Hard budgets, where usage is completely cut off upon hitting a limit, are used only as a last resort across all the companies consulted. There are two reasons for this.
First, if a developer hits the ceiling, cutting off AI access would be devastating to productivity. Neither the company nor the employee wants that outcome. Second, at least some of the highest spenders are precisely the ones who have achieved monumental efficiency gains and are producing a ton of output. Discouraging these people is shooting yourself in the foot.
Instead of a hard cap, most companies have adopted a more progressive approach focused on visibility and increasing levels of friction as spending rises:
- Visibility: all the companies consulted provide near-real-time feedback on ongoing spending, many with tips on how to reduce costs by using cheaper models. It is important that users can see their spending across all tools.
- Spend gates: developers can be prompted to take actions or request approvals as their spending increases. The simplest form is the self-clearable gate, a notification that the spend rate has crossed a certain threshold. Databricks found this mechanism useful for preventing accidental overspending.
- Downshifting: when a developer hits a gate, they can be downgraded to a cheaper model instead of being suspended. Since cheaper models cost significantly less, they can keep producing without racking up outrageous bills.
- Suspension: in the extreme case, systems can still cut off access entirely. But this is usually temporary and serves as a starting point for a conversation about more efficient usage.
Lever 4: Reducing Token Bloat
As we have already seen, a large share of the actual cost comes from context the user never asked to include. Reducing this context bloat is still an emerging area, but several promising approaches are being explored:
- Forcing more frequent compaction and compression of active context.
- Using less chatty harnesses, meaning more token-efficient ones, or tuning existing ones to produce less overhead.
- Auditing the most popular tools and reducing their verbosity.
- Encouraging developers to break tasks into smaller units, reducing the scope of context.
When context gets large, prompt caching also plays an important role in overall performance. Both proprietary and open source models have settings that allow you to enable caching and adjust how long cached content is stored. Writing to the cache costs money, but cache reads can drastically reduce the cost per inference. This balance depends on each company’s workload, so manually tuning these settings can yield massive improvements.
At Databricks, a relatively simple harness and cache configuration adjustment led to a nearly 50% reduction in the number of tokens generated and their associated costs, with no quality drop observed by developers.
The AI Gateway Design Pattern
All of these techniques carry implicit technical requirements. To take advantage of new models quickly, you need a central place where the model menu is managed. To provide budget visibility across multiple tools, you need unified cost observability. To control context bloat, you need to observe tool outputs and enforce compression.
These needs are being addressed by a new class of infrastructure software: the AI Gateway. It is a central hub where four things happen:
- Capacity management and access proxying for models, whether proprietary or open source.
- Budget tracking and enforcement, including complex policies like progressive friction tiers and model downshifting.
- Configuration management for user tools, to enforce allowed model lists and compaction settings.
- Trace logging of coding sessions for efficiency analysis and benchmarking.
At Databricks, the Unity AI Gateway handles all of these functions.
Putting It All Together
The exponential growth of AI costs is not an inevitability — it is an engineering and governance problem that has a solution. The companies that tamed this curve share the same playbook.
They chase the efficiency frontier instead of the intelligence frontier, adopt tools that preserve model flexibility, intelligently route work to the cheapest model capable of getting the job done, swap hard budgets for visibility and progressive friction, and cut the token overhead that dominates actual spending. None of these techniques require giving up the productivity gains that made AI adoption valuable in the first place. Together, they make it possible to satisfy that dual mandate of broad, low-friction access within a predictable cost envelope.
A new set of infrastructure abstractions is emerging to give companies the tools to control their costs. Databricks has released as open source or free the core components of its cost management stack: the Unity AI Gateway for central management and Omnigent for development tooling. Thousands of companies use these components every day.
What becomes clear when looking at these experiences is that there is no single silver bullet. What exists is a set of principles that, adapted to each organization’s context, create the conditions for artificial intelligence usage to scale in a healthy way. Real visibility into consumption, smart model diversification, automatic routing, and a culture of conscious usage are the elements that separate those who keep AI as a strategic asset from those who get trapped in a spiral of rising costs with no clear return. The market is maturing fast, and the practices that were competitive advantages not long ago are already becoming the new expected standard for any organization that takes AI in production seriously. 🚀
