Model prices fell about 13x last year. The savings still to come are in the system that serves the model and the sandbox environment the agent works in, and that’s where Unusual is looking for the next infrastructure winners.
PERSPECTIVES · INFRASTRUCTURE · TEAM UNUSUAL · OCTOBER 2026 · 8 MIN READ
Organizations have spent three years choosing AI models by quality, and for most of that time that was right. Model capability has not saturated. The horizon over which a model can work autonomously, from a single answer to a multi-hour task, keeps lengthening, and we expect it to keep lengthening. What has changed is what happens underneath that frontier: the price of a fixed level of capability now falls for years after the capability itself stops being news.
Epoch AI’s September 2026 study, The plunging price of thought, puts a number on it. Since 2023 the cost of a fixed level of AI performance has fallen about 47 percent per quarter, roughly 13x per year, four times faster than DNA sequencing and eighteen times faster than lithium batteries. In January 2025 OpenAI’s o3, then the leading and largest model, scored 75 percent on GPQA Diamond at an estimated 30 cents per question; under eighteen months later GPT-5.6 Luna, a smaller and cheaper model, matched it for $0.0004 per question, a 725-fold drop.
That example measures the model alone, and the model is now only part of the price. The cost of intelligence is set by the system that serves a model at least as much as by the model itself. Models are called inside agent loops thousands of times per task, and the time a user waits for finished work depends on the queue, the cold start, the sandbox and the test suite as much as on the weights. The metrics providers advertise, price per million tokens and aggregate throughput, measure a single call, and a task is a loop.
Two Unusual portfolio companies, Fal and Lithos, have built their businesses on the serving system and the environment, and their published numbers are the clearest evidence we have of how much slack is left in each.
The cost curve of intelligence is the price of a finished unit of AI work over time, at a fixed level of quality. Most people use the phrase to mean the price of a token, but a unit of work costs more than its tokens. Its price is set by three layers, and the last two now matter more:
| Demand on a falling cost curveTotal spending on inference has risen while unit prices fell, the Jevons paradox. Inference’s share of AI compute roughly doubled, from about a third in 2023 to an estimated two thirds this year, against hyperscaler infrastructure spending expected to exceed $500 billion in 2026. We expect the compute crunch to continue through 2027: demand from agent loops and generative media is growing faster than data-center capacity can be built, so a cheaper token does not mean an idle GPU; it means the same fleet has to produce more finished work. Fal prices per unit of output, so customers’ bills fall as serving gets cheaper, yet its annualized revenue went from $10 million in September 2024 to $400 million by March 2026 and a reported $800 million by September 2026.cost per unit of work = tokens per task × price per tokenPrice per token has fallen by orders of magnitude. Tokens per task have risen just as steeply, and the longer the autonomous horizon grows, the more an agent consumes. Lithos’s founders put the arithmetic plainly: a task that generates one million tokens waits nearly three hours on generation at 100 tokens per second and under 17 minutes at 1,000. As the price term approaches its floor, the time term decides whether a class of work is viable, and time is a property of the serving system. |
01
The price of a fixed quality keeps falling after the quality stops being news, and open weights now cap what any provider can charge for it.
Epoch separates two rates: a level’s cost falls about 66 percent per quarter when it first becomes state of the art, and about 32 percent per quarter two years later, once the launch premium has competed away. The market indexes agree: Silicon Data’s index of realized prices fell 43 percent in the ten weeks to early August 2026, driven by a price war and by low-cost open-weight models from Chinese labs.
The slack here sits in open weights. In July 2026 Moonshot AI released Kimi K3 at $3 per million input tokens and $15 per million output, against $5 and $25 for Opus 4.8 and $5 and $30 for GPT-5.6 Sol, while placing first or second on five of six coding benchmarks in Moonshot’s own reporting. As Lithos’s analysis of the launch notes, for a closed model the API price is the least a customer will pay; for an open-weight model it’s the most, since many providers compete to host the weights. Once the weights are open, the remaining cost advantage belongs to whoever serves them most efficiently, which moves the slack to the serving system.
Buyers have already adjusted. Nobody picks a single model anymore; teams deploy different models for different points on the Pareto frontier of quality, price and latency: a frontier closed model for the hardest step, an open-weight model for the bulk of the loop, a small fast model for routing. The frontier keeps moving outward, but at any moment it is a curve, not a point.
| LEADING INDICATORPrice per completed task at a fixed quality bar, on your own evaluation set, repriced each quarter. | LAGGING INDICATORTokens per task and the blended price your bill reflects. |
02
The largest remaining gains are in the system between the weights and the user, and they’re being delivered on commodity GPUs.
The serving system is everything between a set of trained weights and a user who’s waiting: kernels, scheduler, weight caches, autoscaler and queue. For three years it was treated as plumbing, because the model was the slow part. In 2026 it’s where a new category of infrastructure company has formed, and the market prices it accordingly. The table draws on Contrary Research’s September 2026 review, updated for the rounds announced since.
| COMPANY | FOCUS | LATEST ROUND | VALUATION | REVENUE SIGNAL |
|---|---|---|---|---|
| Fireworks AI | Language-model inference | $1.5B Series D, Jul 2026 | $17.5B; reported in early talks at $30B to $40B, Oct 2026 | Above $1B annualized, company figure, Jul 2026 |
| Fal | Generative media inference | $1.3B Series F, Nov 2026 | $16B | Above $800M annualized, reported Sep 2026 |
| Modal | Serverless GPU compute | $750M Series D, Oct 2026 | $15.75B | Above $1B annualized, company figure, Oct 2026 |
| Baseten | Dedicated deployments, language and agents | $1.5B Series F, Jun 2026; ~$2B round reported Oct 2026 | $13B; new round reported at $26B and $30B in two tranches | About $600M annualized by Mar 2026 (estimate), up 20x year over year |
| Together AI | General inference and GPU cloud | $800M Series C, Jul 2026 | $8.3B | Above $1.15B annual bookings, Q2 2026; inference an estimated 30 to 40 percent of revenue, the rest GPU rental |
Sources: Contrary Research, September 2026; company announcements and press reports through November 2026. Private-company figures are reported, with no audited financials behind them.
Valuations track revenue closely, and the companies are specializing by workload. Lithos, which serves open-weight language models for agent loops and hasn’t announced financing, belongs with the language-and-agents group.
The gains come from a handful of techniques that spread across the industry within weeks of publication: speculative decoding (DSpark, open-sourced in June 2026, was in production at Fal and Lithos within weeks), megakernels that run a whole forward pass as one dispatch (Lithos’s open-source implementation for Apple silicon serves a 27-billion-parameter model at more than 200 tokens per second on an M5 Max), weight streaming (Fal’s FlashPack), new number formats on Blackwell (Fal’s MXFP8 quantizer) and multi-node serving with smart routing.
The physics differ by workload, which is why the specialists have diverged and why we expect them to stay diverged. A video frame from a diffusion transformer is bound by compute per sampling step; a token from a mixture-of-experts language model is bound by memory bandwidth, since the active weights are read once per token. Those are different engineering problems with different winning techniques, and diffusion transformers will continue to need a separate serving stack from MoE LLMs. A platform that treats them as one workload gives up the gains that move the curve.
The two companies we know best sit on either side of that divide. Fal cut Ideogram V4’s generation time from 2.75 seconds to 0.44 seconds with no visible quality loss, and its H3 Max video model generates a five-second clip in under three seconds, the first to run faster than playback at frontier quality. Lithos serves Kimi K2.7 Code, a one-trillion-parameter model, at more than 1,000 peak tokens per second per user on a standard eight-GPU B200 node, against the 80 to 200 typical GPU serving delivers, and it does so at native precision.
One line runs through both: the gains are delivered on commodity GPUs. OpenAI’s launch of GPT-5.6 Sol highlighted serving it on Cerebras at up to 750 tokens per second; Lithos clears that on a standard NVIDIA node. The deeper point is where the advantage lives. The fastest inference companies in the world will circumvent CUDA, writing their own kernels and schedulers against the hardware rather than through the vendor’s libraries, which is what lets the team that saturates a B200 ship a megakernel for Apple silicon months later. An advantage in software compounds with every hardware generation instead of being tied to one vendor’s roadmap.
| A cheaper, faster output at lower quality is a different product. A cheaper, faster output at the same quality is a team moving the cost curve. |
Both companies treat quality as a fixed constraint: Fal ships an optimization only when the model keeps its rank in its quality evaluations, and Lithos serves the weights the model shipped with. They’ve also proposed the measurements. Fal separates inference latency from end-to-end latency at scale, which adds queueing and cold starts (in Fal’s published example, loading weights was 496 seconds of a 524-second start), and Lithos proposes time to useful work, driven by tokens per second per user, since a cluster can produce enormous aggregate throughput while each agent crawls.
| LEADING INDICATOREnd-to-end latency per user under production load, broken into startup (queueing, cold start, weight loading) and execution (the generation itself). | LAGGING INDICATORTime to useful work for your own task class: time to a working pull request, or time to an approved asset. |
03
Once the model call is fast, the time between model calls is the loop, and the first big cut has already been made there.
Once inference runs at 1,000 tokens per second per user, an agent’s loop is dominated by what happens between model calls: starting a sandbox, running tests, saving progress, recovering from a failed attempt. The sandbox providers that grew up alongside coding agents treated those operations as fixed overhead measured in seconds, which was fine while inference took longer and the loops around it were simple, with few tool calls between model turns. It isn’t fine anymore, and the first evidence comes from an inference company: in October 2026 Lithos released LithosBox, explaining that as inference gets faster, managing the environment becomes a larger share of the time to finish a task.
LithosBox gives an agent Git-like control over a running environment: snapshot the live state of a sandbox, including running processes, branch that checkpoint to try alternatives in parallel, and restore it when an approach fails. Source control has offered those operations for code for decades; what LithosBox changes is their cost for a live environment.
| OPERATION | LITHOSBOX | PROVIDER A | PROVIDER B | SPEEDUP |
|---|---|---|---|---|
| New sandbox, first command | 50 ms | 876 ms | 872 ms | 17x |
| Snapshot a running environment (1 GB dirty data) | 95 ms | 6.9 s | 4.1 s | 43x |
| Branch from a checkpoint (512 MB) | 322 ms | 5,169 ms | Not supported | 16x |
| Restore an earlier checkpoint (512 MB) | 140 ms | 845 ms | 875 ms | 6x |
Source: Lithos, October 2026. Only LithosBox restores running processes; the others restore files. Vendor numbers against competitors the vendor chose.
The interesting number is the snapshot. At 95 milliseconds a checkpoint is cheap enough to take routinely, so an agent can try several patches in parallel branches, run the tests in each and roll back whatever failed. At four to seven seconds the same operation is something an agent is designed to avoid. The environment has the same structure the serving system had two years earlier: an operation everyone treated as fixed overhead turns out to be mostly waste, and removing it changes what agents are built to do. Tokens per task will rise again, and the Jevons paradox repeats a third time.
| LEADING INDICATORThe share of an agent’s loop spent outside the model call, measured per task from the agent’s own trace. | LAGGING INDICATORAttempts per completed task, which rises when a checkpoint costs milliseconds while time to a merged change falls. |
Model prices, serving systems and agent environments have all followed the same sequence. We call it the overhead cycle: an operation the market treats as fixed turns out to be ten times slower than it needs to be, a small team with deep systems expertise fixes it, and demand expands into the space that opens up. The cycle points to a few conclusions.
One observation about the people who find this slack: in both cases we know best, the insight came from technical depth in the system itself. Fal’s founders first set out to build compute infrastructure for Python and pivoted to generative media after noticing that their customers’ image models were slow; Lithos’s founders are systems researchers in GPUs, operating systems and hardware-software co-design. We expect the next winners to look like them.
The cost of intelligence will keep falling, and the useful question is where the next drop comes from. Unusual Ventures backs infrastructure founders at the earliest stage, when the thesis is a measurement nobody else has taken yet and the product is a few people who know how to act on it. Fal and Lithos are both in our portfolio because we saw that profile before the market priced it, and working alongside them is how we developed this view. We’ve spent years in the systems layer underneath AI, and we think the next infrastructure winners will come from it: companies that make the serving system and the environment as cheap as the model has become.
If you’re building in any of these layers, or removing slack that everyone else treats as fixed, we’d like to hear from you early, before the round, when a conversation can still shape the company. Reach the Unusual investment team at unusual.vc/contact-us.