Caveat Emptor
“Caveat Emptor” – a cartoon illustrating that the economics of AI are closer to the restaurant industry than traditional software.
For the past two decades, software has been defined by one extraordinary economic advantage in that once built, it could be sold endlessly at almost no additional cost. Serving the thousandth customer was no more expensive than serving the first. But AI fundamentally changes this software business model. Unlike traditional software, AI generates outputs in real time. Every query, every automated workflow, every generated response consumes compute. Intelligence is not stored and reused … it is produced on demand.
This makes the AI industry more like running a restaurant, a notoriously unprofitable business often run for love. In a restaurant, every meal must be prepared fresh, every ingredient has a cost, and success depends not just on attracting customers, but on ensuring each plate can be served profitably.
At scale, AI introduces costs that traditional software largely avoided. For example, a SaaS company providing document search to 50,000 enterprise users. In a traditional model, once the indexing system is built, each additional search costs almost nothing, often less than $0.00001. Even with 10 million searches per day, annual infrastructure costs might remain under $500,000, because the system retrieves existing results. Now consider the same product rebuilt using AI, where each search generates a new answer. If those same users perform 200 AI-powered searches per day, that results in 10 million AI queries daily. At $0.01 per query, annual compute costs exceed $36 million. The user experience may feel similar, but the economics are entirely different.
As Sundeep Goel, CEO of Mavvrik, warned, “My prediction is CFOs are going to have huge cost overruns” as AI pilots move into production.
23
9
3
“Ian, the math here is precise and the industry hasn’t fully internalized it yet.
The deeper structural issue is that LLM inference is stateless by design — every call recomputes from scratch because the model holds no encoded representation of your business logic. That’s appropriate for generative tasks. It’s economically indefensible for deterministic ones.
A compliance validation, a covenant check, a claims adjudication — these have correct answers derivable from documented rules. Routing them through inference at $0.01 per call isn’t an AI problem, it’s an architecture problem. Extract the logic once into a deterministic constraint layer, and marginal execution cost collapses toward zero — reserving inference for the tasks that actually need it. That’s exactly what we’re building at SynapseLayer.ai.
Given what you’ve seen running against live regulatory workloads at scale, I suspect you already know exactly where this is heading. The CFO cost overrun problem is real. For a specific class of enterprise workflow, it’s already solved.”
“Your restaurant analogy nails it. Every query has a food cost, so winning shifts from selling seats to managing unit economics. If you were advising a CFO today, which lever moves COGS most, model choice, context size, caching, or product limits?”
“Interesting that both restaurants and AI depend less on scale alone and more on tight operational control”
“Seat-based SaaS feels outdated in an AI world.
If one AI agent can do the work of five people, pricing per user stops making sense. The shift toward “service-as-software” — paying for outcomes, not access — could fundamentally reshape how software companies build and monetize.
The real question: who adapts first, and who gets trapped by their own model?”
“the real shift is from software margins to operational margins…AI turns product strategy into cost architecture…”
“Great analogy! And it opens up a whole other topic…attribution. Who is consuming the costs? Here (like most companies), the AI costs are showing up in one shared soup pot. And yet, consumption varies wildly across teams, agents, features, users, you name it. Companies are eager to jump to pricing and measure impact of AI, but have to start with actually understanding the costs.”
“If the LLM’s increase costs by 5-10% many AI application companies will fold within 6 months. cortave”
