Notes on Building AI Products In The Probabilistic Era

I greatly enjoyed Building AI Products In The Probabilistic Era. It’s a lengthy read, but so worth your time. I could quote probably a quarter of the article, but I’m going to pop in some things that I think are valuable for coming back to.

This is a great way of stating the change and it’s something my work will be affected by. I work in conversion engineering for DataCamp and we’ll be shipping more and more AI-driven products.

Conversion, activation, and retention are all ratios that require countable, pre-defined inputs and outcomes. The reason why we can count those numbers and construct the ratios is because both the numerator and the denominator are limited and pre-determined: what x and y can look like are known before the function is created! By knowing their cardinality, we can create the funnel.

All these ratios, be them engineering reliability goals or growth conversion targets, are how we make both strategic as well as tactical decisions in building and growing software products. How we measure performance, how we structure our work, how we design and implement our playbooks. The entire operating system of the tech industry relies on them.

The problem is that these rules, in the probabilistic world of AI, have the potential to become actively counterproductive.

This next part, too.

In moving to an AI-first world, we transitioned from funnels to infinite fields.

Stop for a moment to realize what this means. When building on top of this technology, our products can now succeed in ways we’ve never even imagined, and fail in ways we never intended.

These models are discovered, not engineered. There’s some deep unknowability about them that is both powerful and scary. Not even model makers know exactly what their creations can fully do when they train them. It’s why “vibe” is such an apt word: faced with this inherent uncertainty, we’re left to trust our own gut and intuition when judging what these models are truly capable of.

The shift in customer’s expectations will continue to grow.

For people interacting with products harnessing the power of these models, this is a lot to take in, to accept, and to develop an intuition for. Users really dislike the inherent uncertainty of dealing with AI systems. They’re not used to it! They’re expecting a digital product like every other product they know: you instruct the app to perform an action, and the app will perform it. Unfortunately, prompting Replit in the wrong way may very well introduce a bug in your work, depending on your request and on the probability distribution that maps to. Consumers really struggle to understand this: it makes them very mad when the AI doesn’t do what they expect it to do.

The core reason why they get so frustrated is because for the first time in the digital era marginal costs are way larger than zero. In fact, they’re so large that they completely invalidate the business model and growth playbooks that dominated the Internet since the 90s. This may change in the future, depending on innovation and commoditization at hardware level, but for now the cost of intelligence is surprisingly stable (and not really as deflationary as we expected it to be until last year).

This next part is so good, it’s a fundamental shift in how we will build.

to achieve the reliability goal of 100%, classical engineering leadership naturally start adding more and more rails and constraints around the model, trying to reign it in and control it.

This doesn’t work anymore.

The more you try to control the model, the more you’ll nerf it, ultimately damaging the product itself. Past a certain point, intelligence and control start becoming opposing needs.

The goal isn’t perfection: by definition you can’t nor should aim for it. The goal is to manage the uncertainty. As an AI product builder, you should determine what’s the acceptable unpredictability that keeps the model capable of dealing with complexity, given the market you’re operating in and the audience you’re serving. Think in terms of Minimum Viable Intelligence: the lowest quality threshold that is both accepted by the market (some may be more sensitive than others), while preserving its inherent flexibility and generalization capacity.

Another stunning one.

It takes a scientist to build AI products.

The old wisdom of always building incrementally on top of what’s already there doesn’t hold up anymore. If anything, that too is actively harmful. Every time a new model drops, be it a new generation of an existing one (say, from Sonnet to Opus), or a completely new one (say swapping GPT for Gemini), all previous assumptions about its behavior should be disregarded.

In fact, when a new model drops, you should even consider literally tearing down the full system, and building it back from the ground up. There are no sacred cows.

I couldn’t agree more.

V. This Time is Different After decades of technical innovation, the world has (rightfully) developed some anti-bodies to tech hype. Mainstream audiences have become naturally skeptical of big claims of “the world is changing”. There’s now even a popular meme: “nothing ever happens”.

I strongly believe that when it comes to AI, something is happening. This time it does feel different.

“Welcome to the Probabilistic Era.”

It’s a new world, a world of wonder and possibilities, a world to discover and understand.