The fact that 2.4 billion people now use generative AI every month shows just how fast this technology has spread. Yet in that same stretch of time, some teams have torn up the AI-driven workflows they built and started over from scratch. Trace why that redesign happened, and you land on a simple fact: the foundation these tools rest on still hasn't settled.
There's a fork in the road between teams that redesigned twice and teams that got it right the first time. Trace that fork back far enough, and you can see exactly why today's AI infrastructure behaves the way it does.
When API Prices Fall 97%, Your Design Assumptions Collapse With Them
The input-token price for the GPT-4 Turbo API dropped more than 97% in the roughly 18 months after its November 2023 launch. Teams that had originally scoped their automation around that starting price had to go back and re-examine their own assumptions. Handing off quality-review steps that a human used to do — once too costly to automate — suddenly became economical, while other steps made more sense shifted onto cheaper models instead. The very criteria used to decide which tool handled which step had changed underneath them.
Context-window growth had a similar effect. When the early GPT-4 models could only handle 8,000 tokens, splitting long documents into chunks, processing each one separately, and stitching the results back together was simply necessary — and some teams built genuinely sophisticated pipelines around that constraint. Once the window grew to 128,000 tokens, and past a million on some models, the reason for splitting anything into stages disappeared. Collapsing those stages and rebuilding a simpler flow became the better option.
These shifts aren't rare. The coding-agent platform Cursor swapped its underlying model multiple times over the course of 2024, and its output patterns shifted each time. GitHub Copilot changed its base model in the same period, altering how it suggested code and prompting some teams to rewrite their internal guidelines. When the numbers move by this much, a design that was sensible on day one turns into one that's either overbuilt or underbuilt.
Platform Companies Have No Reason to Slow Down
Companies competing in the generative AI platform market have every incentive to push performance forward as fast as possible to defend market share. Along the way, output formats change, default parameter values shift, and features that used to be supported quietly disappear in the next version — often without advance notice.
Individual users barely notice these changes. But it's a different story once the tool is embedded in a team's process. If nobody documented why a given prompt is formatted the way it is, or why a parameter is set to a particular value, then by the time the platform changes, it's no longer clear what actually needs to be re-checked. Eventually, teams conclude it's faster to just review everything from scratch.
Redesign costs arise whenever there's a gap between how fast an AI platform improves and how fast the workflow built on top of it can adapt. Right now, that gap persists across most platforms.
Teams That Separated Business Logic From AI Configuration Avoided a Third Redesign
There's one thing teams that redesigned twice did differently the second time around: they started treating "what gets processed, in what order" and "which AI handles that processing, and how" as two separate layers.
Take the sequence of receiving a customer inquiry, classifying it by type, drafting a response based on that classification, having a staff member review it, and sending it out. That sequence doesn't change just because the AI model behind it does. This is the business logic layer, and it should be designed around why each step exists — not around which AI happens to handle that step best.
By contrast, decisions like which prompt handles classification, which model runs at which temperature setting, and how the output gets parsed are the things that have to change whenever the platform does. This is the AI capability layer — where the model's particular quirks and settings live, and what needs re-examining every time the platform gets updated.
When the two layers are mixed together, a single platform change forces you to re-examine the entire workflow. Understanding why a prompt is formatted a certain way requires understanding the business logic, and understanding the business logic requires also understanding the model's particular behavior. Once the layers are separated, swapping out the AI capability layer alone leaves the rest of the workflow untouched.
This kind of separation is a long-established practice in product design: decide upfront what will change and what won't, then keep the changeable parts easy to swap out. That principle has been applied over and over to keep maintenance costs down, and the same logic applies to AI workflows. At the design stage, the question to ask is: "Does this step reflect logic that holds regardless of which model runs it, or is it a setting that depends on this particular model's quirks?" That question is what draws the line between the two layers.
A workflow that never asked that question at the design stage pays for it at the next platform update. Teams that asked it explicitly, on the other hand, get to choose "we just need to recheck the AI settings" over "we have to review everything."
Most AI platforms in use today are still unsettled. Designing with that fact as a premise, versus designing on the assumption that things will eventually stabilize, makes all the difference the next time a redesign comes around.
