A product team spent six weeks automating a customer workflow. Two days before it shipped, a faster, cheaper model was released. Links flooded Slack. The team lead called a meeting, and they decided to switch. That took three more weeks. In the meantime, yet another model was announced.
Teams who've lived through this scene tend to reach the same diagnosis: "AI is advancing too fast. We need to keep tracking the latest models." The prescription follows the same logic: evaluate every release immediately, and switch if it's better.
But the same scene repeats after the switch. If the diagnosis were correct, following the prescription more faithfully should reduce the disruption — yet for many teams, it doesn't. That mismatch starts with a diagnosis that missed the mark.
Where Did the Six Weeks Go?
In cases covered by MIT Sloan Management Review, teams that used comparable models over the same period sometimes experienced very different levels of disruption when switching. A consistent pattern separated the teams that were hit hard from those that weren't.
The teams hit hardest had built their workflows in direct contact with the model's raw output. Their prompt structure assumed that specific model's output format, and their post-processing logic was written around that model's particular error patterns. Once a team noticed a model reliably returned JSON, they built parsing logic around that format. Once they noticed it tended to over-explain on certain question types, they added logic to truncate responses. A new model fails differently — breaking JSON format on different cases, giving terse answers on different question types. The entire post-processing layer has to be rewritten. That's why the six weeks come due again.
The teams hit less hard had separated their layers from the start. They filtered the model's response through one intermediate step before processing it — a normalization layer that ran first, regardless of the model's output format, with validation logic built on top of it. That validation logic doesn't change no matter which model is running underneath. When a new model comes in, the only thing that needs revising is the normalization layer.
Both teams spent the same six weeks. But one team's six weeks stayed tethered to a specific model, while the other's became a layer that was decoupled from any model.
There's a warning sign you can catch while a workflow is still being built: the moment you start adding exception-handling logic tailored to one model's responses. It usually begins with an observation — "in this case, the model answers like this." As those observations pile up, code gets written on top of them as assumptions. That's the moment to ask: is what I'm building right now becoming dependent on this specific model? If the answer is yes, separating the layers now is far easier than doing it six weeks from now.
The Switching Costs Teams Build for Themselves
In business strategy, switching cost is a concept companies have historically imposed on their customers: raise the cost of moving to a competitor, and churn goes down. Think of the data-migration hassle of switching software subscriptions, the time sunk into learning a platform, or integrations tied to an existing system. A well-designed switching cost becomes a moat competitors can't easily cross.
But as AI model release cycles shorten, this logic starts working in reverse in practice. Teams end up locking themselves into the very tools they use.
When switching costs are high, customers can't easily leave even after a competitor ships a better product. By the same logic, a team that has built high switching costs for itself can't easily move even when a better model comes out. Switching means the last six weeks collapse. Staying means sticking with a weaker model. Either option costs the team something — and that state traces back to an early design decision.
Here's the paradox: teams that evaluate and switch the moment every new model drops can end up keeping their switching costs high. Every switching decision splits the team's attention, and after each switch, the workflow still has to be reworked around the new model. The more often they switch, the more this cost accumulates. A team that separated its layers from the start keeps that cost bounded, no matter how often it switches.
Who makes this call inside the team matters too. When engineers optimize for short-term efficiency, the design ends up coupled to a model. When a PM or director tracks switching cost as a product metric, the design goes a different direction. Making it possible to estimate — "how many days would it take to rebuild this workflow the next time we switch models" — is part of a PM's job.
Without a Switching Standard, Switching Becomes the Job
Every time a new model launches, an argument breaks out on the team: switch, or wait? That argument is itself a cost. Someone writes a comparison doc, runs benchmarks, and the team meets to decide whether to adopt it. If it ends without a conclusion, the whole thing starts over from scratch at the next release. As release cycles shorten, this cost eats up a larger share of everyone's actual work.
Teams with a standard for switching wrap up this process quickly. The starting point of that standard is simple: does the current model actually fail on real cases? If a team keeps a running list of problems from the original workflow that are still unsolved, they can just check a new model against that list when it appears.
The second step is checking whether the new model actually handles those cases — testing it against the team's real data rather than launch materials or generic benchmarks. A high score on a general benchmark doesn't rule out the model failing differently on the industry-specific terminology the team actually deals with. The final step is judging whether the switching cost, including validation, is smaller than the gain. Run through these three steps and the adoption decision comes down to numbers.
"A new model came out" is rarely reason enough on its own to adopt it. A release becomes an occasion to check it against the standard, and the decision closes in thirty minutes without an argument. A team without a standard spends that same thirty minutes just starting the argument.
This standard can't stay right forever. Mismanage the list of unresolved cases and you'll miss a switch you actually needed to make. Once the standard is set, it needs a quarterly check-in. Have a standard — just don't let it calcify.
What Survives the Model, Even When the Model Doesn't
AI model performance is likely to keep climbing. If you accept the premise that cheaper, faster models will keep arriving on short cycles, it's hard to build a strategy around avoiding switches altogether. A more practical question, then, is what should accumulate on the team's side in an environment where switching keeps happening.
Experience accumulated from long-term use of a specific model largely evaporates once that model is replaced. That includes the time spent learning its error patterns, post-processing code tuned to its output format, and internal guidelines written around its quirks. What actually survives, regardless of which model you use, is something else.
It's the instinct for which tasks a model handles well and which it doesn't — not knowledge of one specific model, but an understanding of where language models tend to fail in general. That instinct carries over to the next model. The same goes for the habit of designing with separated layers. Once you've internalized it, the next workflow you build starts off pointed in the right direction from day one.
Here's what having that judgment looks like in practice. When a new model's launch materials land, the team decides within five minutes whether its strengths overlap with the cases they actually handle. If there's no overlap, it's off the review list. If there is, they test it against real data. A team that runs this loop automatically doesn't experience a model launch as a work stoppage.
This kind of asset doesn't disappear when the next model shows up. The model may change every six weeks, but this asset isn't bound to that cycle. The difference between spending six weeks all over again and wrapping it up in three days doesn't come down to which model you're using — it comes down to whether the team has built this asset at all.
