AI Isn’t Failing — Our Operating Systems Are

Executives keep hearing that 95% of AI projects fail, and the instinct is to blame the technology — the models are immature, the data is not clean, the use case was wrong. In a crossover conversation on the Scrum.org Community Podcast, Dave West and I worked through a different explanation: AI is not the thing failing. The operating system most organizations use to manage AI is. Companies pour millions into AI pilots, but strategy, product, and delivery still move on different clocks, and most AI work is being run as a project — fixed scope, plan, milestones — when it is actually high-potential, high-uncertainty work that needs a product approach.

The shift that separates organizations finding real value from organizations stuck in pilot purgatory is this: they use product techniques to decide where to aim AI and how to execute on it. They start from a problem and a strategy (“where do we want to hire AI?”), not from a tool. They run bounded learning loops instead of twelve-month roadmaps. They develop context as carefully as they used to develop code. And they steer with leading indicators tied to a real business constraint. AI is not a silver bullet — it is a forcing function that exposes how slowly your organization actually learns. Before you invest in more models, invest in an operating system that can learn at the speed AI demands.

The conversation started where many AI conversations start these days: with the 95% failure line. I do not think the exact number is the most useful part. The useful definition is failure as money spent without value afterwards. Once you define it that way, the conclusion changes. The problem is less “the model is bad” and more “the organization has no learning system between a promising demo and a changed business result.”

That definition quietly rules out the easy explanations. The models work. They summarize, draft, reason, and code. The failure is happening somewhere between “we have capable technology” and “the business is measurably better off.” That gap is an operating-model gap, not a technology gap.

A lot of this failure is not happening inside traditional IT at all. A sales, marketing, legal, finance, or operations team sees a real problem and tries AI against it. The first version works well enough to create excitement, then the messy long tail shows up: edge cases, adoption friction, workflow fit, data quality, risk, ownership, and the question of what “good enough” actually means. The tool gets labeled a failure, but often the missing piece was a portfolio of bets, a learning loop, and a clear next evidence step. The technology did roughly what you would expect. The operating system around it did not exist.

Here is where I think the deja vu kicks in for anyone who has lived through a couple of decades of IT delivery. We have run this movie before. We spent years getting better at delivering projects on time, on budget, on scope — building better feature factories — and then we discovered that delivering the scope did not reliably deliver the value. A minority of “successful” projects actually moved the needle for the business. That is the lesson product organizations have been internalizing for a long time, and it is exactly the lesson AI adoption has not learned yet.

Most operating systems organizations use to drive change — EOS being the most popular in the mid-market and scale-up world — have a concept of a project baked in as the unit of change. New building, new event, new system: define the scope, build the plan, deploy it. That orientation is fine for work that is genuinely knowable in advance. It is the wrong orientation for AI, because looking for ways to operationalize AI is the textbook case of high potential married to high uncertainty. Will this be a practical use case? Is it feasible? Is it viable? Will people actually adopt it? Those are not scope questions. They are bets. And when your operating system only knows how to manage scope, it manages bets badly — it pushes you to assume the value is achievable, build a plan, deploy, and then act surprised when the plan delivers but the value does not.

The strategic challenges that keep leadership teams up at night — growth, organizational development, where to place the next round of investment — are almost all complex and uncertain in exactly this way. They are bets on whether a new process, a new capability, or a new tool will actually be worth it and actually get used. Treating them as projects to be planned and executed, rather than hypotheses to be tested cheaply, is the deeper dysfunction. AI just makes the dysfunction expensive and visible.

The reframe Dave and I kept circling is the product model — and specifically applying it well beyond the boundary of the product and engineering organization. A product orientation means an organization aligns investment, structure, and empowerment around the capabilities that are the business, with people who own the effectiveness of those capabilities at creating outcomes. That orientation gives you a mechanism for dealing with complexity, because it gives you continuous context instead of a one-time plan.

This is not abstract. In organizations I have worked with — Dyno Therapeutics is one example I keep coming back to — the realization was that improving the product organization only takes you so far. You level up agility inside product and engineering, and then the constraint moves. It shows up in marketing, in partner onboarding, in the customer onboarding and activation experience. The same ideas that accelerate value creation inside product apply to people operations, finance, marketing, partner success, and the interactions between those groups. Decompose the business into a set of capabilities — treat the help desk as a product, sales enablement as a product, the onboarding experience as a product — and you get a roadmap for how each one improves, a clear view of how much you are investing in each, and the ability to review and adjust based on evidence.

There is a real trap in that decomposition, though. The lazy version is that your VP of Sales owns “Sales the product,” your VP of Marketing owns “Marketing the product,” and you have just renamed the org chart. What we actually saw work is that the product is often the cross-functional experience — the onboarding experience, the overall employee experience, the partner experience — and it deliberately cuts across functions. Whether you call it a product or a value stream is a fair debate, with advantages on both sides. I lean toward product language for one practical reason: when I talk to mid-market leaders, “productize” is already a word they are reaching for. They know that scaling to serve customers well means turning bespoke services into more consistent, intentional products. That instinct is the doorway.

There is a second reason the product paradigm matters specifically for AI, and it is about how you actually get good results out of these tools. The most successful uses of AI I have had personally are the ones with a well-bounded problem — a consistent context, the right model, clear boundaries. The output you get from an AI engine that has those boundaries is dramatically different from the output you get without them.

I now spend more time developing the context for my AI usage than I spend on the actual prompt, and that shift has improved my personal AI effectiveness more than anything else. I treat the context itself as a product. I have an instruction set for an advisory-board space I run, and I develop it using something close to a Scrum loop — after a stretch of using it, I stop and inspect with the AI itself: how are we doing, are we getting good results, how could we improve this based on the conversations we have actually had? Then I cut a new version. It is not under source control yet, which it probably should be, but the discipline is the point. Context development is continuous product development.

This is why the boundaries question is not a side detail. The better you understand your products, your customer factory, where the constraints are, and what your strategic focus is, the better results AI will give you — because all of that is the context. An organization that cannot articulate its constraints and outcomes clearly cannot feed AI the context it needs to be useful. The operating-system upgrade and the AI payoff are the same work.

If you take the product lens seriously, the AI question changes shape. The old question was “where do we hire another person?” The new question is “where do we want to hire AI?” — and that is genuinely a strategy question, not a tooling question.

You answer it the way you would answer any product-strategy question: by finding the constraint. Is the business leaking customers? Then the constraint might live in the onboarding and activation experience, which spans both product and customer success. Are you generating marketing-qualified leads who never really adopt the product? Then the constraint is somewhere in activation, not lead generation. Is churn the thing throttling growth — are you spending heavily to acquire customers who leave after three months? Then that is where AI attention belongs. OKRs are a useful mechanism here, because they ground AI focus in outcomes; and if you are using a product model, those OKRs sit inside products that sit inside a coherent business strategy, instead of floating free.

Take the churn example concretely, because it shows how the operating system does the real work and AI rides on top. If you believe churn is your biggest growth constraint, you do not point a team at “reduce churn” and walk away. You form a hypothesis about the leading indicator — say, early product usage, because customers who do not use the product in week one tend not to come back. The team’s product goal becomes improving that usage, with a how-to-win hypothesis underneath it. They run a short cycle — a sprint, a month — on the tweaks they believe will get people back on the wagon, which might be lifecycle prompts, might be a customer-success play, might be a product change. They check whether usage moved. If it did not, they try something else. If it did, they validate that the more-engaged cohort actually churns less — because maybe it does not, maybe the real problem is price, and the hypothesis was wrong. That is a continuous, fast, evidence-driven loop that converges your operating model toward the current constraint. AI can accelerate almost every step of it. But the loop is the thing. AI without the loop is just faster motion.

None of this requires ripping out your operating system and starting over. Dave and I were explicit about that — this is not a call to reformat the organization. If you run on EOS, or scaling-up, or OKRs, or some homegrown system, the question is not “is it the right brand?” The question is whether it is good enough for the kind of work AI now demands. Is it sensing and responding? Is it inspecting and adapting? Does it organize around outcomes rather than scope? Does it help you focus and flow your top priorities? Many operating systems do manage flow and do push focus onto the big rocks — there is a lot of shared DNA there. The harder questions are the next ones. Are those big rocks defined as outcomes that let you steer toward a strategic goal, or as plans you execute regardless? Does your scorecard carry leading indicators you can actually steer with, or only lagging indicators you report after the fact — or no indicators at all, which leaves you free to paint everything green right up until the watermelon splits open?

It is worth being precise about scope here. This operating system is not the thing that runs your business day to day. It is not how you serve customers on a Tuesday or run this month’s marketing. It is the thing that develops the future of the business — the unrealized value, the opportunities, the biggest problems. That is where uncertainty lives, and that is where you need a testing-and-discovery, inspect-and-adapt mindset rather than a plan. One genuine anti-pattern, worth naming because organizations fall into it precisely when they like what they see in the product org: they decide to run everything the agile way. Run the help desk on Scrum, run sales on Scrum. That quickly becomes agile theater, because an operational, repeatable process does not need product-grade inspect-and-adapt — it might just need Lean and good flow management. The skill is telling the difference between what you can simply execute and what is genuinely uncertain and needs to be discovered.

This article is based on my crossover conversation with Dave West, CEO of Scrum.org, on the Scrum.org Community Podcast. We go further on the operating-system question, the product-versus-value-stream debate, and the Toyota production-system-versus-design-system analogy.

The organizations turning AI into business impact share one discipline, and it is not a tool choice. It is agility — not the capital-A flavor with ceremonies and roles, but the practice of aligning, inspecting, adapting, and flowing work toward outcomes. They learn faster and they pivot cheaper, and they have carried that discipline well past the old boundary of the product and engineering organization.

If this sounds familiar, take the AI Traction Scorecard. It is 11 questions, about five minutes, and it will help you see whether your AI effort is creating traction, mostly producing activity, or getting stuck in theater. Use it to find the next constraint to inspect before the next pilot turns into another status update.

AI isn’t a silver bullet. It’s a forcing function — it exposes how slowly your organization learns. Before you invest in more models, invest in an operating system that can think and adapt like one.