Why Your AI Effort Has Activity But Not Impact

Most organizations do not start their AI journey with impact. They start with activity: tool access, training, pilots, hackathons, champions, experiments. That is useful, but it is not the same as value. Some then move to output, especially in engineering, where AI coding agents can help teams produce more software faster. But if engineering is not the real constraint in the business, more engineering output can simply create more work waiting somewhere else.

The better question is not “where can we use AI?” It is “where is the business actually constrained, and how could AI improve flow through that constraint?” AI impact comes from improving the part of the system that limits business results - not from celebrating adoption metrics or optimizing the easiest team to optimize.

When I look at an AI effort, the first question I want to ask is not which model you are using or which tools you have rolled out. It is simpler than that: where is your AI effort right now? Is it mostly AI theater, is it producing more output, or is it changing business impact? I do not ask that as a rhetorical swipe at people doing AI work. Most organizations start in a perfectly reasonable place: they give people access to AI tools, run AI literacy training, sponsor hackathons, start pilots, and create experiments. There is motion. There are dashboards. There are internal stories. Leaders can point to all of that activity and say, “We are doing AI.”

And they are. But that does not mean they are seeing value yet. There are probably spots where value is emerging: somebody is saving time on a document, somebody is using AI to think through a thorny problem, a team is experimenting with a coding assistant, or a function is trying to automate some repetitive work. That is a good start. I am not dismissing it. But it is not the same as AI impact at scale.

This is where a lot of AI efforts start to get comfortable in the wrong way. Activity is visible, easy to count, and reassuring to talk about. You can count how many people have access to the tools, how many people have been through training, how many people attended the hackathon, and how many use cases are in the funnel. Those numbers are not useless, but they are also not enough. They tell you that the organization is moving around AI. They do not tell you whether the business is moving because of AI.

AI activity is especially comfortable when the original reason for doing AI was FOMO. The board is asking about AI. The CEO is asking about AI. Competitors are talking about AI. Every conference has AI in the title, and every vendor is repositioning themselves around AI. People feel like they should be doing something, so they do something.

Again, I am not mocking that. In a fast-moving environment, doing nothing is not a great strategy either. The problem is what happens when “doing something” becomes the goal. At that point, the organization starts celebrating the activity itself. The training happened, the pilots launched, the tools were enabled, the dashboard looks active, the AI council met, the town hall had a demo, and someone shared a prompt library. The organization can now check the box that says AI is underway.

But over time, the questions get harder. The conversation shifts from “are we doing AI?” to “what are we getting back?” Leaders start asking what changed in how the business works, whether customers are seeing anything different, whether decisions are moving faster, whether teams are less overloaded, and whether the organization is actually reducing waste, delay, rework, or risk. These are uncomfortable questions because they expose whether the AI effort has crossed from activity into impact. If the answer is not clear, trust starts to erode.

There is another problem, too. If the AI effort is mostly activity, people in the trenches may not understand the vision for how AI is supposed to change the work. They hear that AI is important, that everyone should use it, and that productivity matters. But they may also quietly wonder whether they are helping automate themselves away. That fear is not irrational. If leaders cannot explain where AI fits, what kind of work it should improve, what boundaries matter, and how the organization expects people to participate safely, people will fill in the blanks themselves.

Some people will experiment. Some will comply. Some will avoid the whole thing. Some may even sabotage the effort, or at least refuse to lean into it. That is why AI theater is not only a value problem. It is also a trust problem. When people see a lot of activity around AI but not a clear connection to meaningful work, cost, value, or strategy, support for AI improvements starts to decline. The organization may still be doing AI, but the energy behind it gets thinner.

The next level after activity is output. This is where AI starts to create visible improvement in a specific area. There are real deliverables, real applications, and teams seeing higher throughput. The most obvious example right now is software engineering.

That makes sense. AI coding agents and coding harnesses were trained on a lot of open source code. Engineering teams are often early adopters of new technology. Developers are used to tooling, and the feedback loop is relatively tight: ask the agent to do something, inspect the result, run tests, adjust, and keep going. So we are seeing engineering teams deliver more. In some organizations, engineers are not doing as much manual coding anymore. They are working through agents. In some places, they are even being asked to only code through agents, or at least to use agents as the default path.

That is a real shift. It is not just theater. But more output is still not the same as impact. The key question is whether that output changes a business result. You might see more code, more tickets closed, more artifacts created, and more work moving through one function. The question is whether revenue per person improves, customers adopt more quickly, retention gets better, learning cycles get shorter, or the broader organization becomes more effective.

Sometimes the answer is yes. Often it is not obvious yet. The reason is simple: engineering might not be the bottleneck. It might not have been the bottleneck to begin with. If engineering was already not the constraint in the system, making engineering faster may simply create more output that waits somewhere else. It waits for product discovery, prioritization, security review, deployment, enablement, customer success, sales, legal, or leadership to decide what matters. The local metric improves, but the business does not move much faster. That is not an AI failure. That is a system failure.

There is also the upstream version of the same problem. Maybe the bottleneck is not downstream of engineering. Maybe it is upstream. Maybe the organization is not able to feed a very hungry AI-augmented engineering organization with enough valuable, validated product ideas.

That can feel like feeding a set of teenagers who do sports all day. You spend all of your time going to Costco or Sam’s Club, buying tons of protein and food, and it is never enough. That is what some product organizations are starting to feel like. Engineering capacity goes up. The ability to code goes up. The appetite for well-shaped work goes up. But the organization’s ability to discover, validate, prioritize, frame, and steer valuable work does not keep up.

Then people look around and say, “Why are we not seeing more impact? Engineering is moving faster.” Yes, engineering is moving faster. But the whole system is not. Sometimes more engineering output even makes life harder for everyone else. More features need to be reviewed, enabled, supported, explained to customer-facing teams, and absorbed by internal groups.

That is the classic danger of local optimization. One part of the system improves, but the end-to-end flow does not. In some cases, the local improvement is neutral to the whole. In other cases, it is actually detrimental. This is why I am cautious when people talk about AI transformation mostly through the lens of individual productivity or functional productivity. Those improvements can be useful. They can also be distractions if they do not connect to the constraint that actually limits business impact.

The more useful question is not “where can we use AI?” That question is too broad and too tool-centered. You can use AI almost anywhere, but that does not mean every use deserves the same attention, risk, funding, or organizational energy. The better question is where the business is actually constrained.

That pushes the conversation into more useful territory. What is limiting flow through the system? Where does work wait? Where do decisions get stuck? Where are customers leaking out? Where are opportunities not turning into revenue? Where are experts overloaded? Where does coordination across functions slow everything down? Where are we generating output that does not turn into outcomes? Once you ask those questions, the AI conversation gets much more practical.

Maybe the bottleneck is still inside product and engineering. Maybe the issue is that you cannot discover valuable ideas fast enough, validate them early enough, or connect engineering work to customer behavior quickly enough. Maybe you are building things, but learning too slowly whether they matter. In that case, AI impact might mean improving discovery, synthesis, prototyping, customer research, experimentation, or product decision quality. It might mean helping product teams feed engineering with better-shaped opportunities, not just more tickets.

But maybe the bottleneck is not product anymore. Maybe it is customer success. Customers start using the product, but they do not stick. The leaky bucket is not because engineering cannot ship; it is because onboarding is weak, customer signals are not being noticed, adoption friction is not being addressed, or the organization does not have the right process for learning from customers after the sale.

Maybe the bottleneck is partner onboarding, if your business model depends on channels and partners take too long to become productive. Maybe it is sales, where marketing can generate interest but the team struggles to qualify, explain, or convert the right opportunities. Maybe it is leadership decision flow: too much work in flight, priorities shifting, and every meaningful cross-functional decision climbing the hierarchy. Work does not slow down because people are lazy. It slows down because the operating model makes decisions expensive. The point is not that one of these is always the answer. The point is that AI impact depends on finding the answer in your system.

The trap is that organizations tend to apply AI where AI is easiest to apply. That is understandable. Easy wins are attractive. They create momentum, show people what is possible, reduce fear, and give the organization examples. But if you stop there, you can end up improving the part of the system that was already strong enough.

This is where the theory of constraints is useful. When there is a bottleneck, you subordinate the system to it. You do not keep optimizing non-bottlenecks and hope the whole system improves. You identify the constraint, exploit it, elevate it, and align the rest of the system around it. The same logic applies to AI. If engineering throughput is already amazing, maybe the next move is not to make engineering even faster. Maybe the next move is to ask how some of that capability can help the real constraint.

That is culturally hard. It cuts across silos and asks people to work outside the area where they are most comfortable. It may mean telling a successful team, “You are doing great things with AI, but improving your own output even more does not help us much right now. We need some of your expertise somewhere else in the business.” That is not how many organizations are wired. Most organizations fund departments, measure departments, reward departments, and optimize departments. Then they are surprised when the end-to-end flow suffers.

AI exposes that problem because it can create local acceleration very quickly. If the rest of the system is not ready, the acceleration does not disappear. It turns into queues, coordination load, review burden, adoption friction, and frustration.

Inside the product development world, we have been dealing with this problem for a long time. Output is not enough. Working software is not enough. Shipping more features is not enough. The product group needs to discover valuable ideas, test risky assumptions, build the right things, validate that they actually work, and keep adjusting until customer behavior and business results change. That is one path to AI impact: use AI to improve the product lifecycle end to end, not just the coding part.

But the broader path is bigger than product. An AI-native organization is not just an organization with AI-enabled engineers. It is an organization that can use AI to improve how the whole company works. That means looking at your key business processes: how you create demand, convert it, onboard customers, retain them, learn from the market, make investment decisions, and develop the organization itself. Then ask where AI can improve flow through the part that actually constrains impact.

That may involve automation, decision support, synthesis, better handoffs, faster feedback, or helping people notice patterns they currently miss. It may involve changing the shape of the work, not just making the existing work faster. This is why AI transformation is not only a technology program. It is an operating-model question.

If you are leading an AI effort, I would start with a simple diagnostic. First, look at activity: what AI activity is happening, who has access, who is trained, what experiments are running, what communities or enablement structures exist, and what the level of curiosity and participation looks like. Then look at output: where AI is creating more throughput, which teams are producing more, which workflows are faster, which artifacts are cheaper to create, and which cycle times are improving locally.

Then look at impact. Ask what business outcome changed, which constraint moved, which queue got shorter, which customer behavior shifted, which decision got better, which costly delay disappeared, and which revenue, retention, quality, risk, or capacity signal improved. If you can answer the first two sets of questions but not the third, you probably have an activity-to-impact gap.

That gap does not mean the AI work is useless. It means the AI work needs stronger steering. The leadership move is to connect the AI portfolio to the business constraint. Not every AI use case deserves the same attention. Not every experiment should scale. Not every local productivity gain should become a strategic program. Some AI activity is just literacy. Some is useful personal productivity. Some is local output improvement. A smaller set should be aimed directly at the constraint that limits business results. That smaller set is where executive attention should go.

This article is based on the solo podcast episode where I walked through the activity -> output -> impact distinction and why the constraint matters more than the easiest AI win.

The goal is not more AI usage, more AI training, more pilots, or even more output in one part of the organization. The goal is AI impact. And AI impact means the business works better because AI helped improve flow through something that mattered. A customer gets value sooner, a team learns faster, a decision improves, a bottleneck moves, or a costly delay shrinks. The organization becomes better at turning ideas, effort, and technology into outcomes.

That is a higher bar than “we are doing AI.” It is also a more useful bar. Because once you frame the work this way, the AI conversation changes. You stop asking only what tools people have. You stop obsessing over adoption numbers as if they are the destination. You stop assuming the easiest AI win is the most important one. You start asking where the system is stuck. And that is often where the real opportunity is.

If your AI effort has lots of activity and even more output, but the business still is not moving faster, do not start by blaming the tools or the people. Look for the constraint.