# You can 10x engineering and still not 10x the business: transcript Automatic transcript of the published podcast audio. Recognition errors are possible. Speakers are unlabeled; do not attribute a passage to Yuval or a guest without checking the audio. Episode: https://yuvalyeret.com/scaling-ai-podcast/you-can-10x-engineering-and-still-not-10x-the-business/ Source RSS GUID: cfd3dd99-a8d2-47fd-b57d-fa5b0cd37146 Source: published Riverside RSS audio enclosure Transcription: faster-whisper base.en, English ## Transcript [00:00:00] AI agents are getting very good at coding, debugging, test [00:00:03] generation, documentation, a lot of activities. [00:00:07] That's real in many organizations. [00:00:09] But what we're seeing is that if engineering was not the [00:00:14] constraint to begin with or now stops the the constraint [00:00:17] because of these improvements, faster engineering [00:00:21] output, faster engineering output may just create more work [00:00:25] waiting somewhere else. [00:00:27] The leadership question is changing from how do we make [00:00:30] teams faster to where does the speed actually improve the [00:00:34] flow to volume? [00:00:35] Welcome to scaling AI from activity to impact. [00:00:39] And today we'll talk about how AI is shifting our [00:00:43] volume. If you look at most AI programs out there, whether [00:00:47] it's within engineering organizations or beyond, they're [00:00:50] still being managed as local productivity. [00:00:54] Leaders are asking their teams or driving their teams to use AI [00:00:59] to leverage it. [00:01:00] We're seeing a lot of activity. [00:01:02] We're giving people the tools that I can save people time. [00:01:05] People can produce more. [00:01:07] And that's a great start. [00:01:09] But it's not really driving the results that that we want to [00:01:13] see the promise that AI brings to organizations is that I [00:01:19] should help human scale. [00:01:20] Organizations should achieve much more with the same amount of [00:01:24] people. [00:01:25] We want to improve the profit, Tb.per person. [00:01:30] We want to see people spending less time on toil and more time [00:01:35] in the flow and the translation from people having less waste [00:01:42] and more leverage should be that every time we see a person use [00:01:49] AI, it drives impact through the fact that these people have more [00:01:54] time to provide judgment, creativity, solve problems. [00:01:59] That our way for AI starts with what you can stop wasting time. [00:02:04] But the reality is that AI is asymmetric. [00:02:10] It's sort of a jagged impact on different areas of the [00:02:13] organization. [00:02:14] One clear example of that is of course coding or engineering. [00:02:18] AI coding assistants are great at coding because a lot of the big [00:02:24] part of the data set that they have is code that's available through [00:02:29] open source and also because code is a very deterministic activity. [00:02:35] Even if you're not sure how to code something, you can try many [00:02:40] times and there's a very concrete target function. [00:02:44] So AI is great at improving engineering output. [00:02:48] But when you look at the end to end life cycle of development and [00:02:53] product, you see that this impact doesn't really repeat at least [00:02:59] not now in other areas of the life cycle. [00:03:03] And what this shows up at as the full engineering speed is improved [00:03:11] significantly. [00:03:12] You can argue it's 10x, x-fract, x-c depends on the environment. [00:03:17] The other aspects that are needed in order to build awesome products [00:03:22] and get to the point that our customers or users, whether internal [00:03:26] or external, are actually using them, are not moving as fast. [00:03:34] They're not moving as fast because AI is not that great at [00:03:39] helping these organizations and also because it is harder. [00:03:42] It is harder to get AI to help us with product discovery. [00:03:47] Or with product adoption. [00:03:50] The observability feedback loop is harder. [00:03:53] There are harder things. [00:03:54] So a lot of organizations are just letting people improve in the areas [00:03:58] where it's easy for them to improve. [00:04:01] And that creates local productivity, which is simply not enough. [00:04:07] You can 10x engineering. [00:04:10] But now if engineering is no longer the constraint that output [00:04:15] will create congestion downstream, or it might create an environment [00:04:19] where the engineering organization is starving. [00:04:22] It doesn't have what to work on. [00:04:25] In the short term, that's maybe okay, because they probably accumulated technical [00:04:30] debt and gaps in test automation that they can spend quite a while addressing. [00:04:38] So I wouldn't be surprised if in your organization, you're not fully seeing [00:04:44] the impact of this starvation bottleneck yet. [00:04:48] One of the other advantages of working on technical debt is that validating [00:04:55] technical debt work is much easier than validating new functionality, new features, [00:05:01] because you're not really changing the behavior of the system. [00:05:04] All you're doing is changing something internal that is observable internally. [00:05:10] Or even if you're fixing quality issues and we know many teams have quality gaps, [00:05:17] that's also relatively easy to validate that the system is meeting a quality, [00:05:22] a quality acceptance data. [00:05:25] Validating value and outcomes is much harder. [00:05:30] So you might be already seeing the congestion or you might be seeing a situation [00:05:35] where that congestion is hidden, it's developing, or is avoided at least [00:05:42] temporarily because the engineering organization is able to leverage their [00:05:47] improvements to catching up on things that they've been struggling with for years. [00:05:53] In any case, at a certain point in time, if your engineering organization is growing [00:05:59] fast enough, they will stop being the constraint. [00:06:04] They will stop being the bottleneck. [00:06:06] The congestion will move elsewhere. [00:06:08] One of the useful things you might do is start to visualize this, start to pay attention [00:06:14] to where is inventory accumulating? [00:06:18] Where do we start to see starvation? [00:06:21] The classic way to look at that is to create a value stream flow for the entire [00:06:27] lifecycle of features, products, ideas that you're developing in your organization, [00:06:35] in your product or technology organization. [00:06:39] And just start to notice what is going on. [00:06:43] That's, by the way, true whether you're using AI codices or spec-driven development [00:06:48] or not, that's something you should be doing already. [00:06:51] But it's even more important to pay attention to this end-to-end flow and [00:06:55] value stream at this point in time. [00:06:59] One of the things that you will maybe start to notice is that things that, for example, [00:07:05] where explored already and built are starting to accumulate either before shipping [00:07:11] to production, depending on how automated that step is for your organization. [00:07:17] Or maybe they're already shipped. [00:07:18] You have continuous deployment. [00:07:20] The dorometrics look good. [00:07:22] But when you look at, is this really adopted, used, creating value for people? [00:07:29] It's either not really used or you don't really know. [00:07:33] And that's an observability gap that, first of all, affects your ability to say, [00:07:41] are we getting impact for this? [00:07:42] Are we not getting impact? [00:07:45] But it also has another effect, which is your AI agents. [00:07:51] If they cannot know whether what they're building is being enjoyed, [00:07:56] they cannot really close feedback loops. [00:07:58] So that's even a worse situation. [00:08:01] So figuring out a way to know whether anything that you're building, is it really [00:08:07] creating the impact that you expected from it or the outcomes that you expected [00:08:11] from it that are connected eventually to business impact is a huge, usually [00:08:17] important to move, to get away from AI activity towards AI. [00:08:22] One of the tools I really like to use for this is Accumulative Flow Diagram. [00:08:26] Accumulative Flow Diagram, if you can imagine it, is over time how much work is [00:08:33] in each stage of the life cycle, your product development life cycle. [00:08:39] And what you will see in these cases is that while there's good flow of work [00:08:46] into the system, there's a certain point in which there's a stage which just piles, [00:08:54] piles, piles work onto the next pile over time. [00:08:57] You'll see that working tested software, for example, is growing 10 times faster than it used to. [00:09:06] But that doesn't automatically result into improvements in the next stage. [00:09:12] It's actually opening a gap between working tested software and shipped and then enjoyed [00:09:19] impact software. And that's a good sign that the bottleneck has moved and you need to do [00:09:26] something about it. Now, what do you do if you want to, let's say you recognized that your AI activity [00:09:34] is maybe turning into an output that's great progress, but you want to shift towards AI impact. [00:09:40] Being aware is the first step, it's an important first step. [00:09:44] But then what do you do? You use this view of the end-to-end flow [00:09:50] to start to focus on the bottleneck. So now that you found the bottleneck, you can start to apply [00:09:58] principles and frameworks like the theory of constraints, which basically says you want to [00:10:04] subordinate the focus and the capabilities of both humans as well as agents around AI [00:10:14] to the current bottleneck. So if the bottleneck, for example, is adoption of features that you're [00:10:20] building or training users on them, then what you should be focusing your AI efforts on is not [00:10:28] accelerating coding, but thinking about how can we use AI to drive adoption? How can we use AI [00:10:38] to make it easier for people to use our features? At the minimum, make sure that the features that [00:10:45] you're building are very easy to use, that they don't require a lot of training and change management [00:10:53] effort, that maybe you're using AI itself as part of the feature in order to train and [00:11:03] onboard and activate usage for these features. Focusing on activating users and getting to the [00:11:10] point that people actually use features is something product management has been focusing for a while, [00:11:17] especially when we talk about product-led growth. It's not as common when you look at internal [00:11:23] systems and internal capabilities, but it's as important in these stages because the challenge [00:11:30] of adoption of systems and features is an issue, whether it is your B2B SaaS or whether you're [00:11:38] enterprise software or internal systems, or even if you're building AI companies, even if you're [00:11:47] building capabilities that are delivered via MCPs in skills, there's the question of [00:11:56] are people aware of them? Are people using them? Even if you're, for example, an internal [00:12:01] organization creating skills for users in the organization and sharing them, are people actually [00:12:08] using that? Are people actually leveraging what you're building for them? So that's the question [00:12:14] that you want to ask. You want to ask, where is our bottleneck and how can we help the [00:12:20] bottleneck scale? One interesting way to do that is to give all of this context about the [00:12:27] bottleneck, about what's flowing, give all of this information to your favorite AI, and [00:12:34] assuming it's a smart one, ask it to come up with ideas. Don't just tell it what to automate and [00:12:42] what agents to create, ask it, what do you think? What's going on here? What are some of the [00:12:49] interventions that we might try to elevate the constraint? Let's go back to the premise for [00:12:57] for AI. So if the high level premise of AI is, and I think it's a good one, I think it's a positive one, [00:13:03] to help people move from toil to genius, to help move people from constant multitasking and routine [00:13:13] work to flow mode where they can focus on creative work, have the autonomy for it, mastering new things, [00:13:21] and you know, field purpose about what they're doing. That's true, but we need to apply even this [00:13:30] asymmetrically, not that it's less important for some people than others, but if we want to get [00:13:37] impact in our organization, there are people that we should prioritize doing this with. And those [00:13:44] are the people that are currently in our bottleneck. So for example, if engineers are not the bottleneck, [00:13:51] maybe they're not the first people that we should focus on with this. Let's focus on the people that [00:13:57] that are in this value, maybe it's product people, product professionals, maybe it's people in operations, [00:14:06] maybe it's our users, I don't know, we need to look at that. Don't just help everybody scale, [00:14:12] focus on helping the constraint scale, you'll get to everybody at some point. Now that doesn't mean [00:14:18] you shouldn't give AI tools to everybody, you should, but when you look at your enablement activities, [00:14:25] your focus, your guidance, who do you emphasize supporting, this should be one of the criteria? [00:14:33] And the result when you're doing that is that once you start to actively manage the bottleneck, [00:14:41] work around it, subordinate other areas around the bottleneck to help the [00:14:46] bottleneck be more effective, whether it's improving the quality of stuff that feeds into the bottleneck [00:14:53] or using spare capacity in other areas to automate things that the bottleneck [00:14:59] that needs to happen in the area of the bottleneck would improve and that would create [00:15:05] better flow, better throughput throughout your entire pipeline, whether that's by the way the [00:15:10] engineering pipeline or any important value stream in your organization. And at that point, you're [00:15:17] shifting from just AI activity anywhere in the cycle to actual impact, to actual value, [00:15:24] you're seeing overall throughput of value improve. And by the way, at that point, there would still [00:15:31] be a bottleneck. It might be in the same place, it might have moved elsewhere, so you need to go [00:15:37] through this cycle, continuous. As a bottom line, AI by itself doesn't create new value necessarily. [00:15:45] It can create new speed, but that speed only matters when it improves end-to-end flow and [00:15:51] achieves impact. And that's not automatic. That's something that you need to focus on. You need to [00:15:59] use flow thinking, flow metrics, end-to-end systemic thinking to get to the point that your AI investments [00:16:08] are actually the winners in the race to shift from AI activity to bottom line impact for the [00:16:16] organization will be these companies and leaders that pay attention to how AI has moved their bottleneck [00:16:25] and focus their best human and AI investments and efforts in these areas. If this episode helped [00:16:34] you see the gap between AI activity and impact a little bit more clearly, [00:16:41] it would be great if you can think of one colleague or friend or leader who you think this [00:16:48] or you think would find this useful as well. This is killing AI from activity to impact, [00:16:55] where we look at different ways to go beyond the tools, pilots, and activity theater and ask [00:17:03] what actually changes the flow of value through an organization. I'm Yvaly Eric, thanks for listening. [00:17:11] All right, so that's the thread I wanted to pull on today. As always, the question I'm focused on [00:17:18] is not just are we doing more AI because we are and we should. It's easy helping us create more impact. [00:17:26] If this resonates, please subscribe or even send the episode to someone who's trying to move [00:17:34] from pilots tools and activity towards real value. I'm Yvaly Eric, this is killing AI from activity to impact.