Clemens Adolphs on the Role of Agility in Getting AI Investments Right

Help me think through how this episode applies to my situation. Start by asking what I am trying to change. Separate the episode's claims from your suggestions, and say when the notes do not support a claim. Use the transcript to find passages, then check the audio before quoting. Transcript: https://yuvalyeret.com/podcast/episodes/clemens-adolphs-on-the-role-of-agility-in-getting-ai-investments-right/transcript.md ## Published episode notes In this episode of the Scaling With Agility podcast, host Yuval Yeret welcomes Clemens Adolphs, co-founder of AIce Labs, a company specializing in helping organizations successfully implement AI initiatives. Their conversation dives into the intersection of AI and agility, exploring how to avoid the all-too-common proof-of-concept trap and instead focus on delivering genuine outcomes. From internal market validation to adapting agile methods to the unique context of AI projects, this discussion is essential listening for leaders who want to scale AI initiatives without falling into process theater. Highlight Quotes / Concepts * Why most AI proofs of concept never scale, and what to do instead * Internal market validation as a critical success factor * Adapting agile practices to the unique uncertainty of AI * Recognizing and preventing common anti-patterns when combining AI and agility Chapters (01:28) Clemens Adolphs’ Background and Journey into AI Implementation(03:46) Challenges Organizations Face When Bringing AI into Production(09:02) The Role of Prototyping and Experimentation in AI Projects(11:09) Why Internal Market Validation Beats Metrics Theater(16:29) Collaboration Models and Execution Patterns for AI Work(23:10) Agile Practices and Anti-Patterns: What Works and What Doesn’t(38:13) Closing Remarks and How to Connect with Clemens Notable Quotes “The proof of concept is where AI projects often go to die. You need to design for internal market validation early on.” , Clemens Adolphs “Agile is not a one-size-fits-all recipe, especially when you’re dealing with the inherent uncertainty of AI.” , Yuval Yeret “Metrics are useful, but if they’re not tied to actual adoption or impact, you’re just performing success, not achieving it.” , Clemens Adolphs Guest’s Links and Resources * AIce Labs * Follow/Connect to Clemens Adolphs on LinkedIn Yuval Yuval Yeret helps leaders maximize outcomes through strategic, nuanced agility. These days, he’s focused on helping business leaders of mid-market/scaleup companies improve time to impact on strategic investments such as AI transformation. As a Product, Scaling, and Agility-focused management consultant, he’s worked with companies across tech, R&D, biotech - supporting both product/tech organizations as well as the broader business in delivering better outcomes using agility. * Developing your AI capabilities using Agility * Free Email Course: Scaling w/ Agility Crash Course * Follow Yuval on LinkedIn To hear more, visit yuvalyeret.substack.com Clemens Adolphs on Getting AI Investments Right – https://yuvalyeret.com/blog/clemens-adolphs-on-the-role-of-agility-in-getting-ai-investments-right ## Transcript This transcript was edited from automatic speech recognition for readability. Speaker turns may be wrong or absent, and it may contain recognition errors. Check the audio before quoting or attributing a passage. Episode: https://yuvalyeret.com/scaling-ai-podcast/clemens-adolphs-on-the-role-of-agility-in-getting-ai-investments-right/ ## The friction around safe / scrum Welcome to the Skilling with the Gility Podcast. I'm Yuval Yeret, here to help you explore the nuances of scaling with agility and applying agility to new frontiers. And that's what we're gonna, I think, talk about today. With me is Glemens Adolph, who's working on helping organizations look at AI the right way, leverage AI effectively. I'm gonna let Klemens introduce himself and what he's focused on. I'm looking forward to the conversation. I've been reading what Klemens has been writing about AI. He's looked at my stuff and I see a lot of interesting connections that we were looking forward to explore. Welcome Klemens. Yeah, thank you so much for the kind of introduction you are. Yeah, really enjoying your content as well. So yeah, very short background. I'm one of the two co-founders of ASLabs, and we have enterprises get AI initiatives done right from end to end, and not just have everything die in the proof of concept stage and gather dust there. And now, a lot of overlap there, and like a lot of the pitfalls that happen there, overlap with what agility would be about and tries to help with. So what I'm curious, what brought you to the point that you're helping organizations get AI right? What's your journey to doing this? Yeah, so in a way, straight out of grad school, I started as first as an intern and then individual contributor customer solutions lead at a company that was doing fairly advanced research development projects and very exciting work. And there I saw that it takes a lot of work and can I, apart from the core technical challenge, a lot of getting things right in the line to make sure that once you hand something over to a client, that they keep using it, that they're confident that it brings them value from the point that you deliver it and then forward. And not just, oh, here's this cool demo. And so in our case, it was quantum computing and you can use this algorithm to use this quantum computer to do stuff. But it needs to keep doing that. And not just once in the demo, in like, oh, we built this two-preter notebook in Python and nobody in the company can actually run that. That would be a good failure point. In a way, I've developed this passion of seeing things through and not just having it be a one-off. So that's interesting. I'm listening to you and I'm coming right off a conversation with, You might call it an initiative owner for trying to use AI in the context of customer success. I think one of the big CRN tools that you're probably familiar with and how can we leverage AI to answer more calls in the same hour, get to better customer satisfaction. I connected to your comment about getting things right, as well as your history. And I'm wondering what have you seen as the biggest risks, surprises, pitfalls in the projects or products that you've worked with? Would you say it's, can it even work? Like what you call feasibility? Is it around? Is it viable? Is the investment worth, like on an ongoing level, is the additional license we pay to the AI tools worth the time savings or revenue growth, whatever it is? Or is it will people actually use it? If we build it, will they come? The field of green is questioned your desirability. What do we see? Where have things failed? Probably all of the above and then in different mix of flavors. So I feel like, yeah, so basically with everything you ever build, there is that market risk, broadly, like will anybody actually you want it once you build it. And market risk doesn't necessarily need to mean, oh, you're building a consumer facing product and they have to buy it, but it could also just be an internal tool that you roll out and you want your people to use, but they don't use it. There's too much friction that just rather copy paste from chat GPT, then like click through your five layers of internal whatever. That's always there. And then through the nature of a lot of the AI initiatives and the uncertainty that is in there, the product risk is a bit or a lot higher depending on how ambitious your aim than it happened to be with more standard software. If it's like you have an idea for a new CRM because it has a different workflow that you think is better, you're like fairly certain you can't build it, but will anyone want to use your workflow? Whereas if you say, oh, I want to use AI that pulls out all these amazing insights out of my CRM or maybe you, let's say that's the fancy buzzword build an AI native CRM. So people don't even have to mess around with the entry and data entry so much. They, can you even do it? And that requires a bit more focused experimentation. And there's some small experiments you can do. But if those don't succeed, that doesn't mean it's not feasible. might just mean you have to work even harder to succeed. And so I mean, I'm tempted to, I'm tempted to say a I native CRM that doesn't require the salespeople or the Salesforce, the service force to enter information manually, that's great. everybody's gonna love that. But I've seen so often that even these obvious assumptions eventually fall flat. I'm wondering like is there, how often do we think things are really desirable, a field of dreams that people will really come and we jump to the technology feasibility. Even for great ideas, do we want to test them? And how do you test something like this without building it in the AI world? Do you have an example of how you help somebody do risk and AI investments recently? I mean, all the AI. Nothing like the R.O.C. concrete comes to mind in that market risk part, because let's say like the last big project that I already had some internal validation because they had a basic tool that did the thing and that had adoption. And so then if you're approving on something that already exists along the dimension that tool was being used, then that typically has very low market risk. It's like, oh, we were paying for this tool because it saves us five minutes. So then obviously they're going to, you know, obviously they're going to pay for a tool that saves them an hour. So that's, you know, one way to de-risk is go along dimensions where it's already proven that, you know, there is demand for improvement in that dimension and maybe where the way of engaging with it so it doesn't require complete different adoption, different ways of working. Just, you do that. And I think there's an interesting opportunity there, right? Which is, and I think we even exchanged some emails about this. The way the whole AI ecosystem is evolving, if you look at vibe coding or even things that aren't vibe coding. So there are so many opportunities to pre-to type, to create preliminary versions of things that work. They're not proving the feasibility from a technology perspective. You know, they're not necessarily gonna be production code ever, but almost like the new version of what we call the concierge MVP or the sort of other MVP that human in the behind the screen, the wizard behind the screen that is making things work, the new version is Cheche-Piti or Cloud or whatever it is with some, you know, some gum and glue and whatever we create some prompts and things work and we can prove some things very cheaply. And then if those things work, we can make it a more robust solution. Yeah, very much agree that for not the experimentation and prototyping and even before that, it empowers the person with the idea or removes some friction. You have the idea, you just flap something together really quickly and you don't have to go through the slog. Now I have to expect it out and then get my das and we have to build even a smallest row way prototype. So that's promising for accelerating your learning loop if you have a learning loop to begin with that is. ## What is really happening in the system But I think to go back to this language that we're both using of the market, this realization that you have an internal market and you need to treat it like a market where people have options where it's we're not developing systems of records most I mean even when we are developing systems of record how people use them like your CRM is a system of record but are people really going to use this feature or that feature are they going to do the minimum that they have to to get away with it, we need to apply a lot of the B2C product or B2B as well but the product management techniques, the product discovery techniques for the internal market as well. People have a choice to use the product that we're building for them internally. They also have a choice not to use these products. Yeah, you can mandate that they use it, but you can't mandate that they embrace it. And then they'll find their workarounds and back channels. And that should tell you something. And not everybody does their best. And so if they're not using your shiny awesome tool, maybe worth digging into that and treating it as, oh, we're getting a bad NPS and let's do some user interviews. Yeah, or we're not seeing usage or let's look at our R metrics, the pirate metrics, internally, like our people aware of R2. Are they activating? Are they starting? Once they're activating, are they actually using it? Do they continue using it? Are they paying for it is an interesting one because internally people don't typically pay for something? So that's one of the interesting ones to learn on how do you know that people are actually willing to invest in that tool and maybe it's enough that they invest their time as a replacement for, you know, they're willing to pay for it. Maybe there's a like a new version of paying for consumption. Maybe at some point we'll see people paying for AI tokens for usage. Are they really willing to pay for the eye consumption usage of their tooling of their department? But I think that might be interesting. I don't know what's What's the answer there? Yeah, what you measure, I guess, a sort of willingness to pay, which has been interesting in the developer space where famously developers are cheap when it comes to tooling. But with a code and cursor and that and that, everybody's shilling our money, even if it's not reimbursed by the employer because they don't see so much value in what the tools do for them. So that's an interesting thing. And I still think that business should pay for the tool, but it would be an interesting thought experiment, like, oh, if we took this tool away from you, how much would you pay to get it back? It makes your life easier. And then I'm sure a lot of people in the enterprise can think of tools where they would pay money to not have to use that every weekend. Yes, yes, that's, and that relates to the PMF survey question. How dissatisfied will it be if we took this tool away from it? If we took away the I-CAP abilities on your CRM, how unhappy will you be? Or will you actually be very glad? Like I know that from my space, the space of agility, a few people that are running agility transformations are willing, not to say, actually asking the teams that they work with, the groups that they're working with, are we a keeper? Like, would you fight to keep using the process, the operating system? And will you fight to keep having access to us as a internal consulting service? If you're an internal consultant, whether that's an AI consultant or an agile consultant, it would be very interesting to see what the people that you work with, what do they think? Are they willing to tolerate you? Are they gonna fight for your time? Are they gonna use the first opportunity to pro you on the bus because you're not useful? It might be a tough mirror to face, but, you know, it's better to know early on and have the transparency than be surprised when the budget for the neighboring function is cut, which happens very often. Makes sense. All right, I missed it. So, who are the people when you work with an organization? Who do you work with? Who's your partner inside? What kind of organizations do you work with? And who's your partner inside those organizations? What does that look like? Yeah, so technically, or not technically, it is like the person that would want to know to work with us is non-technical business owner or business function owner had in one way or another director of a department or if the company isn't that big just founder CTO if it's even a company with a CTO which you know these days pretty much any company is also a tech company in addition to what they do. But yeah, so those will be the truly good person we work with. And then they might have, technically people want not trained in AI machine learning, that can also work with us to basically design a product or an initiative come up with a plan what they actually want to do. Now we work on implementation, ongoing refinements and rollout testing on those parts. And how does that initiative typically look like? Like what's the what people from the organization does it involve? What are the sort of trust? Is there any... Maybe one caveat is that... So we as Lebs haven't been around for all that long, so it's not like I have a huge sample size, but both from the projects we have done and the ones where we're in the pipeline talking about what is involved is the business owner stakeholder of the entire initiative has some vision of where they think there are ways where AI should be able to help but they're not maybe they're not entirely sure because again with AI there's some uncertainty of it's a bit like so Andre, Andre Carpathy, great machine learning, and he calls it Swiss cheese model of AI where it's very good at something and then you slightly change the parameters now it's really bad, a bit bit of a tangent, but you don't know when it would actually do what you wanted to do until you try it in some fashion. So they have this vision and they feel like, you know, we're doing this thing and it's kind of like what we feel like, you know, even CHET-GPT could be good at, but we don't know, we need to figure it out. And so, you know, that's one point where they come in. And then of course we need to work with the people doing the actual work that needs to get enhanced, augmented, improved the subject matter experts. Imagine we're working with an insurance company than whoever is doing the contract review, something like that. We work with them to really understand. and now data expected behavior. So kind of the AI specific parts, but also how does this need to be deployed? Can it be on the cloud? Does it need to be mobile? Should it be a web app? What about this cloud versus that? Yeah, all these points that any software project has to do. And how do you run this software project? So we're small, it's just my co-founder and I, which these days you can actually get quite a lot done. If there are parts of the software engineering stack, like where we have some need for additional help, we can hire contractors from our network. So it's a project that you take and do separate and the people in the organization, or is it typically like you team up with people inside the organization? So we've watched it through the internal lines. How does it do? So the typical way would be, like we do the actual, like if it's a delivery project, then we do the actual delivery. because typically then the client organization wouldn't have the capacity capability to deliver it themselves, otherwise what would they hire us. ## The practical shift We've had a case where we did like an initial phase just what just does and then for the next phase the client added on extra support from a from a consulting company, not one of the big parts, but somewhere in that ballpark, probably all I can say there, but just in a way to kind of de-risk and feel good about it, but then we were still involved as the principal software engineer and machine learning engineers to provide their guidance. So I can also see, so that was kind of like a three way thing with the client and the consulting and the analyst, And I could also see that the client has a IT function, a tech team, they're just not, that's just not their bread and butter. And I guess that's a typical model, right? You bring in certain experts, whether that's cyber security, whether that's machine learning, generator, AI, those can then be somewhat smaller scale because we can just do the work and guidance and point people in the right direction and they figured out from there pretty much independently or well just with guidance. So in your interactions with these business owners and technology people in the organizations that you work with, what would you say have been the ingredients for successful projects the anti-patterns that kind of know where it's going, when those happen. Things to be careful. Yeah, so we haven't had anything, the kind of germs of anti-patterns hadn't had anything go horribly wrong at this point, but kind of both from previous work and seeing other projects tangentially involved on the sidelines. It's yeah what's the anti-pattern. So what works is if kind of the objectives and values are clear what we want to achieve but there is freedom and leverage and flexibility in the how. And so an anti-pattern that I would see is the typical for edge-eye waterfall thing where you decide ahead of time that this is going to be a six-month project because you want to show it to your stakeholders, to the board, to the CEO, to the industry conference, doesn't matter. And like a fixed that land is fine. But then because you also know exactly what you want, you map that all into user stories and you say, well, six months divided by this and that is that many sprints because this is how many sprints we have. And that means you need to put these many stories into these sprints. And then... And you have a gun chart without calling it a gun chart. Yeah, and it's exactly that. And the whole idea of estimates... What does it matter that we estimate something you told us? This needs to happen then. So, like it needs to happen then. Or velocity charts. What does it matter that we track velocity you told us? How many stories and which ones do you need to be done? Let me poke you on that for a second. So if I play, here's all I'm with you, but if I'm the business owner here or whoever is the sponsor, I want to have some sense of where this is going. And the problem that I'm facing, if I'm playing the devil's advocate for a second is, I'm gonna pay you or fund my people. It doesn't have to be an external vendor. and they're gonna go away and do their things and they're telling me, no, no estimates, no velocity. I'm gonna give you a black box or I'm gonna look like a black box. And I'm gonna do, you know, I'm gonna do sprints. I'm gonna do Kanban, whatever, but you won't really know until you know. That's like what I'm hearing I was just having a conversation yesterday with the CTO, who's CEO is struggling with the lack of delivery ownership, delivery culture. For there are in the organization but the same could be sort of project. And I think, or what I've seen, it's not just what I think. My experience is that there are ways to give this transparency that doesn't create that gun chart. It's not about the gun chart, it's about what is it that we know? We know we need to achieve a certain outcome. What does that outcome really look like? What are the hard points and soft points of the hard points? the use cases that we want to show, how many of those use cases are working already at each point in time? How do we think those use cases will spread over the six months that we're going to work on this, the two months that we're going to work on this, whatever, measuring or providing some sort of indication, how are we doing? Is this tracking in the right direction or not? Is useful, I've seen it being useful against scope creep. Like, when you don't have anything like this, it's tempting for people to add more and more use cases, to say, oh, this is cool. While we're at this, let's add this capability and that capability, because we don't know anyhow and this is going to finish. So let's just keep delivering value. But they are diminishing returns. Do we know is it still more valuable to deliver the base outcome after two months or to deliver with these additional use cases after three months? If we're not even having those conversations, we're just letting the team figure it out, which might be okay if they have the right context. If they know all of the parameters, what are we trying to do? we gave them an outcome function of the, you can invest as much as you'd like as long as each week that you work on this, delivers this reduction in F4 or whatever. If we don't do that, then they might be making decisions the wrong way. They might not be really optimizing for the North Star. So it's a nuanced, I think it's one of those nuanced things where you want some way to measure. Velocity is not the best one if you have to stick to velocity. Okay, fine. What I'm trying to do more and more with people is use outcome based leading indicators. No, we won't have a gun chart, we won't. We can track velocity, whatever we can do for a good whatever's the easiest. But More interesting, we will show you over time that output function, the key result that you want to achieve, we will show you evidence along the way how far are we along in getting there. That's the best way to structure projects, to structure products, but it's not trivial to structure them so that you can actually show progress on the real leading indicator every couple of weeks. Yeah, I think to reveal that trust and that transparency that I'm going to talk about, I think the stakeholder would be way more impressed by a demo that shows this does what we wanted to do. Oh, you know, last time we showed you it couldn't do this, now I can do that. That's way more interesting than just giving them a report that says last week we completed 37 story points. Yeah, I try to talk about the difference between velocity and traction. Velocity is output is activity. It's output. It's not activity activity as we spent, it's hours on this velocity is Outfit traction is this is moving the needle towards where we want to be. We're not just showing you working stuff, it's also working stuff that does what you want to do. At least a small part of it. Yeah and that of course then requires that the team is apart enough to say like hey this this use case you added or this this user story you threw in there. are we sure it is going to lead to the outcome you want? Or is it going to change? And that's where the differentiation you made, I think you nailed it with the best practice of, tell us what you need, tell us what you want, what you really, really want, and we will figure out how. You don't have to tell us the user's story. The team that is gonna work on this is gonna slice it into the work that they need to do. Tell us what's the big story of what you're trying to achieve. ## What leaders should pay attention to What's that all? Yeah, and it's interesting that because recently there was this, I guess, development or pushback against even using user stories. I think that linear that issue trackers software, they have their whole philosophy, but system have written up online. And their point is, oh, right, issues don't write stories because you can't read the piece and see if you agree with it or not. but maybe that's the backlash against how they have just become a bit bastardized. And so another recent read was, I mean, it's not that recent as a book, but I just recently read it with Robert Martin's Clean Agile, where he gets it. User stories are supposed to be very simple, just as a bit of a placeholder for a conversation that needs to take place just in time when you actually work on the thing. But they have become this thing where they're in Jira and then like three pages long and there's 15 different acceptance criteria and it's already specked out by the product owner. And they're ready months in advance. Many months in advance. And which creates a lot of waste because by the time you get there, half of the ACs might not be applicable anymore anyway. So then I had to expect out and then the designer already made the design for like the fully functional thing in Figma. And so now you have the super complicated lots of taps, dashboards, prop ups, whatever. And now good luck doing nice modular vertical slicing on something like that. So then you say, Oh, my estimate for that is, I don't know, eight points. What comes up to eight? 13 points because everything is all mashed together into a giant story and then because of that you have your feature branch working on that story that lists for two weeks. Everything gets so... There's a lot to fix. A lot to fix in these tickets. But I guess, and maybe that's where that's maybe a good place to kind of converge. I guess what I'm hearing from you. And maybe that's just because that's my experience and what I want to hear. So you let me know. But what I'm hearing from you is these sort of projects, they're the classic place to be agile in how you develop them to think from a product perspective, think about an internal market, try things, see what works, focus on the outcomes, let the details emerge along the way. Yet, it's another place where the agile theater is either making things hard for people that are trying to use it, or as left such a distaste from using it in IT, that people are not using it for these projects. and are losing the benefits of the discipline of the right level of agility. And I'm with you on that and a lot of it comes down to... I was like, you were going to make a punchy quote of to say, these projects, they're the place to be agile, but they're not the place to do agile. I don't think there's any place to do agile, to be honest. It depends on what you mean by do as well. Yeah, in this case in particular, the principles very strongly still apply the principles of agile. And but if you take those principles and you apply them to a given context and use that to derive practices, those practices might make sense in that context. So you say, great, this is how we should do it. and then you port those to a different context, it doesn't work anymore and you have to go back to the principles. So an example that comes to mind here is that four standard software development, test-driven development is awesome. Small unit tests, red, green, refactor, really great, I wouldn't do it any other way. If you're working on an AI system, how do you even write a unit test? You know, that says like, Oh, if I ask Chet GPT to do this or like the GPT AI API, do this, it will do that. Like it doesn't work. But the principle of working in small steps, making sure those work and then integrating that into the behavior you want totally still applies. You just need to do it in a different way. And similarly, with a lot of the edge theater, it all came out of context where it makes sense, estimate this and start it. and things because the principalists, business owners want to know where things are going, we want to know if you're on track, or true, it just helps to just go back to those principles and apply them in their context, instead of just buying the off-the-shelf way of doing things, how it has been codified. and maybe that's also the criticism level that Scrum and adjunct coaching is, you're supposed to discover better ways of working, but actually the best way of working was discovery, you know, 20 years ago, and it's written in the Scrum Guide. That's a bit of... Listen, Clemens, I think you just earned your badge for being a friend of the podcast, friend of the no BS nuance filling with agility podcast that's that's exactly what what we believe in here I think that's a great a great place to to pause if people want to look you up to see what you're up to get more of your insights on how to do those AI projects rights where do they find you? Yeah, so the website is ASlabs A-I-C-E. Slabs.com. On there there's articles, there's the week daily newsletter themes. Which is great. I'm reading it and loving it every day. Thank you so much. Likewise. And yeah, that's about it. And then there you'll find my email too. if you want to ask more questions. I'll provide all of those details in the show notes. If you are interested in my emails, I'll launch the new crash course to scaling with agility, an email crash course for scaling with agility with some of the best insights from the space trying to think about principles, focus on outcomes, a lot of the good stuff that we talked about here and how to apply it, not just to product development, to other frontiers like AI. Thank you. Thank you, Clemens, for being here. Thank you. Listen to the conversation about how to do AI projects right. What does that look like? How does that relate to agility? For Clemens, I'm Yuval. Thank you for listening to this episode of of this Skilling with Agility podcast. See you the next time. ## Source boundary These are the published show notes from the podcast feed. They are a starting point for discussion, not a verbatim record of the conversation. The transcript is machine-generated and may contain errors or unlabeled speakers. Check the audio before quoting anyone.