Vibe, Buy, or Automate? Deciding How to Build AI Agents

[00:00:00] Isaac Sacolick: Greetings everyone. Welcome to this week's coffee with digital trailblazers. I'm sorry for the late start. Had a bit of a technical difficulty. And the good news is, unlike the last time this happened, which was about two years ago, I have figured out a workaround and what works to get LinkedIn and Zoom reconnected. So we are back. Again, sorry for the late start. Thank you for hanging on. We have a great topic here today, Vibe, buy or Automate, deciding how to build AI agents. I have a good friend of my family, Greg Hack, who is going to be our AI agent expert, technical expert today. And I'm going to give you a sense of why I wanted to talk about this. In fact, the reason I got things screwed up, I actually started the Zoom an hour early to Joe's My Witness. I started an hour early. I was just so excited to be talking about this one. And then of course I couldn't get Zoom to talk to LinkedIn, but here we are. Thank you for being patient. I want to walk through you this research slide so that you get a sense of why I wanted to talk about this. And the reason is that if we had this discussion, oh, I don't know, a year ago, you were just being introduced to the language around AI agents. And if you had this conversation, oh, about, I don't know, eight months ago, we'd be talking about how CIOs have to find ways to bring more of their AI experiments into, from, from experiments into production and figuring out how to get those ones that are in production to deliver business value over the Q2. I went to 10 different conferences just to see how the technology companies are launching from dozens to hundreds of available agents. And what you can see here on the left hand side after talking to some CIOs is they're starting to get to a point where they're going from dozens to hundreds and thousands of agents. And so now I'm scratching my head from, you know, where we were 10 months ago, where we were barely getting these things out into production, to now that we have a fleet of them running in production. And the question is, are we doing things in a governed, safe, smart way? Now what this really makes me think of what you could see in this chart and the research that goes with it. We have millions of spreadsheets in enterprises. It's not countable. When I asked and got some help around, how many proprietary applications does the average enterprise have? They have about 1700 applications that they're hosting either in the cloud or in their Data center, about 60% of them, they develop in house. Research from Zylo says that we have about 680 apps running as SaaS apps. About half of them are approved, half of them are in some form of shadow state. Again, research around that and what Gartner is saying is that within the next two years the average enterprise will have 150,000 AI agents. So if we don't figure out exactly how to manage these, which ones to release, which ones to build versus buy, we're going to get ourselves into some trouble. Now I've gotten a look at a couple of AI governance platforms over the last four to six weeks. I've also spoken to some of their end users and what's happening with some of these platforms is they're really extending the word agent to also mean applications and also mean automations. And the danger around that is because if you take something that's an application or an automation and automatically implement it with AI, chances are you're going to take something that could have been built faster, cheaper, using traditional technologies and end up in a situation where you're building it using AI and paying a lot more for it and taking something that could have been deterministic in terms of its behavior and made it judgment oriented with the risk of ambiguities and hallucinations driving those decisions. So what you're seeing on the right hand side, I got some help with Claude around this. A decision framework on the X axis you have use cases that are more deterministic on the left side and on the right side ones that require more judgment. The ones on the bottom requirements are things that you're more often likely to buy or use, pre built ones as SaaS agents and the ones on the top start getting into areas that you're going to build or customize. And so when we talk about when we are going to build. I'm sorry, when we're going to buy, we're going to use things like RPAs and simple integration platforms and agents built into our SaaS platforms to do things like root cause analysis and IT draft emails in our CRM deal with PDF invoices, kick off a workflow when the CRM finishes, a closed one is marked and then we start getting into custom work. When there's a lot of customized rules, you could see those on the left hand side around workflows, around billing and invoicing, around compliance reports on and off boarding, those are very much rule generated types of automations. Not sure you really need AI for them. And on the right hand Side you start getting into places where there's judgment we probably can't hard code rules around underwriting and pricing and fraud around construction, how we bid on different project types and higher ed, how we help students find their first year housing based on their personal preferences of who they want to live with and the types of housing they want. Every college is different and then a lot of companies that are either SaaS companies or operate platforms that are like SAS companies building agents into their front end experience. So I have a very high bar in terms of what we might build as an AI agent. But inside platforms like Coply, Claude and, and inside ChatGPT, you're likely to be configuring and vibing an agent based on its capability. So that's our discussion today, right? Our discussion is going to get into first, when does it make sense to be building and second, how do we control the sprawl? I want to thank everybody who's been commenting on the on the stream here as I struggled to get today's coffee hour started as and then walked you through our research slide. I want to welcome Greg Hack. Greg is the head of a product development at a startup called Haystack. Greg, hold up to mid session and I'll ask you to talk us about Haystack a little bit. But Greg, I want to talk first, this first question around traditional automation versus AI agents. There's tokenomics costs around running AIs, there's rules versus judgment driven. I want to get your sense of when do you think it makes sense to be using AI to build a capability building an AI agent other than cost and this, this decision factor around rules and judgment, what are some of the other factors that suggest hey, this would be a good area to build an AI agent. Greg, welcome to the floor and say hello to our team here. [00:08:21] Greg Hacke: Good morning everyone. So this is an interesting conversation. I was actually at an AI governance conference at a law firm on Wednesday and after discussing the agentic process and agent driven AI and explaining to let's be fair lawyers on how AI works and what this really means, the number of hands that went up for people that were actually working or developing their own agentic solutions skyrocketed. It's there, it's there now. It's a, you know, the shadow AI structure is out. What we really need to do is figure out how to rein that in, how to manage that process and how to standardize on our structure. So we need to understand that it's happening and it's happening now. So how do we go Back to that and say, with that happening, how can we back up a step and say in our technology stack, what sources and store strategies do we need to approach based on that behavior of our solutions? Whether it's deterministic, whether it's a judgment call, whether it's probabilistic, and what is our true risk? What's going to happen when the risk authorities associated with our business are going to come in and say, hey, how did you come with this decision? How did you achieve this state? So before we get to the tokenization question, which is next, how can we ensure that what our team is doing meets the requirements that we have as a business? And risk is probably the biggest piece people move past, they think, well, there was a comment, Tony and my business came up with this great process and if it makes a decision, how do we know Tony's decision and his tool does what it really needs to be doing? What is the risk if it answers that question improperly? And how do we as a business own that when it happens? So it's happening. The first thing everyone needs to understand is people are doing this every single day right now. And if they're doing it, how can we manage that process in this space? How can we control the tokenization? And I think that's the first question. Tokenization is monetary. Right now, many people paying it out of pocket and they're doing it, they're doing it as we speak. So how can we step in and control that? I think that's the biggest space to start with. And I'll let you hand it over because I see a number of hands up. [00:10:47] Isaac Sacolick: Greg, that was great. I mean, what you're, I think, suggesting, if I'm going to paraphrase here is, you know, this question of should I build an AI agent? Is number one in everybody's hands okay? Number two, yes. There's a cost value equation. The tokenomics of it, does it make sense? You know, I introduced this notion of is this something that requires judgment versus rules? This is an implementation question. You're adding a third variable called risk. Right. Don't build an agent in an area that's high risk for the organization or that we can't validate the results very well. I think that's a good area for us to consider. Liz, I need another dimension here we can't repeat. So where do you want to go to repeat risk? [00:11:36] Liz Martinez: Obviously, I'm going to talk about value. I mean, the value. Business value is the ultimate test. And frankly, the value of piloting is still our best Way to make sure that this comes under control. Now we talked a little bit about the tokenization and the monetization problems that we're dealing with. And that is actually the perfect lever to keep control over the sprawl. If you actually make sure that people only have access to the budget with AI that is actually associated with pilots, you can actually see if that's money well spent. And then you can take a look if, okay, we did this pilot, we think that this is the business case, we think that this is the value we're going to get out of it. [00:12:23] Joanne Friedman: We. [00:12:24] Liz Martinez: But now we can go back to is it judgment based or is it deterministic, as you mentioned before? So we can actually do a whole entire pilot based on AI with a contained financial spend and then come back and say, is this really what we wanted to do? Is this really going to get us the value we needed? Can we do this in a deterministic way that is actually going to get us a better result and cheaper in the long run? [00:12:52] Isaac Sacolick: I think you're adding a separate dimension from cost value. I think you're adding a personal accountability to an employee saying you've got a hundred bucks to spend. Spend it wisely. Right? [00:13:05] Liz Martinez: The only lever we have right now, [00:13:07] Isaac Sacolick: I think that's really smart. I mean, having looked at some of the tools Microsoft has around copilots, I mean, essentially anybody who is using Office 365 can go prompt and start creating an agent out of something that they're doing. And sometimes that makes sense and sometimes it's just creating a lot of clutter. Right. Meaning that it's reporting it as an agent in the tools that our governance leaders will learn how to use. And now they have to sift through and decide which of these are going to be making sense that should be managed. It's. It's creating a little bit of a mess and going back to end users and saying, you know, spend your money wisely. I think that's a really smart suggestion. Hi, Joe, where are you going? We're trying to get more added to the list. Go horizontal for me. [00:14:00] Jow Puglisi: I'm gonna, I'm gonna say three things about whether I'm building an application or going down the agent path. First is can you really state what it is that we're trying to do Very concisely. Vibe coding can be inexpensive and on target if you know precisely what it is you're trying to do. But it can also be runaway cost when you know you're sort of exploring as you go the second Thing is, is the process stable or being invented? Right? Sort of. Along the same lines, if it's stable, you're going to automate. You know the rules, you know the inputs. Exceptions are rare. In that case, you know you're going to automate. Agents are good where there's judgment involved. As we've said, maybe the inputs aren't as clean. They're. There are some exceptions, decisions that need context. I'm sure Joanne will bring up the, the notion of context and, and data lineage and all of that. And the other thing is, and I'm not sure this has been mentioned, who owns this agent or swarm of agents when, when it starts to, to grow? Every agent needs three things. An owner, a cost line, and a kill switch. [00:15:24] Isaac Sacolick: Joe, like always, you're giving me more than I asked for. You've given me like a bunch of these. Right? Accountability, beyond just value can actually define what the intended outcome is and the success criteria. Does the data support the underlying hypothesis? Don't waste your time building agents if you don't have data to support it. Joanne, where are we going? We got a pretty good list going here. [00:15:52] Joanne Friedman: We're going to a very basic criteria. How do you determine if you need the agent, which is if the path is known, automate it. If the path has to be interpreted, adapted, discovered or judged. And that could be judgment on any number of levels because there's not just making a decision judgment, there's judgment about the access, the privilege, the data, the context, etc. So that's basically where I want to start. So if the path is known, just automate it. You don't get brownie points or extra, you know, extra tokens to vibe with by using an agent where you can use a simple API. Okay, anything that's repetitive, you don't need an agent for. Where the work of agency actually gets interesting and agency itself is. How are we giving an agent something that can act on our behalf as a proxy the right to do so? That's another question. But the, when it has, when it requires levels of judgment and types of judgment, different types, like trade off management is one kind of a judgment. When it has to interpret context, when the path changes consistently, is consistently meaning over a period of time or by step, by step, by step, that context is going to change. That's when agents are the better way to go. And that's where the tokenomics will back you up. When it crosses systems, functions, organizational groups or, or organizations. That's where agents have their best value. And also I would say when there's too many exceptions. If this, then that, or else it's too big to be hard coded, it's going to get lost, it's going to cause a problem to a system. So the problem is not just about a decision. The problem is what's standing between the signal and the action of the decision. That's where agents become very useful and their cost can be optimized. Depending on how you write that code. It can start as Vibe, but then it has to have some actual programmatics, some actual determinism and probabilism and stochastic values put into it. And then we have something that can be talked about as pilot ready and then over time and iteration ready for production. [00:18:41] Isaac Sacolick: I think you've given me a couple things here. I think the one I would say is, you know, we talk about rules. I mean, exceptions is a separate area of rules. And a lot of processes have this right, it's this judgment call that's baked into a legacy process that says we need to get somebody to look at this. Or this has, you know, 100 branches to it because it can go to a lot of different departments when you start seeing that level of complexity in your process flow. We covered this with the folks from Camunda a few weeks ago. This is time to re engineer and maybe AI is going to be your friend thinking about a process in terms of coordinating agents instead of coordinating handoffs. Now, Martin, before I bring you up, you know, some of the agents that I'm seeing are really low level stuff and I'm going to share you a couple of examples of things I worked on earlier this week. That magic quadrant that you saw was a dialogue back and forth with Claude. It did not create an agent, it was just doing work for me. But it will show up sometimes as a task inside Claude, depending on what phrase, how you phrase things. And same thing is happening in copilot. I have a copilot agent that when I finish an article will pull out the list of people I quoted from it. Very helpful. I don't have to go sift through it. A lot of the stuff we're seeing is using AI for low level tasks. So I didn't have to do that work myself anymore. So with that in mind, you know, I'm going to give people a budget. What is. How would we go back to people again and start talking about this? Makes sense for you to start building agents out to help you do your work. Martin, you're on mute. [00:20:44] Martin Davis: I was waiting for you to finish. You didn't Give an indication you actually finished up. Let me carry on. Yeah, so I, I think first of all what I'd like to kind of distinguish between is use of AI in a structured manner versus ad hoc and I'll call it shadow AI usage because I think we need to distinguish between the two because yeah, we know that in a more structured usage we're going to take account of what is good data that we can use, make sure that we have the risks properly managed, make sure we're not having hallucinations, make sure that we have the appropriate guide rails and guidelines in place, making sure we have kill Switch, other things like that. So I'm less worried about those kind of more corporate products and corporate projects like that because we're going to do them in a well thought out way as I'm sure Wayne Anderson will going to chip in on as well. Yeah, and I think that is less concern to me. I'm more concerned with the shadow AI side of things which is some guy has got access to Claude and decides to use whatever data and get their hands on and create something. Then he sells it to. Yeah, to his compatriots and they're all busy making a whole load of decisions based on something which may have flawed logic, may have flawed data feeding it and they could be making business decisions based on that as well and consuming a large amount of resources and tokens. So I'm more concerned with that whole shadow AI side of things. And like any other time we have had AI. Yeah, it's so not my lots of other times we've had shadow it. Sorry is what I meant to say. We have a lot of risk there for, for the organization. So I think then you get into the things of how do you educate people about good ways and bad ways to do things, how do you govern the usage, govern the access to the data. And there are tools out there that actually restrict the AI gateways to make sure that you're not getting into dangerous water. [00:23:11] Isaac Sacolick: Dangerous water is not binary anymore, Martin. I mean I'm going to build a small agent that does some analysis of my content that only I get to see that might work for my organization. It's personal. I'm using the tool of choice, whatever that is. And then the next day I'm writing a different AI that's taking content that's not been published yet and sending it off to a third party tool, maybe a third party tool that I actually have a license with. So it's again not ShadowAI but taking IP and exposing it before it's been published. Maybe there should be a policy against that and our end user doesn't know that. So we have a lot of shades of gray here just around AI and data governance. [00:24:05] Martin Davis: Definitely. But all I'm saying is, is that. Yeah. And I'm not usually one that goes all on the risk side of things. I think there's massive opportunities here. There is a lot of. And I use Claude all the time. [00:24:17] Isaac Sacolick: Yeah. [00:24:17] Martin Davis: Even when I'm just writing some PowerPoint decks, getting Claude to format them. Getting Claude to reword things or help me to reword things. Yeah. Discussing how the deck should look for a certain audience. Even refreshing a deck with new content without to worry about making it fit, formatting it properly and everything else. Yeah. The tools are brilliant and save a massive amount of work, but it's when you start getting into some of that dangerous ground of where you, where you're using tools to help you make decisions or making decisions for you. [00:24:49] Isaac Sacolick: Very cool. Let's bring on John. John, you said when we were just getting started, you could probably speak for the whole session. [00:24:57] John Patrick Luethe: Yeah. [00:24:58] Isaac Sacolick: Now tell, you know, give a clue of what you're doing now because this is your job now, right? [00:25:04] John Patrick Luethe: Yeah. Yeah. You know, there's, there's, there's a few. But I've, I've really been working on, on AI coaching and enterprise setting and this is so near. And, and then in the evenings I talk to a lot of my friends that are at companies and, and I'm, I'm starting to see everyone starting to put, look at deploying agents in their, in their environment, for personal use, for team use, for. And, and they're, they're, they're going to be put almost everywhere because the companies are starting to buy people, you know, Codex licenses, Claude code licenses, GitHub co pilot licenses, and that gives people ability to deploy agents. Amazon's released AWS Agent Core makes it super easy to host agents and they have bedrock. And Vercel has made it super easy to deploy things in general. Their platform's so easy to use and, and they've made, they have a new agent platform that, that I've used and I really, I really see that. You know, there's, there's also a new product that came out a couple weeks ago that was actually called the Classifier, which is Jev. And I see the future is really like a hybrid of all these things. It's really hard for me to believe that almost everything going out in the future won't have a combination of anything that's deterministic. You use code to run so you get a known result. But maybe before that you have a classifier like Jev to try to see which way do you route this thing and if it's deterministic, send it to the code or if it needs human judgment, send it to the human for the superhuman exception cases. If it's an exception, but maybe it could be handled by the language model, send it over to the language model for processing. Maybe drop a ticket or something for somebody to monitor it too. The future I really see on this is almost everything some type of hybrid where the people are really sophisticated, are using all the tools and there's a classifier that determines which tool to use at what time and an exception processes. So if something's not working with exception process, it can kick it out. [00:27:15] Isaac Sacolick: Interesting. John, I'm going to bring you back for our next discussion too. Folks, welcome to this week's coffee with Digital Trailblazers. We meet here every week to talk about things facing transformation leaders in their organization. And in 2026, about three out of every four topics has been AI oriented. This one's really getting into the details around AI agents should you vibe, buy or automate deciding how to build your AI agents as we're going from dozens to thousands of them. Our upcoming coffee hours on the 9th, we'll be talking about change management, YAI adoptions and how to measure progress. Sandy McCarran will be back to talk to us about that. On the 16th we'll be talking about AI infrastructure depth strategies for capacity, resiliency and modernization. I have a leader from Hitachi Ventera joining us to guide that conversation. On the 23rd, we'll be getting into the end of 2026, we'll discuss how to finish strong, how digital leaders are are closing out the year. And on the 30th, honestly, this is my favorite conversation that we'll have this month around AI agent strategy. When are we using human augmentation? When are we using human in the middle? And when are we debating agentic AI where we're really aiming for fully full automation. So our upcoming topics, if you'd like to be a speaker, suggest a topic or sponsor an episode, do reach out to me on LinkedIn. Greg, let's bring you back. Greg, first I'd love you to just share about 15 seconds about your startup, what you're doing, what problem you're trying to [00:29:11] Jow Puglisi: for. [00:29:12] Isaac Sacolick: And then I want to get into this other decision part, right? And here's how I'm going to sort of Use some terminology around it. What I call buying an AI agent essentially means it exists already in one of your tools that you already have licensed and you can just go out and use it. All right, so it's in your CRM, it's in your erp, it's in your data processing platform, it's in your infrastructure. They are all launching AI agents built on their data. Some have data fabrics to connect to third party data and you are buying essentially an AI agent they've developed for you. What I'm calling vibing is essentially a lot of what employees are doing. They are using a tool, natural language to come up with an agent that gets deployed as a reusable agent that they can go out and run. Okay? And sometimes those are built into a non dev platform, sometimes these are being built into dev platforms or what I'm calling build Build is being done inside Codex into and Claude code and pave and replit the ability to use lovable. I can just keep going on. These are platforms that are letting you use AI to build out AI applications, automations, agents. And you're clearly taking ownership of the underlying build, the construction, the life cycle of that process. And so, you know, when I think of ten hundred thousands of agents getting to 150,000, Greg, you know, I think organizations need to revisit their build Vibe versus buy rules before things get out of hand. So Greg, first tell us about your startup and tell us when are you building AI agents? What's the criteria? [00:31:11] Greg Hacke: So Haystack is a role based access controlled enterprise AI search tool. Our focus is on the other half of this side of this discussion. How do I allow my agents to get access to very atomic level of information to answer these questions? So when we look at this from our side, it's one of the questions that was brought up by Wayne Anderson, I believe, who said one of the key factors is or Amit, I think was Amit who said what does it know? What knowledge does it have? And that's part of what I refer to as the four questions, you know, and this is part of that Vibe build conversation. It is what kind of automation is required? Are we looking at something that is a script and we're presently running as a script, or are we looking at something that's a truly autonomous agent with no human interaction? Where, where on the spectrum of true automation are we? And the second question is why AI? A lot of people look at me funny when I ask this, but why AI? If you've got a, an RPA now that does this or you've got a process that runs what. Why is transitioning to AI other than a buzzword, what is it going to provide the organization? What benefit do we gain? And a lot of people move past that. They're like, well, you know, we've used an RPA to do this process for decades. Why would we want to transition? That's the second question. Why would we want to transition? Third on the build, buy, configure or hybrid, I put another one in there. So when we talk about buying Vibe or Build, part of that is also the hybrid. You know, that's where we've got the shadow AI structure that's coming up with steps now through Vibe. When do we step into that and say, you know what, this is something we need to leverage at a higher level. How do we reach into what they're developing and what they're using now and build better frameworks, a better guide, a better guideline around it and make that more of an enterprise solution and less of a one off. And that's where honestly, I get the most traction from business. They're like, wait, my, my employees now who are doing this in the shadows, if we help them bring it live, they'll be happier. The answer to that question is yes, 100%. And then the last question is the one that, that a lot of organizations come to is the last and I think is really the first. How much autonomy can we safely allow? When are our regulators going to bite us as a business? How do we prevent that fault? You don't want the FCC walking into your business. You don't want the state's attorney generals walking into your business to go through your stuff. You don't ever want to be in a position where you've got to deal with the lawyers on anything other than I signed another big contract. And those are really kind of the four questions we've got to step behind. Around, Shadow, around the buy Vibe and build. What are we going to gain and what are we going to lose? And I think a lot of people think, well, it's AI, so it's got to be better than rpa. And it doesn't. And it's a question we need to answer. I'll hand it over. [00:34:12] Isaac Sacolick: I like this notion of a hybrid, you know, because it, you know, brings into what's the entire solution that you're actually trying to construct. And this idea of, you know, If I have 30 people in my company building 30 agents that all roughly do the same thing, we know what that looks like. That looks like spreadsheets to me, you know, it looks like macros. It looks like something that has a life cycle that is going to be doing work inconsistently. And I'm going to let Liz, when it's her turn, comment on this because this is where I think this is going. I think it's looking like a free for all because we don't have mechanisms and tools that to properly say, you know, what do we need to discuss before you start building something? And when do we need to have a discussion? I'm going to let Joanne go into this. We're talking about buy Vibe, build Hybrid, you know, lots of different ways to solution your idea around an agent. And I'm trying to tease out when does it actually make sense to build something? [00:35:17] Joanne Friedman: Joanne, it makes sense. I'm so glad you asked because build is relevant to context. And one of the earlier comments that you made was about agents that are built into existing software or bolted on, which is actually more appropriate because the software existed before the AI was brought to the surface. My point is that context matters. And this also goes to when you build, when you buy, when you vibe, an idea will come up, it can be vibe coded, not a problem. As I said earlier, it needs to go through some form of an sdlc. To Greg's point, you don't want to get sued over it, et cetera, et cetera. The notions of hybrid as well speak to how many different aspects of a process can we look at and see if there's a correlation between what the agent is being built to do or vibed to do, or purchased to do. And if you think about business process, the it's not as linear as the input of one is the output of the next, or so forth and so on. It's not a linear thing. You know, if I use a manufacturing example, if I'm going to schedule, if I see something's wrong, I have an anomaly on a production line that can mean a lot of different things. It could be that a machine needs maintenance. Well, if that's the case, do we have spare parts if we don't have them in house, Can I get them from another factory owned by the same company? Do I have to go to the supplier? Do I have to find alternative suppliers because the one I normally use can't furnish me the spare part in time to matter? All of those kinds of things are when you start looking at agentic build, because what you can now do is look at the various elements of what are involved in a process, look at the various departments and what they need. And the same agent can actually start to choreograph not only the steps that are being used and the information that's being pulled from deterministic sources like a database, but it can solve all of the issues of information, meaning to give to a human for approval or to give to people on a shop floor to say, hey, this is not one little isolated step in a process. This has a lot of ramifications. So if we build it, can we make sure that everybody is kept up to speed, that we don't have a latency between insight and action? That's when build makes the most sense because it's going to optimize the cost and maximize the value that you're going to get from the agents. [00:38:17] Isaac Sacolick: You know, Joanne, as you're saying this, I'll let John comment on this. John, can I build an AI agent that says should you build this AI agent? Sounds like a search function that Greg [00:38:29] John Patrick Luethe: could talk about too, you know, and there's some great things for that in like wayfinding and, and in the spectrum and development things that really help you decide if you should choose to build something that'll take you through that logical exercise. And I was actually looking at doing that because my sister in law wanted me to build an app that gives you the voiceover narrations as you walk down the street of all the historical things. And so I was going through that process with one of the things that was really helping do product discovery on the spectrum and development. [00:39:00] Isaac Sacolick: So have you seen something like this? I mean I have not seen it deliberately built into platform that says, you know, here's the agent you requested. Before you go do that, maybe you should look at your friends over there. [00:39:13] John Patrick Luethe: Yeah, yeah. So there's this really neat thing that's been happening. A white paper came out for this two years ago in 2024 on AI delivery life cycle. And some of the big players in this like you know, a year ago GitHub release Spec kit which helps you when you want to build something, it really kind of gives a framework for the, the delivery life cycle for the agent so that when you ask it to build something it gets, you get what you want. But the problem with these was that they started when you actually wanted to build something. You've identified the thing and so then what's happening is they're starting to have a lot of more stuff that's earlier and upstream that's more for like should you build this? Or I have this idea. And so there's some really Great things. AI in their delivery lifecycle has something that's kind of like a feature identifier. In their Cairo platform, Matt Pocock has Wayfinder and Grill with Docs that really kind of help you explore your idea of wayfinders for the earlier things. And then just Anthropic with its new release and the newest ChatGPT, the, they've been trained on all these spec driven development processes and so they actually, a lot of this stuff is starting to be built natively into the models and so if you ask it those questions, it can really help you identify those things. And so like I was doing it with actually with Codex, not that it matters, but yeah, it really wasn't me. It actually looked at all the competitors, it looked at like the features. And so that was a fun weekend exercise. [00:40:44] Isaac Sacolick: I don't know, John, I'm going to give the trophy out to either Anthropic or OpenAI or Gemini or Microsoft that says, you know, we're going to put this governance step against our better interest. Right. Remember, they're in all favor of you burning tokens up to build the same thing 35 times. And instead of just saying, you know what, whether it's a spec or it's a vibe, you know, we're going to go search the corpus of the enterprise and say, you know what, you've built it already. You know, go use this or let me go contact the owner of this agent and see if they'll do an enhancement to it. [00:41:25] John Patrick Luethe: Yeah, that's, that's, that's the, the power of like when you can start putting like all your enterprise information in like a rag system, those are the types of searches you can do. And that's like before you go out and build this, the thing, when you're, when you're doing spec driven development, it like step one is, is to go find like all the existing information. It's, it's really neat what, what you can do with a really good language model and then some of the patterns that are out there for getting really good context. And with mcp there's connections to everything and there's tools that can help you ingest everything. And so there's so much neat stuff out there with a combination of vector databases, graph databases, MCP connections to all the different data sources in an enterprise. There's, it's just, it makes the information so much more accessible. And then when, when you tell it what you want to do, you can have the AI go and start to find the inconsistencies in the Documentations, the questions that you're going to have to clarify with a human. And so it's a whole, whole new world out there. And then the, the very last thing I was going to say is like Vibe coding to me is dead. That's the Vibe coding is horrible. Like I tried to build a financial analysis tool based off vibe coding and 150 releases later, like I didn't get where I wanted. I wrote all the requirements down with the system and I one shot it overnight. And so the future to me is really like trying to use tools that really structure the development and it's really starting with good requirements. And if you have good requirements, you can get a good outcome. [00:42:54] Isaac Sacolick: John we're going to bring back some of our other developers who have been on the program to talk about that. This notion that like I can either vibe an app or even specific driven an app ignores something that we know fundamental about software development, which is don't create a monolith, right? Break this down, the architecture down and solve this one chunk at a time. And somehow that's getting lost in this conversation that we can just put either a developer or a business user in front of a tool and they can start building apps out without deconstructing what the problem is. I, I just love this conversation. Joe we're intermingling a few concepts here. We got about 15 minutes left, so you can, Joe, Liz, Martin and everybody who else has follows. We could talk about how do we balance what, what employees are doing and whether or not they should build or vibe or buy something that's an AI agent. We can talk about build versus buy for the organization and which types of agents it does make sense to build, whether you're vibing or specking, but you're going to build something truly proprietary and scalable. Or we can talk about, you know, the CIO and the PMO and other roles in terms of controlling the chaos where you want to go? Joe? [00:44:20] Jow Puglisi: Well, when it comes to controlling chaos, that's, that's my belly button. I am the chaos to order guy. So I want to say something about this whole conversation. Isaac we have five or six of the finest minds on the panel here and another 25 or 30 people listening, all of whom are expressing a lot of really, really valid points and highlighting the fact that, oh my Lord, this is a complex problem. We as the leaders of technology have to be the department of Know K N o W. We have to be that guiding light to take a look at the potential uses that people are putting forward and to say, yes, AI makes sense in this context, or no, that's just a short program we can vibe, create and, and you know, it's well defined, it's a small task, it has a, a low risk, a small blast zone. Something goes wrong, we have the data that's necessary or oh my goodness, that's a rather neat, complex array of agents that will be collaborating on a particular project to create new value for the company. Let's really engineer the living daylights out of this thing and make sure the data are available, correct and up to date and we've got the right guardrails in so it doesn't fly off the handle on us and we've got validation, we've got the human involvement that's necessary. I mean we understand these things and I think on a case by case basis it's important to look at it and say, yeah, this makes sense to do it in this way. As you just pointed out, we can't just give these tools to the employees and say here, create value, save time. You know, you can give it to them in a sandbox and let them play to become familiar. But, but it's not going to generate the intended purpose. Let me give you two quick examples of something that I've personally done. I've used Chat, GPT and Claude and Perplexity and all those, all those tools, I don't remember which one it was, but I have a spreadsheet and I track the participants in a show, in a session that I do every Thursday morning. And as people sign into the Zoom session, I update the date they've last been on the, on the call I used to manually sort and add new attendees and it became rather tedious. Well, I'm not very good at writing Excel macros, but it wasn't a job to create an agent. I used AI to generate the macro that I inserted into Excel and now my Excel spreadsheet does the sorting and adding of a new participant by itself, which was, which was a great use of AI, not in the spreadsheet, but to generate the code I needed the deterministic code that I needed to make the spreadsheet behave the way it did. Contrast that to yesterday. I'm playing with muse, the new AI agent from Meta, and I gave it access to LinkedIn and I said, every week I post a template of an invitation to that very meeting and the only thing that's different is the date. And oh, by the way, if it's a holiday, we don't meet on, on that particular Thursday So for example, Thanksgiving, we don't have it, okay. And I said, muse, can you just post this on my behalf in LinkedIn on Monday, I'm sorry, on Friday in advance of the following Thursday. And it did it. And since I was playing with it yesterday, you know, it posted. [00:48:10] Isaac Sacolick: And [00:48:13] Jow Puglisi: when, when Friday today came around, it said, oh, I don't need to post it because I already posted it yesterday, but I will post it next week. It had that intelligence that I needed to do it properly. So just two quick examples and I'll turn it back to you. [00:48:29] Isaac Sacolick: Isaac, I think this question of department of no versus no is a very interesting one. Right. I can imagine what it would have looked like 20 years ago if we set up a communication and collaboration cadence so that we knew what spreadsheets people were creating before they became institutionalized as business processes. Liz, you herded cats your whole career. So I'm wondering which door you want to go to. Working with employees, deciding build buy on workflow agents or controlling the chaos. [00:49:13] Liz Martinez: So I, I'm going to go back to your comments about spreadsheets and the proliferation of spreadsheets and shadow AI. So a couple of things there. Number one, of course, because we have this sort of central location of the money that has to pay for AI, we have an opportunity to govern and actually allocate almost like, I don't know, like I give my kid an allowance, right, that every person gets so much money to do whatever shadow AI they want to do and not, not, not for anything. Shadow it has been, has long been the inventor of, or the, the place where you go to see where the need is for real. It, if, if something isn't getting done. So giving people that autonomy to work on a limited basis in a controlled way will actually generate a tremendous amount of value, even if it's repetitive. And then you can come through and say, okay, well we have our rag systems. Let's see what's repetitive. Let's see where there's a real need, let's see where we can actually build something that's more enterprise wide. And then of course, then you have to do all the change management around that to make sure people know about it and use it, et cetera, et cetera, and that it's robust. But what's interesting about this whole thing is, you know, people are worried so much about roles being lost. There's a whole new set of responsibilities that has to be done. And I would say that's a value realization office to take a look at the entire ecosystem to actually do the management associated with leveraging all this amazing brain power and thought that people are, they're out there playing with it. The whole idea that they're playing with a tool actually generates so much more value because they're happy to do it, you know, and when they're happy to do it, that's where you end up getting like the best value for a company. People are really interested. So it starts with the money, but the governance has to be right behind and those, those roles are going to be more and more valuable for figuring out how to manage it in a way where people want to do the work and want to do the investigation and the innovation and it's still not being replicated 1800 times. [00:51:41] Isaac Sacolick: Let's go on a. Martin. Martin, you could. How are you going to control the chaos? [00:51:47] Martin Davis: Well, I'm gonna, I'm gonna throw a couple of things out there. So the first thought is yeah, we did a fantastic job of controlling the chaos. When Excel became popular. You look at how many finance people, finance departments created whole applications in vba and a lot of which still exist today. So I'm not sure on some of this you really can control the chaos. You know, history has shown, shown that we're not very good at it. The other piece I would throw in there is it's got to be around the value and, and trying to actually control access to some of the tools in ways that allow you to focus on the value and allow people to be productive in their own work and do small stuff locally. But if you're getting into things that bigger things try and actually control it and focus on the value, which we already said about that. And then the last thing I was going to say, I know we're running out of time, so I was just making it quick was there is some interesting ways you can get value. So for a company I've been working with, I was looking at AP automation and yeah, that's an old topic. Accounts payable automation. It's been around for decades. The interesting thing is nowadays AP automation tools can do AI based invoice reception that it doesn't care where an invoice comes from and it can read an invoice quite happily, interpret the data on there and put it into your AP system quite happily. And it can use judgment calls, you set exceptions, et cetera, et cetera. Not really the point, but it's very, very good at that. You no longer have to do anything that takes a lot of setup. It's intelligently interpreting what's coming in. But Then on top of that it can also deal with queries coming from vendors. So a vendor says, yeah, here's my statement. It can check that statement quite easily and see the invoice, which one's been paid. It says I haven't had payment for this invoice yet. It can intelligently look at what's coming in and actually deal with that query without AP team having to get involved. So I think there is the kind of growing ability to automate a lot of these tasks and deal with them intelligently and only have a human in the loop when something that can't be easily answered from the. Yeah, the good source data that you have. [00:54:16] Isaac Sacolick: Thank you, Martin. We're going to go a few minutes over because I got a late start today. John, Joanne. And then we're going to end with Greg. Go ahead John. [00:54:25] John Patrick Luethe: If you want to control this, you got to get ahead of what the users are doing and putting out patterns for them and, and then you have to put out patterns for guardrails and stuff for the organization. You're going to have to have network security so they can't use any tool that they want and you're going to have to have data protection running on the laptops, purview or other tools. If you don't have that stuff, it's a wild west. [00:54:46] Isaac Sacolick: Thank you John. Go ahead Joanne. You're on mute. [00:54:54] Joanne Friedman: Yeah, thank you. I would say you also have to have the higher level questions not only about what value are you getting from the agents, but how are these agents and this whole operating paradigm going to move your business forward? What where are you relating the strategic outcome or the business outcome that you're trying to evolve to? From the point of view of now I have a new tool. That new tool is not constrained that to a particular monolithic software frame. It is a tool that goes beyond boundaries and can go cross organization as well in that regard. How am I using this to gain value in my ebtda value, top line, bottom line or even resilience and corporate worth. That's where a good starting place should be because that'll at least give you the impetus to look across functional units and across boundaries. [00:55:58] Isaac Sacolick: Yeah, I think Joanne, there's two kind of fundamental questions here. One is, you know, what is strategic? You know that, you know what's a part of a workflow or a business process or customer facing or this Osman on the comment stream, you know, involves intellectual property. We're going to put that in one bucket. And then what are the stuff where you need agents or assistants to do your personal work just to get things done. And that's potentially a very different category of work. Under both of those is this question of reuse. Right. I don't want to buy 100 different hammers. Right. I want to. [00:56:38] Jow Puglisi: Right. [00:56:39] Isaac Sacolick: And that's, you know, and that applies to both strategic work and it applies to personal work. And be careful about how many spreadsheets you allow in your system before we have better tools to manage it. I think we're going to have to bring back our AI governance platforms to talk about how we're going to enable reuse. And that's the second level question we're getting at. We're almost at. We're basically out of time. Greg, I want to bring you back in your last thoughts for today. So [00:57:09] Greg Hacke: much here I agree with across the board. But there's one question none of us have asked and this is going to lead into your AR governance. Who owns it? Who owns it? At the end of the day, our investors, our company and our business needs to define who owns this entire process. Who's going to keep their hands on the controls as we move forward. And it's something that everyone needs to know. Everyone in this decision channel needs to know where that buck stops. Obviously with the CEO level, but who's going to lead the charge. And that's something that CISOs of the world and CIOs of the world are currently struggling with today. [00:57:56] Isaac Sacolick: That is such a softball. Greg, I am currently writing an article that's exactly that. The 12 new responsibilities of C level leaders in this agentic orchestrated world. Going from dozens to thousands of AI agents. This is going to come out on Monday on drive.starcio.com I literally have roles for 12 responsibilities for 12 different leaders. It is not just the CIO's job. Don't tell me you're hiring a C, A I O and they're going to take care of everything for you. You can't just sit and punt and say it's the CEO's responsibility either. And I'm going to lay out at least one key responsibility for 12 C level roles in that. Go to drive.starcio.com on Monday or sign up for my newsletter at starcio.com driving digital and you'll get access to all of my content. I write about eight articles every single month. Really great comment stream here. I haven't been able to go through it all. I will. We will come out a framework that comes up with some of the ideas here about when to build by Vibe or hybrid now and how to avoid shadow AI. Just too much good stuff here. So thank you everybody for your participation this week. We'll be back next week to talk about change management and AI adoption. The 16th about AI infrastructure debt. We can't just keep launching AI and saying it's going into the cloud. We'll be talking about this idea of where your data has to live. On 23rd, we'll be talking about finishing strong, how digital leaders close out the year and on the 38th, AI agent strategy human augmentation versus in the middle visit drive.starcio.com coffee and from there you get a few things. Number one, you can go and sign up for any of these coffee hours. There's links to every single one of them to their LinkedIn pages. Number two, add a button on your calendar 11am Eastern time, put it on your calendar. A link to be able to go to every single one of these so you don't have to think it through. And number three, access to the recent episodes. If you missed one, you can go to drive.starcio.com Coffee and I put them up on there roughly Saturday or Sunday. Over the weekend, folks, everybody have a great weekend. Enjoy the baseball. Go Yankees. I'll see you all here next week.