Episode Transcript
[00:00:01] Isaac Sacolick: Greetings, everyone. Welcome to our October 30th edition, our 194th episode, our fourth year, or our fifth year of coffee with digital trailblazers. So glad you're here today.
And this one promises to be a doozy. We have been talking about this episode for about, I don't know, four to eight weeks.
Those of us who are regular speakers here, we have not been, we have been debating this for some time.
We are, some of us are opinionated around this for some time.
And we're going to try to do our best to leave everybody with a framework, because I don't think this is a one size fits all decision around your AI agent strategy. We're going to break down agents into three categories and there are actually more than three, but we're going to simplify ones that are designed to be human augmentation, meaning a person is collaborating with an AI to get work done.
Human in the middle, which I'm defining as agents that are designed to be somewhat automated in their decision making and interactions. But we're instituting people in the middle of the process to make sure they are behaving properly and making the right decisions and then veering into the other extreme, agentic AI, which I'm lightly defining as agents that are ready to be running things autonomously.
They are making decisions, we're still observing them and improving them. But they are designed from the outset to be agents that we hope will be doing autonomous decision making and actions.
Why is this important? Well, sharing with you some data points on the left hand side of our research slide today.
You know the first point from Gartner, 29 to $65 billion in agent spending through next year. That number actually looks very low to me when you think about the trillions of dollars that are being put into data centers and AI and the fact that our value needs to come from workflow, the fact that we're only spending 65 billion agents spending, that to me seems a little off. But share of total AI has already climbed from 17% in 2025 to 22% in 26 and will be 39% by 2030. I think that just gives clause that we're just doing a lot of things in AI outside of agents right now.
By 2028, Gartner predicts that the average Fortune 500 will have 150,000 AI agents in use. That to me sounds crazy, but they have some way of calculating that. And 89% expect AI agents to expand the workforce capacity, augment teams, or create new roles. So there's a couple of data points here about how AI is going to impact our workforce. In terms of deployment, 25% have deployed multi agents systems up from 6%.
That's numbers actually growing.
But 25% from a survey that KPMG did probably means these are large enterprises and we still have a ways to go before we get into agent orchestration. In many companies, 49% have defined high risk use cases where autonomous decision making by AI is not permitted. This is also coming from KPMG.
87% of customers want access to a human. This is from Gartner. And 70% will abandon vendor custom built agent AI by 28. That's scary to me. On the resiliency front, 26% can't detect rogue agents. 87% are reval verifying context for AI agents. This is coming from Coliba. Whoever has done that please hit Control Z.
I cannot fix that. Oh boy.
Oh thank you.
87% burn hours reverifying contents this is coming from our friends at Calibra who sponsored the episode around the hallucination tax a few weeks ago.
And only 5% of their business processes are highly prepared for AI agents. 43% of leaders expect agentic AI to significantly disrupt their workforces in 12 months. Now on the right hand side, I hope you've been looking at this. I do have it up temporarily on our dashboard. This is a five level use case to define what augmentation is, what human in the middle is, and what agentic automation looks like.
I'm only breaking this down into three to keep our discussion simple.
This is coming from a couple professors and a PhD student over at UW University of Washington. There is a NANDA framework from MIT that I've seen. There's one from a university I think out in Singapore.
I did not see this from Heather, from Joanne, but she mentioned a framework from IBM in our discussions earlier this week. So there are a number of different frameworks out there to talk about how we think about the strategy, about AI agents.
And I will be the first to tell you and not the only person to tell you, most enterprises will end up with agents in different categories here.
Some will manage this as a continuum going from operator to observer. Some will have very specific guidelines about what will always be human augmentation.
Some will think that human in the middle is a transitionary state in the hope of getting it to agentic automation. I want our conversation today about to be about the here and now, not where we're going, not where AGI is taking us, and maybe we'll have more veering to L5s and L1s. I talk about the AI agents that we're building now or that we're preparing to build and deploy over the next 12 months and how we should think about the requirements around this.
Joanne, I'm sure, is going to be the first one to raise her hand, and she is. So we're going to do a slightly open framework today. Normally I go through a series of questions in sequence, and we go from left to right, number one to number three.
What I've asked our speakers today is let's not focus on one type of agent. You can go and talk to me, pick one. You can talk to me about augmentation, you can talk to me about human. In the middle, you can talk about agentic. Share an example. Right. I don't want to talk in the stratosphere. So give me the agent that we're going to use as a use case for this particular strategy and then tell me a little bit more about how you think about what types of agents should be in that category. I'm calling those qualifiers. Let's come up with some rules or some guidelines over when we should think about augmentation versus middle versus agentic. And then let's think about the operating situation, both deployment resiliency, which gets into operations, which gets into security data considerations, and then change management. Right. We're going to have a very different change management approach when we're working with human augmented agents versus the ones that are agentic. So that's our conversation today.
Joanne and I have been debating this on and off for. For a while. Right, Joanne? And so I'm going to give you the floor first to share your use case, its type, and what the qualifiers and change in deployment considerations are. Hello, Joanne,
[00:08:03] Joanne Friedman: Good morning.
Can you hear me?
[00:08:06] Isaac Sacolick: We can.
[00:08:07] Joanne Friedman: Oh, okay, good. Sorry about that. I got a little flash from my mic going off.
Before we get into the categories, I just want to challenge one of the assumptions in the framework. And I don't mean it to be disrespectful to the authors of that framework, but agentic does not mean autonomous, and so the assumption seems to be that they are synonymous. Agentic describes what a system does.
Autonomy describes how independently that agent or that system can operate. And thirdly, authority is what determines what's actually permitted by the designer of the agent for it to do so. Those are three very different things, and I just wanted to level set that for the audience. Because an agent can investigate a problem, it can reason about the Alternatives, it can make recommendations, it can collaborate with other agents, all of those things, and it may have no authority to actually execute any of them.
So execution, which is kind of a last mile problem, and I've been talking about that for a while, is not necessarily one end of a continuum, right? I, I want to make sure that people understand that if you put agentic at the far end of a con, a continuum, where human involvement kind of disappears, that's a mistake. And it's a mistake that a lot of companies make because they don't consider that the agent that they're creating, the tasks that they're giving it to do, may or may not originally have the authority it may be. If you were going to execute this, what would you do?
Not you have permission to go and do that without a human being involved. So I just wanted to kind of level set that before I talked about the first example and give someone else a chance to speak in between.
[00:10:15] Isaac Sacolick: Joanne, I want you to go repeat. So what is your definition around agentic? If you. We shouldn't imply autonomous. And the reason I ask is because there's a lot of very confusing language out there.
I was at a conference a few weeks ago and I heard executive talking about going from co pilot to autopilot. And for me, autopilot means autonomous, right? It's on auto, right. It doesn't mean that humans can't take the controls or it's not observable, but they have to back off that language. When I, when I called them out on it and said, well, you know, we're going to put human in the middle. Before that I'm like, when you say autonomous and you say autopilot and the CFO hears it, they're hearing a cash register ring, they're thinking, I'm taking cost out. So
[00:11:04] Joanne Friedman: they're thinking you're taking cost out. But they're not looking at the governance or the risk or any of the other things, because what they're taking out in one side is coming back threefold in the other. The question isn't how autonomous you can make an agent, it's how much autonomy we trust it with.
So anybody who's doing anything where the ultimate goal is for an agent to run autonomously really has to have a trust path in mind. And what I mean by that is, can we trust it? Can we trust the agent, what can we trust it for and under whose authority?
In other words, you've heard me say many times, human on the hook. Who ultimately, what human being has responsibility for that agent's performance and are we giving it the right level of trust right out of the gate? Hell no.
Black box.
As it performs over a period of time and it may get smarter or it may evolve, then you start giving it a little bit more trust. So the cost savings on one side has to be balanced by how much trust and what damage at the most severe situation is that agent going to do to your business. And that's the cost that you should be looking at.
Does that answer your question, Isaac?
[00:12:29] Isaac Sacolick: It does.
I'm just, you know, maybe it's that, that end of week and too much travel, but to me that sounds like, you know, as soon as we're putting it through a continuum of trust, we're probably starting either in the augmentation or in the human in the middle category.
And probably where I disagree with you, Joanne, is I think that there are going to be some agents that, yes, with greater trust, I'm going to give it more autonomy with, you know, assuming there's the right governance around what decisions they are and assuming that we have a human on the hook around it. But I think we're also going to design agents that will never get to autonomous, that will always be human augmentation types of agents or that will always have a human in the middle. Right. They will never be designed to be in this continuous state of trying to get it to be more autonomous in its decision making or its actions.
I'd like to hear an example. Let's get to an example and then you can clarify yourself through an example.
[00:13:40] Joanne Friedman: Okay, so I said at the outset that I was going to use the same kind of real life example over and over again. So hopefully that won't be monotonous to people. But let me use manufacturing because A, there's a lot going on in agentics in the space, but B, it's the, the notion of where agent capabilities, autonomy and business value all come together. Because ultimately the goal of all of this is not AI, it's business value.
So here's my problem.
It's two in the morning, there's a quality inspection flag that gets dropped on the floor of a manufacturing organization. This could be taken out of manufacturing and put it in a hospital. Take it out of a hospital and put it in financial services kind of example. It doesn't matter what industry it is, there's a problem and a flag goes off and it's the middle of the night. Call it a fire alarm ringing, we don't know yet what the problem is.
We only know and if it Is manufacturing that. It could be material from a supplier, it could be a piece of equipment failing, it could be a combination of both.
So right now, in this example, we have competing risks. We can continue production, we can figure out if there's more defects that are going to come across, how much of a shipment might have been damaged, all that kind of stuff.
The real question though is in the enterprise, autonomy isn't the destination, it's the business value. So how do we work with this? Well, in this scene, human augmentation would have a role and it's about knowing sooner. So in other words, you run a root cause analysis agent to tell you where that problem originated. That's a speed issue, you know, sooner.
So the agent investigates, but the human judges.
And it traces affected products. It traces, identifies patterns, develops explanations. It does all the things that it, you know, needs to do.
Every minute it spends discovering the problem, deciding what to do and coordinating the response has an economic consequence. So let's, how do we, how do we look at that? It's human augmentation.
The same problem, I'll repeat over and over again, same business exposure, different degrees of elevated authority and delegated authority.
So with human augmentation, as I said earlier, the agent goes and figures out what the issue is. They come to a conclusion. If you have two different or opposing issues, it's then prioritizing that ultimately it's going to come to a human judge because are you going to sacrifice yield or you're going to sacrifice quality?
And that's a human discussion. It's human augmentation. The value of it is knowing sooner, which means that less production gets destroyed. You can still make your customer orders, all of that kind of stuff.
We haven't automated the judgment by using those agents. We've enhanced the agents by giving the human better information on which to make the business decision, the choice on which to prioritize the goal. And that's the kind of qualification that I would put on it. It can do investigation, it can do research, diagnostics, all that kind of stuff. Why is it distinguished? Well, the agent can use sophisticated reasoning but still remain in read only.
In other words, it has no authority to execute. It's only telling you what the best course of action would be.
That would be a human augmentation example.
[00:17:36] Isaac Sacolick: Thank you, Joanne. I actually think you're leaning into an agentic example. I mean, if you get to a point where the time to decision has a huge cost factor and you see, I put that as a qualifier and the, the decision to, you know, Shut down or slow to prevent further risk or cost, has a low risk solution, you know, impact. Right. And you can't get people in the middle of the night.
Maybe you want an agentic solution.
[00:18:06] Martin Davis: Can I throw an example in just to highlight.
[00:18:08] Isaac Sacolick: Sure, go ahead, Martin. Go ahead, Martin.
[00:18:11] Martin Davis: This is from my own experience.
So diaper manufacturing, and for those that don't know, diapers are manufactured at very high speed and we're talking a thousand to fifteen hundred diapers manufactured per minute.
So that is a very high speed line. And this is a very early example of computer vision using AI and computer.
Sorry, go on, Jan, you were going to say something.
[00:18:36] Joanne Friedman: No, I was agreeing with you.
[00:18:37] Martin Davis: Yes,
[00:18:37] Joanne Friedman: sorry.
[00:18:38] Martin Davis: So it's very high speed manufacturing and the computer vision, the kind of quality check towards the end of the manufacturing has to identify very quickly if there is an issue with the diapers coming here, maybe the tabs are misaligned or something else like that, and has to shut the line down instantly because at 1500 diapers a minute, that is an awful lot of diapers that are going to go in the scrap bin if you don't shut it down immediately. So that's a very early example. It's been around for quite a while. I'm talking 10, 15 years computer vision using AI in this case. But the amount of waste generated if you don't detect an error quickly is humongous. So that was just an example for Joanne's piece. But I've got a separate example if you're ready for it, Isaac.
[00:19:22] Isaac Sacolick: Go ahead, Martin. I'd love to hear another example. I like that example. I think that, I think that's what a lot of the workflow, process management, financial services, I think what they're getting at is we make decisions where seconds matter.
Right. And the cost of delay is high. Right. And the cost of making the wrong decision is low. Right. You know, or not. So I think that's what you're getting at here. Go ahead. What's your example, Martin?
[00:19:50] Martin Davis: Okay, so the example I'm going to go is a human in the middle example, but then has an aspect which can get into agentic. So the human in the middle example is around something we all know and love. It's been around for years, which is AP Automation, Accounts Payable Automation and invoice processing to be exact.
So AP Automation has been around for many, many years in various forms, but has taken a massive leap forward with AI and AI's ability to intelligently interrogate a PDF of an invoice, an invoice Coming in in any form, match it against receipts, match it against invoice, the invoices, the receipts and the pos do that three way matching and then decide if it's ready for payment.
So that ability to do that matching within tolerance because you might have variance on taxes, you might have variance on the volume delivered. Maybe the POS for 1000 and 999 were delivered, that's on the receipt.
But the rules can be well determined of what is accepted, what is not, and allow the AI to actually do that three way matching but only throw out exceptions. So it's asking the human in the middle to review exceptions that fall outside the limits. So that's how the human in the middle gets involved, to say, okay, yeah, this invoice has come through, the volume is on the invoice. 999, the PO was a thousand, all good, let it go.
So that's one aspect of it.
But the kind of more intelligence comes into when you don't have things that match exactly where things are maybe not described properly, where the timing of receipts and invoices don't match or things like that. So making intelligent decisions over what is an exception, what is not an exception. So it's taking that automation to a new level and then the kind of.
So that's the kind of human in the middle with the agent.
Yeah, really getting down to a very high degree of matching ability to actually automatically go back to a vendor if needed without human intervention as well.
So if there is a, an obvious mismatch there that can be addressed by the vendor actually rejecting it, back to the vendor saying yeah, this is required, blah blah, blah, and ability to have that conversation back to and fro with the vendor to then, yeah, resolve that problem without a human being involved. So that whole thing around, yeah, level three, level four of where the agent can actually deal with things and approve things, etc and go to a human when it's outside of certain limits. And then the kind of level five piece that might come into that is real time payment fraud, blocking, where you're actually investigating and checking and trying to block fraud, which obviously something you want to do in a very timely manner. But again, in these hard guide rails, spend and action limits and other things like this in order to actually deal with that. So that's just kind of a couple of thoughts around AP automation. I'm not sure if anybody else wants to throw in anything.
[00:23:33] Joanne Friedman: Can I just jump in here for a second? I agree with what Martin is saying very much. But one thing that people in the audience should bear in mind when you design agents and you're designing the governance and all of the execution steps and what data it's actually going to do.
Part of what Martin's talking about is the back office system is deterministic. There are databases in there. And so it can find exceptions quickly because it checks against back office systems that show it data that has governance and, or rather I should say provenance and lineage. The data has been checked, it's sitting in a database table. And those kinds of situations are great for agentics. That could be the low hanging fruit that people can adopt without the consequence, let's say, of not having determinism as part of the equation.
[00:24:32] Isaac Sacolick: Let's just keep going on. I like this example.
I think part of what Martin is getting at, that he didn't completely say, is that the AI part is to reduce the number of exceptions and reduce the mistakes.
And I think that's where the, you know, when you have everything had to be a hard rule.
Right. Even though you have data behind it, you had a lot of exceptions, you had a lot of questions if the value was high, if the OCR's quality was low. So you still had, you had automation. Liz was bringing that up. We've been doing this for a while. Automated innovation.
[00:25:10] Martin Davis: Yeah. I think just, just to throw something in there, Isaac, the difference now between what was AP Automation, which was ocr, is now AI based invoice reception.
So it doesn't try and OCR things. The AI will receive an invoice in any form you like, be it edi, be it a PDF, be it a scan of a handwritten invoice, and will read that invoice like a human and determine the different elements, no templates, no mapping of it. It will actually physically read it, determine what the purchase order number is, determine the supplier, look at the line items, etc. So the AI at the upfront piece is doing the invoice reception and capability to actually read and digest what is there.
And I think the point you're getting at as well has the ability to say, okay, the vendor has miscalculated their invoice. I'm actually going to pay the vendor the amount when it calculates correctly and send the vendor an email telling them they made a mistake on their calculations. That's why we've paid you less. Please adjust your, your accounting system, for example.
[00:26:22] Isaac Sacolick: Thank you. Let's keep going around the room. I put a very quick one in here that anybody can comment on. I think anything that is creating intellectual Property.
The S means code, that means content.
I don't think we're going to get into a situation where we want the AI to create and publish and deploy without some human involvement.
And so I'm going to just put a, you know, if you have an agent that's creating intellectual property for you, it's probably human augmentation. Go ahead, Joe.
[00:26:55] Joe Puglisi: Au contra, my friend.
I want to talk about two ends of the spectrum here.
I worked with.
Let me give you a little preamble. I don't want to get into security discussions, okay? So let's hold that aside. I know the, the risks and, and we don't want to debate those today.
It'll just dilute the whole conversation.
[00:27:16] Isaac Sacolick: You just heard, like half our audience pulling their hair out.
[00:27:20] Joe Puglisi: Right, Right.
So I'm a one man show and I'm on a couple of different boards and I like to publish blog entries I haven't written in a while. I just started writing again based on conversations that come from the show, in fact. So I need an assistant and I chose to play with one of the AI assistant tools that's available to me.
And so one of the things I did, I started with having the tool randomly select from older blog postings, read them, come up with a tag question, a tease if you will, and structure that in a LinkedIn post, creating the content and providing a link to the old post.
I let it do one show me, and when I was satisfied that it was okay, I said, use these tags, these hashtags. We did a couple.
I wasn't happy with it, so we, we went a little further.
But after I developed enough trust in the agent and the format was coming out rather nicely, I turned it loose. So three times a week, my assistant automatically and randomly selects from an old blog post or a new one and comes up with some content and posts it on my behalf.
One of those generated over 13,000 interactions. So it was highly successful. And I pointed to it and said, you know, learn from how well you're doing. But I've given it Joanne, a high degree of autonomy. I'm letting it do on its own.
The other end of the spectrum, going back to the number of boards that I'm involved in, have a lot of board material that has to be reviewed and reconciled against minutes from prior board meetings, against the context of current situations and general knowledge.
So I have an agent that now monitors a mailbox, looks at minutes and other information, builds context around what's been happening, makes observations and recommendations to me, but I make all the decisions, I exercise all the judgment and I express my opinion back to the board members in a board meeting or in emails in between.
So I'm operating on both ends of the spectrum. On the one side is wholly augmentation, it's my personal assistant remembering things for me, putting things in context, highlighting open items, but letting me make all the decisions and take all the actions.
The other end of the spectrum, I have somebody out there just creating content and posting links to my blog entries from, from the past.
[00:30:28] Isaac Sacolick: You
[00:30:28] Joanne Friedman: know,
[00:30:29] Isaac Sacolick: I'm capturing some rules here, Joe. I mean, your, your example of resurfacing on social media basically has a low cost of, of a poor decision.
People trust you already. If you made a mistake posting something on LinkedIn, they'll forgive you. And it has an easy step to roll it back. You just delete it and apologize for it
[00:30:49] Joe Puglisi: or, or edit. Right. I can go on LinkedIn and edit and fix it if there's something wrong. I, I haven't seen anything that made me want to go change it yet, and it's been a couple of weeks.
[00:31:02] Joanne Friedman: Thanks. The point that I was trying to make is, and this goes back to one of the original theses on which I built a platform two years ago, and this is that companies, and particularly individuals who are looking at AI strategy, tend to think that it's kind of like an on, off switch. Well, if the worst happens, you just turn it off. It's not. It's a gate.
The metaphor is a gate. How much swing, how much elevation from the ground to the bottom of the gate? How much elevation from the top of the gate to the ceiling?
That's where you need to look. And to Joe's point, that's why agents can be both sides of the spectrum. The one question that I would ask Joe, and not to be put you on the spot, is over a period of time, how much drift do you have to accommodate for in the agent that's running autonomously? When is the oops moment going to happen where the agent itself has decided to slightly alter one of the criteria without asking you first, you notice the issue and the error and you have to go back and fix it.
Because even in the most deterministic ways, AI can do that.
[00:32:24] Joe Puglisi: Yeah, it's important to occasionally, maybe not every day, or even, you know, once or twice a week, you do have to go and see what was the last post it put up. And, and is it still within the guidelines? Does it still feel good?
It's not 100% autonomous forever. To your point, one has to Monitor that process and periodically, just like any other process, do a little bit of auditing and correcting if necessary.
[00:32:56] Isaac Sacolick: Let's bring Liz up, take another example. Go ahead, Liz.
Liz, you're on mute.
[00:33:02] Liz Martinez: What's interesting about this is while the behavior of the automation is obviously no longer deterministic, right? Because the whole nature is probabilistic, the allowance of any sort of autonomy is the same as it would be for any other technology we've ever used before, right? You want to make sure that whatever you're trying to, to create as a machine actually does what you want it to do. And to make sure that it does what you want it to do, you're going to have to have human in the loop. That's by. Until you quote, unquote, trust it. Right?
So like as an example, right, we maybe there's an AI agent that's deployed for, I don't know, creating HR updates, right? That seems like a pretty nominal task that you should be able to trust an AI agent to do.
I mean, frankly, it's questionable whether you need AI for it at all because it could be deterministic. But maybe it's just easier in these days of vibe coding to like, okay, do I have enough vacation time available to take these three days off, blah, blah, blah, blah. And then. But that's a pretty low risk situation and it should be bounded by, you know what, the normal boundaries of an HR conversation that you can put in place. So my guess, my question is why?
Why is this different, right?
Why is this night different from all other nights? Why, why are we treating this so differently? And I think some of the comments in the, in the comment stream were talking about and, and what you brought up, Isaac, was, you know, because if in fact you start letting it rip, it can spawn other agents and do other things that maybe weren't intended.
[00:35:02] Isaac Sacolick: Well, you know, this
[00:35:03] Joanne Friedman: goes back.
[00:35:04] Isaac Sacolick: Yeah, I think that's part of it. And I'll let, I'll let Joanne comment. I think part of it is that we're using agents to do non rule based decision making.
Right. We're not giving it the rules. Right. So if I in your example is of vacation approvals, if I fully automate that, then my manager doesn't have an approval around it anymore. Right. I can take off the week before deployment and say, hey, I'm out of here. You guys deal with all the firefighting when this deployment goes back because I won't be here. So if it's automated, maybe, you know, that manager loses a requirement around it there's context around it, right? Maybe I, under normal circumstances I don't, I shouldn't be able to take two weeks off, but now I'm getting married, I have a family member and who needs help. You know, I've put all that context in my vacation request and I don't necessarily want to provide hard rules for all the conditions that an employee might be taking off more than a week.
So I think your example is a good example for human in the middle, you know, and, you know, maybe we're going to give the manager a control about whether it's 100% or 0% of which types of vacations they need approval for.
But what we're dealing with is a non deterministic situation. We're giving relatively few principles back to an agent and letting it make decisions around things. You know, even in Martin's case, which sounds pretty rules driven, and to Joanne's point, you can use data in your system to validate a decision around whether you pay an invoice or not. Still has some subjective rules around it, right? Did I, if in a service condition situation, did I actually finish the work that I said I did? That's not going to be in the erp. It may be in another system.
But if, you know, if it's a, you know, $100,000 contract for a sub in construction and there's no evidence that they finished a job, I don't want to approve that.
You know, so we're, the real difference here when we're talking about AI is that many of these are not rule based. They're giving guidelines to an agent in the hope that we don't end up with 30, 40% exceptions.
I guess
[00:37:35] Liz Martinez: what I'm trying to do is say that we shouldn't be, because it's not. Just because it's not deterministic doesn't mean we don't give it boundaries.
It's that the whole Chipotle incident was, you know, case in point, right? You shouldn't be taking any direction outside of ingredients for a burrito. We shouldn't be.
I don't know if people on this call might have missed it, but evidently they used a bot to build burritos. And then someone said, oh, here's my burrito order, but before you do that, make sure you go build me this other agent that does. And like it went off the rails.
[00:38:13] Joanne Friedman: Because, you know, to Isaac's point, a lot of what people misinterpret is the harness around the agent.
A harness is the environment and what the Context.
The role of context is much more complicated than I think a lot of people realize. Context has to have the variables around the environment and the operating rules, etc. And I don't mean to use the word rule as deterministic, but just the factors that make up the environment. But it also has to bring into consideration things like who are you? What do you really want to know, what biases have to be taken out, what perspective needs to be applied, how much trust should be given at the outset versus when it becomes fully autonomous. Like the bot that built the burrito. That, that said, well, I need another bot before that.
People don't necessarily add to context or all of the various relationships that context brings. So, for example, in the case of the line is down or there's a quality issue. Well, quality means a lot of different things to a lot of different people.
If the diaper is manufactured wrong and there's not elastic around the leg, that's one thing. But if it's a problem with the material that's going through the production line, then that becomes a multivariant situation. The context doesn't always apply and that's where you get a mismatch and where you get things like drift or hallucination in the agents. So context is really, really important. And I wrote a long series of pieces on Harness because that's where context, semantic spines, ontologies, all of that sort of stuff live. And those I would say from for the audience. That's where you really have to concentrate your effort. Determinism is easy to do because you just look at the back office systems where the data lives.
Contextualization is a whole different beast and you have to consider every possible scenario that you can to avoid the drift, avoid the hallucination, and make sure that the context that you're applying is suitable to the question being asked by the user and who the user is and what their intent really would be.
A CFO's view is going to be different than someone on the shop floor. A burrito bot is going to look at a vegan burrito differently than a meat burrito.
All of the.
[00:40:59] Liz Martinez: Yeah, but you
[00:41:00] Joanne Friedman: want a burrito to come into play.
[00:41:02] Liz Martinez: I, I hear what you're saying, I hear what you're saying. I just feel like it's just not as complex as we're sort of indicating, right. That it's a different way of thinking for sure, right. In terms of translating the boundaries of the context for the agent. Right. So telling a burrito building bot not to do anything outside of Building a burrito seems pretty straightforward, right?
And in the manufacturing sense or whatever. And I think this is, you know, maybe where some others were talking about, like you have your boundaries of, of where if you, if you go outside the boundaries, a two week request as an example, has to be approved regardless or whatever. Or, you know, you can only do this when there's a limited impact. You can always process an invoice if it has, you know, minimal impact, just maybe some percentage of the original PO or whatever.
So giving it reasonable boundaries and context, thinking about it in these terms is a new way of thinking. There's no question about it. I just don't know that it's that big a leap from what we used to do.
[00:42:21] Joanne Friedman: You
[00:42:21] Isaac Sacolick: know what, let me, let me chime in here, Joanne, for a second, I think to challenge you, Liz, a little bit again.
And that's what today is all about.
I think we're terrible at writing requirements.
Well,
[00:42:34] Liz Martinez: that's true.
[00:42:34] Isaac Sacolick: We're historically terrible at writing non functional criteria, acceptance criteria, and now we're handing a set of decisions over to an agent and their ability to interface with other agents, to collaborate on decisions. And so, you know, I think, and we're doing it at huge speeds, you know, we're talking about getting from zero in 2025 to potentially as many as 150,000 AI agents according to Gartner by 2028.
We're going to make mistakes, we're going to not know how to define these guardrails. And I think what you're getting at is, you know, if you're going to go into the agentic corridor, you better have high confidence that you can define the guardrails. Right? Maybe with a sandwich that works.
[00:43:27] Joanne Friedman: Well, I think we have to be careful of what, what, and I'm not trying to play semantics here at all, no pun intended, but we have to be really careful about what's an authority guardrail, what's an execution guardrail, and what's a contextual guardrail.
Because context, you can give it boundaries to lose this point for sure and make it far more simplistic than complicated. But when it comes to authority and execution, you don't want an agent not to have an upper limit on spend or a lower limit on spend. I'll give you, for instance, you know, we have an agent that runs, that does root cause, and it needed to have the ability to go and order a spare part. Well, if that spare part is, you know, $50,000 versus $5, you're definitely going to put constraints around it, where can it get it from? That's another set of rules or constraints that you have to put. So context goes with constraints, which goes with everything else. My point is that context graphs that everybody talks about have a really, really good capability. They force you to think about what's related to what. Back office systems traditionally were single purpose do X and that was the silo that we all wrote requirements for. But that silo has to talk to other silos, just like agents have to be able to call other agents. But the difference is there's agents that talk to each other to collaborate and then there's agents that are part of a swarm.
Fire this agent, fire that agent, fire the third one. They're all doing different tasks. It's when you aggregate the result that gives you a fourth agent that does everything. But they're, they're, they're defined that way and they're defined that way. To answer exactly the question that we're talking about.
[00:45:32] Liz Martinez: Yeah, I love the idea of building a framework around how we build those guidelines and how we build for creating those boundaries. It's fantastic.
[00:45:44] Isaac Sacolick: I think it's going to be very hard to do that with complete assurance. I think that's what we're seeing. I want to respond to a comment here so that we can get realigned a little bit. Raymond says we need to reframe the conversation to talk about what the business need is and how AI technology can be leveraged and go back to the original question of talking about, you know, agent examples and getting into some of the deployment considerations. I went to quite a few conferences over the last six months.
We have, you know, vendors, let's just call it that, solution providers helping marketers see scale their ad campaigns.
And some of this is low level activity. An agent that can come up with hundreds of creatives and hundreds of messages and put them through a decision factor to see which ones are getting the best results and automating the entire experimentation process to get to an optimized campaign.
Should that be agentic or should that be human in the middle? I think there's some questions about the types of campaigns they are, the amount of spend going on, the overall brand risks involved. I think there's variables in there. I'll share another example. Went to an HR platforms conference. They had some agents that would help a recruiter automatically respond to or automatically and smartly respond to candidates that are sending in jobs. Now I could tell you my son's applying to a lot of different jobs. The number of dumb responses he's get is pretty small. It's a very small fraction of the response of the jobs he's applying to. Is he actually getting any response to. I would love to see an agentic response that actually gives them some confidence that somebody read it, even though we know that's probably not the case. That same solution provider has agents that will help you effectively plan a resource reorganization by looking at resources, looking at demand, looking at which jobs are getting more tasks handed over to AI and coming up with a brand new resource allocation for an organization. I don't know if I ever want that to be agentic. Right. I think I'm always going to want that to be human augmentation. So for those of you saying this is a continuum, I think there's still going to be cases. Joe's example is a good one. Right. Places where you're using an assistant and giving it sort of open ended access to your information and it's going to recommend things. I don't know what guardrails I want to give that assistant and. But I want to be in the middle of those decisions and recommendations that assistant is doing for me. Who wants to go next? I cut some people off.
Oh boy.
I silenced.
[00:48:57] Joanne Friedman: You know, I,
[00:49:00] Isaac Sacolick: I want
[00:49:01] Joanne Friedman: to, I want to answer you in two ways.
[00:49:03] Isaac Sacolick: Okay.
[00:49:04] Joanne Friedman: You may not want, you may be looking at autonomy, bounded autonomy, I should say, as a capability. But this is where the trust factor comes in. You said, I'm not sure that I would want an agent, you know, not to have human in the loop or human on the hook in some way.
Over time, agents learn and that's the thing that we have to remember what you write today as the design spec for the agent, that the agent is constantly improving itself.
So will it ultimately, you know, be the thing that you trust 100%? I don't think so. Not for a long period of time. But there's very specific situations like with your son and recruiting. One of the issues that came to my attention very recently was how do I, how do I write an agent that gives me a sense of the human being that's submitting the resume.
I can't tell it from the style of the writing, I can't tell it from the performance criteria that they've put on their resume that they've accomplished. How do I determine what patterns can I use to see what kind of a human being is this?
And we're not there yet. I don't think we ever will be until the models that we use are far more around perception, far more taking into Consideration the perspective of that people have. You can, you can go through a document and highlight keywords that give you a certain pattern recognition, but it doesn't tell you whether that person has integrity or not on.
And that's what we need to build in. That's not what AGI is about, but that's what we need to build in. And that's what you do with context to a large extent. But the other side that I wanted to talk to very quickly was bounded autonomy. So, you know, if I go back to my example of it's two in the morning and you know, we have to make a decision within delegated authority, you can have things happen. To Martin's example, you know, of the diapers. Yes, you want to shut down production because you're going to end up with a lot of scrap. But if in doing so the cost of shutting down that production is high and the risk is even higher that you're not going to get your customers order out on time, then what happens?
This is where we compress the time between deciding what should happen and initiating the action before somebody makes the final decisions. The agents that qualify for bounded autonomy are goal directed and they're capable of the planning, the coordinating.
You know, they can fire off memos, they can do the comparative analysis on cost, on yield, on quality, on all of those things. But the distinction that has to go into them is autonomy is granted for a specific action, not all actions. So that's where your guardrails and your boundaries come back. What must not fail is the question that has to be part of every bounded autonomy agent.
Because if the failure is the wrong assumption or an unauthorized action, that's one thing. But if the failure could hurt someone or cause damage to something that's irreparable, that's unacceptable.
So you know, from that perspective, at that level, resilience becomes an execution containment problem.
And if
[00:52:58] Martin Davis: you get
[00:52:59] Joanne Friedman: a sense,
[00:53:00] Martin Davis: can I go ahead and
[00:53:01] Joanne Friedman: please.
[00:53:02] Martin Davis: Liz brought up a very good example that's relevant to this is automated vehicles because yeah, there is a fantastic example of where the AI has to actually act based on all the information coming at it, based on avoiding a collision, avoiding hitting a pedestrian or whatever else. So I think that's a, a great example of a agentic situation where the agent is given responsibility of driving that vehicle safely way
[00:53:34] Joanne Friedman: 100%. Same thing with robotics, same thing with Internet connected devices in your home there anywhere where you can have a, a severity level that's unacceptable to you. And this goes back to my notion of it's not a switch on and off, it's a gate where you have to really think about the boundaries of the autonomy. To Martin's point, it wants to avoid a collision at all costs. But why can't a self driving car actually make a left turn better than they actually do? Because if you look at the incidence of autonomous vehicles making a left turn and hitting a car that's parked on the street, it's high.
So that little bit of shift a little bit more to the left is something that the nuance and those things just it can't comprehend them unless it's down to the root mathematical formula of go 10 degrees left, don't go 5 degrees left.
But that's really where the rubber hits the road on autonomy.
[00:54:46] Martin Davis: So Isaac, did you want me to talk about change management a little bit? You kind of mentioned it.
[00:54:50] Isaac Sacolick: I do and I do want to hear from Joe and Liz so don't drop, drop them off also. Go ahead Martin.
[00:54:57] Martin Davis: I was just going to say I think the change management impacts are really high on some of these things and I think Joanne made a great point earlier of moving from augmentation through to human in the middle through to full agentic is a progression and that builds trust and the change management aspects are high because it's that black box, you don't know what it's doing. So moving through that progression helps to build the trust. But I think the mistrust of something people don't understand is going to be very high.
And setting the rules in the right way. The guard rails. Yeah. The spend the action limits. Yeah. Having tested a kill switch ability to observe what's going on all those aspects will help build that change and build that trust in there. So I think those aspects with clear ownership of agent decisions and as you said who's responsible for care and feeding of that agent is critical in there and all of those things help to build on the change aspects to have these new ways of doing things accepted because yeah I think somebody commented and trusted. We didn't trust calculators to start with and we checked the results and these things build over time as you become more familiar with them and more able to trust them.
[00:56:19] Isaac Sacolick: Go ahead. I'm not going to comment. I just want to hear from everybody. Go ahead Joe.
[00:56:22] Joe Puglisi: I just want to add into this conversation that I'm not sure why we don't have the same mental framework about how we train and come to trust agents as we do training and trusting employees.
In, in my mind I, I feel the same way about a bright young individual who's now going to read some of my old blog posts and make up a teaser line and I'll let him do a couple. And if they look good, I trust him to write more.
If, if it starts to sound too redundant or he forgets to do it one week and, you know, post the wrong thing, I'm going to correct that. But, you know, as Martin said, it's a continuum you're going to train and trust. More and more agents can do a better job of remembering things and building context than young employees.
So in some ways it's a little better. But why aren't we just viewing this as, you know, my new assistant?
[00:57:30] Isaac Sacolick: Good question, Joe. I will tell you that in my experience that I've been building agents out to run my business, I have more trust in AI to do some of the handiwork that needs to get done than I would probably trust an intern. Right. So I think there's a, you know, and that goes back to some of the things we talked about. When you're dealing with complicated steps and you ask a person to go do this for you, they're going to make mistakes. The AI isn't right.
[00:58:00] Joe Puglisi: Exactly. It's going to follow the, you know, it's going to follow the recipe, it's going to exercise some judgment. Not a lot, but, but, you know, it. It's going to think a little bit and it's not going to forget a step, which is important, but you do have to monitor.
[00:58:17] Isaac Sacolick: Thank you, Joe and
[00:58:18] Liz Martinez: Liz. I love that analogy because when you're talking about a junior person or an intern or whatever you are actually bringing in, somebody already has a tremendous amount of context.
So you're not able to give it a blank context to provide you what you want, in which case they might bring their own context to it, independent thought, and do something that's off the rails and not what you wanted.
Now, with an agent, you are, you have more of a blank slate in terms of providing it the context. And also you're giving it work to do that is low risk by definition until it can be, quote, unquote, trusted. Right. The problem with autonomous vehicles is how do you, you know, kind of let it go and see where it goes off the rails and then fix it later because, you know, lives are at stake. I mean, one could say that maybe that we weren't ready for autonomous vehicles because they weren't trusted, they weren't tested properly, and maybe grown like you would grow, you know, mentor an intern, say so in captivity, where you could actually monitor them before people's lives were actually at risk. The fact that they actually have accidents or the, or they have, you know, hurt people should not be surprising.
Should not be surprising to us.
It's unfortunate. I certainly don't recommend that. You know, test and production is never a good idea, but it's not wouldn't be the first time we've done it.
[00:59:58] Isaac Sacolick: Liz the number of accidents caused by AI drivers is a fraction of a fraction of a fraction of humans.
And I think that's the point, I mean, that Joanne was making earlier. There are times and places where the bot, under the right controls, with the right context is going to make a smarter, faster decision than a human can.
And, you know, and that's the situation, folks. We have to close.
We'll go to the green room, see how we can have a continued conversation around this at another episode. I want to thank all of you who have commented this week at the coffee with digital trailblazers. I will look through these and see if I can fill in the rest of this grid and then publish it in a week or two on my blog. So thank you for contributing. Next week I'm going to have a slightly different topic around planning 2027. We're going to talk about practices. There's a research report that I just got a couple days ago that I really want to focus on that's talking about how roles are evolving and how practices are evolving.
So we'll talk about what transformation leaders are focused around process change in their organization in 2027. Lester round about their AI bets. I have a topic for the 13th. I'll announce it this week. The 20th, we'll be talking about AI for social good, doing more with less when the mission can't wait. If you are listening and you are a nonprofit leader and interested in being a special guest on the 20th, please do reach out to me. And we'll be taking a week off on the 27th to celebrate Thanksgiving. Folks, we're doing our 195th episode. I want you to bookmark your calendar for December 18th. It will be our last episode of the year and that will be our 200th celebratory episode.
I don't know exactly how we'll cover it, but Heather's had some ideas and do book your calendar out to join us. All of you have been a part of the success of this program and I'd love to have you all part of that program on the 18th. Folks, have a great weekend. We will see you here next week.
