0:00 A new technology invoice line item is growing in your business right now. 0:05 It isn't a license, it isn't a seat, it's AI tokens, and it's the raw fuel that large language models like Claude, ChatGPT, and Microsoft Copilot run on. Today's guest is a procure tech tinkerer at heart. 0:19 He recently caught my attention again by pricing benchmark procurement tasks with different models. 0:25 Uh, one pretty striking example showed that a task run through default settings costed about eight hundred dollars a month. 0:31 However, when right-sizing the models for the job at hand, it turned into forty dollars a month, same output. 0:38 He says the rate card for these tokens is constantly moving, and that there's a hundred and twenty-five-fold price gap between suppliers at times. 0:46 He also points out that almost nobody in procurement is managing any of it as it stands. So today we get into AI token economics and why reading LLM model rate cards might be the next skill our profession has to learn. 0:59 Let's get into it. [upbeat music] Technology presumes there's just one right way to do things, and there never is. AI will do what you tell it to do, but that may be very different from what you had in mind. 1:11 Computers are useless. They can only give you answers. For a list of all the ways technology has failed to improve our quality of life, please press three. 1:22 Welcome to Procure Tech Unpacked, the show for procurement professionals who want to master the technology shaping our function. I'm Joel Kani-Demas. Today, I'm joined by Gaurav Sharma. 1:32 Gaurav is a global supply chain and procurement general manager with a background spanning private equity and large complex operations in telecommunications and manufacturing. 1:41 In my opinion, he's one of the sharpest independent voices experimenting with AI in procurement. 1:47 Gaurav has spent years doing the granular boots-on-the-ground work of category management and operations, and then taught himself Python and machine learning so he could solve problems with data instead of by hand. 1:57 He's the founder of Supernegotiate, an open source procurement technology lab where he runs live experiments with the newest AI models and shares the results with no hidden agenda. 2:08 I first got acquainted with him in the first edition of the ProcureTech Cup tournament, and I've been following his work ever since. 2:15 For a couple of years now, he's been making the case for what he calls UI-less or headless procurement technology. 2:22 Practically speaking, this means AI agents doing the work via APIs and databases instead of humans having to click through screens and software vendors needing to develop them in the first place. 2:33 Lately, he's also been publishing hard numbers on AI token economics and the real cost of running AI inside a procurement function. That's what I wanted to dig into with him today. Gaurav, welcome to the show. 2:45 Joel, thank you for having me. Really excited to be here. Yeah, it's long overdue, right? Like, I feel like we've been playing, uh, social media tag for a couple of years now. Yeah. 2:55 We both are terrible planners, as you put it. We have scheduled call with it, then rescheduled it again. So finally we are here. Thank you for having me. Yeah. And where, where are you joining us from today, uh, Gaurav? 3:07 I am joining from Dubai. Beautiful. Beautiful. And I know you as one of the biggest procure tech nerds on the planet, which is exactly why I wanted you to have this conversation with me. 3:18 Uh, I know in our chats, you know, you've, you've started out as skeptical with AI in procurement- Yeah... like a lot of us. Uh, what actually changed your mind over time? It's a really good question, right? 3:30 So I think when you look at AI in procurement from a very forty thousand, sixty thousand feet perspective, I would still be very s- uh, skeptical if I be candid. 3:41 But I think over the years, what I have seen, especially in the last eight odd months, the ability to execute the tactical use cases for me has been quite an enabler. So 3:55 previously I could form up as a conceptual level, for example, my supply performance management, I can do AI, but the path to execute that was missing. 4:06 What has changed recently in the last six to eight months, the script has been flipped. AI is now able, now able to execute. 4:17 So I can design my playbooks if I structure correctly and ha- have AI execute it, and that has fundamentally shifted. 4:25 And over a period of time, in the last six or eight months, ability to execute autonomously has advanced significantly. And, and for me, that, that is quite an enabler that 4:39 unlocks a lot of conceptual bottlenecks that we were having previously, and now we can execute it. That's what ch- changed my mind. And I'm curious to dig into like the, the, the what you think enabled that. Yeah. 4:52 Like my mind went immediately to MCP servers, which we've been discussing on the show in the past couple weeks. Like is that- Yeah... is that one thing? 4:59 Are there other, other enablers that you feel have changed the, the dynamic? 5:03 So, so glad you brought the MCP context because for me, and you rightly mentioned in the beginning, I am still the biggest probably advocate of a UI-less procurement digitization because 5:16 UI is the number one reason why your change management efforts are significantly complicated. 5:23 Not everyone gets the UI in the right place, and then people have to struggle where is that button and where is that button, and that slows down the digitization objectives. 5:33 What MCP enables us is, for me, the UI stays the same, that is the chat. So I can have the same chat interface like I have for ChatGPT or, or CoWork. 5:44 But instead of me going on to a different platform every single time, it will now be, uh, let's say, uh, APIs talking between themselves and have the task executed. 5:56 So for me, it really doesn't matter which platform I'm using. Okay? Mm. That means my execution path to execution is quicker, and UI, which was previously, let's say a, a blocker, is now gone. That's one. 6:10 On the second point, Joel, I think why I, I, let's say, changed my mind, AI now or is the mainstream in procurement is because, like I mentioned, 6:21 d- the platforms such as, uh, I'm pretty sure you have heard about, uh, uh, Grok Bot or, or always-on autonomous agents. These are the capabilities that enables us to have like set and forget kind of- Yeah... 6:36 procurement, uh, uh, use cases. Once you organize your setup clearly, like 6:43 you organize your knowledge repository in a smarter fashion, you organize your playbooks on, on how, which I call as the brain of the AI agents, and then you task the AI agents to perform some activities sequentially. 6:57 The combination of these three setups is quite powerful. Now, I've ran, for example, few experiments on supply performance management, right? 7:07 Which previously was largely, uh, somebody giving you access to a, a, a supply performance management platform, which is basically a UI, 7:17 and have people input SLAs manually, and then reminding people to enter the actual values manually, and then somebody calculating the late delivery clauses manually. Now, I don't need any UI for it. 7:30 I can throw a very well an always-on AI agent. I'm trying to be, uh, m- not mentioning the names. Uh, but let's say you have variety of always-on AI agents platform. 7:41 You can very well structure a workflow in under a minute now, which says, "This is my contract. Extract the SLA clauses every quarter on first of every month between 2:00 to 6:00 PM. 7:54 Reach out to my supplier, extract the SLA clauses, ask them to submit the actual performance, take that data, analyze it, sends to my-- send it to my 20 different business stakeholders, take their feedback, compare the two activities, go to my contract LD performance clause, and recommend is there a penalty note because of the underperformance, or I need to reward my supplier because they have over-delivered." 8:21 I can just set this once for all, forget about it. Mm. And this is the power of always-on AI agent that I've been experimenting on, on my super negotiate experiments. 8:33 And that ability to delegate and forget and operate within the rules that I defined is actually the game changer. Mm. 8:41 Which again accelerates the need for not have UI-based procurement platform rather than execution based through MCP servers or through these always-on AI agents. Yeah. 8:54 So I hear two, two things, MCP servers and the, the ability to schedule, uh, different tasks, right? Which also, yeah, it, it has only come up in the last six, eight months, right? 9:05 Which is, which is crazy to think, but it, to me it's always, it's also a sign that like, you know, we're all-- it's always going back to like the foundations of a business projec- projec- uh, process. 9:13 There's always a trigger, and that trigger can be just, you know, every week at X time or every day at X time, execute this playbook, this process, these set of tasks. Um, very cool. 9:24 So I, I love to, to hear your perspective on that. I also wanted to get us into, to, uh, to AI token economics. 9:30 Um, and I know that can be a scary conversation for some procurement pros because we start getting a bit technical and I think, you know, we, we've thrown out terms like UI and, and things like that, uh, in, in, in just a couple little sentences here. 9:44 But can you start us off on some good foundations with AI tokens? Like what is an AI token, and why do I, why have I not been exposed to it, uh, necessarily if I'm a user who has a premium, uh, seat on one of these LLMs? 10:00 Yeah. No, thank you for bringing it up. So let me just start with the basics, right? As procurement people, we are always used to now in this license-based software negotiation, right? 10:12 So previously it used to be, uh, a perpetual license. Then from perpetual license costing, we moved to usage-based licensing, which is that's a per user licenses, et cetera. 10:23 Then we went on to a, a cloud kind of a licenses where the cloud consumption was monitored, and now it's even more granular, which is token based. What is a token? 10:32 Token for me, it's let's say the smallest unit of, of let's say a word. So if I say, "Hi Joel, how are you?" It could very well be five tokens. So every word on essentially could be a token, but 10:47 the way the LLM providers do, uh, I have my, my own way of saying that rules of thirds. So 10:55 we have 100 words, I would roughly end up at 130 tokens because they, they count characters as more than a token at some, some of that mechanic. So for me, 100 word would roughly be 130 tokens about it. That's good. 11:10 And why this is more important now, we are entering into a phase which is based upon my token inputs, token outputs, tokens processing. 11:21 It becomes extremely difficult for us to do any kind of forecasting in terms of how many tokens will I consume. 11:27 That means for me as a procurement manager, I don't know how much wallet am I going to spend or what my spend is gonna look like. 11:35 In its current shape or reform, it's very difficult topic for a procurement manager to say, "I can't do a TCO based cost comparison." 11:45 My business stakeholders, and understandingly so, can't really forecast how many tokens I would be consuming. Yeah. 11:52 So it becomes a cost management problem where nobody knows how much bill I'm gonna be ending up with if I don't really ring-fence the way I'm buying. And, and for me, that presents a golden opportunity. 12:06 Others see it might as a threat, but I see as a opportunity because this is where procurement can come in and say, "I will create an environment where my token costs would be managed and also I can save-" If I structure my negotiations properly. 12:21 Going into a bit more, more depth into it, Joel, so 100 words roughly, rules of third, 30% is, is going to be the tokens. We have three kinds of tokens. 12:33 One is input token, meaning that's me giving the first input to my ChatGPT or Claude or whatever, asking what I need it to be done. 12:42 The second is the output token, which is basically ChatGPT giving me the feedback to say that, "This is the answer to your question." And recently, what we have seen now is about reasoning tokens. So all the-- 12:56 I call it as an entertainment because it doesn't provide me any value, but all the thinking tokens or reasoning tokens, which is AI describing you what the AI thinking, what the step is, 13:08 um, are called as reasoning tokens. Now, previously, and I'm still learning this category because it's really new, I used to think that's probably free of cost. 13:18 But now I'm going onto the surface, I'm being actually charged for that entertainment, if I use that word, right? So I have input tokens, which are, let's say, cheaply priced as compared to the rest to do. 13:32 Then I have output tokens, which are five to ten times more expensive, and I have reasoning tokens which are sitting in where. So these are the three tokens, and as a procurement manager, 13:43 I'm not in command of any of the three. And, and we, we'll go into those aspects separately. But yeah, this is what the token looks like and the types of tokens I have. Cool. 13:53 And, and I'm-- So one of the things I'm hearing as well or, or I wanna call out to listeners is you're using tokens whether you're buy- you have like a, you know, a Claude Pro or a Claude Max seat or not. Yeah. 14:05 Uh, you're, you can see it, right? You-- In the detail, it'll say, "Tokens being used up." Yeah. Uh, but once you start being charged on a token basis is when you start using the API, right, uh, Gaurav? Yeah, correct. 14:18 So for example, uh, the API just accelerates the token consumption because, let's be fair, right? Even as a user, I will never get my final output from the ChatGPT in one go. Okay? Yeah. 14:32 That means I will have to do iterations before I say I'm happy with the output. 14:39 So that means, and this is one of the learning that came to me, uh, once you go under the hood, the commercial model is so, so one-sided here. Let me give you an example. Uh, 14:53 my first conversation could be, "Analyze this RFQ for me," okay? Or, "Tell me the latest news on this supplier." So let's say that's 50 input tokens. 15:04 I'm just using rough examples, and the AI will come back to say maybe 500 tokens as an answer. Supplier is doing this, supplier is doing that, et cetera. Now you say, "Tell me more about one specific area." 15:17 Like, "Tell me about this factory," or, "Tell me about their, uh, business operations based upon what you have told me." Now, what I found out was it's not really one input anymore. It just cascades. 15:31 So your conversation one, which is the first input, and the answer one, which was ChatGPT's output to you, is reloaded again every time you ask a question, so it becomes a chain, 15:45 and that just exponentially consumes your token every single time. So every time I ask a question to my agent, it has to preload everything which you have already asked and ChatGPT answered as a fresh conversation 16:04 for the AI to understand the whole conversation history. Hmm. So that means every turn is an exponential increase. It's just not like me asking 50 more tokens or 50 more... No, no, no. It has to preload everything again. 16:19 So you don't pay once for the question asked. You pay maybe multiple times because you have to preload everything, and that shoots up your token consumption, token cost significantly. 16:32 Yeah, and, and I would imagine, like, the longer the question, the more tokens you consume. The, the, the longer the conversations, to your point, right, the more exponentially you consume tokens as well. 16:41 And then- Yeah... however, if you're asking for more details of the steps happening in between in the logic and you want- Yeah... 16:47 that explicitly with those reasoning tokens, you're also consuming exponentially more, right? Correct. And it's fundamentally wrong because why should I pay for the thing that I have already paid for, right? 16:57 So I asked you one question, you charged me, let's say a dollar for it already. Now I'm asking follow-up question you needed for your context, but I'm paying a dollar again on that question which I've already paid. 17:10 So it's double counting in my view. Like- Mm-hmm... from procurement perspective, if I had a choice, I will never pay for that question again, but I don't have a choice today. 17:18 For us to really understand that these are the mechanics, I think this is why I, I, I am trying to spread as much information as I'm able to understand by going through the contracts of the LLM providers today. 17:32 And so we've covered kind of the different token types and the fact that, you know, you're, you're consuming them ever, ever more increasingly and exponentially- Yeah... 17:41 as you get deeper and deeper into a conversation because it's like starting a new conversation every time, but your question gets longer and longer and longer. Correct. Yeah. 17:48 Even though you may have only asked, you know, a couple follow-up things. Yeah. 17:53 Um, maybe we dig into the difference between, you know, let's say we can, we can actually, uh, benchmark or standardize a conversation, right? So maybe we use your SRM example from earlier, right? 18:05 I think you've also been trying to do that, right? Experiment between different models of different levels of sophistications, uh, different, different consumption, uh, AI token consumption models. 18:16 What ha- have been your, your large top-line findings in terms of trying to do that? Yeah, couple of them, and thank you for asking this one. 18:24 One, I think if you see an, an enterprise scale-- By the way, one of the shocking finding for me for now is two things. 18:33 One, when you are going for an enterprise deal versus when you are going for a, let's say, a normal consumer, me going out on personal capacity, the pricing for enterprise deal would be probably 18:48 expensive, if not better, uh, if not the same, than the consumer one. Which quite is ironic because generally we are used to have bulk discounts when you go for an enterprise deal versus a consumer. 19:01 So when I as a-- in my personal capacity will go to a ChatGPT, I would probably get a 20, 25, $30, whatever is a subscription fee. 19:10 But when you go out on enterprise scale, your unit economics will be much more expensive, not just cheaper. That's one, quite strange. The second one is 19:21 the, the context window that you would often see pe- the LLM providers will compete on the context windows. For example, somebody would say, "I have a, a million token context window." Now, w-what is a context window? 19:34 Context window is like your desk space, right? How many things can you put in one conversation on my context window for me to really understand what you're asking? 19:43 Now, I can have a larger desk space so that I can ask you many things in one go. I can have a smaller context window, meaning if you put more, I would probably drop many things, uh, and, and retain only the latest one. 19:58 Claude has a really nice context window. It's probably a million, uh, in the best out there, if I remember correctly. Now, what I really found out, the longer the conversation, 20:09 the higher is the drop rate of how much the LLM can actually retain. So it's, it's almost like a normal distribution curve, so it's really good at remembering in the start of the thing. So let's say the top 10%. 20:24 It's really good at remembering the last 20%, which is, I mean, 30%, but it's terrible at the bottom 60, 65%. It, it just forgets thing. 20:37 That's why you struggle every single time, and you have to remind LLM to go back to what it already delivered. So as much as people sell the context window as a feature, 20:51 the better output comes in when you are having shorter conversations. So every time, if your context window's becoming quite big, I found it cheaper and much more accurate to start off a fresh conversation. Hmm. 21:05 That, that, that's one of the aspect. Like context window is not really a blessing, but let's say it's, it's a feature that is meant to be not utilized if you're trying to get a focused effort out of the LLM. 21:18 Uh, the another example, or rather the learning for me is, especially in procurement. Now, I'm comparing to the 21:28 capabilities and complexities of the non-procurement world, like in the data science, the biology, and all those industries, versus the complexity that we have in procurement. The difference is night and day. 21:42 We do not need the flagship models all the time to solve for procurement challenges. 21:49 The procurement challenges are fairly, if I use the, the complexity of one, which is, let's say, simple to a complex five based upon the, uh, various industries. Most of the procurement use cases would be two, 22:03 some would be three in terms of the complex. Now, why is it important? Because if you see, uh, I would not recommend anyone to apply today the normal sourcing principles to manage the token economics. Why? 22:21 Because the cost of the model which was launched previously versus the cost of the model that launched today, it's one-tenth. 22:30 I would never advise a category manager to fix the cost like we generally are used to for a two-year period or three-year period or a five-year period. And I'll tell you the reason why. 22:42 If you compare the cost of the model that was launched last year versus its cost today, it will be a factor of five to ten difference. The model gets cheaper within one year for every time a new flagship model comes in. 22:57 And we already talked about on the complexity scale, procurement would ra- largely be sitting at a scale of two and only some use cases at three, meaning I don't need flagship models every single time to solve my procurement use cases. 23:13 People are solving much more complex problems with flagship models such as Fable 5, uh, which is, uh, Claude. Uh, I can use a Sonnet, which is 10 times more cheaper and still get the same quality of the output, right? 23:28 And that has been, let's say, my fundamental reasoning behind why I think we need to rightsize which problem of procurement needs to be solved using what model. The last learning that I want to share is, I think 23:46 with the experience that we get from ChatGPT is a real-time feedback, right? You type in a question, the answer comes in, uh, uh, immediately. Um, 23:58 if you batch your processing, meaning you will only get output every four hours or you will only get output for this use case next day, the cost can reduce by a factor of 10 to 20 times. Okay? And 24:15 why am I saying this? These are the experiments that I was trying to run by myself to understand, are there any ways to optimize the cost? So the answer is 100% yes. Hmm. One, figure out which use case needs what model. 24:32 Figure out do you really need a real-time output or can you do the processing every four hours or one day, et cetera, right? And the-- there are other techniques such as caching so that 24:45 if me as a procurement manager has already run a use case, instead of asking someone else to rerun the similar use case, use my output, which is the caching portion. 24:56 And, and these are the techniques which I'm not seeing right now in the mainstream advice. 25:00 And, and these are the things which I think should become one of the aspect of whenever you're negotiating an enterprise deal, have these established right up front because 25:11 if I'm doing my spend cleansing, I don't need a real-time input. I'm more than happy to get an output monthly once, right, and then cache it so that other people can reuse that output. Yeah. 25:24 And my suddenly the cost is one tenth if not more. And for someone to really establish discipline around the build-- And it's very similar when you get an intake request today 25:39 to say which supplier you are going to use. And instead of that, I'm just saying, which model are you going to use? Yep. And what's the delivery time? It's not going to be instant. 25:53 I will only give it to you after two days. And that is the area for me it's quite interesting because I see next year this becoming quite, quite important. Yeah, absolutely. 26:04 Like token budget management, but also right sizing to the use case. Um, and it's so funny, right? 26:11 Because every time I have these t-types of conversations around the era of AI, the, the skills, the reflexes you need, like I always go back. 26:18 I'm a process guy, like I see the world through business processes, and I'm just going like, "Hey, this is process optimization principles at heart," right? 26:26 It's like, let's get to the same output, but let's be a bit more efficient about how we get there, um, using the, the tools that we have at our disposal. 26:36 So one other thing that came to mind, uh, as you were talking about AI tokens is I have this, this hypothesis or this theory that eventually AI tokens will be free, uh, because I'm already seeing in the procure tech space where certain providers are providing certain use cases out of the box for free as part of licensing because they know, you know, it's not gonna cost that much to run them. 26:59 Um, and over time, you know, th-they can-- they'll recoup the cost through the licensing fees. 27:05 And I'm also thinking back to, you know, the mobile phone era where at the beginning, right, minutes were super expensive, and then eventually we got to these plans where you had free, free minutes, global minutes, et cetera. 27:16 And so I think it's really about the CapEx amortization schedule, right? 27:20 And so we're, we're all these big CapEx costs for data centers, but eventually that, that'll be amortized over time and we'll be, uh, able to offer, you know, free AI tokens. 27:30 Maybe it's in a decade or fifteen years, twenty years, something like that. But I think that's the, the trajectory that we're on with this. Would you agree? No, a hundred percent, Joel. 27:38 I think sh-we should get there eventually, right? Uh, right now if I see a lot of procurement tech providers are exposed, and I'll tell you why. Uh, 27:48 their licensing model is still, for example, uh, based upon the number of users, okay? 27:55 And they are-- Some of them are already offering these AI agent which are basically an API wrapper and in the back end they're calling all the models and everything. Now, if they're 28:05 using that as a feature which is not tagged to the tokens cost today, they are basically absorbing the cost, to your point, right? 28:14 Uh, eventually when-- if the adoption happens, then their cost will go up and you will see cases where, uh, the cost for them is probably higher than the licensing at certain point. 28:26 So there are only two opportunities here. If not the procurement team, but the procurement tech providers will start replacing the flagship models with what we just discussed in terms of the workhorse models. 28:40 The long-term future, and you correctly put it, is free tokens. Now, the way it will happen is using the open weight models, which is published by many, many providers already, right? 28:54 Using these open weight models, they will use the customer data, which is us, to train these offline models and then offer these models to the procurement, let's say CPOs. Example would be, uh, a Qwen model, for example. 29:10 A three billion parameter model which is free to download, has open weights. 29:16 Uh, that's one of the experiment I was running is you can download this model, host it on your private server, train it using the data, and still get about eighty percent of the same output as a Claude Sonnet, but free of cost. 29:32 Hmm. And it makes more sense because like I mentioned, procurement use cases are not that complex as compared to the rest of the industries. So you're spot on. 29:41 We will see eventually, uh, the procurement tech providers opting for open source models and get permissions from their clients so that they can train those open source models and feed it back into the, uh, the platform that we are getting today. 29:56 And so you, you did point to spend classification as being, you know, one thing that you can use probably a lower type model for. Um, I think you also, uh, kept the premium models for bid scoring. 30:08 Do you see that as being one of the more, the more tough things, you know, the three-level difficulty tasks that we do in procurement? Yeah. So let me take a step back, right? Because I think 30:20 a lot of skepticism, including me in the early days, was coming through, I threw AI at my problem and it gave me a really bad output. Yep. Right? 30:29 And w-with this experimentation, I figured out there's a method to this madness. And you almost have to, let's say, form a golden thread before you can throw problems at AI, right? 30:43 For example, uh, let's take example of spend classification. If I take my spend file today and ask AI to classify it, I'm pretty sure most of the people will be disappointed how it classifies. However, 31:01 if you build a playbook, which is the brain, like I was mentioning in the beginning, to say, analyze, for example, my last 31:10 50 emails or last 30 reports and tell me what's the best way of-- for me to categorize the information of those numbers. Or you say, "This is my taxonomy, and I want... 31:27 I, I classify my rule, rule book to say, 'If you see this description, this is the bucket I want you to put it in.'" Okay? And once you define this, this is the way you can check for the accuracy. 31:41 My accepted parameters for success are these, right? And then you throw the spend data, not its entirety bit, but in batches like I mentioned before. Mm-hmm. 31:53 And iterate it till you have, what I call it, as a golden subset to say, "This is what a really nice spend looks like." And then use that as an input on the wider set of the data. 32:05 So you're almost giving a golden key to your spend classification system, because now AI knows what to back test against its accuracy. 32:16 If you do that in this structured way, the accuracy of your spend classification improves significantly. Mm. Okay? A-and the cost to do it based on- And the cost to do it as well... how you spend. That's right. Yeah. 32:28 Right? And then at a certain point, you, Joel, will-- we will eventually figure out a rule-based engine. AI is really good at dealing with uncertainty, right? 32:39 And through hoops of this iteration, we will end up with a rule-based scenario in some of the parts, not all. And when we are able to figure it out with some of the parts, 32:51 you can ask AI, the heavy lifting is done by this rule book. So if you see this, classify that. Again, AI is really good at execution, not really at making decisions, right? Yeah. 33:04 So the iterations help me to define this rule-based engine. Let AI execute and you decide kind of a theory. And then the remaining becomes the unclassified one, which is solved through iteration. 33:16 So the moment you apply this playbook principle, the classification accuracy increases significantly, uh, on that one. And like I mentioned, you almost need to have a panel of models, like I call it. 33:32 You will have your workhorse model, which is probably 80% of u-use cases going on. So this is where you fix Claude Sonnets, GPT 4.5, uh, as your day-to-day go-to models. 33:45 Then you have your premium flagship model, which is your Fabes of the world, which are like 15 times more expensive. And to your point, why I think 33:55 technical evaluation, for example, or scoring is, is one of the complex use cases is because of the document overload. 34:03 If your document has a rich data, it contains text, it contains tables, it contains diagrams, technical architectures, and you need a higher processing capacity, I would probably use my flagship model, which happens, by the way, in technical scoring use cases. 34:23 But if you have a structured data in Excels or the CSVs only, one format, and there is no cross-hybridization of you're not asking, uh, the AI to combine multiple files and then classify, 34:39 like a ERP extract is one singular file. I would use my, use my workhorse models to- Mm... get to the, that stage. 34:47 If I have a well-defined rule book or the playbook on all I want AI to execute it, I would even go for the cheaper model like the Geminis of the world, right? 34:59 So-- And, and these are the decisions I think your procurement excellence, the data science, and the category managers have to take. 35:05 For this use case, phase one, I'm gonna use Fable-05, but in the next six months, I want to develop a rule-based playbook so that I can move from Fable-05 to my workhorse model, to eventually the Geminis of the world, and then save it from there. 35:22 So what I'm hearing, Gaurav, is there's a couple different things, right? Where if the process is more complex, it has more inputs, you have more documents that need to be processed, like you need more processing power. 35:34 That's a good use case or a good place to start is with those more flagship, more expensive models. Yeah. And the simpler your business process is, the more explicit it is, the more... 35:44 you know, the, the, the less outputs or inputs you have, rather, uh, along the way, the more it's, it's fit for purpose for your daily drivers, right? Your, your, your base models that are at least less expensive. 35:56 And then the most insightful thing I think you said is, you know, when you have something you're crafting from scratch and you don't really know the complexity, like you can use the more expensive models, but then you have a path to getting it to the least ex- the more, uh, the less expensive models because you can get it to a rules-based playbook type process. 36:15 Is that fair? Yeah, hund- 100%. It's a really nice way to put it. Like if you're dealing with text, start with the basic or let's say a workhorse model, never with a flagship one. Because let's say 36:29 your 70-pager text-based document would maybe, uh, get 3,500 tokens or 5,000 tokens, right? The moment you put document into it- Your tokens increases by seven to eight times. 36:43 So now you're talking in terms of 80,000 to 150,000, uh, 100,000 tokens. The more documents you put, the complexity increase, and I think it's quite a, a straight up advice. I'm not giving anything technical out here. 36:58 Again, if you have only one file to deal with, go for the workhorse model, which is the medium ones, uh, Claude 4.5, Sonnet 4.5, Gemini 4.5... Sorry, uh, ChatGPT 4.5. 37:11 Uh, if you have two files where you want AI to do some sort of analysis by joining the files and then coming up with some analysis, and then on top of that analysis you want AI to execute something, yeah, go for Fable 5. 37:25 But then the intention should be: how do I establish a playbook, meaning I decide the rule and AI just executes it. That should be the goal. Yeah. How do I optimize this business process? [laughs] Correct. Yeah. 100%. 37:40 Um, a- and so, you know, all this work that you do, I know you do it kind of on the side outside of your daily activities, right? Yeah. Uh, your, your daily job, which is, uh, in and of itself pretty impressive. 37:50 Um, but it's the second thing that's impressive is that it's all open source, right? You've built different applications through these experiments. Uh, you share all the knowledge as well- Yeah... 38:00 for free with folks who, who follow your content. And so what, what, um, keeps pulling you back to, to building in this fashion and, and sharing in this way? I'm curious. 38:09 In a world where, you know, everybody and their grandma is starting a procurement tech company. [chuckles] No, John, I think the motivation comes from the amount of things that are possible, right? And I think-- 38:22 And, and it's getting-- We are getting to a closure where procurement tech will become free. 38:27 And I'm, I'm putting my word out there, I know, but eventually the tech will become free, uh, because how fast and easy it is now becoming. 38:36 Uh, especially with now these always-on agents, you will see the category managers experimenting with their skill set because they have the domain knowledge. The, the issue was they didn't know how to code. 38:51 Now the coding part is commoditized. 38:53 So we will all see our teams internally developing their own dashboards, their own agents, and those agents will be very, very highly effective because they are working only one small niche use case. 39:09 So you would eventually see a lot of build versus buy decisions coming out next year. Why can't I do this in-house? Because it takes me 30 minutes instead of me running an RFP and buying a very expensive tech platform. 39:24 So I'm almost of the opinion that the tech will become free. The, the value that AI can unlock is we need to solve better problems, and the better problems are in not me running another reverse auction event. 39:39 The better problems are how do I enable two teams in procurement come together and, let's say, create value, whether it is joint sourcing. The risk management needs to be shared across. 39:54 It's a fantastic area where I think all of us can benefit from the collaboration. The soft skills will get a lot more important. 40:04 I know to a certain extent you can commoditize negotiation, but you cannot commoditize the relationship with suppliers. 40:11 You cannot commoditize those strategic negotiations where, uh, uh, it c- it's just not about price haggle. I think what gets automated is the price haggle portion. 40:23 You would quickly see it will be very restrictive in terms of how fast you can scale it, because it will always be left to the very commoditized aspect. 40:34 The things that will take center stage in would be the risk management, supplier relationship management, and the strategic aspects. And I also believe because 40:44 the, the world is moving towards more collaboration in non-procurement world, uh, I see a tremendous value operations where AI can help, let's say, us to collaborate more. 40:57 I don't see a reason why two organizations buying the same thing, for example, cannot work out a joint w- uh, buying solutions together. 41:06 The, the issue was connecting them, and with the help of AI, we can reduce a lot of friction. 41:12 For example, judging the similarity, judging, uh, how, what's the past path for fast execution and all these avenues that open up. Uh, you would see a lot of gain sharing models, right? 41:26 Why I started Supernegotiate is because a lot of problems in procurement tech are still the same. We are just changing the UI layers. 41:35 I mean, if somebody is still talking about getting you cost saving using reverse auctions in 2026, that's a really solid indicator that we need to do a better job. So 41:45 a, a lot of things, there's a great community out there. Everyone is putting, uh, so much content around it, and I really love it when people talk about soft skills. But 41:56 the, the agenda behind a lot of procurement tech platforms are the, the master key of solving procurement. I think by reselling the same situation and same problem over and over again, we are unable to graduate. 42:09 So I need better problems to be solved. So once the-- Hopefully, when procurement tech becomes free, like I'm putting Supernegotiate for free, we all migrate to better class of problem solving. 42:21 That's the whole motivation. I love it, and it's, that's why it's always so much fun talking to you. You're an optimist, an idealist. 42:28 Uh, and, uh, and what I heard was three, three distinct things that still have value, right? So soft skills, digital literacy, right? 42:35 So being able to understand the tech, which is what you're doing with your experiments, and then the domain expertise in procurement, right? 42:42 And those things, what I heard through our entire conversations is, are not going away and are actually getting more valuable in the era of AI. 42:49 Yeah, I-I-I still see category managers defeating the large e-sourcing suites next year. 42:56 So once we get always-on AI agents, it's gonna be a very interesting time because people can then delegate most of the stuff to AI agents, and I don't need UI for it. So it's a very interesting space to be in next year. 43:09 Yeah, and would you say, maybe to, to wrap up here, uh, you know, you say interesting. I, I hear your curiosity when you speak. Um, how would you recommend procurement pros think about having a fulfilling long career? 43:23 Because you seem to have one yourself. I, I see procurement as deal makers, right? 43:28 I, I would classify the future of procurement role as a combination of, you know, like merger and acquisition teams and sales team put together. 43:36 So I, I think now the execution gets, let's say, autonomously executed, and I don't have to worry about follow-ups because a lot of procurement things are is about follow-up and making sure execution happens. 43:48 When you get confidence in terms of execution that there's an AI agent that will execute according to your standards and rule books, you can focus on, in my view, questioning the cost models itself. 43:59 Why do I have to pay you upfront? Why can't this be a risk-sharing mechanism? So I see a future where a lot of cost models are being challenged. 44:09 If I, for example, in raw materials commodity, um, if I have to do a index-based pricing, I can have a very fair and open conversation to say, "I'm going to monitor this index on quarterly basis and have a gain-sharing mechanism. 44:24 So neither you lose nor I, I lose in terms of the scenario." 44:29 I can, for example, work with my supplier to launch new projects because I have now capacity to understand consumptions and bring my sales team into the picture, develop a revenue-share mechanism, and there's, there's whole lot of variety in terms of how do I de-risk in terms of me paying upfront and not getting the value. 44:49 Uh, how do I prevent the nest- next over inventory problem? And these are really good problems to solve, and the AI just takes the noise away from what we are solving today. 45:01 And so I'm hearing, uh, accepting that you're always gonna have problems to solve, and then- Yeah... developing the skills to improve the quality of your problems over time, right? Hundred percent. 45:11 That's a really nice way to put it, Joel. Very cool. And so outside of procurement tech, uh, what are you most excited about right now these days? Largely it's around experimentation. 45:22 So, uh, the, the reason I, I enjoy my work at Supernegotiate is I get an idea, and then I put it out there and then try to have fun with it. So whether it's about why can't I just do bidding using WhatsApp? 45:37 And that was an idea, and 72 hours later we have a prototype working and people testing it. And because it is, let's say, 45:45 m- many people coming together and making that use case happen, that's why I'm very big advocate of let's use AI to bring CPOs together. That's a risk-sharing element. 45:55 So I spend a lot of time talking to people about ideas, what's blocking, uh, clear out their visions, and, and when you remove the sales portions of it, the quality of these conversations is, is extremely well. 46:09 So I speak, uh, quite a lot outside my working hours with CPOs, what's bothering them, and ideas and brainstorming. Yeah, that's what I really enjoy. That's cool. That's funny. 46:20 And, and it, uh, your answer, um, ties back to what I said at the intro, which is, you know, you're, you're a procure tech nerd true and true, and like you... 46:30 I'd ask you what your hobby outside procurement technology is, and you said it's procurement technology. [laughs] It's really fun doing it, trust me. Yeah. No, I agree. I mean, I'm in the same boat, right? Just- Yeah... 46:40 I'm a bit more on the content front. Um, so Gaurav, this has been, you know, a great conversation. Uh, so many insightful pieces of, uh, you know, of, of, uh, value and ideas that you shared with us today. 46:52 So I really appreciate you taking the time to, to come on. 46:54 I know you're halfway across the globe, and it's always a, a challenge for us to find some time in the calendar, uh, but really appreciate you, you doing so, and I hope listeners, uh, also appreciated, which I'm sure they did. 47:07 So any, any parting words before we let people go here? Joel, let me thank you because you have been one of the person whose work has inspired me to keep going on. 47:16 I know it's not easy out there, but to constantly put high quality content, I admire the, the, the contribution you have made to the community. So thank you for giving me this opportunity. 47:28 Really, really enjoyed it, and I'm pretty sure we'll collaborate in future again. Yeah. I want a, I want a some conclusions from some future experiments here. So we'll have you on again soon, I'm sure. [laughs] Great. 47:38 Thanks everybody. Talk to you again next week. That's a wrap on this episode of ProcureTech Unpacked. 47:43 If you enjoyed this one, give us a follow on your podcast app or YouTube and sign up for the Pure Procurement Newsletter to get procurement technology insights delivered straight to your inbox. 47:54 Otherwise, what's the single best way to tell us you're enjoying the show? Leave us a five-star review on Apple Podcasts or Spotify, like the video on YouTube, or drop us a comment. 48:04 Whatever platform you're listening on, your feedback means absolutely everything to a niche show like this. All the links, reports, and resources we mention in the episodes are in the show notes. 48:15 And as always, a big thank you to our production team over at Resonate Recordings for making us look and sound good. I'm Joel Clermont. We'll see you next week. [outro jingle] Resonate.
