Every Big Vendor Is Great. And Terrible.

Hi {{FIRST_NAME|Readers}},

Yes, today’s edition of the newsletter looks different.

After our website rebrand and premium offering update…

We’ve updated our newsletter design.

Same sections and content you love. Updated look.

Hit reply and tell me what you think. One line is plenty. Love it or hate it?

And while you’re there, I have a question for you:

What worries you most about a ProcureTech implementation?

Maybe it’s getting the B-team on your project? Maybe it’s the “out-of-the-box” integration that turn out to need months of custom work? Or what happens after the implementation team leaves?

Tell me in your reply… I might have something interesting for you depending on what you say.

Onwards!

The reference call went perfectly. 45 minutes with a director of procurement who loved the platform. Go-live landed two weeks early, adoption was strong, and the integration "just worked." You hung up ready to sign.

Now picture how that reference got chosen. The vendor has hundreds of customers. Its sales team picked one for you to talk to. Which one do you think they picked?

Nobody lied to you. You just heard about one draw from a very wide distribution, and it was the best one available.

A small provider with ten customers is easy to judge. The founders sit in on every implementation, the same handful of people configure every tenant, and the outcome tracks the quality of those people.

A large provider is a different animal. Dozens (if not hundreds) of implementations run at the same time, across regions, partners, industries and project teams. Some of those projects are trainwrecks: runaway scope, a lead consultant who quits in month three, an architecture that never fit. A large middle gets delivered "well enough," a few weeks late, a bit over budget, but live. A small group goes flawlessly: clear requirements, a sharp team, no surprises.

Nobody publishes the exact split. But this shape shows up at every vendor large enough to have one.

So when someone tells you "X is great" or "Y is a disaster," they're both right. They're describing different draws from the same distribution.

This cuts both ways, and it applies to you too.

Your implementation is a sample of one. So is your former colleague's horror story, and so is the glowing post on LinkedIn. A single project tells you the different parts of the tail exist. But it tells you nothing about where the middle sits.

And the stories you hear aren't a random sample either. Disasters get retold at every occasion... "It went fine" doesn't make a good story, so nobody tells it. The loudest data points come from the tails.

If outcomes were a neat bell curve like shown above, you could shrug and accept the average. They aren't.

Look at the upside first. The best possible project finishes a few weeks early and a little under budget. That's the ceiling.

The downside has no ceiling. Flyvbjerg and Budzier (Oxford) studied 1,471 IT projects and found an average cost overrun of 27%. One in six projects, though, averaged a 200% cost overrun and a schedule overrun close to 70%.

Their conclusion?

IT projects aren't especially prone to high overruns on average. The danger is a disproportionate number of “Black Swans” (projects at the "bad” end of the distribution).

So the good tail saves you weeks. The bad tail can cost you multiples of your budget. When you choose a vendor, you're managing the bad tail, not maximizing the average.

As software vendors grow, this typically pulls the whole curve toward "bad" unless the vendor works very hard against it. Two forces do most of the damage.

The better vendors fight this explicitly with things like standardized delivery methods and asset, formalized project hierarchies, internal “centers of excellence” that review every project periodically, partner certifications for SIs, etc.

When you’re validating for quality, ask what they do from this list, and ask for proof.

(That’s not to say smaller vendors are automatically better at delivery… I’ve seen plenty of small organizations who are too ambitious for their skill level… All I’m saying is they have a statistical advantage, all things being equal.)

Another thing to consider is that the software vendor isn't the only variable. Your own maturity shifts where your project lands on the curve (often more than anything the vendor does…)

Picture two companies buying the same platform, with the same partner, in the same quarter. The first has one process owner, formally documented requirements and a steering committee that decides things in the meeting. The second has three business units that each want their own approval flow, a data cleanup requirement nobody funded, and a sponsor still fighting finance over the budget. Same vendor. Different curves.

An immature organization drags its own project toward the left tail. Requirements change every sprint because nobody owns them. Decisions wait for the next steering committee, then the one after that. The vendor's best consultant can't settle your internal politics (and won't try, since it's not their call).

If that sounds familiar, shrink the bet. Go for the “minimum viable scope”: one process, one business unit, one region. Start where the sponsor is strong and the new operating model can be proven (here's how to pick that first site). Get a win, build the muscle, then expand. A small scope also caps how much the bad tail can cost you.

The value from a vendor's projects isn't spread evenly either. A small number of flagship implementations produce the case studies, keynote stories, award submissions and reference calls. The rest pay the bills.

That's social proof at work, and it works on procurement people too. When you see a logo wall and three polished case studies, you're looking at the vendor's best few draws.

They show what's possible. They don't predict what you'll get.

Now flip the question around. Vendors and partners have a limited number of top people, and they decide where those people go. So if you want the flawless tail, ask yourself honestly: why would they give their best to you?

You already know this concept from the other side of the table. Procurement spends years working to be a customer of choice for key suppliers. Your software vendor and your SI run the same calculation on you.

Here's what makes an account worth the A-team:

→ Strategic value beyond the contract. A recognizable logo, a new industry or region for them, a reference they can use for years.

→ A growth path. A credible roadmap of next modules, business units or regions, so the first project opens a relationship instead of closing a deal.

→ Design-partner potential. Willingness to adopt new modules early and give open, frequent and structured product feedback.

→ Being easy to work with. Fast decisions, an available executive sponsor, invoices paid on time. Partners remember which clients burn out their consultants.

Then put it on the negotiating table. Reciprocity works best when it's explicit.

Offer to be a public reference, co-present at their conference or co-author a case study, conditional on the outcome: named senior resources in the SOW, an executive sponsor on their side, and agreed success criteria.

You trade what they want (a showcase from the right tail) for what you need (the people who produce right-tail results).

Drop "is this vendor good?" It has no useful answer at scale. Ask "what decides where my project lands on the curve?" Then control as much of it as you can.

→ Evaluate the team, not the logo. Name the key resources in the SOW. Add a key-person clause, and require your approval before any substitution. The people on your project matter more than the company on the contract.

→ Ask for references that match your draw. Same implementation partner, same region, similar scope, live in the last 18 months. Then ask for one project that went sideways and if/how they recovered. How a vendor handles the bad tail tells you more than its best case.

→ Check how thin your team will be stretched. Ask how many concurrent projects the vendor’s lead consultant and solution architect are carrying.

→ Size the scope to your maturity. Clear requirements, favorable end users and a named decision owner sit in the "flawless" tail for a reason. If you don't have them yet, start with the smallest “viable scope”.

→ Make yourself worth the A-team. Know what your account is worth to the vendor, and trade reference rights for named senior people and executive sponsorship.

→ Plan for the tail you can't control. Phase the scope, set exit criteria (we covered how in Pilot vs Proof of Concept vs MVP), and hold real contingency. Flyvbjerg and Budzier's own advice is to break big projects into smaller ones of limited size, complexity and duration, plan for unavoidable risks, and use reference class forecasting.

Next time someone asks you whether a big vendor is any good, try answering with three questions:

For whom? With which team? On what scope?

At a certain size, every provider is great, average and terrible all at once.

Your job isn't to find the "good" one…

It's to stack the variables so your project lands on the right side of the curve:

Then, work on putting in place the mechanisms to protect yourself in case things lands somewhere else anyway.