Picking the wrong software partner is one of the most expensive mistakes a company can make. Around 70% of software projects exceed their budget, and McKinsey, studying more than 5,400 projects with the University of Oxford, found that half of all large IT projects massively blow their budgets, running 45% over budget and 7% over time while delivering 56% less value than predicted. Those numbers don't mostly come from bad luck or hard technology. They come from the partner being chosen on the wrong evidence: a polished pitch, a low headline price, a logo on a slide. A good partner is the single biggest lever you have against those failure rates, which is why the selection deserves more rigour than most companies give it.
The hard part is that everything looks fine in the sales process. Vendors are practised at the demo and the proposal; they are far less practised at the parts of delivery that actually decide whether your project ships. So a useful evaluation is designed to surface the things a vendor would rather not discuss until after the contract is signed. Here is the checklist we'd use as a buyer.
1. Proof they've shipped at your level
Slide decks are cheap. Ask for named clients, live products and references you can actually call. A team that has delivered for large, demanding organisations has already survived the security reviews, the procurement gauntlet and the scale problems you're worried about. Be specific: ask to see a product that resembles yours in shape (similar integrations, similar user load, similar regulatory exposure) not just an impressive client in an unrelated domain. (For context, see our case studies, with work for enterprises including Microsoft, Google and National Instruments.)
When you take the reference call, skip the softball questions. The ones that matter are: Did they hit the dates they committed to? What happened when scope changed? Would you hire them again for something harder? A reference who hesitates on the third question is telling you something.
2. Senior people on your account
The classic agency trap: senior engineers in the sales meeting, juniors on the delivery. Ask who specifically will work on your project, see their work, and get the named team written into the contract. The cost of getting this wrong is real. Replacing a single bad hire runs at least 30% of first-year salary, and a thin team on a critical build is the same problem wearing a different hat.
3. A clear, honest engagement model
Fixed-price, time-and-materials, or a dedicated team: each has a place, and the right one depends on how well-understood your scope is. A partner who pushes one model for every problem is optimising for themselves, not for you. Watch for the firm that insists on fixed-price for genuinely exploratory work (you'll pay for the risk premium and fight over every change order) or one that only offers open-ended T&M for a tightly-defined build. (We break the trade-offs down in engagement models.)
4. Code and IP ownership in writing
You should own the code, designs and documentation outright on payment. Read the contract for the quieter traps too: components the vendor licenses back to you rather than transfers, "shared" repositories you can't actually export, or proprietary frameworks that make leaving expensive. If a contract is vague here, walk.
5. Security and compliance maturity
Vendor evaluations should cover data ownership, encryption in transit and at rest, and secure-coding standards before anything is signed, not bolted on later. A mature partner talks about this without being prompted and can describe how they handle your industry's specific regime. A team that treats security as a checkbox at the end is a team that will hand you remediation work at the worst possible moment.
6. A scoped pilot before the big commit
The single best de-risking move is a paid pilot: a small, real slice of work that proves fit before you commit to scale. Try before you buy at scale. A good pilot is a thin vertical slice of the actual product (one real workflow, end to end), not a throwaway prototype, so the work and what you learn about the team both carry forward.
7. How they communicate when things go wrong
Every project hits trouble. The differentiator is whether your partner tells you early and clearly, or goes quiet and surfaces the problem when it's already expensive. Reference calls are where you find this out, and so is the pilot, where you get to watch their communication under a little real pressure.
A quick red-flag list
A few signals that should make you slow down:
- A quote that's dramatically lower than everyone else's. It usually means a different (smaller) scope, juniors, or a plan to recover margin through change orders.
- Reluctance to name the actual delivery team or let you talk to references.
- Pressure to skip discovery and sign for the full build immediately.
- Vague or evasive answers on IP ownership and what happens to your code if you part ways.
What to actually do
Shortlist three vendors. Send all of them the same one-page scope so you're comparing like for like, and check their transparent, comparable pricing before the sales call, not after. Run a paid pilot with your top one or two. Decide on the evidence the pilot produces (shipped work, communication, how they handled the inevitable surprise) not on the proposal. If you're already feeling some of the signs you've outgrown your dev agency, or specifically vetting an AI vendor, our questions to ask before hiring an AI development company covers the AI-specific version of this checklist. This sequence costs a little time up front and saves you from the far more expensive cost of unwinding a bad full-build engagement.
The cheapest quote is rarely the cheapest project. Re-doing a failed build costs far more than choosing well the first time.
If you're evaluating partners right now, tell us what you're building and we'll give you a straight read on scope, cost and timeline within 48 hours, pilot included.
Sources
- Acquaint: Software budget overrun statistics
- McKinsey & University of Oxford: Delivering large-scale IT projects on time, on budget, and on value
- Netguru: How to evaluate AI vendors: a guide for CTOs