The expensive AI mistakes I see don’t happen in production. They happen in the premise. A team gets the mandate to “build an agent,” picks a use case that sounds impressive, spends a quarter and a meaningful budget, and ships something that demos well and gets quietly switched off two months later.
The frustrating part is that almost every one of those projects was answerable before it started. Here are the five questions I’d want answered before anyone writes a line of code. If you can’t answer them yet, that’s not a reason to stop — it’s the actual first piece of work.
1. What decision or task does this replace — and what does that cost today?
“Use AI” is not a use case. “Cut the time our support team spends triaging tickets from six minutes to one” is. An agent has to displace something measurable: a manual task, a slow handoff, a decision people dread making.
If you can’t name the current cost — in hours, dollars, or delay — you have no way to know whether the agent is worth building, and no way to prove it worked. Start from the thing that’s expensive now, not from the technology.
2. Is the data the agent needs actually reachable?
Agents are only as good as what they can see. The question isn’t whether you have the data — it’s whether it’s reachable: in a system with an API, reasonably clean, and permitted for this use. A huge share of “AI projects” are really data-access projects wearing a more exciting hat.
The honest version of this question: if I asked your team to hand the agent the same context a new employee would need to do this task, how long would that take to assemble? If the answer is “months,” your first project is data, not AI.
3. What does “wrong” cost, and who notices?
LLMs are probabilistic. They will be wrong sometimes. The viability of an agent depends almost entirely on what a wrong answer costs and how fast someone catches it.
Drafting an internal summary a human reviews? A wrong answer is cheap and visible — great fit. Auto-approving refunds, or telling a customer something with authority? A wrong answer is expensive and may go unnoticed for weeks. Same technology, completely different risk. Know which one you’re building before you build it.
4. How will you measure whether it’s working — before and after?
If you can’t measure it, you can’t defend it, and it will be the first thing cut in the next budget review. You need a baseline now (question 1 gives you this) and an honest metric after. “People seem to like it” is not a metric.
This also forces a useful conversation: what accuracy is good enough? Ninety percent is a triumph for some tasks and a disaster for others. Decide the bar up front, not after the demo.
5. Build, or buy?
The reflex is to build. Often the right move is to buy a managed model or an existing product and spend your effort on the part that’s actually specific to your business. Custom build only earns its keep when the task is core to what you do and nothing off-the-shelf fits. Most agent value lives in plumbing and context, not in the model itself — and you can rent the model.
The cheap version of learning this
Every one of these questions is answerable in days, not quarters, and answering them is far cheaper than discovering them halfway through a build. The output isn’t a no — it’s a ranked list of where AI actually pays off for you, what each one needs, and what to do first.
That’s exactly what a vendor-neutral AI Readiness & Agent Strategy Assessment is for: a short, discovery-only engagement that pressure-tests your candidate use cases against value, feasibility, and risk, and hands you a credible roadmap before you spend real money. No system access, no build — just the answers to these five questions, in writing, so the build you do start is the one worth finishing.