Five questions to ask before any healthcare AI pilot
Health systems are running a lot of AI pilots right now. Ambient documentation, prior authorization, coding support, call center tools, forecasting. Some of them are working well.
More of them are agents now: tools that take a series of steps on their own, such as building a prior authorization packet or working a denials queue. That makes these basics matter more.
Many others stall. The pilot ends, the results are “promising,” and nothing happens next. The vendor moves on to the next customer, and the team that ran the pilot goes back to its day job.
In my experience, you can usually tell which pilots will make it before they start. The ones that succeed have clear answers to a few plain questions. The ones that stall usually skipped them.
Here are the five I’d ask before approving any AI pilot.
What work changes, and who does it today?
Start with the person, not the tool. Name the role whose work will be different if this succeeds. A coder. A nurse. A scheduler. A denials analyst.
Then ask that person, or their manager, whether they want this. If the answer is lukewarm, the pilot will be too. Tools that solve a problem the front line already feels get used. Tools that solve a problem only leadership sees tend to sit idle.
How will we know it worked?
Pick the measure before the pilot starts, and pick one you already track. Minutes per note. Denials overturned. Days to authorization. Calls handled per hour.
Write down where that number is today. That’s your baseline. Without it, every pilot looks good at the end, because the only evidence is how people feel about it.
Also agree on what “good enough” means. If the tool improves the measure a little, is that worth the cost? Decide that now, while nobody is attached to the result.
Pick a second measure that would show if something got worse. AI meant to reduce work can slow things down instead. In Washington state, under Medicare’s AI-assisted prior authorization pilot, UW Medicine said approvals that used to take one to three days were taking 15 to 20. Speed is easy to forget when the pilot is about accuracy.
What data does it need, and do we trust that data?
Every AI tool reads something. Notes, claims, schedules, the item master, the provider list. Ask your data team where that data comes from and how confident they are in it.
If the tool depends on data you already know is messy, the pilot will mostly measure your data quality, not the tool. That can still be worth doing, but go in knowing it.
This is also where privacy and security come in. Know what patient data leaves your environment, where it goes, and whether the right agreements are in place, including a business associate agreement under HIPAA where one is required. Your privacy and security teams should be in this conversation before the pilot starts, not after.
What happens when it’s wrong?
It will be wrong sometimes. Every tool is. The question is whether anyone will notice, and what happens if they don’t.
For a tool that drafts a letter, a person reads it before it goes out. That’s a natural check. For a tool that flags patients or suggests codes, a mistake might not be visible to anyone. Decide who reviews the output, how often, and what they do when they find a problem.
Be clear about who is accountable. If the tool makes a bad call, it should be obvious whose job it was to catch it.
Agents make this question sharper. A person reads a suggestion before acting on it. An agent that sends a request or updates a record doesn’t wait. Before the pilot, write down what the agent may do on its own, what needs a person’s approval, and what it may never touch.
If it works, who owns it, and can we afford it at full scale?
This is the question that kills the most pilots, because nobody asks it until the end.
A pilot is small. It runs on one unit, with extra attention, often at a discount. Full use is different. Ask what the tool costs across the whole system, year after year. Ask who supports it and who watches its performance over time. Ask which budget pays for it.
If nobody can answer, the pilot will end in a nice summary and no rollout. Better to know that now.
Using the questions
None of these questions is hard. That’s the point. They’re the questions a careful operator would ask about any new tool. AI gets a pass on them more often than it should, because the demos are impressive and everyone feels pressure to move.
You also don’t have to invent the oversight from scratch. In June, the Joint Commission launched a voluntary AI certification that looks at how an organization governs and monitors AI and trains its people. In late July, Hackensack Meridian Health became the first system to earn it. Your answers to these five questions are the kind of record that work depends on.
Asking them doesn’t slow you down much. It usually takes one working session with the people who would own and support the tool. What you get back is a pilot with a clear owner and a clear path to full use if it works.
Before you approve the next AI pilot, ask the sponsor to answer these five questions on one page. If they can’t fill it in, the pilot isn’t ready yet, no matter how good the demo was.
Discussion