There is a lot of AI fatigue in healthcare right now, and it's earned.

Every new project sounds manageable when it is introduced: a pilot, a few meetings, some feedback, a limited rollout.

But operators experience those projects on top of everything else they are already managing: staffing, referrals, compliance, quality, reimbursement pressure and day to day fires.

The real cost of a new AI project is often not the software. It is the attention required to absorb it.

I think that gives us a better way to evaluate AI projects. Not just:

What value could this create?

But also:

How much organizational attention will this require before it becomes routine?

A good pilot should have an attention decay curve

One implementation made this especially clear to me.

At a home health agency with roughly 600 patients, initial scoping took about two weeks. We then held a kickoff with clinical and operations leaders, aligned on the workflow, and ran a 60 day pilot.

That required real attention. People needed to understand what the technology was doing. Teams had questions. We reviewed interactions and made adjustments. The agency needed confidence that the information being surfaced was useful and that exceptions were handled appropriately.

That is normal. The important part came after.

Once the workflow was established, the agency's involvement dropped sharply. Our 30 minute review meeting went from weekly to every two weeks, and we expect it to move to once a month.

And the program is doing real work. In one recent week, it made 676 calls and had conversations with 400 patients, about two thirds of the agency's census.

The agency is no longer actively managing an "AI project." The technology has become part of the operating model.

That, to me, is the goal.

A good pilot should have a visible decay curve in organizational attention. If maintaining a workflow on day 90 requires as much attention as launching it did on day 10, something is probably wrong.

Before starting a pilot, I would ask four questions

1. Does this solve a problem we already care about?

AI projects struggle when the use case is interesting but not operationally important.

The problem should already be on the agency's agenda. If leadership does not already care about the outcome, the project will eventually lose oxygen.

2. How many people need to change behavior for this to work?

This is one of the most underestimated implementation costs.

If a project requires clinicians, schedulers, branch managers, intake teams and administrators to all learn a new process, the burden is much larger than the software contract suggests.

The best first project often creates meaningful value while asking relatively few people to work differently. That does not mean avoiding change. It means being deliberate about how much change is required to produce the outcome.

3. Is the technology filling a capacity gap or disrupting an existing workflow?

Those are very different things.

If AI enters a process that already has established roles, ownership and handoffs, there will naturally be more change management.

If it adds capacity where the organization already wants to do more but does not have enough staff or time, adoption can be much easier.

That was the case at the agency above. The program made outreach calls the team already wanted to make but did not have the capacity for. Nobody's role was being rerouted, which made the change much easier to absorb.

That distinction is worth making before the pilot starts.

4. What does routine operation look like after the pilot?

This may be the most important question.

Who still needs to be involved? How many meetings are required? Does someone need to check another dashboard every day? Does the vendor still need to chase feedback? Does the organization still feel like it is running a project?

If the answer is yes to all of those, the pilot has not really become operational.

Vendors should be measured on the burden they remove

I also think vendors need to take more responsibility here. As a vendor, this is the standard I hold us to.

It is not enough to say a product is easy to implement. The vendor should prove it.

The vendor should own the project management burden, define the workflow, surface what needs review, make the adjustments and reduce the number of decisions the operator has to make.

We need the operator's judgment. We should not need their project management capacity.

The best technology should become less demanding over time. Not more.

Maybe this is the better test

Healthcare organizations do not need more AI pilots.

They need more AI projects that make it through the pilot stage and become boring. Routine, trusted, useful and quietly operating in the background.

So when evaluating the next AI project, I would not only ask:

How much value could this create?

I would also ask:

How quickly can this stop feeling like a project?

Because the best AI implementation may not be the one with the biggest theoretical upside. It may be the one that solves a real problem and then gets out of the organization's way.