Who Will Build the Systems to Oversee Clinical AI?

Who supports a patient with heart failure between appointments? ADVOCATE, a program run by the U.S. Advanced Research Projects Agency for Health (ARPA-H), plans to develop patient-facing AI alongside AI that monitors it. Its intended scope includes support with appointments, medications, diet and exercise, working in coordination with the clinical care team. Official program ↗

The September 9, 2026 award announcement set out a $62.7 million, four-year program, with up to $33.7 million in first-year funding. This does not mean that the full amount has already been spent. The program funds not only technology that interacts with patients, but also oversight and evaluation after deployment. Award announcement ↗

Roles and teams are listed in the award announcement ↗.

Researchers may want to look closely at the specific methods used to oversee AI decisions. Stanford plans a staged process of outlier filtering, rule-based checks and audits involving more detailed investigation, retaining rationales so that individual decisions can be examined later. This goes beyond showing one model's answers to another model. Which checks to perform, when to perform them and how much of the reasoning behind a decision to record are themselves part of the development work. Supervisory AI award details ↗

Evaluation in healthcare settings is a different task from research to improve a model's performance. Kaiser's plans include shadow-mode trials, in which AI operates without directly influencing patient care, and randomized trials embedded in routine clinical practice. Improving accuracy is not the same as investigating how patients' health and healthcare professionals' work change after a system is introduced. Implementation award details ↗

These roles suggest that technologies with a contribution to make are not limited to conversational models. Other possibilities include anomaly detection, implementing clinical rules, audit trails, integration with electronic health records and trial designs that measure the effects of an intervention. Asking “Whose decisions could our technology help check—and which ones?” may offer a different way to frame a research project. This is, however, an interpretation of the awards, not an announcement of a currently open funding call.

For a similar research effort in Japan, collaboration with healthcare providers cannot be left until the end of development. How to handle missing records, who receives alerts and how to hand over responsibility at night all need to be decided alongside the evaluation methods. Supervisory AI can also miss problems or generate false alarms. Will anyone notice when failures to detect problems compound? Could too many alerts add to the burden on clinical teams? It is important not to leave these evaluations solely to the developers.

The patient-facing AI teams are required to submit an FDA authorization package within 24 months of their contract awards. What has been announced is a development plan aimed at securing regulatory authorization, not products that have already obtained it. Award announcement ↗ What we want to follow next is not just progress toward submission, but results showing which kinds of errors the systems can detect and how well they engage healthcare professionals when their involvement is needed.