Successful AI implementation in healthcare requires guidance and oversight in the form of organized AI governance.
As healthcare AI solutions grow, the question of how to implement the technology for the benefit of patients, staff, and an organization’s bottom line remains top of mind. For many healthcare systems, AI governance is the answer.
Still, nearly half of CIOs and other tech leaders find that health systems lack a formal AI governance committee to oversee AI tool development and deployment, according to a recent study from hospital automation and AI software Qventus. Considering how just 5% of enterprises that adopt AI see real transformation, experts agree that AI governance is required for diverse healthcare environments made up of clinical, technical, and financial professionals.
“If you think of any kind of policy or anything that touches a clinical protocol, it’s necessary because you want to make sure things are happening in a standardized way, and there’s a way to improve,” says Mudit Garg, Qventus CEO and co-founder.
One example of AI governance at work is at the University of Arkansas for Medical Sciences (UAMS), where Dr. Joseph Sanford works as chief clinical informatics officer and director of the university’s Institute for Digital Health and Innovation. UAMS has established a university-wide governance committee that brings together leadership from the clinical, research, and education fronts to set policy, ensure alignment across initiatives, and execute AI vetting processes. All of this is underpinned by their standard cybersecurity, data security, and HIPAA protocols.
Sanford, who spends much of his time making AI work within the context of patient care and health system operations, has key questions that guide their AI governance: How do we make it impactful for clinicians and patients on a day-to-day basis, how do we measure success, and how can we understand what’s worth scarce time and resources.
A range of expertise
The AI governance committee at UAMS includes experts from the educational side, the biomedical informatics department, translational research institutes, and the CIO and CTO. It also includes stakeholders like IT professionals who focus on implementation.
Because of the healthcare organization’s principal technological focus to responsibly use clinical data, the governance committee focuses on compliance with the law and their own data privacy and transparency goals, as well as protection for patients and the organization at large.
Garg, whose company works with hundreds of hospitals and health systems across the US to automate care operations, has witnessed firsthand what leads to AI implementation success. AI governance and the committees that keep it in check are key factors.
“AI governance is still very much in its infancy,” he says. “This multidisciplinary aspect of pulling clinical, tech, and finance leaders together is critical and beneficial.” The first question, he adds, isn’t an AI question but can the health system unlock real benefits for this use case within their limited bandwidth. This is true in traditional ML mechanisms, as well as with more contemporary AI agents that act as niche assistants across the healthcare system’s workflow.
Garg adds that the leap from theoretical AI to operationalized implementation is massive. In other words, moving beyond AI demos to a place where these systems have real ROI, from finance to staff and patients, requires figuring out how well the tools work, whether there’s sufficient human-in-the-loop oversight, whether escalation is built in, and how to manage what the AI itself can’t tackle.
Possible side effects
One downside to AI governance committees, Garg says, is they sometimes only focus on the point in time when they’re reviewing it. But the health system and models are dynamic. Everything is changing all the time, so a monitoring mechanism has to be in place on an ongoing basis.
Another fault often lies in the multi-disciplinary approach. While the benefits of diverse perspectives are immense, it’s those same voices that can stall innovation. “What you also don’t want is it being so multidisciplinary that everyone ends up with 100 reasons to say no, but no one’s actually responsible to say yes,” he says. That’s where a deeply rooted AI governance framework comes into play. “It requires a bit of clarity to say, we need to understand the value, workflow impact, and safety, and make the decision under these use cases to move forward,” he adds.
In the Qventus study of healthcare CIOs, two-thirds of respondents reported they consider their organization’s AI strategy limited or fragmented. This is a major gap considering today’s AI technologies have the capacity to help hospitals save upward of $48 billion in a five-year period, assuming they have the right frameworks, implementation, and iteration strategies in place.
“Policy codifies principles,” says Sanford. “We were lucky to have highly principled individuals in alignment with what we wanted to accomplish. This made policy iteration relatively straightforward and collaborative.”
Even so, Sanford recognizes that innovation outpaces regulation. “The moment you get a policy revision through the committee, he says, it’s outdated. Because of that, policies and frameworks must be adaptable while respecting those hard lines laid out for the committee to follow.
Sanford and Garg still recognize how difficult all this is to put into practice. “Implementation of AI requires a lot of social work because gen AI advancements are popular and seemingly everyone has a notion of what they hope or fear the future will look like,” says Sanford. For his part, Garg adds it’s not an easy or straightforward process. “In a field that’s changing rapidly, and in principles that aren’t yet well-established, and in the degree of impact that’s possible in healthcare, there’s a desire to move fast,” he says.
The most successful healthcare systems, Garg says, are the ones that weigh the return for patients, staff, and finances in all their AI decisions, which is where the AI governance committee does its best work.
