Explainable and trustworthy AI for healthcare

Xin Sun’s research addresses two key challenges: first, how to make large language models (LLMs) explainable when used as digital therapists or healthcare tools — currently these models often act as “black boxes” with no insight into their reasoning. He introduces the Script-Strategy Aligned Generation (SSAG) approach, where the model predicts a conversational strategy (like “reflection” or “advice”) before generating any dialogue. Second, he explores how user trust can be built through a combination of human-computer interaction, natural language processing, and psychological measurement methods — offering new insights into how users perceive and evaluate AI systems. The impact is significant: without transparency or reliability, healthcare AI risks being misused, misunderstood, or rejected by end users.

These insights offer wide-ranging applications. The SSAG approach, for example, can enhance digital therapeutic tools — such as mental health chatbots or chronic care assistants — by making the reasoning behind system responses more understandable to users. His research also shows that people tend to trust LLM-powered conversational search systems more than traditional search engines, provided their workings are transparent. These methods can also help developers evaluate and improve trust in AI healthcare tools using behavioral and physiological data, which is crucial for regulation and broad implementation.

To read the full news release from CWI, visit: CWI – How can we make AI understandable and trustworthy for healthcare?