Reliability, accuracy, observability, and auditability are crucial in building LLM workflows in healthcare. All of these rely on the ability to measure LLM automations at scale. But as the metrics we care about in GenAI applications (e.g. hallucinations, adherence to a policy, etc.) are complex, traditional machine learning or NLP metrics are not relevant anymore. These measurements can only be conducted by other LLMs that are tuned specifically for judging, i.e., LLM‑Judges. As the evaluators are LLMs themselves in this paradigm, they also need to be observed, measured, and tuned to prevent drifts from expected behaviour. This talk will delve into LLM‑Judges in the context of healthcare LLM workflows and agents.
Reliability, accuracy, observability, and auditability are crucial in building LLM workflows in healthcare. All of these rely on the ability to measure LLM automations at scale. But as the metrics we care about...
The integration of Natural Language Processing (NLP) and Generative AI in healthcare holds transformative potential, enabling efficient diagnostics, personalised care, and streamlined administrative tasks. However, these technologies present unique challenges...
Image series analysis represent a common way of analyzing patient condtion by practitioner. Vast amount of parameters and image acquision techniques may lead to various ways of reprsenting volumetric data...
The integration of Artificial Intelligence (AI) and Large Language Models (LLMs) into simulation-based education is transforming healthcare training by enhancing scalability, adaptability, and accessibility. AI‑driven technologies, including LLMs such as...
Demos
See More