Care that measures itself: Quality systems, protocol-driven practice, and where AI actually helps

Care that measures itself: Quality systems, protocol-driven practice, and where AI actually helps

by Dr John Tilley · View the channel on PSI

Applied Sciences & Engineering Medicine & Health MBPHealth systems and servicesMBPMMedical administration and management
🎙 Dr John Tilley 👥 2K 📅 October 8, 2026 ⏱ 33 min 👁 44 📄 expert opinion 🧭 2026-10-15
Available in: English (current) Français

Keywords

quality systemsLLMdata interoperabilitySNOMED CTFHIR

Summary

The presentation discusses the challenges of measuring and improving quality in healthcare, emphasizing the need for closed-loop systems that link protocols, outcomes, and adjustments. Dr Tilley highlights the high cost and labor-intensive nature of traditional quality assurance, citing examples from his bone marrow transplant unit’s JC accreditation process. He critiques electronic health records (EHRs) for merely shifting costs to clinicians and failing to solve fundamental reasoning gaps, and illustrates the fragmentation of medical data across silos. He reviews major coding systems (ICD-10, SNOMED CT, LOINC, RxNorm, NAPPI) and the FHIR data standard, explaining their roles and limitations. He also discusses the UMLS project and its NLP approaches, now largely superseded by large language models (LLMs). He notes that LLMs can extract and codify free text efficiently, though they remain nondeterministic and struggle with context window limits. He proposes a pipeline where all patient data are transformed into text, processed by LLMs to create structured datasets, enabling scalable quality measurement. He concludes that routine data extraction is now easier, but scalability issues persist for very large patient records.

177 words

Critical Evaluation

Value of the Information & Strength of the Argument

The talk provides valuable practical insights into the real-world application of quality systems and informatics from a seasoned clinician. The argumentation is coherent, moving from the importance of quality measurement to the bottlenecks of manual data extraction, then to the limitations of existing systems and the potential of LLMs. The speaker supports claims with concrete examples (e.g., heart failure, BMT unit) and references to recognized standards, making the reasoning persuasive. The emphasis on the need for closed-loop systems and the caution regarding LLM limitations demonstrates balanced critical thinking.

Scientific Rigor, Source Quality, Title Accuracy

The presentation draws on authoritative sources such as JC, WHO’s ICD-10, SNOMED CT, LOINC, RxNorm, and FHIR standards, as well as Kahneman’s ‘Thinking Fast and Slow’. While no explicit academic citations are provided, the speaker’s hands-on involvement in building informatics systems lends credibility. The title accurately reflects the content, focusing on the theme of quality self-measurement and AI’s role, without overstating technical details. The talk is opinionated but grounded in practical experience, though it would benefit from deeper referencing.

182 words

Title / Content Match

The title aptly captures the core themes: quality measurement, protocol-based practice, and the role of AI in automating data extraction and performance evaluation.

Quality & Reliability

8/10

The talk is based on the speaker's extensive clinical and informatics experience, with references to established frameworks (JC, ICD-10, SNOMED CT) and well-known works (Kahneman). While primarily opinion, it offers practical insights grounded in real-world implementation.

Key Moments

Markers derived by PSI from the transcript: the creator did not define chapters.

Contribution & Novelties

The talk offers a clinician’s perspective on bridging quality measurement and AI by automating data extraction using LLMs, grounded in hands-on implementation experience. It synthesizes existing standards (ICD-10, SNOMED CT, FHIR) with novel workflow ideas, emphasizing closed-loop quality systems that are often neglected in practice.

Pour aller plus loin :

  • ICD-10 — Overview of the WHO coding system, its population-level design, and limitations in clinical detail.
  • SNOMED CT — Description of the clinical ontology and its role in semantic interoperability.
  • FHIR — Fast Healthcare Interoperability Resources standard for exchanging healthcare data.
  • UMLS — The Unified Medical Language System and its metathesaurus.
  • Large language model — Background on LLMs and their applications in natural language processing.
  • Thinking Fast and Slow by Daniel Kahneman — Reference on cognitive biases affecting feedback loops (no URL).

132 words

Radar Profile

The radar shows balanced scores across quantity, quality, and technical level, with reliability slightly higher, reflecting the speaker's practical expertise. The presentation is information-dense yet accessible, aligning with the intended academic audience.

Reliability 8/10