A future role for health applications of large language models depends on regulators enforcing safety standards

• Verfasst von: Freyer, Oscar [VerfasserIn]
• Verfasst von: Wiest, Isabella [VerfasserIn]
• Verfasst von: Kather, Jakob Nikolas [VerfasserIn]
• Verfasst von: Gilbert, Stephen [VerfasserIn]
• Titel: A future role for health applications of large language models depends on regulators enforcing safety standards
• Verf.angabe: Oscar Freyer, Isabella Catharina Wiest, Jakob Nikolas Kather, Stephen Gilbert
• E-Jahr: 2024
• Jahr: September 2024
• Umfang: 11 S.
• Illustrationen: Illustrationen
• Fussnoten: Online verfügbar: 21. August 2024 ; Gesehen am 11.10.2024
• Titel Quelle: Enthalten in: The lancet. Digital health
• Ort Quelle: London : The Lancet, 2019
• Jahr Quelle: 2024
• Band/Heft Quelle: 6(2024), 9 vom: Sept., Seite e662-e672
• ISSN Quelle: 2589-7500
• DOI: doi:10.1016/S2589-7500(24)00124-9
• Datenträger: Online-Ressource
• Sprache: eng
• K10plus-PPN: 1905437102

Abstract

Among the rapid integration of artificial intelligence in clinical settings, large language models (LLMs), such as Generative Pre-trained Transformer-4, have emerged as multifaceted tools that have potential for health-care delivery, diagnosis, and patient care. However, deployment of LLMs raises substantial regulatory and safety concerns. Due to their high output variability, poor inherent explainability, and the risk of so-called AI hallucinations, LLM-based health-care applications that serve a medical purpose face regulatory challenges for approval as medical devices under US and EU laws, including the recently passed EU Artificial Intelligence Act. Despite unaddressed risks for patients, including misdiagnosis and unverified medical advice, such applications are available on the market. The regulatory ambiguity surrounding these tools creates an urgent need for frameworks that accommodate their unique capabilities and limitations. Alongside the development of these frameworks, existing regulations should be enforced. If regulators fear enforcing the regulations in a market dominated by supply or development by large technology companies, the consequences of layperson harm will force belated action, damaging the potentiality of LLM-based applications for layperson medical advice.