Data extraction from free-text reports on mechanical thrombectomy in acute ischemic stroke using ChatGPT

• Verfasst von: Lehnen, Nils Christian [VerfasserIn]
• Verfasst von: Dorn, Franziska [VerfasserIn]
• Verfasst von: Wiest, Isabella [VerfasserIn]
• Verfasst von: Zimmermann, Hanna [VerfasserIn]
• Verfasst von: Radbruch, Alexander [VerfasserIn]
• Verfasst von: Kather, Jakob Nikolas [VerfasserIn]
• Verfasst von: Paech, Daniel [VerfasserIn]
• Titel: Data extraction from free-text reports on mechanical thrombectomy in acute ischemic stroke using ChatGPT
• Titelzusatz: a retrospective analysis
• Verf.angabe: Nils C. Lehnen, MD, Franziska Dorn, MD, Isabella C. Wiest, MD, MSc, Hanna Zimmermann, MD, Alexander Radbruch, MD, JD, Jakob Nikolas Kather, MD, MSc, Daniel Paech, MD, PhD
• E-Jahr: 2024
• Jahr: April 2024
• Umfang: 8 S.
• Illustrationen: Illustrationen
• Fussnoten: Online veröffentlicht: 16. April 2024 ; Gesehen am 27.11.2024
• Titel Quelle: Enthalten in: Radiology
• Ort Quelle: Oak Brook, Ill. : Soc., 1923
• Jahr Quelle: 2024
• Band/Heft Quelle: 311(2024), 1 vom: Apr., Artikel-ID e232741, Seite 1-8
• ISSN Quelle: 1527-1315
• DOI: doi:10.1148/radiol.232741
• Datenträger: Online-Ressource
• Sprache: eng
• K10plus-PPN: 1909635545

Abstract

Background - - Procedural details of mechanical thrombectomy in patients with ischemic stroke are important predictors of clinical outcome and are collected for prospective studies or national stroke registries. To date, these data are collected manually by human readers, a labor-intensive task that is prone to errors. - - Purpose - - To evaluate the use of the large language models (LLMs) GPT-4 and GPT-3.5 to extract data from neuroradiology reports on mechanical thrombectomy in patients with ischemic stroke. - - Materials and Methods - - This retrospective study included consecutive reports from patients with ischemic stroke who underwent mechanical thrombectomy between November 2022 and September 2023 at institution 1 and between September 2016 and December 2019 at institution 2. A set of 20 reports was used to optimize the prompt, and the ability of the LLMs to extract procedural data from the reports was compared using the McNemar test. Data manually extracted by an interventional neuroradiologist served as the reference standard. - - Results - - A total of 100 internal reports from 100 patients (mean age, 74.7 years ± 13.2 [SD]; 53 female) and 30 external reports from 30 patients (mean age, 72.7 years ± 13.5; 18 male) were included. All reports were successfully processed by GPT-4 and GPT-3.5. Of 2800 data entries, 2631 (94.0% [95% CI: 93.0, 94.8]; range per category, 61%-100%) data points were correctly extracted by GPT-4 without the need for further postprocessing. With 1788 of 2800 correct data entries, GPT-3.5 produced fewer correct data entries than did GPT-4 (63.9% [95% CI: 62.0, 65.6]; range per category, 14%-99%; P < .001). For the external reports, GPT-4 extracted 760 of 840 (90.5% [95% CI: 88.3, 92.4]) correct data entries, while GPT-3.5 extracted 539 of 840 (64.2% [95% CI: 60.8, 67.4]; P < .001). - - Conclusion - - Compared with GPT-3.5, GPT-4 more frequently extracted correct procedural data from free-text reports on mechanical thrombectomy performed in patients with ischemic stroke. - - © RSNA, 2024 - - Supplemental material is available for this article.