Collective human intelligence outperforms artificial intelligence in a skin lesion classification task

• Verfasst von: Winkler, Julia K. [VerfasserIn]
• Verfasst von: Kommoss, Katharina [VerfasserIn]
• Verfasst von: Müller-Christmann, Christine [VerfasserIn]
• Verfasst von: Toberer, Ferdinand [VerfasserIn]
• Verfasst von: Enk, Alexander [VerfasserIn]
• Verfasst von: Abassi, Mohamed Souhayel [VerfasserIn]
• Verfasst von: Fuchs, Tobias [VerfasserIn]
• Verfasst von: Blum, Andreas [VerfasserIn]
• Verfasst von: Stolz, Wilhelm [VerfasserIn]
• Verfasst von: Coras-Stepanek, Brigitte [VerfasserIn]
• Verfasst von: Cipic, Robert [VerfasserIn]
• Verfasst von: Guther, Stefanie [VerfasserIn]
• Verfasst von: Hänßle, Holger [VerfasserIn]
• Titel: Collective human intelligence outperforms artificial intelligence in a skin lesion classification task
• Verf.angabe: Julia K. Winkler, Katharina Sies, Christine Fink, Ferdinand Toberer, Alexander Enk, Mohamed Souhayel Abassi, Tobias Fuchs, Andreas Blum, Wilhelm Stolz, Brigitte Coras-Stepanek, Robert Cipic, Stefanie Guther, Holger A. Haenssle
• Jahr: 2021
• Umfang: 7 S.
• Fussnoten: Gesehen am 04.09.2023
• Titel Quelle: Enthalten in: Deutsche Dermatologische GesellschaftJournal der Deutschen Dermatologischen Gesellschaft
• Ort Quelle: Berlin : Wiley-Blackwell, 2003
• Jahr Quelle: 2021
• Band/Heft Quelle: 19(2021), 8, Seite 1178-1184
• ISSN Quelle: 1610-0387
• DOI: doi:10.1111/ddg.14510
• Datenträger: Online-Ressource
• Sprache: eng
• Sach-SW: artificial intelligence
• Sach-SW: collective
• Sach-SW: convolution neural network
• Sach-SW: skin lesion classification
• K10plus-PPN: 1858742358
• K10plus-PPN:
  Universitätsbibliothek
• Lokale URL UB: Zum Volltext

Abstract

Background and objectives Convolutional neural networks (CNN) enable accurate diagnosis of medical images and perform on or above the level of individual physicians. Recently, collective human intelligence (CoHI) was shown to exceed the diagnostic accuracy of individuals. Thus, diagnostic performance of CoHI (120 dermatologists) versus individual dermatologists versus two state-of-the-art CNN was investigated. Patients and Methods Cross-sectional reader study with presentation of 30 clinical cases to 120 dermatologists. Six diagnoses were offered and votes collected via remote voting devices (quizzbox®, Quizzbox Solutions GmbH, Stuttgart, Germany). Dermatoscopic images were classified by a binary and multiclass CNN (FotoFinder Systems GmbH, Bad Birnbach, Germany). Three sets of diagnostic classifications were scored against ground truth: (1) CoHI, (2) individual dermatologists, and (3) CNN. Results CoHI attained a significantly higher accuracy [95 % confidence interval] (80.0 % [62.7 %-90.5 %]) than individual dermatologists (75.7 % [73.8 %-77.5 %]) and CNN (70.0 % [52.1 %-83.3 %]; all P < 0.001) in binary classifications. Moreover, CoHI achieved a higher sensitivity (82.4 % [59.0 %-93.8 %]) and specificity (76.9 % [49.7 %-91.8 %]) than individual dermatologists (sensitivity 77.8 % [75.3 %-80.2 %], specificity 73.0 % [70.6 %-75.4 %]) and CNN (sensitivity 70.6 % [46.9 %-86.7 %], specificity 69.2 % [42.4 %-87.3 %]). The diagnostic accuracy of CoHI was superior to that of individual dermatologists (P < 0.001) in multiclass evaluation, with the accuracy of the latter comparable to multiclass CNN. Conclusions Our analysis revealed that the majority vote of an interconnected group of dermatologists (CoHI) outperformed individuals and CNN in a demanding skin lesion classification task.