| |  Online-Ressource |
|---|
| Verfasst von: | Ferber, Dyke [VerfasserIn]  |
|---|
| | Wölflein, Georg [VerfasserIn] |
|---|
| | Wiest, Isabella [VerfasserIn] |
|---|
| | Ligero, Marta [VerfasserIn] |
|---|
| | Sainath, Srividhya [VerfasserIn] |
|---|
| | Ghaffari Laleh, Narmin [VerfasserIn] |
|---|
| | El Nahhas, Omar S. M. [VerfasserIn] |
|---|
| | Müller-Franzes, Gustav [VerfasserIn] |
|---|
| | Jäger, Dirk [VerfasserIn] |
|---|
| | Truhn, Daniel [VerfasserIn] |
|---|
| | Kather, Jakob Nikolas [VerfasserIn] |
|---|
| Titel: | In-context learning enables multimodal large language models to classify cancer pathology images |
|---|
| Verf.angabe: | Dyke Ferber, Georg Wölflein, Isabella C. Wiest, Marta Ligero, Srividhya Sainath, Narmin Ghaffari Laleh, Omar S. M. El Nahhas, Gustav Müller-Franzes, Dirk Jäger, Daniel Truhn & Jakob Nikolas Kather |
|---|
| E-Jahr: | 2024 |
|---|
| Jahr: | 21 November 2024 |
|---|
| Umfang: | 12 S. |
|---|
| Illustrationen: | Illustrationen |
|---|
| Fussnoten: | Gesehen am 28.04.2025 |
|---|
| Titel Quelle: | Enthalten in: Nature Communications |
|---|
| Ort Quelle: | [London] : Springer Nature, 2010 |
|---|
| Jahr Quelle: | 2024 |
|---|
| Band/Heft Quelle: | 15(2024), Artikel-ID 10104, Seite 1-12 |
|---|
| ISSN Quelle: | 2041-1723 |
|---|
| Abstract: | Medical image classification requires labeled, task-specific datasets which are used to train deep learning networks de novo, or to fine-tune foundation models. However, this process is computationally and technically demanding. In language processing, in-context learning provides an alternative, where models learn from within prompts, bypassing the need for parameter updates. Yet, in-context learning remains underexplored in medical image analysis. Here, we systematically evaluate the model Generative Pretrained Transformer 4 with Vision capabilities (GPT-4V) on cancer image processing with in-context learning on three cancer histopathology tasks of high importance: Classification of tissue subtypes in colorectal cancer, colon polyp subtyping and breast tumor detection in lymph node sections. Our results show that in-context learning is sufficient to match or even outperform specialized neural networks trained for particular tasks, while only requiring a minimal number of samples. In summary, this study demonstrates that large vision language models trained on non-domain specific data can be applied out-of-the box to solve medical image-processing tasks in histopathology. This democratizes access of generalist AI models to medical experts without technical background especially for areas where annotated data is scarce. |
|---|
| DOI: | doi:10.1038/s41467-024-51465-9 |
|---|
| URL: | Bitte beachten Sie: Dies ist ein Bibliographieeintrag. Ein Volltextzugriff für Mitglieder der Universität besteht hier nur, falls für die entsprechende Zeitschrift/den entsprechenden Sammelband ein Abonnement besteht oder es sich um einen OpenAccess-Titel handelt.
kostenfrei: Volltext: https://doi.org/10.1038/s41467-024-51465-9 |
|---|
| | kostenfrei: Volltext: https://www.nature.com/articles/s41467-024-51465-9 |
|---|
| | DOI: https://doi.org/10.1038/s41467-024-51465-9 |
|---|
| Datenträger: | Online-Ressource |
|---|
| Sprache: | eng |
|---|
| Sach-SW: | Cancer |
|---|
| | Computer science |
|---|
| | Diagnostic markers |
|---|
| | Machine learning |
|---|
| | Oncology |
|---|
| K10plus-PPN: | 1923737155 |
|---|
| Verknüpfungen: | → Zeitschrift |
|---|
In-context learning enables multimodal large language models to classify cancer pathology images / Ferber, Dyke [VerfasserIn]; 21 November 2024 (Online-Ressource)
