Source code and data for the PhD thesis "Measuring the contributions of vision and text modalities in multimodal transformers"

• Verfasst von: Pârcălăbescu, Letiția [VerfasserIn]
• Titel: Source code and data for the PhD thesis "Measuring the contributions of vision and text modalities in multimodal transformers"
• Verf.angabe: Letitia Parcalabescu
• Verlagsort: Heidelberg
• Verlag: Universität
• E-Jahr: 2024
• Jahr: 2024-10-14
• Umfang: 1 Online-Ressource (5 Files)
• Fussnoten: Gefördert durch: bwHPC and the German Research Foundation (DFG): INST 35/1597-1 FUGG ; Gesehen am 14.10.2024
• DOI: doi:10.11588/data/68HOOP
• Datenträger: Online-Ressource
• Dokumenttyp: Forschungsdaten
• Dokumenttyp: Datenbank
• Sprache: eng
• Sonstige Nr.: Grant number: DFG INST 35/1597-1 FUGG
• Sach-SW: Computer and Information Science
• K10plus-PPN: 1905638248
• K10plus-PPN:
  Universitätsbibliothek
• Lokale URL UB: Zum Volltext

Abstract

This dataset contains source code and data used in the PhD thesis "Measuring the Contributions of Vision and Text Modalities in Multimodal Transformers". The dataset is split into five repositories: Code and resources related to chapter 2 of the thesis (Section 2.2., method described in "Using Scene Graph Representations and Knowledge Bases"). Code and resources related to chapter 3 of the thesis (VALSE dataset). Code and resources related to chapter 4 of the thesis: MM-SHAP measure and experiments code. Code and resources related to chapter 5 of the thesis: CCSHAP measure and experiments code related to large language models (LLMs). Code and resources related to the experiments with vision and language model decoders from chapters 3, 4, and 5.