Status: Bibliographieeintrag
Standort: ---
Exemplare:
---
| Online-Ressource |
Verfasst von: | Damrich, Sebastian [VerfasserIn]  |
| Böhm, Jan Niklas [VerfasserIn]  |
| Hamprecht, Fred [VerfasserIn]  |
| Kobak, Dmitry [VerfasserIn]  |
Titel: | Contrastive learning unifies t-SNE and UMAP |
Verf.angabe: | Sebastian Damrich, Jan Niklas Böhm, Fred A. Hamprecht, Dmitry Kobak |
E-Jahr: | 2022 |
Jahr: | 3 Jun 2022 |
Umfang: | 29 S. |
Fussnoten: | Gesehen am 18.10.2022 |
Titel Quelle: | Enthalten in: De.arxiv.org |
Ort Quelle: | [S.l.] : Arxiv.org, 1991 |
Jahr Quelle: | 2022 |
Band/Heft Quelle: | (2022), Artikel-ID 2206.01816, Seite 1-29 |
Abstract: | Neighbor embedding methods $t$-SNE and UMAP are the de facto standard for visualizing high-dimensional datasets. They appear to use very different loss functions with different motivations, and the exact relationship between them has been unclear. Here we show that UMAP is effectively negative sampling applied to the $t$-SNE loss function. We explain the difference between negative sampling and noise-contrastive estimation (NCE), which has been used to optimize $t$-SNE under the name NCVis. We prove that, unlike NCE, negative sampling learns a scaled data distribution. When applied in the neighbor embedding setting, it yields more compact embeddings with increased attraction, explaining differences in appearance between UMAP and $t$-SNE. Further, we generalize the notion of negative sampling and obtain a spectrum of embeddings, encompassing visualizations similar to $t$-SNE, NCVis, and UMAP. Finally, we explore the connection between representation learning in the SimCLR setting and neighbor embeddings, and show that (i) $t$-SNE can be optimized using the InfoNCE loss and in a parametric setting; (ii) various contrastive losses with only few noise samples can yield competitive performance in the SimCLR setup. |
DOI: | doi:10.48550/arXiv.2206.01816 |
URL: | Bitte beachten Sie: Dies ist ein Bibliographieeintrag. Ein Volltextzugriff für Mitglieder der Universität besteht hier nur, falls für die entsprechende Zeitschrift/den entsprechenden Sammelband ein Abonnement besteht oder es sich um einen OpenAccess-Titel handelt.
Volltext ; Verlag: https://doi.org/10.48550/arXiv.2206.01816 |
| Volltext: http://arxiv.org/abs/2206.01816 |
| DOI: https://doi.org/10.48550/arXiv.2206.01816 |
Datenträger: | Online-Ressource |
Sprache: | eng |
Sach-SW: | Computer Science - Human-Computer Interaction |
| Computer Science - Machine Learning |
K10plus-PPN: | 181907434X |
Verknüpfungen: | → Sammelwerk |
Contrastive learning unifies t-SNE and UMAP / Damrich, Sebastian [VerfasserIn]; 3 Jun 2022 (Online-Ressource)
68975327