Beyond one-hot encoding

• Verfasst von: Rodriguez, Pau [VerfasserIn]
• Verfasst von: Bautista, Miguel [VerfasserIn]
• Verfasst von: Gonzàlez, Jordi [VerfasserIn]
• Verfasst von: Escalera, Sergio [VerfasserIn]
• Titel: Beyond one-hot encoding
• Titelzusatz: lower dimensional target embedding
• Verf.angabe: Pau Rodríguez, Miguel A. Bautista, Jordi Gonzàlez, Sergio Escalera
• E-Jahr: 2018
• Jahr: 11 May 2018
• Umfang: 11 S.
• Fussnoten: Gesehen am 21.04.2020
• Titel Quelle: Enthalten in: Image and vision computing
• Ort Quelle: Amsterdam [u.a.] : Elsevier Science, 1983
• Jahr Quelle: 2018
• Band/Heft Quelle: 75(2018), Seite 21-31
• DOI: doi:10.1016/j.imavis.2018.04.004
• Datenträger: Online-Ressource
• Sprache: eng
• Sach-SW: Computer vision
• Sach-SW: Deep learning
• Sach-SW: Error correcting output codes
• Sach-SW: Output embeddings
• K10plus-PPN: 1695324978

Abstract

Target encoding plays a central role when learning Convolutional Neural Networks. In this realm, one-hot encoding is the most prevalent strategy due to its simplicity. However, this so widespread encoding schema assumes a flat label space, thus ignoring rich relationships existing among labels that can be exploited during training. In large-scale datasets, data does not span the full label space, but instead lies in a low-dimensional output manifold. Following this observation, we embed the targets into a low-dimensional space, drastically improving convergence speed while preserving accuracy. Our contribution is two fold: (i) We show that random projections of the label space are a valid tool to find such lower dimensional embeddings, boosting dramatically convergence rates at zero computational cost; and (ii) we propose a normalized eigenrepresentation of the class manifold that encodes the targets with minimal information loss, improving the accuracy of random projections encoding while enjoying the same convergence rates. Experiments on CIFAR-100, CUB200-2011, Imagenet, and MIT Places demonstrate that the proposed approach drastically improves convergence speed while reaching very competitive accuracy rates.