Deep learning in high dimension

• Verfasst von: Schwab, Christoph [VerfasserIn]
• Verfasst von: Zech, Jakob [VerfasserIn]
• Titel: Deep learning in high dimension
• Titelzusatz: neural network expression rates for analytic functions in L2(Rd,γd)
• Verf.angabe: Christoph Schwab and Jakob Zech
• E-Jahr: 2023
• Jahr: March 3, 2023
• Umfang: 36 S.
• Fussnoten: Gesehen am 29.06.2023 ; Im Titel sind die Zahl 2 und der Buchstabe d im Ausdruck "Rd" hochgestellt
• Titel Quelle: Enthalten in: Society for Industrial and Applied MathematicsSIAM ASA journal on uncertainty quantification
• Ort Quelle: Philadelphia, Pa. : SIAM, 2013
• Jahr Quelle: 2023
• Band/Heft Quelle: 11(2023), 1 vom: März, Seite 199-234
• ISSN Quelle: 2166-2525
• DOI: doi:10.1137/21M1462738
• Datenträger: Online-Ressource
• Sprache: eng
• K10plus-PPN: 1851264795

Abstract

Multigrid modeling algorithms are a technique used to accelerate iterative method models running on a hierarchy of similar graphlike structures. We introduce and demonstrate a new method for training neural networks which uses multilevel methods. Using an objective function derived from a graph-distance metric, we perform orthogonally-constrained optimization to find optimal prolongation and restriction maps between graphs. We compare and contrast several methods for performing this numerical optimization, and additionally present some new theoretical results on upper bounds of this type of objective function. Once calculated, these optimal maps between graphs form the core of multiscale artificial neural network (MsANN) training, a new procedure we present which simultaneously trains a hierarchy of neural network models of varying spatial resolution. Parameter information is passed between members of this hierarchy according to standard coarsening and refinement schedules from the multiscale modeling literature. In our machine learning experiments, these models are able to learn faster than training at the fine scale alone, achieving a comparable level of error with fewer weight updates (by an order of magnitude).