Reward-based learning under hardware constraints

• Verfasst von: Friedmann, Simon [VerfasserIn]
• Verfasst von: Frémaux, Nicolas [VerfasserIn]
• Verfasst von: Schemmel, Johannes [VerfasserIn]
• Verfasst von: Gerstner, Wulfram [VerfasserIn]
• Verfasst von: Meier, Karlheinz [VerfasserIn]
• Titel: Reward-based learning under hardware constraints
• Titelzusatz: using a RISC processor embedded in a neuromorphic substrate
• Verf.angabe: Simon Friedmann, Nicolas Frémaux, Johannes Schemmel, Wulfram Gerstner and Karlheinz Meier
• E-Jahr: 2013
• Jahr: 20 September 2013
• Umfang: 17 S.
• Fussnoten: Gesehen am 31.05.2021
• Titel Quelle: Enthalten in: Frontiers in neuroscience
• Ort Quelle: Lausanne : Frontiers Research Foundation, 2007
• Jahr Quelle: 2013
• Band/Heft Quelle: 7(2013), Artikel-ID 160, Seite 1-17
• ISSN Quelle: 1662-453X
• DOI: doi:10.3389/fnins.2013.00160
• Datenträger: Online-Ressource
• Sprache: eng
• Sach-SW: hardware constraints analysis
• Sach-SW: large-scale spiking neural net- works
• Sach-SW: neuromorphic hardware
• Sach-SW: reinforcement learning
• Sach-SW: spike-timing dependent plasticity
• Sach-SW: wafer-scale integration
• K10plus-PPN: 1759240141

Abstract

In this study, we propose and analyze in simulations a new, highly flexible method of imple- menting synaptic plasticity in a wafer-scale, accelerated neuromorphic hardware system. The study focuses on globally modulated STDP, as a special use-case of this method. Flexibility is achieved by embedding a general-purpose processor dedicated to plasticity into the wafer. To evaluate the suitability of the proposed system, we use a reward modulated STDP rule in a spike train learning task. A single layer of neurons is trained to fire at specific points in time with only the reward as feedback. This model is simulated to measure its performance, i.e. the in- crease in received reward after learning. Using this performance as baseline, we then simulate the model with various constraints imposed by the proposed implementation and compare the performance. The simulated constraints include discretized synaptic weights, a restricted inter- face between analog synapses and embedded processor, and mismatch of analog circuits. We find that probabilistic updates can increase the performance of low-resolution weights, a simple interface between analog synapses and processor is sufficient for learning, and performance is insensitive to mismatch. Further, we consider communication latency between wafer and the conventional control computer system that is simulating the environment. This latency increases the delay, with which the reward is sent to the embedded processor. Because of the time continu- ous operation of the analog synapses, delay can cause a deviation of the updates as compared to the not delayed situation. We find that for highly accelerated systems latency has to be kept to a minimum. This study demonstrates the suitability of the proposed implementation to emulate the selected reward modulated STDP learning rule. It is therefore an ideal candidate for imple- mentation in an upgraded version of the wafer-scale system developed within the BrainScaleS project.