Energy-Efficient Recurrent Neural Network With MRAM-Based Probabilistic Activation Functions
Document Type
Article
Publication Date
4-1-2023
Abstract
Herein, we develop a programmable energy-efficient hardware implementation for Recurrent Neural Networks (RNNs) with Resistive Random-Access Memory (ReRAM) synapses and ultra-low power, area-efficient spin-based activation functions. To attain high energy-efficiency while maintaining accuracy, a novel Computing-in-Memory (CiM) architecture is proposed to leverage data-level parallelism during the evaluation phase. We employ an MRAM-based Adjustable Probabilistic Activation Function (APAF) via a low-power tunable activation mechanism, providing adjustable levels of accuracy to mimic ideal sigmoid and tanh thresholding along with a matching algorithm to regulate the neuron properties. Our hardware/software cross-layer simulation shows that our proposed design achieves up to 74.5× energy-efficiency with ∼11× area reduction compared to its counterpart designs while keeping the accuracy comparable.
Identifier
85137922115 (Scopus)
Publication Title
IEEE Transactions on Emerging Topics in Computing
External Full Text Location
https://doi.org/10.1109/TETC.2022.3202112
e-ISSN
21686750
First Page
534
Last Page
540
Issue
2
Volume
11
Recommended Citation
Sheikhfaal, Shadi; Angizi, Shaahin; and Demara, Ronald F., "Energy-Efficient Recurrent Neural Network With MRAM-Based Probabilistic Activation Functions" (2023). Faculty Publications. 1807.
https://digitalcommons.njit.edu/fac_pubs/1807