Quantized artificial neural networks implemented with spintronic stochastic computing

Abstract An artificial neural network (ANN) inference involves matrix vector multiplications that require a very large number of multiply and accumulate operations, resulting in high energy cost and large device footprint. Stochastic computing (SC) offers a less resource-intensive ANN implementation with minimal accuracy loss. Random number generators (RNG) are required to implement SC in hardware. These can be realized through stochastic-magnetic tunnel junctions (s-MTJ), where the energy barrier to switch between the ‘up’ and ‘down’ states is designed to be small, enabling thermal noise to generate a random bit stream. While s-MTJs have previously been used to implement SC-ANNs, these studies have been limited to architectures with continuously varying (i.e., analog) weights. In this work, we study the use of SC for matrix vector multiplication with quantized synaptic weights and quantized outputs. We show that a quantized SC-ANN, implemented by using experimentally obtained s-MTJ bitstreams and a limited number of discrete quantized states for both weights and hidden layer nodes in an ANN, can effectively reduce time (latency) and energy consumption in SC compared to an analog implementation, while largely preserving accuracy. We implemented quantization with 5 and 11 quantized states, along with SC configured with stochastic bitstream lengths of 100, 200, 300, 400, and 500 on neural networks with one hidden layer and three hidden layers. Inference was performed on the MNIST dataset for both training with SC and without SC. Training with SC provided better accuracy for all cases. For the shortest bitstream of 100 bits, the highest accuracies were 92% for one hidden layer and over 96% for three hidden layers. The overall system attained its peak accuracy of 96.82% using a 400 bit stochastic bitstream with three hidden layers. Our investigations demonstrate 9× improvement in latency and 2.6× improvement in energy consumption using the quantized SC approach compared to a similar s-MTJ based ANN architecture without quantization.