BNN implementation using memristors is accurate and efficient than CMOS and FPGA
Date: 13/12/2023
By using a complete memristor-Based Bayesian Neural Network implementation, Researchers at CEA Leti able to handle a real world case of analyzing recordings of heart' health-condition called arrhythmia to classify its types with precise aleatoric and epistemic uncertainty.
Pic above: Wafer-level characterization of memristor crossbar arrays using a prober, highlighting detailed probe card analysis at an enlarged scale.
Bayesian neural networks is said be good option for safety-critical, sensory-processing applications requiring accurate decisions based on a small amount of noisy input data. But Bayesian neural networks require high computation and lot of power when it is done on CMOS-based ASICs and FPGAs, due to huge transfer of data between processor and memory. Implementing the networks in hardware requires a random number generator to store synaptic weights and such probability distributions. It also requires massive parallel multiply-and-accumulate (MAC) operations.
Researchers have innovatively used crossbars of memristors that naturally implement the multiplication between the input voltage and the probabilistic synaptic weight through Ohm's law, and the accumulation through Kirchhoff's current law, to significantly lower power consumption.
CEA Leti researchers have published a new paper in Nature Communications presenting the first complete memristor-based Bayesian neural network implementation.
“Our paper presents, for the first time, a complete hardware implementation of a Bayesian neural network utilizing the intrinsic variability of memristors to store these probability distributions," said Elisa Vianello, CEA-Leti chief scientist and co-author of the paper, “Bringing Uncertainty Quantification to the Extreme-Edge with Memristor-Based Bayesian Neural Networks". “We exploited the intrinsic variability of memristors to store these probability distributions, instead of using random number generators."
Co-author Damien Querlioz, a scientist associated with the University of Paris-Saclay, the French National Center for Scientific Research (CNRS) and the Center of Nanosciences and Nanotechnologies, said the team also had to reconcile the nature of memristors, whose statistical effects adhere to the laws of device physics, with Bayesian neural networks, in which these effects can take arbitrary shapes.
“This work overcomes that challenge with a new training algorithm – variational inference augmented by a 'technological loss' – that accommodates device non-idealities during the learning phase," he said. “Our approach enables the Bayesian neural network to be compatible with the imperfections of our memristors."
Uncertainty quantification involves the network's ability to identify unknown situations out-of-distributions. If a traditional neural network trained to recognize cats and dogs is presented with an image of a giraffe, it “confidently misclassifies" it as a cat or a dog, Vianello said. “In contrast, a Bayesian neural network would respond, 'I am not entirely sure what this is because I have never seen it.' While this example is lighthearted, in critical environments like medical diagnosis, where incorrect predictions can have severe consequences, this uncertainty-capturing ability becomes crucial."?
This capability arises from the fact that synaptic values in Bayesian neural networks are not precise values, as in traditional neural networks, but rather probability distributions. Consequently, the output is also a probability distribution, providing information about its “certainty".
Pic above: Scanning Electron Microscopy image of a filamentary memristor in the back end of line of our hybrid memristor/CMOS process
News Source: CEA Leti
