Mechanism Analysis and Device Specifications Study for Neural Network Training in Analog Crosspoint Arrays using Tiki-Taka Algorithms

Abstract

Resistive memory device candidates for analog crosspoint array-based neural network training accelerators show various non-idealities, making realization of analog AI hardware challenging. Recently, specialized algorithms such as Tiki-Taka algorithms have been developed to mitigate such device non-idealities. Due to the trade-off between immunity to the device non-idealities and hardware and performance overhead, it is important to select the appropriate algorithm based on the selected crosspoint device's specifications. However, the criteria for this selection has been lacking. In this thesis, we investigate the impact of device non-idealities in analog in-memory computing using advanced training algorithms, Tiki-Taka algorithms, and quantify device specifications for energy-efficient neural network training. Tiki-Taka algorithm version 1 and version 2 are known to mitigate the negative effects of device non-idealities in analog in-memory computing, and therefore enable the accelerated training of deep neural networks. We perform simulation study in multi-dimensional parameter space to extract the device specification and quantitatively compare the performance of advanced training algorithms customized for analog AI hardware.