Energy-efficient image processing using binary neural networks with Hadamard transform

Abstract

Binary neural networks have recently begun to be used as a highly energy- and computation-efficient image processing technique for computer vision tasks. This paper proposes a novel extension of existing binary neural network technology based on the use of a {\em Hadamard transform} in the input layer of a binary neural network. Previous state-of-the-art binary neural networks require floating-point arithmetic at several parts of the neural network model computation in order to maintain a sufficient level of accuracy. The Hadamard transform is similar to a Discrete Cosine Transform (used in the popular JPEG image compression method) except that it does not include expensive multiplication operations. In this paper, it is shown that the Hadamard transform can be used to replace the most expensive floating-point arithmetic portion of a binary neural network. In order to test the efficacy of this proposed method, three types of experiments were conducted: application of the proposed method to several state-of-the-art neural network models, verification of its effectiveness in a large image dataset (ImageNet), and experiments to verify the effectiveness of the Hadamard transform by comparing the performance of binary neural networks with and without the Hadamard transform. The results show that the Hadamard transform can be used to implement a highly energy-efficient binary neural network with only a miniscule loss of accuracy.