2019 Volume 16 Issue 2 Pages 20180909
Convolutional neural networks (CNNs) are being widely used in computer vision tasks, and there have been many efforts to implement CNNs in ASIC or FPGA for power-hungry environments. Instead of the previous common representation, the fixed-point representation, this letter proposes a short floating-point representation for CNNs. The short floating-point representation is based on the normal floating-point representation, but has much less width and does not have complex cases like Not-a-Number and infinity cases. The short floating-point representation, contrary to the common belief, can produce a low-complexity computation logic because the operands of the multiplier logic can be shortened by the exponent concept of the floating-point representation. The exponent can also reduce the total length to reduce the SRAM area. The experimental results show that the short floating-point representation with 8-bit total width achieves less-than-1-percentage-point degradation without the aid of retraining in the top-5 accuracy on very deep CNNs of up to 152 layers and gives more than a 60% area reduction in the ASIC implementation.
