Structural exploration of stochastic neural networks for severely-constrained 3D memristive devices

Abstract

Neural networks architectures will be soon required to be deployed on IoT-edge systems, which will demand small and low-power circuits and application-dependent customizability. Focusing on the power-efficiency and internal structures of 3D memristive devices, this paper proposes neural networks architectures that are implementable in those devices, together with an existing transfer learning technique. Our architectural model is stochastic and device-conscious in handling noise/variation-vulnerability and suppressing inter-layer connections. Through experiments, we quantitatively explored appropriate structures under device constraints and demonstrated comparable performance as conventional work holding complex connections. Also, we revealed an important challenge on device technologies for further performance improvement.