DiSC: A Distributed In-Storage Computing Platform Using Cost-Effective Hardware Devices

Abstract

The ability to efficiently process exponentially increasing data remains a challenging issue for computer platforms. In legacy computing platforms, large amounts of data can cause performance bottlenecks at the I/O interfaces between CPUs and storage devices. To overcome this problem, the in-storage computing (ISC) technique is introduced, which offloads some of the computations from the CPUs to the storage devices. In this paper, we propose DiSC, a distributed in-storage computing platform using cost-effective hardware. First, we designed a general-purpose ISC device, a so-called DiSC endpoint, by combining an inexpensive single-board computer (SBC) and a hard disk. Second, a Mesos-based resource manager is adapted into the DiSC platform to schedule the DiSC endpoint tasks. To draw comparisons to a general CPU-based platform, a DiSC testbed is constructed and experiments are carried out using essential applications. The experimental results show that DiSC attains cost-efficient performance advantages over a desktop, particularly for searching and filtering workloads.