ROVN: Replica placement for distributed data system with heterogeneous memory devices

Abstract

Replication is an integral technique for distributed data systems. By maintaining multiple copies of the same data over a network, it increases their availability, throughput, and responsiveness. However, this benefit inherently demands increased memory usage for the replicas. This paper presents a novel replica placement policy called ROVN (Replica-over-NVM), which exploits the fact that for cost savings, the performance requirements are disparate across the data nodes in distributed systems. Replicas typically conduct asynchronous writes for rapid responsiveness, and thus it has little impact on user-perceived latency. In this regard, ROVN places replicas in a slow but inexpensive memory (NVM), while maintaining the master in a fast but high-cost memory (DRAM). This policy can reduce the TCO (Total Cost of Ownership) of data centers without compromising user experience. We implement ROVN in Redis, a commercial-level distributed NoSQL database, by addressing a set of challenges that arise during its realization. Performance evaluation with the Memtier benchmark shows that ROVN achieves significant cost savings in distributed data systems, with neither performance loss during normal operation nor a continuous slowdown even in the event of a failure.