Design and Implementation of a Metadata Management Scheme for Large Distributed File Systems

Abstract

Most large-scale distributed file systems decouple a metadata operation from read and write operations for a file. In the distributed file systems, a certain server named a metadata server (MDS) is responsible for maintaining the metadata information of the file systems. In this paper, we propose a new metadata management scheme in order to provide the high metadata throughput and scalability for a cluster of MDSs. First, we derive a new metadata distribution technique. Then, we present a load balancing technique based on the distribution technique. Several experiments show that our scheme outperforms existing metadata management scheme in terms of scalability and load balancing.