Distributed parallel generation of large-scale random graphs based on Watts–Strogatz model

Abstract

In recent years, there has been a growing demand for the development of programs that perform analysis on large-scale graphs such as SNS and Web graphs. In evaluating the performance of such programs, we need a number of input graphs with the desired number of nodes and specific features such as the small-world property. However, the generation of large-scale random graphs by sequential programs is very time-consuming, and may cause memory shortage. In order to solve this problem, distributed parallelization of large-scale graph generation is desired.
In this research, we propose distributed parallelization of the random-graph generation based on Watts-Strogatz model, which is one of well-known graph models that provide characteristics similar to large scale graphs in the real world. We implemented our proposing distributed parallel algorithm by using Hadoop MapReduce. In the distributed parallelization, some restrictions are introduced into the original generation method to improve its efficiency. We have shown that the restrictions do not break the the characteristics that the original model has.