Eliminating unnecesarry communications in the vertex-centric graph processing by the Fregel compiler.

Abstract

Vertex-centric graph processing is a promising approach for processing large graphs in parallel and distributed ways. Pregel is a well-known model and framework for large-scale graph processing. However, the programmer has to write a Pregel program with explicit message passing between vertices and explicit state controls over vertices, which make the programming more difficult. To resolve this problem, Fregel, a functional domain-specific language, which supports declarative description of vertex-centric computation, was proposed. A Fregel program is translated by the Fregel compiler into a program runnable on existing systems such as Giraph and Pregel+, but the generated program is not sufficiently optimized and accordingly has large overheads, compared to a handwritten one. This paper proposes optimization methods for a generated program by removing unnecessary communications between vertices on the basis of algebraic properties of the operation used for combining messages. The extended Fregel compiler judges whether the operation satisfies the properties and applies the optimizations, if possible, to generate a more efficient program. Through some experiments, we confirmed that the execution times of optimized programs are within 1.3 times of those of handwritten programs.