A Design of Scalable Computing Platform for Continuous Data

Abstract

Recently, many researchers focus on the studies of management and analysis for continuous data such as time series and geographical location data. To manage the large continuous data, the structured overlay network technologies are proposed. In addition, to analyze the large continuous data, MapReduce platforms are proposed. However, it is difficult to analyze the continuous data by MapReduce platforms on overlay. Because general overlay uses hash functions and these hash functions do not preserve the continuousness. In addition, general MapReduce platform generates a lot of communications and synchronous operations in continuous data processing. To handle these problems, we propose the scalable computing platform for continuous data.
Our platform achieves the asynchronous computing and high parallel performance for continuous data analysis. In concrete terms, our platform builds the balanced tree based on SkipList for all nodes. This architecture enables each node to manage its children nodes' states, analysis results, and synchronous operations. In addition, our platform can balance load by gathering all nodes' load information. Therefore, our platform realizes high performance MapReduce computing for continuous data.