IEICE Communications Express
Online ISSN : 2187-0136
ISSN-L : 2187-0136
A high-speed data processing technique for time-sequential satellite observation data
Ken T. MurataHidenobu WatanabeKazunori YamamotoEizen KimuraMasahiro TanakaOsamu TatebeKentaro UkawaKazuya MuranagaYutaka SuzukiHirotsugu Kojima
Author information

2014 Volume 3 Issue 2 Pages 74-79


A variety of satellite missions are carried out every year. Most of the satellites yield big data, and high-performance data processing technologies are expected. We have been developing a cloud system (the NICT Science Cloud) for big data analyses of Earth and Space observations via spacecraft. In the present study, we propose a new technique to process big data considering the fact that high-speed I/O (data file read and write) is important compared with data processing speed. We adopt a task scheduler, the Pwrake, for easy development and management of parallel data processing. Using a set of long-time scientific satellite observation data (GEOTAIL satellite), we examine the performance of the system on the NICT Science Cloud. We successfully archived high-speed data processing more than 100 times faster than those on traditional data processing environments.

Information related to the author
© 2014 The Institute of Electronics, Information and Communication Engineers
Previous article Next article