Abstract
A variety of satellites for space investigation and Earth observation has been launched and are yielding a large amount of data. Easy and effective parallel processing technique is required to analyze such scientific big data without heavy programming. In the study we analyze a set of waveform data measured by the WFC-L receiver onboard Japanese lunar orbiter “KAGUYA” for 9 months using our original program. The total data size is as small as 144G B, but it takes long time (230 hours) to survey all data files to detect specific waveform patterns. The practical issue is that it is not easy for many space scientists to rewrite a program via parallelization library such as MPI (message passing interface). Herein we import our original program, without rewriting, on a science cloud system on which a task manager is ready for use for development and management of parallel data processing. We demonstrate that easy task scheduling and parallel processing is effective and practical for big data analysis even in case that the data set is heterogeneous.
