A variety of satellites for space investigation and Earth observation has been launched and are yielding a large amount of data. Easy and effective parallel processing technique is required to analyze such scientific big data without heavy programming. In the study we analyze a set of waveform data measured by the WFC-L receiver onboard Japanese lunar orbiter “KAGUYA” for 9 months using our original program. The total data size is as small as 144G B, but it takes long time (230 hours) to survey all data files to detect specific waveform patterns. The practical issue is that it is not easy for many space scientists to rewrite a program via parallelization library such as MPI (message passing interface). Herein we import our original program, without rewriting, on a science cloud system on which a task manager is ready for use for development and management of parallel data processing. We demonstrate that easy task scheduling and parallel processing is effective and practical for big data analysis even in case that the data set is heterogeneous.

View full abstract

　The National Diet Library collects and maintains a database (Collaborative Reference Database) of reference service questions and the answers given to them. The questions are submitted to public and university libraries by users, and the answers are given by the libraries. This project support reference service and research activities. In Collaborative Reference Database, Property of “Nippon Decimal Classification (NDC) codes” is optional, and the NDC codes are assigned to only about 2/3 records because such codes can be burdensome to reference librarians. This paper proposes a method for automatically assigning Nippon Decimal Classification (NDC) codes to them using machine learning. NDC codes allow users to easily find reference service records and are therefore useful. We propose following two methods. (1) Using the NDC codes of the reference materials in reference recodes, (2) using frequency of each NDC of morpheme in question in reference recodes, and (3) using both method. For our experiments, we used 62,328 reference records. Experimental result showed that our method achieved a 45.6% precision about (1). And our method achieved a 53.8% precision about (2). And our method achieved a 45.6% precision about (3). These figures are significantly higher than 32.4% precision of the preceding studies.

View full abstract