Abstract
Astronomy Data Center constructs a system to analyze a huge data set, such as large-scale survey data obtained by wide-field cameras including Hyper Suprime-Cam at the Subaru Telescope, that require much computing resources for processing, and operates the system in collaboration with the Subaru Telescope since September 2019.
The system consists of a login node, a management node, 35 compute nodes, a storage with 5 PB capacity and high speed I/O, and file servers with redundant configuration. All nodes other than the management node are connected by InfiniBand with bandwidth of 56 Gbps or more. Computing resources of the system are managed by a job scheduler. Interactive use of compute nodes are prohibited.
Although the computing resources at the start of operation were 280 CPU cores and 5 TB memory, the significant addition of compute nodes in spring 2020 expands the total resources to 1,976 cores and 18.5 TB memory. The additional nodes are currently used for the analysis of HSC-SSP that also serves as trial operation. After adjusting of queue settings, the nodes are to be shortly released for HSC open-use observers.
The performance evaluation tests, conducted before the start of operation, reveal that the speed of reading and writing files is 24–26 GB s−1, and that the processing speed of HSC data is four times faster than that on the batch servers of Multi-Wavelength Data Analysis System.
