BBView: A burst buffer for MPI file views that performs bulk writes

Abstract

Large-scale simulations periodically enter an output phase to save restart images or dump multidimensional arrays for post-processing. In these phases, multi-dimensional arrays are distributed across MPI processes in a scattered manner, so that each process owns many small, non-contiguous regions of the global data. Although each rank holds its data contiguously in memory, this layout maps to disjoint offsets in the shared file. When encoded using MPI file views, the resulting I/O is decomposed into millions of tiny, non-contiguous write requests. Such fine-grained writes incur high per-request overhead, amplify metadata traffic, and ultimately prevent applications from saturating the available bandwidth of modern NVMe devices and parallel file systems.

We present BBView, a lightweight Open MPI plug-in that addresses this inefficiency by buffering MPI file views at the process level. Instead of immediately scattering a rank's contiguous in-memory data across the many disjoint file offsets prescribed by its view, BBView defers that scatter. Each output phase first performs a bulk write, writing the data exactly as it resides in memory, as a single contiguous stream to a node-local file. When the process later closes the file or installs a new view, a background daemon reconstructs the original file view and scatters the buffered data to the parallel file system. This design removes fine-grained writes from the application's critical path while ensuring that the final shared file layout is identical to what the application specified.

BBView is seamlessly compatible with applications built on HDF5 or PnetCDF and functions as an effective I/O acceleration mechanism without requiring source-code changes. An evaluation on 128 nodes demonstrates up to 74.2× higher throughput than UnifyFS on local NVMe SSDs, achieved solely by increasing the effective write granularity through bulk writes.