Abstract
For large-scale granular simulations based on Distinct Element Method (DEM), a simulation code has been developed for GPU supercomputers equipped with GPUs. We propose an effective method for dynamic load balance among the decomposed domain assigned to each GPU. Applying the slice-grid method, we maintain the equal number of particles in the domain. Several techniques for particle counting and data movement between GPUs are implemented. The frequency of the memory de-fragmentation on GPU is examined from the view point of the cost of the data communication between CPU and GPU. A link-list technique of the neighbor particles list is introduced to save the memory drastically. The granular simulation works well with the dynamic load balance and good scalability has been achieved with increase of GPUs. The sand behavior of golf bunker shot is demonstrated with over 130 million particles using 256 GPUs. An agitation simulation using billion particles is also carried out on TSUBAME 2.0 at GSIC, Tokyo Tech.
