Implementation of Hybrid CPU-GPU Programs in a Backtracking-based Load Balancing Framework

抄録

General-purpose computing on graphics processing units (GPGPUs) has become increasingly prevalent, with hybrid CPU-GPU systems at the forefront of parallel computing. Dynamic load balancing is highly effective in maximizing CPU and GPU utilization in such environments. Backtracking-based load balancing, utilizing work-stealing, offers a promising strategy for task parallelism. However, Tascell, a task-parallel language that implements this mechanism, currently lacks GPU support, limiting its potential for hybrid CPU-GPU parallelism and constraining its application in computationally intensive tasks. In this paper, we demonstrate an experimental study on enabling Tascell to fully utilize the computational power of CPU-GPU hybrid environments by writing both CPU-oriented and GPU-oriented code for workers, allowing each worker to selectively execute either implementation based on the task size and GPU availability. Using this technique, we implemented hybrid CPU-GPU programs for three applications using Tascell: recursive block matrix multiplication, 2D stencil computations and Mandelbrot set calculations. The GPU-oriented code was implemented using OpenACC or the NVBLAS library. We conducted performance evaluations on both high-performance and workstation-grade CPU-GPU hybrid computing environments. The results demonstrated that in the workstation-grade environment, the hybrid approach outperformed both CPU-only and GPU-only configurations. Notably, hybrid CPU-GPU executions achieved performance improvements of up to 12.9% and 25.2% in 2D stencil applications compared to GPU-only and CPU-only executions, respectively. These findings provide valuable insight into the effective use of hybrid CPU-GPU systems within a backtracking-based load balancing framework.