In this paper we introduce a new system called “Collective Body” which combines the scanned human body with action detection to generate real-time special effects. The system is equipped with an automated human body scanner. As a person moves inside the monitored area, multiple cameras are used to scan the human body in real-time to obtain the 3D visual hull. The visual hull of the user is then visualized with point-based rendering and a particle system. We also apply special visual effects to the 3D model of the user in response to his motion. We implemented two types of point-based visualization modes and two types of special effects inspired from biological activities, namely respiration and metabolism. This system allows users to visualize, from any viewpoint, 3D models generated from their own body and actions, providing them with a new experience of interaction and immersion.
View full abstract