Acquisition of correct punching form is the most important training for beginning boxers. However, it is difficult for them to acquire basic punches with correct forms without direct instruction from an instructor. In addition, they may acquire basic punches with wrong form because of the less-judgment knowledge for punching forms' qualities. The aim of this study is to solve the above problem by our proposed solo boxing training system with AR. We developed some functions such as AR display of the target motion, comparison with the user’s motion, correct/incorrect judgement and positive feedback for each joint of the user by using MagicLeap1 and Kinect^TMV2 with machine learning. We confirmed that the effectiveness of our system via experimental evaluations through these functions.

View full abstract

Although calligraphy is a traditional Japanese culture, calligraphy exhibitions are being nearly monopolized by experienced calligraphers. It is primarily because appreciation depends largely on the level of calligraphic skills. In order to support the calligraphy beginners, we have developed in this study, a system called Caps（Calligraphy appreciation system）that provides seal script character recognition and conversion to running script. For recognizing seal script characters, a convolutional neural network-based method was introduced. Respecting the feature that kanji is composed of parts such as the radicals, unreadable characters used in calligraphic works can be recognized. This results in higher performance character recognition with small numbers of data and classifications. For converting to running script, a generative adversarial network-based method was introduced. A calligraphic work with arbitrary characters can be generated by a trained model that assigns calligraphic styles to the character skeleton. A skeleton image with stroke order and attribute information was used to achieve effective learning of calligraphic styles. It was empirically proven that Caps, which incorporates these two methods, enabled an intuitive appreciation experience ranging from recognizing the characters used in calligraphic works to generating calligraphic work in a different style.

View full abstract

As a recent trends in 3D modeling environments, they implement simultaneous editing through network by multiple people. Furthermore, multiple people are able to create 3D model with communicating online or in VR space by using these 3D modeling environments. On the other hand, in the case of modeling by collaborating with some users in real space, users need to share the display or multiple work environments are required. In this study, we propose FabKUI, a modeling method that enables multiple people to share a single interface in real space and collaborate on 3D modeling by using smartphone AR. In the proposed method, the concept of shadow user interface enables to control 3D movement of the entity by manipulation of shadows. And 3D models can be generated from 3D trajectories of the movement of the entity. Because of the application of KUI, FabKUI enable to create 3D models as if handwriting. In addition, by superimposing the 3D model generated in the real space using smartphone AR, it is possible for multiple people to view the 3D model. In this paper, we show the way of implementation of the proposed method and demonstrate the effectiveness of the system from the results of evaluation experiments.

View full abstract