We propose a tele-echography system linking between the patient's home and a hospital because of the demand due to increase in the number of patients by aging society and recent progress in portable echography enabled us to develop this system. In previous researches three-dimensional relationship between the ultrasound probe and internal organs was difficult to specify because a remote doctor observe the patient through a video camera. Therefore we have developed a visualization system of 3D shape of internal organs based on AR and VR technology using ARToolKit. We have also designed a doctor side interface including echogram, patient scene and CG of internal organs. As a result of evaluation experiments, guided position was satisfied to reproduce the echogram for diagnosis.

View full abstract

In the real world, the Doppler effects are caused by moving object like emergency vehicle. We introduce the way to reproduce the Doppler effects caused by moving sound source into the existing audio-visual mixed reality space system. We find the Doppler effects which caused by artificial sound made by full-sized virtual object and thus the results make us satisfaction. Additionally, we tried to reproduce the Doppler effects on virtual object of miniature, some subjects felt unnatural by the condition, however we noticed they feel natural in virtual reality space removed real background. We defined this phenomenon effect on perception of Doppler effects as "Gulliver-Doppler Phenomenon". This paper presents the results of the experimentation, "Gulliver-Doppler Phenomenon".

View full abstract

This paper proposes an augmented telepresence system which generates large-scale omnidirectional AR scenes in outdoor environments using recorded omnidirectional videos captured from an unmanned airship. The proposed system realizes geometric registration by estimating extrinsic camera parameters using a video sequence and GPS data. Photometric registration is realized by environmental map completion and illumination environment estimation of real scene. This paper describes a system for aerial imaging using an airship and an omnidirectional camera, geometric and photometric registration methods, and experimental results of generating AR scenes from an omnidirectional video sequence captured on a historic site.

View full abstract

Human beings perceive three-dimensional objects with simultaneous convergence and lens accommodation in binocular vision. Virtual images are perceived via the same mechanism, although, as we previously reported, the focus is not always fixed on the surface of a display when stereoscopic images are being viewed. In this study, we developed a method to simultaneously measure accommodation and convergence in order to provide further support for this theory. We also measured accommodation and convergence in natural vision to confirm that these measurements are correct. Then, we measured simultaneously them in gazing 2D and 3D images.

View full abstract

This paper is about optical camouflage 2.0, a mirror bringing the experience that one's own figure is transparent. When the user wears a retro-reflective material coat and stands in front of the mirror, the part of the coat is transparent. This design realized the experience that was imagination to appear in the SF conventionally in the reality world. This is the revised design of Optical Camouflage by RPT to be self-transparent. The system utilizes with Retro-reflective material, projector and beam splitter as same as previous works, but the position of observer is placed the same position of masking object. Additionally, this article describes about the optics design of the system, and also the result of exhibition which called "The Interfaces of Dr. Jekyll and Mr. Hyde" at National Museum of Emerging Science and Innovation.

View full abstract

In this paper, we describe the information transmission system that uses the contact between users and ball devices as an information channel in interactive contents. When the ball device with built-in transmitter comes in contact with the user who wear the receiver, this system transmits information with the human body communication through user's body. We aim at the improvement of the augmentation of the interaction in such a way that presenting information on user's body according to the contact between each ball device and each user by this method. This system also enables use in a wide range field in the same network by managing contact information of both collectively. We realize the application development mixed the contents environment of the real world with the virtual space by regarding ball devices as input interface. Moreover, we developed a interactive game content intended for children with this system, and conduct a user study.

View full abstract

These days, location information services such as navigation or location awareness are used with cell phones and PDA. However, these mobile terminal devices restrict the use of one hand and demand that a user keep a close watch on the small display. We consider that floor projection by a hip-mounted projector could solve the above issues. In this paper, to make user easier to look information while walking, we propose a projection stabilization method which refers to user's walking periodicity. As a result of usability test, we confirmed that the proposed stabilization method improved the user viewability.

View full abstract

This paper presents a new high color reproduction method for proximity optical see-through display. This method corrects the color appearance of cultural heritages when users browse them closely in Mixed Reality environment like a MR museum. The many conventional MR displays suppose a distance between a real object and half-mirror is far and discuss only an environment light and a reflected light from half-mirror. However, these methods are not possible to realize the high color reproduction when a half-mirror is close to a real object because they disregard the light going through a half-mirror. In this paper, we propose the strict optical model for the color restoration and experimental results for changing the color appearance of some patterns and a real Emakimono painting.

View full abstract

This paper proposes a method to control robots using an imaging display. In this method, a multi-robot controlling system can be constructed with a simple method. Graphic displays are no longer screens just for human, but are also for robots. It is able to consider display as a device that provide visual information to human and robots simultaneously. Additional to that, people who don't have any programming skills can manipulate robots by editing images using an animation software. We think this method enhances and enriches interaction around table-top displays. Tangible devices that can be manipulated by humans can also be controlled by this system.

View full abstract

In this paper, we introduced a new system for designing and evaluating the user interface of handheld devices on table-top environments using the SPIDAR-Hand system which is developed for tracking each fingertip position and giving force feedback to each fingertip with a graphical display superimposing virtual handheld devices to real background. Our system calculates 3D positions of each fingertip using the lengths of three strings and, depending on the positions, tensions of the strings are controlled to give suitable force feedback and tactile sense of manipulating a virtual device. An AR technique of optically monitoring the hand movement by camera is also introduced to trace the hand in motion images. Some simple experiments are conducted to evaluate the controllability of the virtual device using the proposed system. The reality of manipulating an actual device is still hard to generate. However, with force feedback, it is shown that the presence of the trial device is improved when the virtual trial device created by 3D CG techniques superimposed on the real images is controlled by a hand.

View full abstract

This paper describes an ID assignment and error detection/correction method for Two-tone similar colored markers, which we have proposed for geometric registration method on mixed reality. We assume that these markers' color are similar to that of the background object to make them unimposing. However, there are trade-off between robustness and inconspicuousness. In general, any image recognition method cannot avoid recognition errors caused by external factors. Therefore, our proposed markers occasionally get not enough identification result by these reasons. To solve this problem, we have developed an ID assignment and recognition error detection/correction method. First, we clarified the total number of IDs for marker-units that consist of some small markers. We formulated single marker-unit system, in which we attach importance to the total number of IDs, and composite marker-unit system, in which we attach importance to the robustness. Secondly, we developed a recognition error detection/correction method. Basic idea is that the system assigns some numerical value to each small marker and judges the difference between the sum of the assigned value and previously given check digit. The level of recognition error detection and correction can be adjustable based on the mapping between the numerical value and each small marker.

View full abstract

In this paper, we describe design issues of sound effects for a workshop called "HERE/HEAR," which was aimed to recognize relationship between localization of sounds and sense of presence. Sound devices used in the workshop could be classified into two types: that for sounds localized in the listener's body, such as ear phones, and that for sounds localized out of the listener's body (in the environment around the listener), such as hypersonic speakers and normal speakers. Novel sound effects were achieved by suitably combining these devices according to the types of sound contents (environmental sounds, sounds of interaction with body, sounds of action, inner sounds, and voices of words). Reports by the participants supported that the sound effects functioned properly in the workshop.

View full abstract

Virtual Diorama (VD) interface is a new Mixed-Reality interface for mobile device, which enables users spread in large outdoor environment to share their point of interest (POI) among users. The outdoor environment is modeled in CG and POI is attached onto the CG model. Users will see POI by maneuvering the CG model (VD model). In this paper, we conduct user study and experiment for usability of our VD interface for better recognition of POI location. We compare two repositioning methods of VD model; Free shot and Dolly-round/Crane (D/C) shot. We also evaluate the usability of superimposing VD model onto the real image. As a result, D/C shot is more suitable than Free shot on our VD interface. We also found that about half of the subjects prefer superimposing of VD model onto real image.

View full abstract

In this research, we consider the manipulation of 3D virtual objects consisting of many parts in mixed reality space. We propose a method in which users can pick up only one part of an object they need and observe the relative positions of the parts and the connections between them from various directions. Our method uses metaphors in the real world. User studies showed that our approach helps users to avoid incorrect operations and improves the operational feeling and responsiveness. This paper presents the details of our proposed method and a system that can realize it using gestural inputs.

View full abstract

Compared to Augmented Reality/Mixed Reality, which refers to technologies to superimpose Computer Graphics to real environments to enhance the real world, Diminished Reality (DR) refers to visually diminishing or eliminating obstacles in the real world, or seeing through objects such as walls. The techniques and difficulties to implement DR will vary to adapt to various environments. Hence, we first set one clear goal of DR to completely eliminate existences in the real environment from observed scene. Then, we conducted a survey to show guidelines for researchers to achieve the goal. We systematically arranged existing papers with some concerns for DR implementation.

View full abstract

In this paper, we describe high-speed 3D position and normal measurement of an arbitrarily specified point of interest using a high-speed projector-camera system. A cross line pattern is adaptively projected onto the point of interest and the reflected pattern on an object surface is captured by the camera and analyzed to obtain the position and normal information. Using this measurement system, we describe new game designs in which real-time interaction with non-structured 3D real environment is featured.

View full abstract

Mixed reality (MR) is the technology which merges real and virtual worlds in real-time. In MR space, a real object can be changed its visual appearance by superimposing a CG image (CGI) on it. Because it is said that the feeling of the weight is affected strongly by visual stimulation, we believe that it is affected similarly, in the case of presenting MR visual stimulation. If the behavior and extent of such an influence are well investigated, one real object can be perceived differently. In this study, we focus on the center-of-gravity (COG), and verify the influence of MR visual stimulation on sense of COG in MR environment. In this paper, we describe the systematic experiments of the influence. As the result, we obtained the interesting and promising result: (1) the sense of COG can be changed by MR visual stimulation, and (2) although the different feelings of COG between vision and sense of force, the feeling of COG can be represented by MR visual stimulation under certain conditions, (3) the influence from MR visual stimulation becomes smaller when COG positions of a real object and a CG image is distant.

View full abstract

In mixed reality (MR) environment, an appearance of touchable object can be changed by superimposing a computer generated image onto it (MR visual stimulation). Meanwhile, when human sense a hardness of real object, it is known that the perception is influenced by not only tactile information but also visual information. In this paper, we studied psychophysical influence on sense of hardness by using a real object which is superimposed by a computer generated image (CGI). In this experiment, we deform the CGI animation on the real object extremely, while the subject pushes the real object by his/her finger. As the results of the experiments, it found that human subjects sensed different hardness by emphasizing dent deformation of the CGI animation.

View full abstract

We have developed a haptic display for rotational tasks that involve functions of the human wrist. We represent the torque using a motor and a brake. Reference torque curves are obtained by the measuring torque required for each actual task using a torque sensor. The brake represents the stop condition. We have confirmed the effectiveness of the display by comparing the actual tasks with the haptic display experiment.

View full abstract

This paper describes a telepresence system to provide users with a highly realistic sensation using a motion base with limited degrees of freedom (DOF). Conventional studies using a limited DOF motion base have never investigated the generation of various inertial forces such as a transitional inertial force and a continuous inertial force. This is because the motion base can not realize the same motion as the real ride. This paper proposes a new telepresence system which can provide a user with an inertial force sensation using a motion base with rotational DOF and an immersive display. In our research, the various inertial forces are generated virtually with a component of acceleration of gravity and a visual stimulus which induces self-motion perception. The inertial force is generated with acceleration of gravity by inclining the motion base. The visual stimulus is given to the user by translating a view-point virtually with an image-based rendering approach. The magnitude of inertial force given to the user is determined from a ride motion estimated using an image sequence captured by an omnidirectional camera on the running ride. In experiment, a prototype system has successfully produced a highly realistic sensation in various environments.

View full abstract