This paper proposes a wallpaper replacement method in a free-hand movie by using Mixed-Reality (MR) technique. Our method overlaps virtual wallpaper onto a real wall region in a movie that is taken by handheld camera looking around a room. In an ordinary way, 3D models of the target room are required. However, such special and expensive 3D survey instrument is not available in homes or offices where many users demand for a wallpaper replacement simulation. We solve this problem to extract a wallpaper region by using image segmentation technique with user interaction. Image segmentation technique can extract a target objects that include a thin one from a source image with an appropriate user interaction, and requires no special equipment like a 3D survey instrument. By applying our method, wallpaper replacement can be easily realized by using only a handy camera and a PC.

抄録全体を表示

Vibration is useful information for various applications including vibrotactile reproduction. However, conventional contact-type vibration sensors sometimes obstruct the original motion, and noncontact-type vibration sensors cannot track a dynamic object like a ping-pong ball in a game. In this research, we propose a non-contact vibration sensing system for dynamic objects, using high-speed optical axis control and laser Doppler vibrometer. In our system, low-frequency large motion of the target is tracked by 1000-fps visual feedback, and high-frequency minute vibration is measured by the Doppler shift of a coaxial laser. Then, the prototype of this system realized non-contact vibration measurement with tracking the object.

抄録全体を表示

In this paper, we present a system for Diminished Reality (DR). In this system, we have two ways to perform DR: one is for a static target scene, and the other is a dynamic target scene. In the case of a static target scene, we create 3D environment model beforehand. Viewpoint Generative Learning is performed based on this model, and a database of feature descriptors is constructed. When DR is performed, feature descriptors of a smartphone image are compared with those of the database, then matched and a relative position between the smartphone and the model is computed. The model which corresponds to the position that the user touches on the smartphone is hidden and the model behind the position is drawn on the smartphone image. In the case of a dynamic target scene, feature values of a smartphone image and an RGB-D camera image are extracted for every frame, then matched and a relative position between the smartphone and the RGB-D camera is computed. Point cloud data is used instead of a model. The area specified by the user is hidden and the point cloud data is drawn on the smartphone image. We can perform real-time processing because we send the smartphone image to the PC which returns obstacle-removed images for every frame. Our experiment revealed it was possible to perform DR on the smartphone in approximately 8fps when the target scene had 3D shapes.

抄録全体を表示

In this paper, we construct a prototype of a wearable text input device that can be attached to a user's torso or pants and conduct a pilot study on its use with a head mounted display (HMD). The user can enter text while mobile without having to look at his or her fingers. The layout of the keys is similar to that of a QWERTY keyboard, but the device is separated into two halves so that they can be positioned and oriented for comfortable typing on clothing. It is flexible and conforms to the shape of the user's torso. In our pilot study testing input speed, a total of 7 participants conducted various typing tasks, and reached an average speed of 30.1 words per minute (WPM) amongst all tasks in a 45 minute session. We then conduct an analysis of a large number of recent text entry methods that can potentially be used with wearable displays, including the results from our experiment. We also discuss a lighter, wireless prototype reflecting feedback in the pilot study, which can be easily attached to clothing with magnets.

抄録全体を表示

Wide-view HMDs allow users to use their peripheral vision in a wearable augmented reality (AR) system. However, there have been few studies on information display methods that make use of a wide FOV in AR environments. In this study, we discuss and implement types of information display methods that make good use of a few types of wide FOV HMDs, and then we evaluate the effectiveness of these methods through a user study with real and virtual wide FOV see-through HMDs. In one of these methods, an annotation is presented near the border of the HMD's view with a lead line connected to the annotated object (outside of the HMD's viewing plane). An immersive CAVE-like environment is used to simulate a virtual see-through HMD as well as a virtual walk-through environment, and a prototype of a hyperboloidal head mounted projective display (HHMPD) is used as a real see-through HMD. Experimental results show that the methods with lead lines improve target discovery rates compared to a method that overlays annotations directly over the target's position. Additionally, it was shown that annotations with an added blinking effect have little effect on target discovery rates regardless of viewing angle.

抄録全体を表示

In this paper, we propose a diminished reality (DR) system by which target objects in front of planar structure are effectively removed from the user's view. Our approach is to use a view-dependent image set captured before target objects are placed in an environment and to use common region (surrounding region of interest; SROI) between the view-dependent images and the user's view. First, in the proposed method, a target object in the user's view is overlaid by a selected view-dependent image using homography warping. For accurately projecting the view-dependent image, feature points are matched to ones only in SROI. During real-time process, lightings can change and can be different from that in the advanced capture of the view-dependent image set. Therefore, the projected view-dependent image is compensated using color information in SROI. In experiments, we used a 6DOF robot arm to capture ground truth image sequences, and compared them with our DR results. The quantitative results showed success and limits of the proposed method.

抄録全体を表示

In this paper, we propose a novel interaction technique called Pinch-n-Paste. Pinch-n-Paste allows a user to touch or pinch one part of an object, copy and move its texture, and paste it onto another object, directly with his or her hand, in an augmented reality environment. In this way, a user can interactively change and modify object texture in the scene using texture of other objects for a variety of applications such as interior design, garment simulation, and entertainment. Physics-based hand interaction such as touching, pinching and pasting is implemented by using the Microsoft Kinect. To transfer texture appropriately from one part of an object to another, two texture images are generated from source and target objects by the Least Square Conformal Map (LSCM) technique at an interactive frame rate. Two regions in the texture images corresponding to source and target areas of interest are then obtained using cross-boundary brushes. Target texel values are sampled from corresponding source texels by Moving Least Squares (MLS), and are finally mapped onto the target object. A selected part of texture is rendered as a virtual handkerchief with soft body dynamic cloth simulation so that the user feels as if he or she really extracted the texture from the object. Our system also supports interactive 3D reconstruction so that a real object can be reconstructed on demand and subject to Pinch-n-Paste interaction, in addition to pre-defined virtual objects. A preliminary result indicates that Pinch-n-Paste is more fun to transfer texture between objects, compared to a GUI-based interaction. In this paper, we will describe the basic idea, implementation details, and example interaction results and the preliminary user study.

抄録全体を表示

We propose a waist-mounted projector-camera (ProCam) system for asymmetric remote collaboration. A wearable camera is often used to transmit a worker's situation to a remote instructor, however 3D structure of the worker's environment is not always available and the instructor has a minimal flexibility in changing the camera's viewpoint. Our waist-mounted ProCam system reconstructs worker's environment in real-time without occlusion from the worker's body. The remote instructor can give instructions simply by drawing annotations on the reconstructed environment on screen, and they are properly projected in front of the worker. Structured-light based reconstruction, vision-based localization, and visual annotation projection, are processed synchronously with a camera and modified shutter glasses so that both the camera and the worker observe only the information they need. Experimental results show that users prefer our system to a stationary ProCam system.

抄録全体を表示

This paper proposes a wearable Hyperboloidal Head Mounted Projective Display (HHMPD), and reports on three user studies on the evaluation of visual quaity of a wearable HHMPD. Using a hyperboloidal mirror, an HHMPD can provide a wide field-of-view (FOV), a large observational pupil, and optical see-through capability. We propose a simple head attached screen that is both retro-reflective and semi transparent thereby allowing a HHMPD to be available in a wearable condition. Our user studies have shown that our wearable HHMPD provides a virtual image with a visual acuity of around 20/200 at perceptually 2 to 3 meters away from the user, and that users are able to identify visual stimuli in the periphery of the visual field up to ± 50 degrees in horizontal, while paying attention to a real object in frontal direction.

抄録全体を表示

In this paper, we propose an image and tactile presentation system for quickly moving objects, high-speed dynamic information environment, and a calibration method between a camera and Airborne Ultrasound Tactile Display(AUTD), which can present tactile sensation noncontactly as focused ultrasound. We realize one example of high-speed dynamic information environment that even if an object is moving, we can use the object as a tool for human interfaces without delay by combining a high-speed gaze controller by rotational mirrors. An accuracy of position, however, is not sufficient against difference threshold between images and tactile sensation because of mainly error of position between AUTD and camera. We calibrate the AUTD-camera system by visualizing the invisible focal position using Frustrated Total Internal Reflection(FTIR). Experimental results show that estimation accuracy is better than the threshold.

抄録全体を表示

Smart-tourism is one of the important applications in the trends of Mixed Reality System. A MR Bus system, a mobility platform, is designed for the large-area contents based tourism. In this paper, we propose a novel sound system to support the MR Bus platform which can play the sound realistically in the real environment mixed with virtual objects. Our method consists of three main components: 1) HRTF Open-Air Headphone, 2) Acoustic synthesizer using the image processing with simple sound models, 3) Sound Source Management by GPS. Our method is evaluated by the questionnaire from tourists. The result shows a good performance on the enjoyment of sound effects.

抄録全体を表示

This study proposes and evaluates the effectiveness of visualization modes for an AR based context-aware assembly support system in object assembly. Although many AR-based assembly support systems have been proposed, few keep track of the assembly status in real-time and automatically recognize error and completion states at each step. Naturally, visualization modes and their effectiveness for such context-aware systems remain unexplored. Our test-bed system automatically displays guidance information and error detection information corresponding to the recognized assembly status in the context of building block (LEGO) assembly. In our first evaluation, we compared the performance of the test-bed system in different AR visualization modes with a traditional assembly instruction style-paper manual in assembly tasks. Experimental results show that although subjects took longer to complete the assembly tasks with the test-bed system, accuracy was dramatically improved and subjects also felt that the visualization modes proposed were easier to understand and more useful than the traditional assembly style with a paper manual. Based on these results, we conducted the second evaluation to explore effectiveness of new visualization modes in situations where the best visualization mode proposed in the first evaluation does not function well. We explored two visualization modes; one having guidance information rendered on a virtual model adjacent to the real model, and the other having guidance information and the topmost layer of the virtual model rendered directly overlaying on the real model. Our experimental results suggest that both these modes had the best performance in the contexts considered respectively.

抄録全体を表示

This paper presents a novel shadow interface which has the looser spatial relationship with real shadow. Shadow is a natural extension of our bodies and often used as devices which operate large displays. On shadow interface, users can make shadow as a cursor bigger or smaller by moving their own bodies. Particularly, we focus on the environment in which users operate large displays on walls with staying seated. In proposed interface, we set a virtual screen nearer to users position than a real screen so that users can close their hand to a virtual screen easily and can operate detailed works quickly. We evaluated our interface by pointing experiments and confirmed that the operability could be improved on the proposed interface when comparing with conventional interface.

抄録全体を表示

This paper describes design and implementation of a distributed framework for creating mobile mixed reality (MR) systems. The goal of the framework is providing the same MR space for a variety of mobile devices which connect via wireless network. This paper discusses the following three topics which are essential for designing our framework: system architecture for supporting diverse mobile devices, an easy-to-implement script language for developing applications, and a communication method which takes into account the operability of the users. We implemented our framework based on the proposed design and investigated its performance empirically. As a result, we confirmed that various kinds of mobile devices can share the same MR space using our framework.

抄録全体を表示

In this paper, we propose the method for creating multi-view movable fogscreen, and then implement it in our system called "HANASUI". "HANASUI" displays handheld-like fireworks through a fog screen instead of sparkles. Our method generates projection data dynamically from a virtual space and then casts it with multiple projectors, tracking the marker attached to the device which spouts fog at the fogscreen with infrared cameras and infrared floodlights. Finally we conducted a survey to verify the capabilities of "HANASUI" and its potential for art and entertainment purposes.

抄録全体を表示

We propose a novel knowledge-based method for rough scale estimation in SLAM using only a single camera. The proposed method assumes that there are multiple planer and periodic-textured objects whose category is given a specific spatial frequency instead of the conventional strict registration of markers or 3-D geometry data. As cues for scale estimation, the method extracts the specific spatial frequency of each object and classifies the object in the corresponding category. A proof-of-concept prototype shows the feasibility of the proposed method through the experiments conducted in a limited scene.

抄録全体を表示

Nowadays, taking advantage of information science and technology for tourism is promoted by the local governments, and the feasibility of Virtual/Mixed Reality (VR/MR) technology applied for tourism is well studied in the state-of-the-art work. In this paper, we propose a new portable VR/MR system to support the large-area contents reproducing the historical scenes. We applied this system into a guide tour in Asuka-kyo for the tourists and volunteers. Our system consists of three elements: 1) Portable VR/MR Terminal using HMD and Tablet-PC with the backbone system for MR, 2) Simple user interface software, 3) Historical restored objects for VR/MR Contents. Our system is evaluated by questionnaires from the tourists and volunteers in a long term experience. As result, we obtained valuable comments and useful information.

抄録全体を表示

When humans sense the weight of real objects, their perception is known to be influenced by not only tactile information but also visual information. In a Mixed-Reality (MR) environment, the appearance of touchable objects can be changed by superimposing a computer-generated image (CGI) onto them (MR visual stimulation). In this paper, we studied the psychophysical influence on the sense of weight by using a real object that has a CGI superimposed on it. In the experiments, we show CGI representing the inertial force caused by the objects placed inside, while the subject swings the real object. The results of the experiments show that the subjects sensed weight differently when being shown the CGI animation.

抄録全体を表示

These days, AR (Augmented Reality) is widely used in various fields. Interacting with AR objects that are superimposed on a real world environment, by working with watching the real space in which AR object exist, it is possible to operate directory. In the case of AR objects are positioned out of reach, using the extension method of arm, it is possible to operate. However, there are several problems such as reducing the flexibility and accuracy. Then, we propose the relative input method for quick operation can be accurately an AR object with grasping the touchpad with still looking at the real space in which AR object exist.

抄録全体を表示

Our study centers on providing tactile feedback in Mixed Reality (MR) environment. While most studies focus on the use of vibration and temperature to provide tactile feedback, vibration and temperature are not the only sensations a human can perceive. In this study, we focus on the psychophysical influence of MR visual stimulation on pain sensation. We conducted an experiment where we induce pain on the subject's forearm and display visual stimulation on a different position than where we induced the pain. We found out that the position where the subjects perceived pain, differs from the actual position according to the displayed visual stimulation.

抄録全体を表示

We research the remote collaboration in which a local worker works with real objects by a remote instructor. The goal of this research is to achieve an interaction which allows a remote instructor to provide a local worker with clear and accurate instruction. We focus on occlusion problem that occurs in remote collaboration. To solve the problem, we deployed the camera in front of the instructor. Also the system captured the instructor from front face. In this way, gestures of the instructor are not occluded by the instructor's body. At the same time, the mirror image of the instructor is superimposed on the display deployed in the instructor interface to give visual feedback to the instructor. In this paper we develop and evaluate a system with the proposal method. Finally, we discuss the effectivity of the proposal method.

抄録全体を表示

When a head is equipped with a hanger made of wire sideways, and its temporal region is sandwiched by the hanger, the head rotates unexpectedly. We named the phenomenon "Hanger Reflex", and we have studied this phenomenon to understand its mechanism and to show the possibility of utilizing the phenomenon as a human interface or for rehabilitation. In this paper, we evaluated that the necessary conditions of the Hanger Reflex are compressions to temporofromtal region of the head and its counter position. We also developed an interface that can induce head rotation by Hanger Reflex, without giving pain to the user.

抄録全体を表示