The goal of this research is to formulate the design criteria for developing a successful collaboration support system to be used among different professionals. As a first step to meet this goal, this paper investigates how humans communication acts are influenced by the availability of non-verbal communication modalities. We conducted two types of experiments in two environments and observed the achievement ratio of the given task and subject utterances. Our experiments have revealed the following: (1) the availability of deictic references (e.g., pointing with a finger) affect the achievement ratio and the content of subject utterances; (2) the visibility of other subjects does not affect the achievement ratio; and (3) the differences in the environments do not affect the utterance frequencies, their length, or their content.

View full abstract

We propose a novel glasses-free 3D display of 360° optimized for tabletop tasks. The display employs circularly arranged projectors and a cylindrical- or a conical-shaped optical device that has a directional optical characteristic. Those devices are installed underneath the table not to obstruct the working space. One of the novelties of the proposed method is forming a ring-shaped viewing area above the table. When users look in this area, they can observe individual 3D images from each direction with correct perspective on the table such as a centerpiece.

View full abstract

This paper proposes overlayable transmissive color markers for stacking virtual objects and information in Augmented Reality (AR) environments. The proposed markers have a pattern which consists of a transparent color and an opaque white. And the marker has regions for recognizing an order in the layer in addition to an ID pattern region. Firstly, the order in the layer is estimated by observing the order-recognition region. Then, an overlaid marker is resolved into each marker pattern by separating the camera image into each channel. All ID patterns of the markers are recognized as a two-dimensional marker pattern. Experimental results have demonstrated that the markers in the layer can be detected and split into each marker. In addition, a stackable interior design simulator was implemented and the application has shown that the user handle the virtual objects intuitively by using the proposed markers.

View full abstract

This paper describes "Round Table with t-Room", a system to share a round table among people in remote locations. The system takes the image of the tabletop surface at the remote site and overlays it on the local table surface, and vice versa, through a projector mounted above each table. This setting allows users to naturally appreciate the objects on the table, and their reference by finger pointing, regardless whether the objects and users are in the same location or not. The paper describes also an approach to eliminate the infinite video feedback caused by the projector-video camera setting, and furthermore, an example of use of the round table is presented.

View full abstract

I have developed a multi-user canvas-based drawing system called "Co-draw". Through sharing a canvas, users can add drawn pictures to those that another user has drawn. The users can then receive stimuli from the picture, producing a consequent emergent experience that is expected to increase the enjoyment of drawing. However, if this environment were to be merely presented to a user in the form of a shared canvas, it would not generate any new dialog. Therefore, in order to increase the user's awareness of drawing upon the canvas, I implemented a computer-simulated artificial intelligence system. The system distinguishes the pattern of each brush stroke, then based on that pattern adds an animation or other adapted material to it, creating a designed interaction called "Animated Embellishments." By interposing this computer process, the system seeks to allow the user to not draw upon the canvas, but rather engage in dialog with the canvas. In addition, the system aims to have users draw at the same time together, or choose to add on to each other's work, mutually producing creative work that is registered by the computer, then used to create special decorative animations that provide the foundation for encouraging further dialog.

View full abstract

We propose gCubik+i as a new interactive platform that naturally interfaces a 3D display with a tabletop display. A virtual 3D aquarium application that showcases its group-sharing capability and its interactivity potential is presented here. Multiple users, around the table, can use the gCubik 3D display to naturally pick up static 2D fish images, from the table into their hands, and see them transforming into full parallax interactive 3D fish images. The gCubik+i platform introduces two novel interaction paradigms to existing tabletop display applications: 1) natural switching between the shared working spaces of the table and the users' hands, thus effectively combining the interaction modes of both them; and 2) transforming static 2D images into interactive 3D images that can be viewed and manipulated as if holding a real object. This paper describes the conceptual design and prototype implementation of the gCubik+i platform along with a description of its 3D virtual aquarium application.

View full abstract

We present our new interaction technique for tabletop system and video game application using it. This technique is for recognizing pinching gestures performed with the thumb and forefinger and tapping gestures with them by a ceiling camera above the table. The positions and orientations of multiple gestures are recognized at 200Hz, and the users around the table are tracked by their arm positions. This technique is useful for large-sized tabletop system for multiple users because it does not require any additional equipments to them nor registration process befor using the system. Some pilot studies demonstrated the robustness and accuracy of the proposed technique with displaying apropriate guide on the display. We also developed a video game application called "Pac-pac". A player can shoot a bullet allong the orientation of his hand by tapping, from any side of the table. We exhibited Pac-pac several time and well-received.

View full abstract

For intuitive interactions on tabletop, we present a novel tabletop display system named UlteriorScape. This system can project images both onto held screens and the tabletop display. This time, we have also adopted vision-based interaction on UlteriorScape using a marker recognition method. According to the position, rotation and ID of the object, the projection image changes interactively. In this paper, we describe the system overview of UlteriorScape and three types of interactions. Moreover we introduce several applications on UlteriorScape for single user and for multiple users.

View full abstract

We propose a variant of the multi-touch display technology that introduces an original way of manipulating three-dimensional data. The underlying metaphor is that of a deformable screen that acts as a boundary surface between the real and the virtual worlds. By doing so, the interface can create the illusion of continuity between the user's real space and the virtual three-dimensional space. The developed system presented here enables this by employing three key technologies: a tangible and deformable projection screen, a real-time three-dimensional sensing mechanism, and an algorithm for dynamic compensation for anamorphic projection. This paper introduces the concept of the deformable tangible workspace, and describes the required technologies for implementing it. Also, several applications developed on a prototype system are detailed and demonstrated.

View full abstract

In manufacturing factories, it is important to achieve a good balance between cost reduction and product quality in increasingly competitive markets. With the massive retirement of expert engineers in aging society, an effective training scheme for new workers is desired. However, cell production system is widely used lately and each worker is supposed to carry out every step in the process and thus the worker should learn more and more kinds of manufacturing skills and tasks. It is also desired that fresh workers can learn the skills by themselves, however, transferring skills is a very difficult issue. Learning by print-based manuals is not effective, because it is not easy to read the physical operations and the process transitions from paper documents. Besides, flipping the pages to find the information during the work is troublesome. In this research, a new training system is built with a movable head-up display, which can be adapted to actual workspaces and allows the user to access digitized guidance contents with a semitransparent touch-sensitive display. The effectiveness is experimentally shown by applying the proposed system to a typical skillful manual work of origami paper craft.

View full abstract

We propose a new Wall-and-Tabletop-based Reconfigurable Interaction (WATARI) system, which is an evolution of our previous system. It provides both table and wall as its working spaces, enables gestural interaction with them, and can reconfigure their usage flexibly. In this paper, after describing the concept and design scheme of WATARI, we introduce the system configuration and, as an example of typical application, the fashion coordinate system on it. Especially, using metaphor of cards and document handling, the "Candle on the Table" subsystem is implemented at the level of its table-top operation. We critically discuss the results of our evaluation and comment on the limitations and advantages of WATARI.

View full abstract

Virtual Heritage is able to give the real feeling of cultural legacies as if one was actually there. This can be achieved by restoring those legacies which are difficult to experience in our daily lives. In this research, Virtual Heritage construction and evaluation of human model distribution towards its impression were performed. The preference degree of Virtual Heritage according to human model distribution was sought. The evaluation on the change of preference corresponding to the density of human model distribution was performed by focusing on the relation between the human model distribution and the impression change. As a result, it was verified that the human model distribution influences the preference degree, sense of reality, and etc. Under the condition of high density human model arrangement, it was confirmed that there are cases of decreased sense of reality due to the increased sense of pressure.

View full abstract

Photometric consistency is one of the most important issues in mixed reality (MR). Many works achieved a seamless MR by rendering virtual world under the same photometric conditions including illumination and image quality as real world. As a next step, we focus on a challenging trial to Look-Change of MR space. Look refers to the feeling of an image provided by illumination and color tone. In this paper, we introduce a relighting method for the Look-Change. The proposed method allows an MR space to have additional virtual illumination. The effects of virtual illumination are applied to both real objects and virtual objects while keeping photometric consistency. We utilize the relighting method for MR based pre-visualization for filmmaking.

View full abstract

This paper makes a comparison between two-way transmission and one-way transmission of the sense of force in terms of easiness of work in a remote haptic control system by subjective assessment. In the system which is applied to an instruction system, a teacher teaches a student how to write characters. We have two cases about how to attach a pen and how to transmit video. In one case, a pen is attached to only the student's terminal, and video is transmitted from the student's terminal to the teacher's terminal. In the other case, a pen is attached to each of the two terminals, and video is transmitted bi-directionally between the terminals. In both cases, we also investigate the influences of network latency on the easiness of work.

View full abstract

Three-dimensional sound field reproduction using directional microphones and wave field synthesis can be used to synthesize wave fronts in a listening area using directional microphones and loudspeakers placed at the boundary of the area; the position of the loudspeakers is the same as that of microphones in this technique. Thus, it is very difficult to construct an audio-visual virtual reality system using this technique because the screen or display of the visual system cannot be placed at the position of loudspeakers. In order to reproduce the 3D sound field of the listening area even when the loudspeakers are not placed at the boundary of the area, this paper proposes a 3D sound field reproduction system using directional microphones and boundary surface control. In the proposed system, the 3D sound field in the listening area can be accurately reproduced even when the loudspeakers are not placed at the boundary of the area by introducing inverse filters based on acoustic transfer functions. Results of a computer simulation show that the proposed system can reproduce the 3D sound field in the listening area more accurately than the conventional system.

View full abstract

In this research the authors have developed the high accuracy Hand Motion Capture System using electromagnetic tracker (Polhemus) of human body, which is very effective to determine any types of hand movement like pianist. We described the technique for using magnetic motion capture data to determine the movement of fingers, receiver position with six degrees of freedom (6DOF) and to analyze the receiver velocity and acceleration for our subject. We have also developed a method to calculate the finger joint angles for human body by observing piano playing. We have observed that during contact of the keyboard of the piano the metacarpophalangeal (MCP) joint flexed, distal interphalangeal (DIP) and proximal interphalangeal (PIP) joint extended and when the finger lifted the MCP joint extended, the DIP and PIP joints flexed, creating a closing and opening reciprocal motion respectively.

View full abstract

This paper deals with a remote haptic control system which enables a haptic interface device to be controlled with another remote haptic interface device while watching video. We propose switching control which dynamically switches the transmission direction (one-way or two-way) according to network latency. The control chooses two-way transmission when the network latency between the master and slave terminals is small, and it selects one-way transmission when the network latency between the terminals is large. We demonstrate the effectiveness of the control by subjective assessment. We also propose automatic selection of switching time according to the contents of work. This proposal selects the optimal switching time automatically by identifying the contents of work when the haptic transmission direction is switched. In this paper, we further illustrate the effectiveness of the proposal by subjective assessment for work of writing multiple characters and work of drawing a figure without lifting a pen from a sheet.

View full abstract

We propose a substitutive tangential force and slip display method by controlling position of the vibrotactile phantom sensation (VPS). The tangential force is substituted by the VPS position displacement and the slip is displayed as the oscillatory displacement of the VPS position. A 20mmx20mmx20mm fingertip-wearable device with four pins has been prototyped. The evaluation experiments demonstrated the feasibility of the substitutive display of tangential force and slip, and the assistive effect in cognition of contact state change.

View full abstract