Tele-existence is an advanced type of tele-operation system which enables an operator at the control to perform remote manipulation tasks dexterously with the feeling that he or she exists inside an anthropomorphic robot as a virtual exsoskeleton robot working in the remote environment. A tele-existence manipulation system was constructed and was evaluated quantitatively by comparing tasks of tracking a randomly moving target under several operational conditions. The effects of various characteristics, e.g., binocular vision and the effect of natural arrangement of the head and the arm, were analyzed by comparing quantitatively the results under these operational conditions. Human tracking transfer function was measured and was used for comparison. A manipulation task of block building was also conducted under several operational conditions including motion stereo. The time elapsed for the completion of the task was used as the criterion for the comparison. The results revealed the significant dominance of the binocular vision with natural arrangement of the head and the arm, which is the most important characteristic of tele existence.

View full abstract

Teleoperation is valuable for steel makers to mechanize or automate highly what cannot have been done. One of the most important problems in teleoperation is to give operator a sensation of presence.
We made a Teleoperation master slave manipulation system to be composed of an active binocular display, a 3-D camera, a master arm and a slave robot. Some vision types were experimented the capacity of remote operation. It is proved that the active binocular display is superior to the 2-D plane visual one and that R-R value and beta wave are possible to estimate an effect of operation, as a result of the experiments.

View full abstract

Recently, there has been much work in the field of teleoperation and telerobotics. A branch of research in this field has moved toward telepresence or tele-existence, where a human operator is made to feel that he is located at the task site. In order to construct a human-robot system operating in the real world by utilizing the tele-existence technology, the relationship between human and robot, i.e., the task allocation between them has to be made clear, and to be matched with the purpose of the constructed system.
In this paper, we propose a novel method for allocating tasks to a human operator and a slave-robot by constructing a cognitive information processing model of the human operator when he is engaged in controlling the tele-existence system. By utilizing this method, we introduce an architecture for a semi-autonomous tele-existence mobile robot, where the human operator generates a path for the robot while the robot moves along the path avoiding the obstacles based on the proposed fuzzy control. This architecture enables us to make a prototype system which is to be used in the security system operating in the real world.

View full abstract

An indoor distribution simulation system based on virtual reality technology has been developed. The system is designed to give the user the feeling as if he is actually touring in a substation for routine inspection. Various states of the substation facilities, such as the opening and closing of the doors and the disconnecting switches as well as the relocation and extension of the installations, can be simulated. In the event the user approaches the energized components of the equipment and fails to keep the specified insulation distance from them, the system first gives an alarm sound, and then a flashing red light on the display to indicate he went beyond the warning point and suffered an electric shock. Validation of equipment operating procedures, training of unskilled workers and work schedule preparation can be upgraded by using this system.

View full abstract

This paper deals with a learning environment for maintenance of plants and equipments based on virtual reality technology. First of all, maintenance tasks for operators are analyzed and the results of cognitive psychology are discussed from the viewpoint of understanding support system. Then design philosophy of the learning environment is proposed based on the analysis. The prototype system is designed and implemented using both EWS (Engineering WorkStation) and GWS (Graphic WorkStation). This prototype system is applied to the maintenance of the Gas Insulated Substation, which is one of power equipments, and its performance is evaluated by subjective assessment.

View full abstract

This paper proposes a three-dimensional computer animation method for behavioral description of crowd, a throng of people, in a virtual city. In the method, each human character in the computer generated city is assigned his/her own personal space, which prescribes the regions around the character where another character may stay or cannot. As for any other environmental object in the city, such as roads, trees, and cars, a similar “personal” space is also specified. Associated with these personal spaces, the vector field, called attractiveness, is generated and enables to describe various behavioral animations of crowd.
The uniqueness and efficiency of the proposed method is well illustrated by the animations obtained, which include the description of psychologically based personal space, walking on a pavement, and behaviors of avoiding collision with cars.

View full abstract

In this papar, we propose a Virtual Reality (VR) Relax/Refresh system employing physiological feedback whose purpose is getting rid of stress and recovering from mental fatigue in a short time.
First, the concept of the system and the necessary functions are discussed from the viewpoint of physiology and VR technology.
Secondly, a prototype system developed on the basis of the concept is shown. It is composed of a massage lounger, a heart rate sensor, a head-mounted-display (HMD), a VCR, and an interface circuit which controls the massage chair and generates threedimensional images. The developed system also has functions such as sleep judgement algorithm and physiological feedback mechanism which varies the stimulations to the living body according to the measured heart rate data.
Finally, the system is evaluated by measuring electrocardiograms of 5 subjects and calculating power spectrum of fluctuation on R-R intervals of the waveform. The result shows the system more effective than just having a rest with eyes ope

View full abstract

Two-handed interactions are efficient to design a three dimensional (3D) object, hence a two-handed spatial interface device must facilitate such a design process in a virtual environment. In order to realize a virtual environment where an operator can use his/her both hands efficiently, it is necessary to implement a virtual reality system that is suitable to human's ability of sensory integration. We can unconsciously integrate many kinds of sensory information, e.g. visual and kinesthetic sensation.
In this paper, we investigate human's ability of sensory integration between visual sensation and kinesthesis of hands when they are in disagreement spatially, by changing the spatial relation among the operator, a visual display, and a 3D spatial interface device. Then, we implement a virtual reality system for two-handed interactions that can facilitate human's smooth sensory integration.

View full abstract

To obtain highly realistic sensation of presence in virtual environment, natural three dimensional sensation of presence, real-time interaction between the operator and the virtual environment, and self-projection to the virtual environment are required. To implement these requirements, we have proposed the concept of “virtual human”, which exists in the virtual environment and acts and feels as the operator in the real world would do. However, this concept has not been realized because of lack of systematic interface to access virtual environment data. In this paper, an object oriented method for constructing virtual environment using class library of the C++ language is proposed, to implement the concept of virtual human. This method was implemented on the system by human motion measurement and visual display, in order to show that the virtual human can be implemented using the proposed method.

View full abstract

When we design 3D objects in virtual environment, we combine and deform various primitives such as line, triangle, polyhedron like a cube and sphere. This method is effective in constructing simple objects. However, it requires enormous effort to construct more complex objects. We propose a new method by using autonomous free-form surface. Users intervene in autonomous deformation process. Cooperative work by two users often leads to more effective work. In this research, we developed a system that enables two users to work simultaneously in virtual environment with force feedback. And we developed autonomous free-form surface which has surface tension. Through experiments in deformation of the surface, the effectiveness of cooperative work in virtual environment is tested.

View full abstract

This paper presents 3D-DDM (3 Dimensional-Direct Deformation Method), which is a new direct deformation method of free-formed B-Spline surfaces. The method allows users to “push/twist” surfaces directly with the 3D cursor and it provides various kinds of deformations that are area-controllable. The basic idea of 3DDDM is rearranging, creating, and deleting form representing parameters from the “pushing” action of a user to obtain appropriate deformations. The method is combined with an input device with force feedback, which the authors have developed as well, into a virtual surface modeler with haptic feedback.

View full abstract

To construct a high performance human interface such as virtual reality, it is very important to evaluate quantitatively the effects of the modality of information presented by a computer such as vision, sound and force for work efficiency. The authors evaluated quantitatively the effects of VISUAL, FORCE and SOUND CUE in the virtual work space using a space interface device for artificial reality named SPIDAR. We can do various works such as pick-and-place of blocks with two fingers in the virtual work space with vision, force and sound presented by SPIDAR. The results were 1. VISUAL CUE had less effects. 2. FORCE CUE had large effects in the occasion in which the displacements of fingers need to be restricted. 3. SOUND CUE had large effects on work efficiency in the virtual workspace. In the occasion in which FORCE and SOUND CUE were simultaneously presented, the largest effects were obtained than in any other occasions with a single cue. It was because of multimodality of the cues.

View full abstract

This paper describes a new see-through display and its applications. The proposed new see-through display system constructed from 2 LCD-TVs and an Electrochromic (EC) mirror, is designed to solve the contrast problem that occurs when two images are simultaneously viewed. The EC mirror is composed of a half mirror with electrochromic layers coated on one side. The transparency ratio of the EC mirror can be specified within a certain range by controlling the amount of voltage on the electrochromic layers. The correct contrast can be obtained for almost every lighting condition if the proper half mirror is selected in the construction of the EC mirror.
As a demonstration of an application using EC mirror technology, a virtual driving simulator was developed for experimenting traffic related problems.

View full abstract

HRTF (Head Related Transfer Function)-based acoustic display systems have been studied in parallel with the development of virtual reality technology. However, their initial intended function was only to display the point sound sources to the user in three-dimensional virtual space. Hence, this provided motivation to design and develop a new sound field display system, which could reproduce complex and widespread Environmental Sound Fields (ESF) with a high degree of realism. To develop this system, the understanding of the behaviour of sound waves is crucial. Thus, an ESF display system along with the application of sound and noise principles was developed and tested. The simulation results were used to generate theoretical diagrams of the sound waves generated by point sound sources. Secondly, an experimental prototype speaker system was developed and accurately tested in terms of auditory localization. Furthermore, the ESF display system was implemented into a conventional virtual reality device. Creation of these graphs and experimental results provided better understanding into the behaviour of the ESF display system and the subjects with respect to integration with conventional virtual reality devices.

View full abstract

A new method of a force display was proposed for presenting the sense of force to an elbow joint using a metal hydride (MH) actuator in order to apply it to artificial reality and tele-existence robots.
A force display made by the MH actuator was attached to a human's elbow joint and two psychophysical experiments were carried out to investigate whether or not the actuator is useful for the force display. First, the differential limen of the force sensation was measured in order to check whether or not the MH actuator can give the smooth force change to the human elbow. Next, the force sensation level produced using the MH actuator was compared with that produced when a real object was put on a forearm.
From the experimental results, the differential limen was found to be about 400 gf and the force sensation level difference between the sensation produced from the MH actuator and that from the real object was found to be smaller than the differential limen. It is ascertained that the MH actuator would be used for one of the force displays.

View full abstract

This paper introduces conceptual level learning (CLL) as a method that can be used to refine human-motion models. CLL imitates human learning processes in which people learn through multi-modal macro-instructions represented by language, gestures, and other means. Our method implements a human-motion model and a macro-instruction interpreter using fuzzy associative memory systems. The model has several nodes, and their connections represent the relationship between upper concepts and minimum components in terms of motion. The interpreter also has several nodes, and their connections represent the relationship between macro-instructions and trends in the minimum components. The interpreter transforms multi-modal macro-instructions to minimum components. Our method provides unskilled people with an easy-to-use design system and provides for useful intelligent human-machine interaction such as virtual reality. This is done by real-time refinement of a model, and visualizing motion so the user can observe it. Here, this method is used to construct and refine a human tennis-motion model.

View full abstract

It is necessary to know how accurately humans can react to sensory stimulation in order to construct a virtual reality system. The motion induced by the sensory stimulation is called ‘sensory-motor coordination.’ We have been investigating the accuracy of gazing and indicating movements to the target such as a flashlight presented around subjects without seeing their arm to analyze the mechanism of the sensory-motor coordination. In this report, the angle error between the indicated angle and the target angle was measured as a function of the target angle. The targets were placed in working space with the distance of 0.5 meter from a subject to the targets and out of extraworking space with the distance of 1.5 meter. From the responses to the target at the distance of 0.5 meter, the degree of the angle error was ascertained to increase in the peripheral area, especially at 80 degrees and the indicated angle inclined to center. From the responses to the target at the distance of 1.5 meter, all indicated angle's responses shifted to the outside compared with the responses of 0.5 meter. We discussed how the virtual reality systems should be constructed based on these findings.

View full abstract

Most VR systems these days use unconstrained motion sensors like Polhemus Tracker, which has deficiencies of low sampling rate and critical latency. Some ad-hoc linear filters were proposed, but are known to be incomplete to rotatinal motion. We proposed the Sensor-Fusion-based method to the problem before. In this paper we improve this method by adopting quaternion and report its real-time implementation. We also propose to use normalized cross-correlation function to evaluate unconstrained sensors quantitatively.

View full abstract

In the present virtual reality system, the user must wear certain sensors (e.g. data gloves, head mounted display) in order to measure the user's head position and orientation and track other body parts such as the user's hand and arm. A more flexible interface paradigm measures the head position and orientation or gesture directly from images captured by video cameras; thus freeing the user from wearing the cumbersome sensors. However, the image processing technique is still experimental because the previous structure-from-motion techniques suffer from lack of robustness against noise, instability caused by sharp acceleration, and uncertainty due to line-of-sight occlusion. To addresses these drawbacks, we describe a Kalman filtering-based technique to recover more robustly 3-D structure and kinematics of the head and arm in view from optical flow. Because of its explicit modeling of measurement noise and modeling uncertainty, the extended Kalman filter (EKF) has been shown to improve the robustness of structure-from-motion techniques. The robustness of the recursive estimation technique is further enhanced by (1) confining feature-tracking within the neighborhood of skeleton sketch of the head and arm, (2) formulating measurement noise of the optical flow as a function of the optical flow's confidence measures, and (3) for the occluded feature points, fusing into EFF the optical flow predicted from the estimated 3-D kinematics, and (4) compensating the feature tracking errors with a feedback loop that tracks feature points from the projections of structural estimates.

View full abstract

Accompanying the miniaturization and lightening of electric products, high density mounting of printed circuit boards is accelerating. Positioning by image processing has become indispensable for mounters such as chip shooters to improve mounting accuracy. Mounting speed is a large factor in determining a chip shooter's performance. Recently, the ratio that IC parts hold and the number of IC pins are increasing, and even faster processing is being demanded by users. Positioning which utilizes template matching by means of a normalized correlation algorithm has advantages of high accuracy and resistance to the effects of lighting changes; how to reduce processing time for the expanding operation amount is the problem.
We are presenting one technique for measuring parts that has the same level of accuracy as the normalized correlation algorithm, as well as being able to conduct, high speed processing. This paper focuses on measuring the position of IC parts.

View full abstract

It is a challenging robotic task to directly grasp a moving object with various types of shape. Previously some researchers have addressed robotic technologies relating to shape recognition by vision sensing and control of some dexterous hands. However, there are few reports to succeed in realizing the task in an actual system because it needs an enormous computation to simultaneously conduct some necessary subtasks to detect the state of the object, generate a suitable trajectory of the manipulator and control to stably grasp it. In this paper we introduce an new automatic handling system for a moving object using a visual servoing technique and a hand with three fingers which can detect contact forces and ranges to the object. Here some experimental results are presented.

View full abstract