This paper introduces several problems in reinforcement learning of industrial applications, and shows some techniques to overcome it. Reinforcement learning is known as on-line learning of an input-output mapping through a process of trial and error interactions with its uncertain environment, however, the trial and error will cause fatal damages in real applications. We introduce a planning method, based on reinforcement learning in the simulator. It can be seen as a stochastic approximation of dynamic programming in Markov decision processes. But in large problems, simple grid-tiling to quantize state space for tabular Q-learning is still infeasible. We introduce a generalization technique to approximate value functions in continuous state space, and a multiagent architecture to solve large scale problems. The efficiency of these techniques are shown through experiments in a sewage water-flow control system.
PID control schemes still continue to be widely used for most industrial control systems. This is mainly because PID controllers have simple control structures, and are simple to maintain and tune. However, it is difficult to find a set of suitable control parameters in the case of time-varying and/or nonlinear systems. For such a problem, the robust controller has been proposed.Although it is important to choose the suitable nominal model in designing the robust controller, it is not usually easy.In this paper, a new robust PD controller design scheme is proposed, which utilizes a genetic algorithm.
Many mechanical systems are subjected to periodic disturbances which may adversely influence control performance. When the plants are nonlinear systems with unknown parameters, this disturbance compensation problem has been considered as adaptive output regulation problem. This paper discusses the problem about adaptive output regulation for nonlinear systems with arbitrary relative degree, which can be transformed into the output feedback form. The nonlinear plant is affected by unknown constant parameters and periodic disturbances with unknown frequencies, magnitudes and phases. The periodic disturbances are considered as signals generated from a linear time invariant exosystem. It is not assumed that the disturbances meet the matching condition for the control input. The design method of the adaptive controller and its parameter update law is based on adaptive backstepping manner for the coordinates-changed extended system, which is generated from the nonlinear plant and the exosystem. Then, it can be achieved that the output asymptotically converges to the output reference and that all signals in the closed-loop system are bounded.
In container yard terminals, containers are brought by trucks in the random order. Since each container has its own destination and it cannot be moved after shipping, containers have to be loaded into a ship in a certain order. Therefore, containers require rearrangement from the randomly stacked initial state into desired order before shipping. In the problem, the number of states for the container stack increases by the exponential rate with increase of containers. In this paper, a new design method of reinforcement learning system is proposed to obtain desirable movements of containers and to reduce the run time for shipping. The proposed method assures that the optimal order of container movements can be obtained. Moreover, the method can reduce the required memory size. In order to show effectiveness of the proposed method, simulations for several examples are conducted.
This paper describes a state space composition technique for the adaptation to environment in the autonomous behavior of intelligent wheel chair (IWC).In the product like IWC with actual sensors, composing state space is difficult problem since environmental information can not be observed sufficiently from restricted sensor inputs.A lot of states observed from same environment position raise the fail of the learning and adaptation with active learning approach.In order to compensate for the effects of the sensor configuration, that is sensor position, angle and precision, a normalization processing of position detector is introduced.In sensor normalization process, IWC scans present environment via range sensors with executing spot-turn, and prepare scan-patterns of each sensor.Then the normalization process adjusts the phase and dynamic range of each pattern to the reference sensor scan-pattern, analyzing phase differences and scale factors of each pattern against reference pattern.Using phase difference and scale factors, automated state space composition is possible.From the simulation experiment with both artificial and real-worlddraft, the automated state space construction is confirmed as a practical approach for pre-processing for environment learning and adaptation.
Ultrasonic motors (USM) which has many useful features such as small, light and high torque at low speed are expected to find many applications in several fields. However it is difficult to control precisely since the USM contains nonlinear properties caused by driving frictions, changes of temperature, fluctuation of load-mass and so on. To date, a large number of studies have been made on the compensation of the USM nonlinear properties using neural network, fuzzy control and other position control theories as well. However, studies aiming at holding the optimal tracking performance of the USM designed for nominal plant have not been reported yet. Accordingly, in this paper, we propose a position control method of the USM using a two-degree-of-freedom optimal servo system with an adaptive technique. The effectiveness of the proposed method is well demonstrated based on the results of our experimental study.
This paper gives a design of an I-P controller for a pressure control system in a furnace of a thermal power plant boiler. The plant has multivariables, interactions among variables, non-linearity and uncertain time delays. Hence the control performance for this plant would be not good by simply using I-P controllers. To obtain a good transient property for a reference signal of a plant such as having the interactions among variables, non-linearity and unknown time delays, this paper regards interactions as disturbances, lets the plant approximate to two first-order independent systems having disturbances instead of interactions and uses a self-tuning I-P controller based on an extended generalized predictive control (GPC) which can redesign disturbance response independently to reference response. Expressions of a self-tuning scheme of the I-P controller for two first-order systems are obtained by approximating the I-P controller to the extended GPC method with identified parameters of two first-order systems. Simulation results are given to show effectiveness of the proposed controller.
The tuning rule of PID gains has been studied for a long time. For example, there is a method referred to as the improved ultimate sensitivity method. This method which utilizes the estimate of delay-time and the characteristic type tunes the gains. However, estimating delay-time and the characteristic type is needed experience of an expert, it is not easy for beginners to estimate these values accurately. Therefore the systematic method for estimating these values is required. In this paper, a new method for identifying the delay-time and the characteristic type of a plant with input delay is proposed. This method only uses frequency responses at two frequencies, which correspond to the critical point and the other point on the Nyquist locus. Then, PID gain is calculated by applying the improved ultimate sensitivity method, and numerical examples show the effectiveness of the proposed method.
This paper describes a novel application of genetic algorithm for navigation of an autonomous mobile robot (AMR) under unknown environments. In the navigation system, the AMR is controlled by the decision-making block, which consists of neural network. To achieve both successful navigation to the goal and the suitable obstacle avoidance, the connection weights of the neural network and speed gains for predefined actions are encoded as genotypes and are tuned simultaneously by genetic algorithm so that the static and dynamic danger-degrees, the energy consumption and the distance and direction errors decrease during the navigation. Experimental results demonstrate the validity of the proposed navigation system.
No paper has been published on the active use of the foot toe of the biped walking robots. In other words, the sole of the supporting leg is usually assumed to be completely contacted to the floor. To maintain this condition, the robot walking has a restriction such as the maximum walking speed limitation. If the point contact of the toe to the floor through walking is available, the variety of the walking can be realized. In this paper, a new control method for biped robots, in which the contact of the sole to the floor becomes a point contact, is proposed. The COM (Center of Mass) and swing leg can track the desired path and each heel joint angle can be controlled by the proposed controller while a biped walking robot is following a forward falling. Therefore the motion of the body has not to be limited to the quiet motion. The proposed method is verified by the simulation and experimental results with 6 joints DOF biped robot“Ken”.
A universal motor, which is mainly used in vacuum cleaners, generates commutation sparks at the moment when the brush and the commutator segment are separated from each other during rotation. This study investigates the mechanism of commutation spark generation by analyzing high-speed camera images and its electrical aspect. We invented a new external trigger method that used laser light as the trigger signal for the shuttering a high-speed camera. This method enabled us to photograph sparks on any desired commutator segments during high-speed rotation, and that made the analysis after photographing easier. This paper shows that commutation sparks in universal motors are generated on every other commutator segment and at the peak of pulses in the voltage between the brush and commutator segment. Other aspects are also clarified, such as the generation of the singular and plural number of sparks on one commutator segment at a time, the time from the moment of spark generation to extinction, and spark generation during a single rotation.
This paper presents a fast electromagnetic field analysis by the 3D geometric multigrid method with edge elements. The multigrid method uses a symmetric Gauss-Seidel smoother with Conjugate Gradient acceleration. The convergence and computation speed of the V-cycle, W-cycle and full multigrid method using this smoother are compared with the conventional multigrid using Gauss-Seidel. Comparison is also made between the multigrid method and the ICCG method which is commonly used in the finite element analysis. The efficiency of the multigrid method is analyzed for meshes whose maximum aspect ratios vary in a wide range. It is proven that the multigrid method with the accelerated symmetric Gauss-Seidel outperforms the multigrid with Gauss-Seidel and the ICCG method. The multigrid method with the accelerated symmetric Gauss-Seidel shows stable convergence rate that does not deteriorate for bad quality meshes. It is robust against mesh distortion and parameter variations and is much faster than the conventional multigrid with Gauss-Seidel and the finite element method using ICCG.
In the case of the sensorless control for the PMSM that estimates position and velocity from EMF using the model of a synchronous motor with the voltage and current, it is desirable for voltage and current to be perfect sine waves. However, in fact, various harmonic components are contained in the voltage and current, and they have bad influence on the estimation performance of a sensorless control. In this paper, we propose a improving method of sensorless control performance for the IPMSM by suppression of harmonic component in current control loop of the vector control using Fourier transform and repetitive control. To suppress harmonic components, compensation signals are acquired using Fourier transform and repetitive control, and feedforward control is implemented with the acquired compensation signals. To evaluate our proposed system, the sensorless control system for the IPMSM is taken for instance. The system is constructed on the basis of the extended EMF disturbance observer with the current and voltage command whose harmonic components are suppressed. As a result, we can have a margin in the pole assignment of disturbance observer, and the performance of sensorless control can be improved. The effectiveness of the proposed method is confirmed by experimental results.
This paper presents a mathematical approach for the analysis of trihalomethane (THM) generation and its spread in water distribution network. THM is a carcinogenic substance, which is produced principally from chlorine and some THM precursors, whereas chlorination of water in purification plants is an indispensable process. Therefore it is important to know when, where and how much THM will be generated in distribution network that has an exceedingly complex structure. In this paper, we propose a mathematically analyzing system of THM generation and its spread that consists of four functional componets, which are a pipe network analysis, an analysis of stay-in-pipe time of water, an estimation of THM generation, and an analysis of THM spreading. We describe in detail the new algorithm to calculate stay-in-pipe time of water, which is needed in the estimation of THM generation, and show a numerical experiment for the proposing algorithm.
This paper treats the 3-phase synchronous inductor motor which is the same structure as the PM type or HB (hybrid) type 3-phase stepping motor. The characteristics of the synchronous inductor motor when it is driven by a single-phase power supply using a phase shifting capacitor are analyzed. Motor performance is examined in the connection with the capacitance of capacitor, and the selecting method of capacitance for obtaining good performance is proposed. Effects of motor constants on the characteristics of the motor are discussed. Moreover, starting characteristics of the motor driven by a single-phase power supply are also examined.
A major problem in a visual servo system arises because of longer sampling period of image processing as compared to that of manipulator control. This paper proposes a new visual servo system compensating the delay time of the image processing. To obtain an image feature of manipulators without delay time, variation of image feature during delay time is estimated by a Jacobian matrix from joint velocity to image feature. Moreover, a criterion function using the estimated image features is adopted so that manipulators can avoid obstacles using redundancy. Here, a strategy that enables less computational time and easy programming by using the Jacobian matrix is proposed. A feature based visual servo system without inner loop speed or position controller and 3D reconstruction is proposed. Here, the disturbance observer in joint space realizes acceleration control in image plane. The system utilizes the estimated image feature without delay in each manipulator control period. The proposed system is applied to trajectory control and obstacle avoidance using redundancy experimentally.
A magnetic bearing system usually contains position sensors that detect the gap length between the electromagnet and the rotor. Employing the position sensor is obstructions of downsizing and cost reductions of the system. To solve the problem, self-sensing techniques are developed. This paper proposes a new PWM (Pulse Width Modulation) amplifier for self-sensing magnetic bearings. A self-sensing magnetic bearing reported in this paper solves the problem that sensing characteristics are affected by the length of the cable between the PWM amplifiers and electromagnets. This methodology is verified by supporting the rotor of a high-speed motor with the proposed self-sensing magnetic bearings in two axes and conventional magnetic bearings with sensors in the other three axes. In the verification, the rotor revolving at 47, 000 r.p.m. has stably been suspended in the various cases of the cable length.
One of the subjects of cold rolling is a flatness of the rolled strip. Conventionally, measured strip flatness was approximated by polynomial (2th, 4th, 6th) equation across the entire strip width. This made it difficult to deal with desired loose edge or any desired flatness across the entire strip width. Also conventional flatness control was done for the entire strip width, so if there is a different flatness error among drive side and work side, conventional flatness control can not control properly. We propose independent strip flatness control among drive side and work side, and also automatic flatness control (AFC) system with arbitrary desired strip flatness. Also some applied results to cold mill are shown.
Motion control is developed in control of industrial machines. However, some kind of machines working in an open environment are required in our future society. Then it is necessary to design general methodology for motion controller in open environment. In this paper, decentralized control system is applied as a motion controller for this purpose. This paper is constructed from two parts roughly: the first is a method to abstract environmental mode from environmental information, and the second is a method to design the motion controller with the environmental information. This paper applied this method to biped robot as a model case. The effectiveness of proposed method is confirmed by several simulation and experimental results.
This paper aims at the improvement in a future life using a mobile manipulator. For example, power assistance of old people and cooperative operation with human. For coexistence of robot and human being, it is required that a person can work in cooperation with robot safely. To address this issue, this paper introduces interactive virtual impedance which determines the cooperative motion between robot and human being. The external force by human added to end-effector of manipulator is given into vehicle and manipulator as force command. The equivalent mass of each system is used as a standard which opts for operation of manipulator and vehicle. By determining the impedance input of the vehicle according to equivalent mass, the virtual force input to the vehicle in consideration of the manipulability of manipulator and vehicle can be performed. This is one of the remarkable points in the proposed strategy. The validity of the proposed method was confirmed by numerical simulation and experimental results.
A novel type of inverter is proposed for grid-connection of new energy power generation. The proposed inverter is based on buck-boost operation well-known in DC chopper circuits. This inverter acts as an interface between the generator and the grid and has a distinctive feature of realizing the inverse load flow with a wide range of input DC voltage. A control scheme of DC reactor current is also introduced in order to reduce losses in the DC reactor. The performance and characteristics of the proposed inverter is discussed in computer simulation.
This paper presents a novel circuit topology to attain ZVS operation of a high frequency inverter over a wide range output power regulation using a PWM control technique by connecting an auxiliary switch to the conventional single ended push-pull (SEPP) ZVS high frequency inverter. A switching current is injected into the main switches via the auxiliary switch only during the short period between its turn-on and off times to supply a current required for its ZVS operation.
This paper presents a novel single-phase current source converter coupling with ac chopper, which is connected in parallel to main bridge. Two switching devices are used in common for main bridge and ac chopper circuits. This paper gives the circuit configuration, the operational principle and a PWM method. The calculated result proves that the proposed converter allows the dc reactor to be greatly reduced and has the smoothing dc current and the sinusoidal ac current.