This article surveys the history of the field of Neural Network research and presents a review of several techniques developed in the field. Attempts at statistical analysis of search dynamics of the optimization methods in Soft Computing and recent advances on implementation in parallel computers are briefly introduced.
This paper proposes a multi-point combinatorial optimization method based on Proximate Optimality Principle (POP), which method has several advantages for solving large-scale combinatorial optimization problems. The proposed algorithm uses not only the distance between search points but also the interaction among search points in order to utilize POP in several types of combinatorial optimization problems. The proposed algorithm is applied to several typical combinatorial optimization problems, a knapsack problem, a traveling salesman problem, and a flow shop scheduling problem, in order to verify the performance of the proposed algorithm. The simulation results indicate that the proposed method has higher optimality than the conventional combinatorial optimization methods.
In many engineering applications, it is necessary to find more than one solution to an optimization problem with complex multi dimensional objective function. For example, energy supply systems require the most attractive solution in cost but also a solution near to the actual regime among multiple solutions. Metaheuristics is paid to attention as a method for solving such a problem. The parallel searching Niche PSO (Particle Swarm Optimization) algorithm can find multiple solutions in multi dimensional problems in an acceptable time limit. In this paper, we propose the repetitive searching Multi PSO algorithm which is based on PSO and detects more precise solutions in the same time. Furthermore, we show the results of applying Multi PSO and Niche PSO to energy supply systems, and evaluate the validities of these methods.
When environments are dynamically changed for agents, the knowledge acquired in an environment might be useless in future. In such dynamic environments, agents should be able to not only acquire new knowledge but also modify old knowledge in learning. However, modifying all knowledge acquired before is not efficient because the knowledge once acquired may be useful again when similar environment reappears and some knowledge can be shared among different environments. To learn efficiently in such environments, we propose a neural network model that consists of the following modules: resource allocating network, long-term & short-term memory, and environment change detector. We evaluate the model under a class of dynamic environments where multiple function approximation tasks are sequentially given. The experimental results demonstrate that the proposed model possesses stable incremental learning, accurate environmental change detection, proper association and recall of old knowledge, and efficient knowledge transfer.
Particle Swarm Optimization (PSO), which has attracted special interest as a global optimization method recently, has a drawback in that its sustainable search can not be executed until the end of computation. In order to endow global searching abilities to PSO, repetition of unstable and stable states of the particles is necessary. In this paper, based on stability analysis of PSO's model, with considering its random numbers, we realize sustainable search by choosing system parameters on boundary region between unstable and stable states, and then introduce an optimization model with global searching abilities as a revision of the conventional PSO.
Optimization problems in which multiple objective functions are optimized simultaneously are called “multi-objective optimization problem”. Multi-objective optimization problems appear naturally in the decision making process for complex systems. Recently, a number of multi-objective optimization methods which search Pareto optimal solutions covering Pareto front have been proposed and have attracted much interests. Such methods are based on meta-heuristics, and a multi-objective optimization method based on gradient dynamics which takes a similar approach has not been proposed yet to our knowledge. In this paper, we propose a new multi-objective optimization method using a coupled discrete gradient dynamics. In the proposed method, firstly, we consider multiple search points driven by discrete gradient dynamics which optimize respective objective functions independently. Next, trajectories of the search points are synchronized by a coupling among the search points. Then, Pareto optimal solutions that cover whole Pareto front are obtained by the modulation of dynamic characteristics of each optimization model. We confirm effectiveness of the proposed method through applications to benchmark problems which have various types of Pareto fronts.
A Hopfield network is a good tool for solving combinatorial optimization problems. But one of its major drawbacks is existence of energy local minima, because iterative state transitions are carried out just reducing the energy defined in advance. In order to avoid such poor solutions, a virtual magnetic diminuendo (VMD) method is proposed recently. Although its effectiveness is confirmed through some computer simulations, its working mechanism has not been clear yet. Then, in order to solve this tough problem, behavior of the Hopfield network is investigated carefully with the help of visualized representation in this paper. As a result, it is confirmed as follows: i) When the virtual magnetic parameter exceeds one of its critical values, easiness of neuron firing chance is changed and it makes improve the score. ii) A configuration of the energy function is changed, e.g. an energy basin becomes a hillside and vice versa in some cases, by controlling the virtual magnetic parameter, and it makes be possible to escape from a trapped energy local minimum. These two facts must be the very essence of the VMD method, and they strongly support the reason of performance improvement by introducing the VMD method.
Today, a lot of automatic programming techniques have been proposed and applied in various fields. Graph Structured Program Evolution (GRAPE) is one of the recent automatic programming techniques. The representation of GRAPE is graph structure, therefore it can represent complex programs using its graph structure. GRAPE succeeds in generating the complex programs automatically. The generation alternation model of GRAPE is usually used minimal generation gap (MGG) which is not considered the evolution of program size. Therefore, it would not be search various program sizes. In this paper, a new evolutionary algorithm for GRAPE, called Evolutionary Algorithm Considering Program Size (EACP), is proposed. EACP maintains the diversity of program size in the population by using particular fitness assignment and generation alternation. We apply EACP to three test problems, factorial, exponentiation and sorting a list. And we show the effectiveness of EACP and confirm evolution of maintaining the diversity of program size.
This paper proposes an automated web site evaluation approach using machine learning to extract evaluation criteria from the existing evaluation data. Evaluating web sites is a significant task because evaluated web sites provide useful information for users to estimate sites' validation and popularity. Although many practical approaches have been taken to present a measuring stick for web sites, their evaluation criteria are set up manually. Thus, we develop a method to obtain evaluation criteria automatically and rank web sites with the learned classifier. Evaluation criteria are discriminant functions learned from a set of ranking information and evaluation features collected automatically by web robots. We conducted experiments and confirmed the effectiveness of our approach and its potential in performing high quality web site evaluation.
In this paper, we propose a new vectorization method for a new generation of computational intelligence including neural networks and natural language processing. In recent years, various techniques of word vectorization have been proposed, many of which rely on the preparation of dictionaries. However, these techniques don't consider the symbol grounding problem for unknown types of data, which is one of the most fundamental issues on artificial intelligence. In order to avoid the symbol-grounding problem, pattern processing based methods, such as neural networks, are often used in various studies on self-directive systems and algorithms, and the merit of neural network is not exception in the natural language processing. The proposed method is a converter from one word input to one real-valued vector, whose algorithm is inspired by neural network architecture. The merits of the method are as follows: (1) the method requires no specific knowledge of linguistics e.g. word classes or grammatical one; (2) the method is a sequence learning technique and it can learn additional knowledge. The experiment showed the efficiency of word vectorization in terms of similarity measurement.
AdaBoost is a method to create a final hypothesis by repeatedly generating a weak hypothesis in each training iteration with a given weak learner. AdaBoost-based algorithms are successfully applied to several tasks such as Natural Language Processing (NLP), OCR, and so on. However, learning on the training data consisting of large number of samples and features requires long training time. We propose a fast AdaBoost-based algorithm for learning rules represented by combination of features. Our algorithm constructs a final hypothesis by learning several weak-hypotheses at each iteration. We assign a confidence-rated value to each weak-hypothesis while ensuring a reduction in the theoretical upper bound of the training error of AdaBoost. We evaluate our methods with English POS tagging and text chunking. The experimental results show that the training speed of our algorithm are about 25 times faster than an AdaBoost-based learner, and about 50 times faster than Support Vector Machines with polynomial kernel on the average while maintaining state-of-the-art accuracy.
Artificial Immune Systems applied to network intrusion detection have been shown to perform relatively well detecting anomalies formed by connections timely and spatially clustered on small datasets. However, the performance of Artificial Immune Systems does not scale up well on large data sets, where anomalies are less likely to appear tightly clustered. In this work, we propose a collective detection with diversity activation method to detect anomalies that need not be spatially and timely clustered, aiming to improve the performance of AISs for network intrusion detection. In the proposed method, detectors are activated based on the diversity of connections that are able to match. In the detection stage, collectives of detectors are used to detect the anomalies. We conduct experiments with data obtained from a real network, verifying that the proposed method can improve detection rates of AISs significantly.
This paper proposes a Particle Swarm Optimization (PSO) with hierarchical structure. In the proposed method, particles are separated into some groups, and besides, in a group particles are parted the particle of the best value from other particles. Particles of the best value in each group are applied to Gbest Model, and other particles are applied to Lbest Model. Then, the proposed method is validated through numerical simulations with several functions which are well known as optimization benchmark problems comparing to the conventional PSO methods.
A method for an optimal design of a surface motor based on integrated optimization is proposed in this letter. While the optimal design problem of a surface motor is formulated as a continuous optimization problem of design parameters, Particle Swarm Optimization (PSO), Radial Basis Function Network (RBFN), and Electromagnetic field simulation are used in the integrated optimization. The proposed approach is applied to an optimal design of a stator of a surface motor, and the advantage of the proposed approach is verified.
GNSS (Global Navigation Satellite Systems) Regression (GR) models are shown. Two kinds of GR models for unknown and known positions are derived. The unified methods of point and relative positioning algorithms are shown based on combining GR models. Further, estimation methods of ionosphere total electron content, and orbit and clock errors of GPS satellites are reviewed by applying GR models and measuerement data at the known positions such as GEONET.
Recently, the wireless sensor network where advanced sensing, and image recognition and a network were connected attracts attention. In this research, the multi-hop wireless sensor network which carries out sensing of the image which used IEEE 802.15.4 ZigBee is built. In the multi-hop network, two or more sets of terminals may be communicating within 1 hop using the same frequency band. In that case, communicative delay occurs by access control (CSMA/CA) of IEEE 802.15.4. In order to solve this problem, in this research, proposal of the image transfer network which applied the Frequency Division Multiplex system to IEEE 802.15.4, and construction were performed. In order to show the usefulness of this proposal system, image transmission by the conventional system and a proposal system was performed, and transmission time was measured. When a proposal system was applied, improvement in a throughput has been confirmed as compared with the conventional system. Furthermore, improvement in a throughput was confirmed by performing a channel setup which avoided the channel currently used by wireless LAN. Moreover, it confirmed that a proposal system improved about 25 [msec] faster than the conventional system by the experiment of the picture transmission by communication of IEEE 802.15.4.
In many metaheuristics, such as simulated annealing or genetic algorithm, the aim of optimization is to obtain better results at the end of the search process. However, It is more useful to be able to get better results, also in the early stage of the search process. In this paper, we propose a new “agent search” method with the goal of obtaining better results not only at the end of the search process, but also in the early stage of the search process. In our method, a number of “search agents” autonomously explore for better solutions in the solution space, by means of several neighborhoods with different sizes. Some “manager agents” modify the status of each search agent under control, by two operations (“transfer” and “transport”) for the improvement of effectiveness of the exploration. The speed of searching of each search agent is measured, in order to decide the timing and kind of the operation. Our method has applied to passive filter synthesis for performance evaluation, and acceptable filter has been synthesized automatically.
This research examined the relevance of the face skin temperature change and the amount change of face blood flows by the degree fall of transient awakening of a driver. First, the data of face expression evaluation, alpha wave power, and LF/HF were compared with the data of precedence research on the basis of nasal part skin temperature. The data of face expression evaluation, alpha wave power, and LF/HF was mostly in agreement. Next, comparison of the amount of face blood flows and nasal part skin temperature was performed using the proved data. A result -- the amount of face blood flows, and nasal part skin temperature every subject some although there was a difference, it detected negative high correlation This was the result of differing from the conventional view. Since the result was summarized this time using the hypothesis of the relation nature of the amount of face blood flows, and nasal part skin temperature.
A simplified IMC scheme is proposed in this paper. By using an inverse system or an approximate inverse system for the IMC structure, the order of the controller leads the high order, meanwhile the high order controller is not practical in implementation. This paper is concerned with the problem of reducing a high order controller to a reduced order one. The disturbance estimation property of the closed loop system is also discussed. The proposed procedure is applied to prototype robotic hand that can hold an object with suitable grasping force without force sensor. The effectiveness of the proposed controller scheme is confirmed by experiments.
This paper presents the methods of the multi-layer control and the graphical feature editing by the server side rendering in Ajax-GIS. Ajax-GIS uses divided raster image file called “tile” in order to keep light handling. We propose that the multi-layer control is realized by means of merging tiled images in the server application as the requests of the client application. Furthermore we propose the graphical feature editing protocol that sent from a client and send back to an image in order to edit a feature such as moving vertices, changing color. In an evaluation experiment of an actual map data, we confirmed the effectiveness of these methods as comparing with conventional methods.
Recently, it was shown that squeezing is good at low information rate. In this paper, we investigate capacities for 3ASK squeezed-state and coherent-state signals. As a result, it is clarified that squeezing is good not only at low information rate but also at high rate as the capacity when the signals are discrete.
In this paper, we propose a method to synthesize whistling sounds using frequency modulation, for musical whistling certificate examination system. This paper shows that the proposed whistling sounds have good sound quality in comparison with MIDI sounds.
Histogram equalization (HE) is one of the common methods used for improving contrast in digital images. However, this technique will cause an effect on brightness saturation in some almost homogeneous area. In this letter, we propose a novel HE with variable enhancement degree. In this method, we can control the enhancement degree by one parameter. We also show the guide to decide the parameter. Since our method is able to control the enhancement degree, all kinds of images are enhanced efficiently.
This paper proposes an Estimation of Distribution Algorithm with partial solution. In the proposed method, partial solutions are generated by estimated value and search local solution by co-evolution. The proposed technique is applied to some optimization problems to verify its performance.