In this paper, we challenge to solve a reinforcement learning problem for a 5-linked ring robot within a real-time so that the real-robot can stand up to the trial and error. On this robot, incomplete perception problems are caused from noisy sensors and cheap position-control motor systems. This incomplete perception also causes varying optimum actions with the progress of the learning. To cope with this problem, we adopt an actor-critic method, and we propose a new hierarchical policy representation scheme, that consists of discrete action selection on the top level and continuous action selection on the low level of the hierarchy. The proposed hierarchical scheme accelerates learning on continuous action space, and it can pursue the optimum actions varying with the progress of learning on our robotics problem. This paper compares and discusses several learning algorithms through simulations, and demonstrates the proposed method showing application for the real robot.

View full abstract

In this paper we propose a new method to obtain transition rules of one-dimensional two-state cellular automata (CAs) using genetic algorithms (GAs). CAs have the advantages of producing complex systems from the interaction of simple elements, and have attracted increased research interest. However, the difficulty of designing CAs' transition rules to perform a particular task has severely limited their applications. The evolutionary design of CA rules has been studied by the EVCA group in detail. A GA was used to evolve CAs for two tasks: density classification and synchronization problems. That GA was shown to have discovered rules that gave rise to sophisticated emergent computational strategies. Sipper has studied a cellular programming algorithm for 2-state non-uniform CAs, in which each cell may contain a different rule. Meanwhile, Land and Belew proved that the perfect two-state rule for performing the density classification task does not exist. However, Fuks´ showed that a pair of human written rules performs the task perfectly when the size of neighborhood is one. In this paper, we consider a pair of rules and the number of rule iterations as a chromosome, whereas the EVCA group considers a rule as a chromosome. The present method is meant to reduce the complexity of a given problem by dividing the problem into smaller ones and assigning a distinct rule to each one. Experimental results for the two tasks prove that our method is more efficient than a conventional method. Some of the obtained rules agree with the human written rules shown by Fuks´. We also grouped 1000 rules with high fitness into 4 classes according to the Langton's λ parameter. The rules obtained by the proposed method belong to Class- I, II, III or IV, whereas most of the rules by the conventional method belong to Class-IV only. This result shows that the combination of simple rules can perform complex tasks.

View full abstract

In this study, a virtual art museum system has been developed. The system, ArtFinder3, has two main functions -- sharing and 3D-visualizing viewpoints of art appreciation -- to support development of aesthetic experience. Using the functions, a user can understand other viewpoints and develop art appreciation skills. Walking through virtual exhibition rooms of the system, a user can not only appreciate art works, but also take notes of his/her thoughts on them. The notes are written using an extended semantic network notation and editted graphically on a page of a sub-system Appreciation Notebook. Additionally, using the sub-system, a user can share his/her viewpoint with others and find other viewpoints related to his/her own. An extended semantic network on a page of Appreciation Notebook, which represents a user's viewpoint, can also be transformed into a virtual exhibition room by another sub-system 3D-Visualizer. A user can walk throuth this virtual exhibition room and appreciate art works in it. In this way, a user can understand the different kind of view of the same art works that he/she appreciated. The process leads a user to an advanced level of art appreciation.

View full abstract