In this paper, we consider profit sharing that is one of the reinforcement learning methods. An agent learns a candidate solution of a problem from the reward that is received from the environment if and only if it reaches the destination state. A function that distributes the received reward to each action of the candidate solution is called the reinforcement function. On this learning system, the agent can reinforce the set of selected actions when it gets the reward. And the agent should not reinforce the detour actions. First, we will propose a new constraint equation about reinforcement functions to distribute the reinforcement values on the non-detour actions. If we use the reinforcement function to satisfy the constraint equation, the agent can select the non-detour actions directing to the destination state. Next, it is shown that the reinforcement function can be constant after learning process to suppress the selection of detour actions. Lastly, in computer simulations for maze problems, we show that the learning performance of agents does not depend on the size of environment.

View full abstract

This paper addresses a framework in dialog systems that can understand users' speech depending on location during outdoor activities. Depending on the outdoor context, systems should be able to correctly interpret spoken sentences in changing locations. A promising approach is to combine the knowledge of users' goal and plan with information of their position since their speech is normally dependent on their goal and plan, which can be related with their location in an outdoor activity. To test this approach, a user support system called G-Assist has been developed for mobile PCs. G-Assist provides pieces of advice to beginner golf players in response to their questions via voice on the spot. By defining the associations of users' goal and plans and spaces in the knowledge base, the system dynamically activates the corresponding part of a goal and plan with respect to the users' position or a location provided by the GPS during the dialog. To interpret a spoken sentence, the system integrates bottom-up information based on analysis of the sentence with top-down information derived from the users' goal and plan definitions. The framework is useful in analyzing incomplete sentences in which information dependent on the location is omitted. Furthermore, the top-down information is utilized to revise the result of voice recognition so that a recognized word can be replaced with another word of similar sound highly related to activating goal and plans.

View full abstract

Recently, many opportunities have emerged to use the Internet in daily life and classrooms. However, with the growth of the World Wide Web (Web), it is becoming increasingly difficult to find target information on the Internet. In this study, we explore a method for developing the ability of users in information seeking on the Web and construct a search process feedback system supporting reflective activities of information seeking on the Web. Reflection is defined as a cognitive activity for monitoring, evaluating, and modifying one's thinking and process. In the field of learning science, many researchers have investigated reflective activities that facilitate learners' problem solving and deep understanding. The characteristics of this system are: (1) to show learners' search processes on the Web as described, based on a cognitive schema, and (2) to prompt learners to reflect on their search processes. We expect that users of this system can reflect on their search processes by receiving information on their own search processes provided by the system, and that these types of reflective activity helps them to deepen their understanding of information seeking activities. We have conducted an experiment to investigate the effects of our system. The experimental results confirmed that (1) the system actually facilitated the learners' reflective activities by providing process visualization and prompts, and (2) the learners who reflected on their search processes more actively understood their own search processes more deeply.

View full abstract

We have been developed a support system to interactively search certain articles of users' interest on the World Wide Web (WWW) without their hesitating over query choices. Especially we have been implementing an effective application system to enable tourists to easily find special event information of their interest and to enjoy their own tours. This system also enables developers of each system to provide them with the means of easily constructing an initial database and automatically updating it.
As events are generally held cyclically, we have assumed events or keywords related to the events will appear in each term. If we can extract keywords that appear cyclically in a corpus including date information, we can obtain event keywords easily. The system can extract event information using the event keywords as queries for WWW information retrieval systems, and update the database automatically.
In this paper, we introduce our support system with a focus on a concept to extract event keywords and event information through the appearance of keywords periodically. We found our approach effective by some experiments.

View full abstract

We develop a new false-name-proof double auction protocol called the Generalized Threshold Price Double auction (GTPD) protocol. False-name-proofness generalizes strategy-proofness by incorporating the possibility of false-name bids, e.g., bids submitted using multiple e-mail addresses. An existing protocol called TPD protocol is false-name-proof but can handle only the cases where marginal utilities of each agent always decrease, while our new GTPD protocol can handle arbitrary evaluation values. When marginal utilities can increase, some bids cannot be divided into a single unit (e.g., an all-or-nothing bid). Due to the existence of such indivisible bids, meeting supply/demand becomes difficult. Furthermore, a seller/buyer can submit a false-name-bid by pretending to be a potential buyer/seller to manipulate allocations and payments.
In the GTPD protocol, the auctioneer is required to absorb the supply-demand imbalance up to a given upper-bound. Also, the GTPD incorporate a new false-name-proof one-sided auction protocol that is guaranteed to sell/buy a certain number of units. Simulation results show that when the threshold price is set appropriately, this protocol can obtain a good social surplus, and the number of absorbed units is much smaller than the given upper-bound.

View full abstract

Problems in the world are classified into two classes; the ones that people can foresee to the details beforehand (to such an extent that making computer programs for solving them is possible) and the others. Because of very the nature of current information technology computers are used only for the former class problems but the latter problems are left to human without exception. Only possible method to deal with unforeseen problems in the latter class is to take trial-and- error approach. When an unforeseen problem comes arise, human tries to make a hypothetical solution and evaluate it. When the scale of problem gets large however, it becomes a very difficult task even for human. Recently, the number and the scale of the latter class problems are increasing. It is anticipated that in near future they go beyond human capability. Then no one can deal with these problems satisfactory and it is expected that many troubles happen to be. An innovative technology is required to deal with these problems in order to avoid the troubles. It is expected that computers have potential to contribute in this process by undertaking evaluation of the hypothesis. Since however the problems are not foreseen beforehand, it is not possible to provide any specific method suited for evaluating the hypothesis in advance. Automatic and dynamic generation of evaluation system in real time is necessary as an innovative information technology. The major objective of this paper is therefore to discuss such an information technology that can aid human trial-and-error approach. An analysis of the process of solving unforeseen problems and a method of generating automatically information systems for evaluation are discussed.

View full abstract

In a dynamic environment, even if an agent makes a plan to obtain a goal, the environment might change while the agent is executing the plan. In that case, the plan, which was initially valid when it was made, might later become invalid. Furthermore, in the process of replanning, it is necessary to take into account the side effects of actions already executed. Nowadays, HTN planning is becoming popular among agent researchers because its task decomposition algorithm is efficient and suitable for joint planning in multi-agent systems. However, the dynamic replanning algorithm of HTN planning has not yet been established. In order to solve this problem, we have previously presented an agent life cycle that integrates HTN planning, action execution, knowledge updates, and plan modification. In that agent life cycle, the plans are always kept valid according to the most recent knowledge and situation. However, when one plan does not work and the agent starts using an alternative plan, it sometimes undoes some actions that do not have to be undone. The source of this problem lies in the need to have total-order plans. In other words, all the actions have to be sequentially ordered even when the order is not important in some parts of the plan. This paper extends our previous agent life cycle and presents new ways to avoid unnecessary action cancellation. This enables the agent to handle partial-order plans and to make use of two kinds of undoing actions: sequential undoing actions and concurrent undoing actions.

View full abstract

Reinforcement Learning (RL) is a promising technique for creating agents that can be applied to real world problems. The most important features of RL are trial-and-error search and delayed reward. Thus, agents randomly act in the early learning stage. However, such random actions are impractical for real world problems.

This paper presents a novel model of RL agents. A feature of our learning agent model is to integrate the Analytic Hierarchy Process (AHP) into the standard RL agent model, which consists of three modules: state recognition, learning, and action selecting modules. In our model, the AHP module is designed with {\\it primary knowledge} that humans intrinsically have in a process until a goal state is attained. This integration aims at increasing promising actions instead of completely random actions in the standard RL algorithms.

Profit Sharing (PS) is adopted as a RL method for our model, since PS is known to be useful even in multi-agent environments. To evaluate our approach in a multi-agent environment, we test a PS RL method with our agent model on a pursuit problem in a grid world. Computational results show that our approach outperforms the standard PS in terms of learning speed in the earlier stages of learning. We also show that the learning performance of our approach is superior at least competitive to that of the standard one in the final stages of learning.

View full abstract

In this paper, we describe a collaborative information retrieval method among personal repositorie and an implementation of the method on a personal agent framework. We propose a framework for personal agents that aims to enable the sharing and exchange of information resources that are distributed unevenly among individuals. The kernel of a personal agent framework is an RDF(resource description framework)-based information repository for storing, retrieving and manipulating privately collected information, such as documents the user read and/or wrote, email he/she exchanged, web pages he/she browsed, etc. The repository also collects annotations to information resources that describe relationships among information resources and records of interaction between the user and information resources. Since the information resources in a personal repository and their structure are personalized, information retrieval from other users' is an important application of the personal agent. A vector space model with a personalized concept-base is employed as an information retrieval mechanism in a personal repository. Since a personalized concept-base is constructed from information resources in a personal repository, it reflects its user's knowledge and interests. On the other hand, it leads to another problem while querying other users' personal repositories; that is, simply transferring query requests does not provide desirable results. To solve this problem, we propose a query equalization scheme based on a relevance feedback method for collaborative information retrieval between personalized concept-bases. In this paper, we describe an implementation of the collaborative information retrieval method and its user interface on the personal agent framework.

View full abstract

Recently, agent technologies have attracted a lot of interest as an emerging programming paradigm. With such agent technologies, services are provided through collaboration among agents. At the same time, the spread of mobile technologies and communication infrastructures has made it possible to access the network anytime and from anywhere. Using agents and mobile technologies to realize ubiquitous computing systems, we propose a new framework based on KODAMA and VPC. KODAMA provides distributed management mechanisms by using the concept of community and communication infrastructure to deliver messages among agents without agents being aware of the physical network. VPC provides a method of defining peer-to-peer services based on agent communication with policy packages.

By merging the characteristics of both KODAMA and VPC functions, we propose a new framework for ubiquitous computing environments. It provides distributed management functions according to the concept of agent communities, agent communications which are abstracted from the physical environment, and agent collaboration with policy packages.

Using our new framework, we conducted a large-scale experiment in shopping malls in Nagoya, which sent advertisement e-mails to users' cellular phones according to user location and attributes. The empirical results showed that our new framework worked effectively for sales in shopping malls.

View full abstract

A methodology which enables a flexible and reusable development of mobile agent application to a mobility aware indoor environment is provided in this study. The methodology is named Workflow-awareness model based on a concept of a pair of mobile agents cooperating to perform a given task. A monolithic mobile agent application with numerous concerns in a mobility aware setting is divided into a master agent (MA) and a shadow agent (SA) according to a type of tasks. The MA executes a main application logic which includes monitoring a user's physical movement and coordinating various services. The SA performs additional tasks depending on environments to aid the MA in achieving efficient execution without losing application logic. "Workflow-awareness (WFA)" means that the SA knows the MA's execution state transition so that the SA can provide a proper task at a proper timing. A prototype implementation of the methodology is done with a practical use of AspectJ. AspectJ is used to automate WFA by weaving communication modules to both MA and SA. Usefulness of this methodology concerning its efficiency and software engineering aspects are analyzed. As for the effectiveness, the overhead of WFA is relatively small to the whole expenditure time. And from the view of the software engineering, WFA is possible to provide a mechanism to deploy one application in various situations.

View full abstract

In the vision of the semantic web, agents are defined as the programs that collect information from diverse sources, process the information, and exchange the results with other programs. In order to extend application areas of the agents from the Internet to the real world, we propose CONSORTS, a multiagent architecture for content managements in ubiquitous computing. In this paper, we, first, describe two important concepts in order to realize the agent-based content managements in ubiquitous computing, ``physically grounding'' and ``cognitive resources managements.'' Second, we describe the outline of the CONSORTS (ver. 1) and its RDF-based spatial information representation. Finally, we show an application of the CONSORTS, context-aware information assist services in a museum.

View full abstract

This paper develops a new ascending-price multi-unit auction protocol that has following characteristics: (i) it has an open format, (ii) sincere bidding is an equilibrium strategy even if the marginal utilities of each agent can increase and agents can submit false-name bids. False-name bids are bids submitted under fictitious names such as multiple e-mail addresses, which can be done easily in the Internet. This is the first protocol that has these two characteristics. We show that our new protocol outperforms an existing protocol, which satisfies (ii), with respect to the social surplus and the seller's revenue.

View full abstract

According to the widespread use of Web Services, composition of multiple services is becoming a main issue. BPEL was proposed as a Web Services workflow language in this context. Then, in many cases we need much interactions between a service requester and a BPEL process execution engine. However for the end user under the ubiquitous environment where the service is executed with a cellular phone or PDA, the interaction means simply a burden. The reason is: (1)it is hard to carry out the operation manually, (2)a network connection is unstable, (3)a connection charge is relatively high. In this paper, we propose middle agents between ubiquitous users and Web Services. The agents analyze BPEL, extract the interaction which the ubiquitous users would need, then produce action operators. It makes a plan in order to achieve a service request from the user. Here the planning agent can reduce operations of the user by collecting the necessary information at the beginning which will be inputted during the IP execution. We show the architecture of the agents and an example regarding ATM service, then evaluate the number of inputs, the performance, and the quantity of data transmitted on network. Finally, we consider adaptability of this approach to generic BPEL flow.

View full abstract

A virtual dialogue interface for soft interaction between a system and a user is proposed. It enables the information acquisition as a sub task without hindering the user main task during the main task execution. A virtual dialogue is performed with suitable timing between the two characters used in this interface: the user avatar with a model obtained by observing the interaction of the user and the environment, and an agent having the knowledge about the environment. The user is usually an indirect participant performing ``soft interaction'' as an onlooker of the system behavior (dialogue between the user avatar and the agent), and only when required he can participate through a direct interaction instead of his avatar. A car navigation interface was used to examine the applicability of the proposed technique. A task of memorizing information about one's profile while performing an operation task was given and the rate of main task execution and rate of related information memory were used as evaluation criteria. Our proposed method showed outstanding results comparing to the conventional direct interactive interface. Moreover, the degree of favorable impression for the proposed interface was higher than the conventional one and the subjectivity evaluation clearly showed that it was considered as a sociable interface.

View full abstract