Appropriately designing sampling policies is highly important for obtaining better control policies in reinforcement learning. In this paper, we first show that the least-squares policy iteration (LSPI) framework allows us to employ statistical active learning methods for linear regression. Then we propose a design method of good sampling policies for efficient exploration, which is particularly useful when the sampling cost of immediate rewards is high. The effectiveness of the proposed method, which we call active policy iteration (API), is demonstrated through simulations with a batting robot.

Visual inspection is one of the most important processes in precision instrument factories for screening out products of poor quality. Usually this is carried out by human experts who went through long-term training sessions, so there is a strong demand for automating this process in order to reduce production costs. In this paper, we apply a recently-developed outlier detection method called least-squares outlier detection (LSOD) to this task and demonstrate that inferior products can be successfully detected. LSOD can utilize knowledge of inliers for enhancing outlier detection performance, so it suits well to visual inspection in industries. Furthermore, LSOD is equipped with automatic model selection mechanism and, hence, users do not have to grapple with parameter tuning.

While the existence of different phases in financial markets such as bull/bear or bubble/non-bubble has been widely recognized among investors, few stylized facts about the different phases has been revealed so far due to diffculty of analyzing tail behavior. In particular, the sudden phase shift from bull to bear markets, or financial bubble burst, damages economies very severely, and it is urgent social requirement to provide early warning system for helping policy makers to alleviate the damage. The aim of this study is to classify financial markets into several phases, and warn the change of the phase at appropriate timing. Phase extraction and the statistical analysis is based on moving densities estimated by adaptive kernel density estimator, that exhibits a good performance for fitting fat-tailed and multi-modal densities which are typical to phenomena involving phase transitions. Based on the analysis of New York Stock Exchange index daily returns from 1966 to 2009, the shape of the estimated return density is classified into four phases by using the detected cyclical pattern of risk-return relationship. Moreover, most of the bubble burst or sudden change in the price trend could be warned several months prior to the price peak point.

The ratio of two probability density functions is becoming a quantity of interest these days in the machine learning and data mining communities since it can be used for various data processing tasks such as non-stationarity adaptation, outlier detection, and feature selection. Recently, several methods have been developed for directly estimating the density ratio without going through density estimation and were shown to work well in various practical problems. However, these methods still perform rather poorly when the dimensionality of the data domain is high. In this paper, we propose to incorporate a dimensionality reduction scheme into a density-ratio estimation procedure and experimentally show that the estimation accuracy in high-dimensional cases can be improved.

The goal of sufficient dimension reduction in supervised learning is to find the low-dimensional subspace of input features that is 'sufficient' for predicting output values. In this paper, we propose a novel sufficient dimension reduction method using a squared-loss variant of mutual information as a dependency measure. We derive an analytic approximator of squared-loss mutual information based on density ratio estimation, which is shown to possess suitable convergence properties. We then develop a natural gradient algorithm for sufficient subspace search. Numerical experiments show that the proposed method compares favorably with existing dimension reduction approaches.

Structural equation models and Bayesian networks have been widely used to analyze causal relations between continuous variables. In such frameworks, linear acyclic models are typically used to model the datagenerating process of variables. Recently, it was shown that use of non-Gaussianity identifies a causal ordering of variables in a linear acyclic model without using any prior knowledge on the network structure, which is not the case with conventional methods. However, existing estimation methods are based on iterative search algorithms and may not converge to a correct solution in a finite number of steps. In this paper, we propose a new direct method to estimate a causal ordering based on non-Gaussianity. In contrast to the previous methods, our algorithm requires no algorithmic parameters and is guaranteed to converge to the right solution within a small fixed number of steps if the data strictly follows the model.

Early detection of network troubles such as link down is important in the management of communication networks. However, it is difficult to correctly locate the failed network device since the networks are time-evolving, redundant, and not directly-observable. In this paper we focus on a detection algorithm that allows for detecting the change in a given network structure (topology) and for locating the failed link. After sending a sufficient number of test packets (probes) among multiple pairs of end nodes, we evaluate the likelihood of failure on each of the existing links. Then we use maximum likelihood estimation to detect failure and to infer the most suspicious link. Simulation experiments demonstrate that our method can identify the location of a failed link with good accuracy.

We will establish the VC dimension of the class of translated ellipsoids in d-dimensional Euclidean space is between (d/2+1)2 and (6.53486 ・ ・ ・ )(d2+3d). By this, (d/2+1)2 turns out to be a lower bound of the VC dimension of the concept class that N-component Gaussian mixture models induce through maximum likelihood estimate.

This paper addresses an algorithm for probabilistic inference for Bayesian Networks (BNs). Recently, we proposed the algorithm using Zero-suppressed BDDs for compiling BNs using "separation variables" which provides compact ZBDDs. In this paper, we provide the method for constructing ZBDDs only for related parts of networks. We show some experimental results to compare our new method with the previous one. The experimental results suggest that the new method is superior to the previous method when BNs have not many ancestors for each node, and when the number of instantiations are small.

In this paper, we focus on a single graph whose vertices contain a set of quantitative attributes. Several networks can be naturally represented in this complex graph. An example is a social network whose vertex corresponds to a person with some quantitative items such as age, salary and so on. Although it can easily be expected that this kind of data will increase rapidly, most of current graph mining algorithms do not handle these complex graphs directly. Motivated by the above background, by effectively combining techniques of graph mining and quantitative itemset mining, we developed an algorithm named FAG-gSpan for nding frequent patterns from a graph with quantitative itemsets.

Text classification tasks such as sentiment and subjectivity prediction require a dependency structure beyond a bag of words. The BACT method by Kudo and Matsumoto (2004) extracts frequent subtrees from the dependency trees by a tree mining method, and uses them as features of AdaBoost. It cannot deal with predicate-argument structures (PASs) produced by recent deep parsers, because PAS is represented not as a tree, but as an attributed graph with totally ordered vertices (linear graphs). We propose an efficient frequent mining method for linear graphs based on the reverse search paradigm. In comparison to dependency trees, our graph features were better in accuracy and interpretability,while the efficiency stays competitive.