In this paper we consider an optimal control problem for partially observable Markov decision processes with finite states, signals and actions over an infinite horizon. It is shown that there are <isins>-optimal piecewise-linear value functions and piecewise-constant policies which are simple. Simple means that there are only finitely many pieces, each of which is defined on a convex polyhedral set. An algorithm based on the method of successive approximation is developed to compute <isins>-optimal policy and <isins>-optimal cost. Furthermore, a special class of stationary policies, called finitely transient, will be considered. It will be shown that such policies have attractive properties which enable us to convert a partially observable Markov decision chain into a usual finite state Markov one.

View full abstract

This paper is a continuation of our papers [6] and [7] and is concerned with a continuous time Markov game in which the state space is countable and the action spaces of player I and player II are compact metric spaces. In the game, the players continuously observe the state of the system and then choose actions. As a result, the reward is paid to player I from player II and the system moves to a new state by the known transition rates. Then we consider the optimization problem to maximize the total expected discounted gain for player I and, at the same time, to minimize the total expected loss for player II as the game proceeds to the infinite future. We show that such a two-person zero-sum game is strictly determined and both players have optimal stationary strategies.

View full abstract

Typical queueing problems of general arrival and general service systems are solved by an approximation method. That is, approximate formulae are given for the mean customer number in the G/G/1 (∞) system, the mean customer number in the the G/G/1 (m) system and the mean queue length in the G/G/s(∞) system_ The routine of the proposed method is as follows. The sequences of arrival and departure of customers are approximately replaced by normal stochastic processes. That is, the sequences are characterized by mean and variance only. Next, the diffusion equation is constructed as to the probability distribution of the customer number in the system. Using the solution of the equation, the aimed statistical value for the queue is calculated. Finally, the calculated value may be revised to agree with the exact solution on the area where it is known.

View full abstract

This paper studies, as a continuation of previous work [8] , an optimal stopping problem with-out recall in which the decision-maker observes a sequence of iid bivariate random variables which appear sequentially one by one in a Poisson manner. The problem can be interpreted as deciding to buy a house which has the two-dimensional worth, for example, the values for husband and for his wife. The concept of equilibrium neutral functions is introduced, and by using it an explicit solution of the problem is derived by means of finding a unique solution of some simultaneous differential equations. Some examples are included to illustrate the computations required by an "equilibrium neutral strategy".

View full abstract

This paper treats the multiple-choice (continuous) knapsack problem P: maximize Σ^^n__<i=1> Σ^^<m_i__<j=1>c_<ij>x_<ij>subject to (1) Σ^^n__<i=1> Σ^^<m_i__<j=1>c_<ij>x_<ij>&les; b, (2) 0&les; x_<ij>&les; 1,i = 1,2 ,...,n,j=1,2,.... m_i and (3) at most one of x _<i1> , x _<i2>,..., x _<im_i> is positive for i = 1, 2,..., n, where n, m_i are positive integers ancl a_<i1>, c_<i1>, b are nonnegative real numbers. Two approximate algorithms and an exact branch-and-bound algorithm are proposed, by making use of the property that the LP relaxation of P provides considerably accurate upper and lower bounds of the optimal value of P. Although the multiple-choice knapsack problem is known to be NP-complete, computation results are quite encouraging. For example, approximate solutions withing 0.001% of the optimal values are obtained in less than one second (on FACOM 230/60) for problems with n = 1000 and m_i = 2, which are randomly generated from the uniform distribut[on. Exact optimal solutions of these problems with n = 500 and m_i = 2 are also obtainedin less than 0.2 seconds (on FACOM M190).

View full abstract

This paper deals with the M + I - out-of-N:G system with correlated failure and single repair facility, where correlated failure means that i (1 ≦ i ≦ N) operating units in the system are possible to, fail at the same time. We consider two repair policies, and under each repair policy Laplace transform of point-wise availability and reliability, meantime to the first system failure and stationary availability are derived. Finally some properties of stationary availability for each repair policy are given.

View full abstract

A latin square is an n x n square matrix each of which cells contains a symbol chosen from the set [1, 2, . . . , n] ; each symbol occurs exactly once In each row or column of the matrix. A partial latin square is a latin square in which some cells are unoccupied. We consider the problem of obtaining necessary and sufficient conditions for a partial latin square to be completed to a latin square. For this problem A. B. Cruse has recently given a necessary condition associated with triply stochastic matrices. In this paper two sets of necessary conditions are given, one developed from network flow theory and another obtained from matroid theory. It is shown that the network condition is equivalent to Cruse's condition and that the matroid condition is strictly stronger than either of the former.

View full abstract

This paper considers a generalized chance constraint programming problem having a controllable probability level a; with which the chance constraint should be satisfied. Several properties of this problem are derived and, based on these properties, an algorithm is also proposed.

View full abstract

Some applications are discussed concerning reliability analyses of a standby redundant system with repair. The system consists of a series subsystem composed of N identical units and n spares. Failed units are assumed to be repaired by r repair facilities and repaired units join the spares. For this system, two cases which can be analyzed by means of the semi-Markov process method are considered. The first case is the system with N = 1 in which the failure time distribution of a unit is general and the repair time distribution is exponential. The second orle is the case with r = 1 where a unit has an exponential failure time and a general repair time distribution. An application of the first model to an ion source system for a particle accelerator is investigated. The second model is applied to a centrifuge uranium enrichment plant, error correcting terminals for an automatic newspaper editing system, and an autobus system. The general distribution. is assumed to be a gamma distribution in the applications. For a proton synchrotron, a proposal to use a redundant ion source system in a design study reported in 1965, when the life time of a duoplasmatron ion source was not long enough, is investigated. Concerning a cascade system of uranium enrichment centrifuges, a production plant composed of 100,000 centrifuges is proposed to 'De divided into 20 groups, each of which consists of 5,000 centrifuges, and to have n spare groups. An example of a bus company office possessing N+n+2 (= 61) buses is discussed, where N, n and 2 of them are in operation, spares and in regular maintenance, respectively. A failed bus which is substituted by a spare one is repaired in temporary maintenance. The number of spares needed to attain to a target value of the mean time to system failure (MTSF) is obtained for a given repair capacity.

View full abstract