In this paper we consider an operational software system with multi-stage degradation levels due to software aging, and derive the optimal dynamic software rejuvenation policy maximizing the steady-state system availability, via the semi-Markov decision process. Also, we develop a reinforcement learning algorithm based on Q-learning as an on-line adaptive nonparametric estimation scheme without the knowledge of transition rate to each degradation level. In numerical examples, we present how to derive the optimal software rejuvenation policy with the decision table, and investigate the asymptotic behavior of estimates of the optimal software rejuvenation policy with the reinforcement learning.

In the theory and engineering of reliability, it is one of the important issues for reliability researchers to develop effective evaluation methods of reliability performance of systems. For the case of a binary state system, using the minimal-path or minimal-cut sets of the system, an effective method is given by decomposing a structure function into series or parallel systems. For multi-state systems with partially ordered state spaces, however, sufficient examinations of the decomposition and related subjects have not been given. In this paper, following the definition of a series system of Ohi [28], we show a necessary and sufficient condition for a multi-state system to be a series system, which denotes that a system is series system if and only if the serialisation at system's and component's levels are equivalent with each other,and then presenting the series-decomposition, we show the relationship among the stochastic bounds which is given by the decomposition. Furthermore, some examinations about the pattern of maximal state vectors of a series system are given. In this paper, we omit the discussions about the parallel system, since it is ordered set theoretically dual of the series system.

A trend renewal process is characterized by a counting process and a renewal process which are mutually transformed with each other by a trend function, and plays a significant role to represent a sub-class of general repair models. In this paper we develop another nonparametric estimation method for trend renewal processes, where the form of failure rate function in the renewal process is unknown. It is regarded as a dual approach for the nonparametric monotone maximum likelihood estimator by Heggland and Lindqvist (2007) and complements their result under the assumption that the form of trend (intensity) function is unknown. We validate our nonparametric estimator through simulation experiments and apply to a field data analysis of a repairable system.

Linear consecutive-k-out-of-n: F systems are considered. It is assumed that the components are independent and the component failure times follow an exponential distribution with identical failure rate. It is also assumed that there are only two component states (working and failed) and we can know the component state at any time. If there is at least one minimal cut set consisting of one working component, the system will be preventively maintained after a certain time interval. If the system fails before reaching the preventive maintenance (PM) time, the failed components are replaced by the new ones. The optimal PM interval time which minimizes the expected cost rate is obtained. The performance of the proposed policy is evaluated by comparing the expected cost rate of the proposed policy with those of corrective maintenance (CM) and age PM policy.