A modeling approach of human actions is focused, which designs human action model based on the obtained stored data during long-term monitoring of a person. This approach consists of the following two processes. At first, several kinds of frequent partial time series data are extracted from the stored data and regarded as human action patterns. Next, the extracted time series data are modeled based on a statistical modeling method such as Hidden Markov Model. In this research, it is focused on the extraction method of the frequent time series data in the stored data. A person changes his action according to the change of the situation around him. And, it takes some time for him to perform his action after he recognizes the situation around himself. This time is called delay time in this paper. A human action model considering this delay time leads to greater accuracy in recognition and prediction of human action based on the one. It is necessary to extract time series data of situation and action containing the delay time as learning data in order to generate the above human action model. In conventional methods, multi-dimensional time series data are used as the stored data without distinction between situation and action data. And, some frequent partial time series data are extracted from the stored data. Therefore, the delay time is not considered. In this paper, we propose a new extraction method of a frequent time series data considering the delay time. In this method, the frequent partial time series data with the delay time are extracted by evaluating the repeatability and relativity between the partial time series data with different occurrence time. In the experiment, extraction of a frequent interaction motion between two examinees is executed. The usefulness of the propose method is examined through some experimental results.