Cooking Behavior Recognition Using Egocentric Vision for Cooking Navigation

Abstract

This paper presents a cooking behavior recognition method for achievement of a cooking navigation system. A cooking navigation system is a system that recognizes the progress of a user in cooking, and accordingly presents an appropriate recipe, thus supporting the activity. In other words, an appropriate recognition of cooking behaviors is required. Among the various cooking behavior recognition methods, such as the use of context with the object being focused on and use of information in the line of sight, we have so far attempted cooking behavior recognition using a method that focuses on the motion of arms. Using the cooking behavior rate obtained from the motion of arms and cooking utensils, this study achieves recognition of the cooking behavior. The average recognition rate was 63% when calculated by the conventional method of focusing on arm motions. It has been improved by approximately 20% by adding the proposed cooking utensil information and optimizing the parameters. An average recognition rate of 84% was achieved with respect to the five types of basic behaviors of “cut,” “peel,” “stir,” “add,” and “beat,” indicating the effectiveness of the proposed method.