Estimating the perceived difficulty and the performance in program comprehension using gaze and heartbeat

Abstract

Errors in program comprehension are surprisingly difficult to be recognized, even by the developers themselves. In order to avoid overlooking errors during understanding, it might be useful to provide developers with their own comprehension-status estimated objectively. Previous studies estimated the comprehension-status on a per-function basis by analyzing a program and heartbeat during viewing it. However, this method cannot estimate the comprehension-status for a part of a function because the number of data points for analysis become fewer. In this study, we proposed the analysis method for estimating the comprehension-status on a part of a function by calculating program-characteristics and psychological-characteristics. Program-characteristics are defined by variable coupling between a part of a function and whole function. Psychological-characteristics are defined by weighted heart metrics with the degree of attention obtained from eye gaze. Through program-comprehension experiment, it was confirmed that the proposed method can estimate the comprehension-status. Thus, the estimation of the correctness of comprehension is expected to be applicable especially for inexperienced developers.