抄録
Conventional human activity recognition (HAR) primarily maps wearable sensor time series to predefined action labels. While effective for well-defined actions, this paradigm is limited for open-ended activities in which neither a single correct sequence nor a unique outcome can be specified, and in which the manner of engagement has intrinsic value. Responding to the NHIC theme ``From Labels to Text,'' we propose a process-oriented HAR framework that represents how an activity progresses and describes its characteristics in natural language without action classification or outcome evaluation. Using dominant-arm wrist-worn IMU data, we compute acceleration magnitude, segment it into short windows, and derive low-granularity activity states (movement/rest). From state transitions and their temporal structure, the pipeline extracts meta-level process features: rhythm stability, trial-and-error patterns (short pauses followed by resumption), self-defined boundaries (relatively long pauses), and adaptive switching between early and late phases. These features are mapped to concise post-hoc positive feedback (PFB) statements that are descriptive and non-judgmental; a language model is used only to paraphrase templates. To examine generality, we apply the same pipeline to two contrasting datasets: an instruction-guided salad preparation task provided by NHIC and an open-ended collaborative paper-tower construction task from our prior studies. Across both tasks, we identify common process structures, suggesting that process-oriented, text-based feedback provides an alternative alignment between sensor time series and natural language beyond label-based HAR.
https://ror.org/01bawqf59 
