The Skill Representation of a Multimodal Communication Care Method for People with Dementia

The purpose of this paper is to describe the way to represent the skills in dementia care based on the multimodal communication analysis by a computational approach. With the rapid growth of an aging population, it is becoming extremely important to elucidate communication skills for improving the quality of life of the elderly. Focusing on the dementia care method by HUMANITUDE, we have analyzed dementia care skills to evaluate their effectiveness. We designed three layers for representing skills and named them “Intra-modality,” “Inter-modality” and “Multimodal-interaction.” These results are a part of the findings of empirical analysis of building human relationships.


Introduction
Dementia affects people's thinking, behavior and ability to perform everyday tasks; its symptoms will gradually get worse.Since a person with dementia is still a unique and valuable human being, it is important to help the person to retain his/her sense of identity and feelings of self-worth [1].In recent years, it is found that well-suited caring skills improve the quality of life (QOL) of people with dementia [2,3].As for the surgery, for example, they evaluate surgical skill based on a hierarchical method using motion data in the limited situation [4].For the nursing, narrative case summaries are a core method for analysis of nursing skill [5].However, for caring skills, there remains a need for scientific approach because of the difficulty of evaluation.In this viewpoint, we took a practical approach using video data for evaluating caring skills in the fields, and we have developed a multimodal dementia corpus to create and deepen knowledge about dementia [6].In this paper, we provide a framework for representing skills and show the result of multimodal analysis.

Dementia care for improving QOL of people with dementia
Dementia is a syndrome of multiple different etiologies characterized by a global decrement in cognitive functioning, occurring in a clear sensorium [7].There are common cognitive disorders such as memory impairment, disturbance of orientation regardless of causative factors.Behavioral and psychological symptoms of dementia (BPSD), also known as neuropsychiatric symptoms, shows loitering, agitation, anxiety, irritability, depression, apathy, delusions, hallucinations, and sleep or appetite changes [8].Fig. 1 shows a rough mechanism of BPSD.The causes of BPSD are not clear but frictions of interaction between a person with dementia and his/her environments may trigger the changes in behavior.In other words, BPSD can be improved by removing frictions such as uneasiness and distastefulness through environmental coordination and non-pharmacological approach [2,3].Fig. 1.A rough mechanism of behavioral and psychological symptoms of dementia.

A multimodal communication care method: HUMANITUDE ○ R
The HUMANITUDE is a non-pharmacological intervention, which is a French-origin methodology of caring for the vulnerable elderly with cognitive impairment and focuses strongly on building human relationships [9].It aims at common cognitive disorders in dementia symptoms focusing especially on caring.The HUMANITUDE has four pillars to build human relationships: "gaze", "touch", "speech" and "stand".Table I shows the features of the three of them which are used in almost all the situations.It has also other skills: "come close", "attentional guidance" or "promote wakefulness".These skills seems to be pretty natural ways, but it is difficult to practice using them.The HUMANITUDE standardizes communication skills for the elderly with dementia, but there is a lot of implicit knowledge of care practitioners.As the first step, we discuss the skill representation to evaluate the quality of caring skills using a video analysis.

Skill Representation Using a Tool of Multimodal Behavior Analysis 3.1 A tool of multimodal behavior analysis
We have developed a tool of multimodal behavior analysis as shown in Fig. 2 to represent skills on the caring site.The tool has been constructed as a web application to be accessed easily.The tool supports 1) video annotations, 2) visualization of annotation data, 3) quantitative analysis, 4) create a query of expert's interpretation.The tool is implemented as a web-based analysis to share description data and to use rich visualization technologies.We use MongoDB which is NoSQL database system for data management.The tool provides the description environment and the collaboration with remote places.It is considered that objectification is promoted by describing human behavior on the common platform.

The framework of skill representations
We have designed three layers for the skill representation and named them "Intra-modality", "Inter-modality" and "Multimodal-interaction".Intra-modality: the smallest unit of behavior, such as "gaze", "speech", "touch", "nod" and "knock the door".
Inter-modality: the unit of relationship among intra-modalities, such as comprehensiveness, waiting for elderly people's action and consistency.Multimodal-interaction: the unit of actions developing relationship between actors, such as eye contact and verbal / nonverbal dialogue.The feature of this framework is that we separated it into current situation we can observe and expert's interpretation.It is able to be task decomposition for describers.It's a kind of shallow annotation scheme [10] which means syntactic structures are flat and simple.The skills are represented by the interpretations based on behavior descriptions (Intra-modality).We are able to describe each of the interactions with one viewpoint such as gaze, speech, touch.

The behavior primitive of intra-modality
Intra-modality is the most basic layer of skill representation.Table II shows a part of the behavior primitives which we have designed to represent dementia care skill.And we defined the other actions such as nod, knock the door, visual guidance and actions.The common terms are actor, object person, block time, and describer.These primitives represent actions of single person, and lead to represent complicated behavior and interactions.

Procedures
We evaluated the effectiveness of the framework which we described in section 3 for representing dementia care.The experiment was done by eight subjects: four experienced students (A -D) and four inexperienced students (E -H) majoring in computer science.Two video contents were used for evaluation (shown in Table III).At the first step, a trier explained the way of behavior description using the tool to each participant.The four participants described the video twice by two ways.The one way, the behavior description trial, consists of intra-modality such as gaze, speech and touch using behavior primitive.The other way, the interpretation trial, consists of inter-modality where the synchronism of skills of inter-modality is, and interaction where the eye contact between care practitioner and the elderly is.The order of video description is assigned to the participants at random.The time for description was 20 minutes for one scene with the purpose of comparing the amount of described data per time.The participants were not allowed to consult with others about the test when they conducted it.The participant A, B, and F in interpretation trial spent time of 679 sec., 520 sec.and 561 sec., but the other participants in this trial and all the participants in behavior description trial were not able to complete description in 20 minutes.The result shows that describing the interpretations is difficult despite their experiences.In the interpretation trial, there is variability of the concordance rate regardless of the viewpoints (Table IV and V).Fig. 3 shows the average of the concordance rate in interpretation trial.The average rate of all the interpretation trials is 33 %.It is thought to be due to cognitive overload.On the other hand, the result of behavior description trial is better than that of interpretation trial regardless of modality or participant (Table VI, VII and VIII).Fig. 4 shows the average of the concordance rate in behavior description trial.The average rate of all the behavior description trials is 89%.The result shows statistically-significant difference (Wilcoxon signed rank test between the interpretation trial and the behavior description trial, t (7) = -2.52,p < 0.05).These findings show that the framework we've proposed is useful to represent dementia care skill.

The experimental analysis of dementia care skills
We need to describe our preliminary findings about care practitioners' behavior.Before-and-after study was employed in a geriatric hospital.Participants (nurses and care workers) received 3-hour training by HUMANITUDE caring skills.Students shot the oral care scenes and annotated the video data of intra-modality such as "gaze", "speech" and "touch".Experts made interpretations of Intra-modality, and analyzed the video data cooperatively.Fig. 5 shows the representation of eye contact."Gaze" of HUMANITUDE means that a care practitioner looks the elderly's eye in front of them at the 20cm distance.Thus, eye contact represents the relationships of the interpretation of Intra-modality between a nurse and an elderly person.Further, quantitative analysis shows that a participant who learned the method improved the rate of eye contact 19.5 % larger than before learning.The analysis shows that the caring skills are represented as a basic HUMANITUDE technique.

Conclusion
The result shows that the framework enables us to evaluate the skills acquisition.Our framework leads to appropriate skill representation and contributes evidence based approach on the caring site.Thus, it suggests that our framework objectifies subjective interpretations.Our long term goal is to evaluate the effectiveness of multimodal communication method as a scientific approach.Since the situation of this study is very limited, we will analyze continuously in various situations.

Fig. 3 .
Fig. 3.The average of the concordance rate in the interpretation trial.

Fig. 4 .
Fig. 4. The average of the concordance rate in the behavior description trial.

Table I .
The skill of the HUMANITUDE.
e.g. a care practitioner speaks frequent positive words by gentle voice.touch where to touch, how to touch, stroke e.g. a care practitioner touches the elderly's body softly and widely.

Table II .
The behavior primitive of intra-modality focused on gaze, speech, touch.

Table III .
Test materialsTable IV and V show the result of the interpretation trial, and Table VI, VII and VIII show the result of the behavior description trial.They are the concordance rates between prepared correct data and described correct data of each participant within the time limit of 20 minutes.The hyphens of participant G in Table VIII represent non-quantifiable data.

Table IV .
The concordance rate of correct data in the interpretation trial: the synchronism of the skills (inter-modality).

Table V .
The concordance rate of correct data in the interpretation trial: eye contact (interaction).

Table VI .
The concordance rate of correct data in the behavior description trial: gaze.

Table VII .
The concordance rate of correct data in the behavior description trial: speech.

Table VIII .
The concordance rate of correct data in the behavior description trial: touch.