The present paper describes microslips (disfluencies originally observed for instrumental actions that are corrected before they become fatal errors) in spontaneous gestures, focusing on how discourse is organized by speech and gestures in general and, in particular, how different kinds of gesture viewpoints are connected with each other to make them part of continual discourse. The results are discussed in terms of catchment, the impact of disfluencies on speech, and speech-gesture mismatch.

View full abstract

This paper explores ways in which a particular agreement with a prior speaker can be made recognizable as a strong one. From the conversation analytic perspective of multimodal interaction, this study demonstrates that the agreeing speaker builds an agreeing turn by incorporating the multiple resources currently available, such as talk, prosody and bodily conduct, into an integral whole. Simultaneously utilizing the design features reflected by these multiple resources, the speaker displays strong involvement in the assessment activity initiated by the prior speaker by way of his or her affiliation with what has just been said, and thereby makes the agreeing turn-in-progress recognizable as a strong one. It thus indicates that the strength of a particular agreement should be measured by reference to the strength of the agreeing speaker's displayed involvement in the current assessment activity that is simultaneously indexed by multiple design features of the agreeing turn.

View full abstract

As an illustration of multimodality of social interaction, this study describes how a speaker of perspective-proffering turn creates "negotiation space" right after producing a linguistic expression conveying the main point of perspective. In doing so, the speaker solicits a recipient's reaction before completing the utterance. In addition, the study investigates how the speaker incorporates the recipient's reaction into the design of the utterance after resuming the suspended turn. The findings show that, when a recipient provides a qualified response, a speaker mitigates assertiveness at utterance-ending by a combination of non-assertive modality expressions and other interactional resources. In the case that a speaker modifies her perspective without waiting for a recipient's response, the delayed progress of the utterance by the creation of negotiation space allows the speaker to confirm the recipient's agreement before completing her turn. As a result, the speaker completes the utterance with an assertive modality expression. Based on these observations, this study claims that creating negotiation space following a main point of perspective is a speaker's intra-turn practice to make a fine adjustment of the content of perspective and/or assertiveness in accordance with a recipient's reaction.

View full abstract

This study focuses on situations in which multiple groups performed activities in parallel and analyzes scenes of interruption from one group to the other. Multimodal analysis revealed that: 1. Devices for interruption, such as pointing, utilize the "periphery" of an interrupted group, 2. It is important to take into consideration the activity context of not only the interrupted group, but also of the interrupting group, 3. During an interruptive activity, not all participants of both groups are necessarily involved in this activity exclusively, but each of them can allocate his/her involvement momentarily to multiple parallel activities, for instance, continuing the previous activity, or withdrawing from the interruptive activity rapidly and restarting the original one. The theoretical significance of these findings is discussed in terms of concepts advocated by Erving Goffman.

View full abstract

In this paper, by focusing on a rock climbers' discussion of a "fall" accident caused by one of the parties, I will examine the process in which the spatial relations between the landmarks on the routes are connected and constructed in situ into a shared mental map through multimodal resources such as language, gaze, gesture, and posture. It was found that coordination of utterance and body is both intricately achieved and carefully differentiated in terms of the intrinsic/extrinsic perspectives mediated with character/observer viewpoints based on different experiential bases. A major factor for facilitating the process is argued to be "intersubjectivity," emerging from different viewpoints taken in the narrated scene. We then confirm that mutual understanding is made possible by distributed cognition via the process of "trading places" (Duranti, 2010) among the participants.

View full abstract

This paper describes the IMADE (Interaction Measurement, Analysis, and Design Environment) project to provide an environment for recording and analyzing human conversational interactions. The IMADE room is designed to record audio/visual, human-motion, and eye gazing data to build an interaction corpus, which can be used to provide understanding of the nonverbal behavior of humans. In this paper, we describe the notion of interaction corpus and introduce iCorpusStudio, a software environment for browsing and analyzing the interaction corpus. We present the utility value of iCorpusStudio through an analysis of the gesture of 'pointing' in examples focusing on hearers' visual attention. The paper also attempts to show how nonverbal behavior can be used to quantify the activeness of individual participants in consensus-building type conversation.

View full abstract

In this paper, we investigate gaze behavior in Japanese multi-party conversation and elucidate its relation to turn-taking. In particular, we focus on those cases where next speaker's self-selection is employed and aim at answering two research questions: i) Will the participant gazed at by the current speaker be the next speaker, and ii) if there are any exceptions, how do they occur? Based on quantitative and qualitative analyses of casual conversations produced by 12 different triads, we show i) that, even when an adjacency pair is not affiliated, there is observable regularity that the gazed participant is most likely to select herself as next speaker, and ii) that self-selection by the non-gazed participant is only possible a) when the gazed participant declines a next turn or fails to produce an expected response or a sequencially-relevant turn; or b) when the non-gazed participant belongs to the same category as the current speaker by sharing background knowledge about the topic or aligning a stance, like agreement/disagreement, with the current speaker. These results indicate participants' orientation toward the preference of gazed participants as next speaker.

View full abstract

This article focuses on two interactional episodes between two-year-old children and their caregivers in which the emergence of some trouble with understanding each other is publicly recognized and dealt with. More specifically, I analyze in detail how the children organize the action of repairing their own utterances when the co-participant's response to their utterance shows inappropriate understanding. The analysis shows that young children, who are at an early stage of language development, are capable of organizing their actions in multi-modal ways to deal with trouble in understanding which potentially endangers the participants' intersubjectivity. I conclude with a brief discussion of the implications that empirical studies of young children's actions operating on intersubjectivity, such as this study, can have for studies of the development of understanding others, and, in particular, 'theory of mind.'

View full abstract

In this paper we discuss some of the issues related to the continuity of modality in a sign language conversation, such as holding the shape of the hand or the positioning of hands during the interlocutor's turn. It is difficult to determine if these physical expressions function as linguistic or as non-linguistic elements in a sign language conversation. In our first analysis, we explored how turn-taking mechanisms and sequence organization occur when conveying an expression with the hands. In our second analysis, we found that sometimes held expressions did not work effectively in interactional sequences when the speaker did not get the interlocutor's attention. In our third analysis, we encountered a case in which the speaker changed the direction of his hands to add more information. We used the results of these three multimodal analyses to determine how specific aspects of sign language expressions are used to organize conversational structure in an interaction.

View full abstract

Attention to the nonverbal behaviors of students in a classroom is important for gaining a full understanding of classroom interactions. Shifts in participant's gaze are good cues for clarifying their interactions as listeners. We examined the shifts of teachers and students gaze to identify how students listened to speeches during Japanese language lessons. Classroom sessions in two first, third, and fifth grade classes of a public elementary school in Japan were videotaped. We extracted successive three-minute sequences from the videos and coded the gaze direction during each second. Results of examining the interactional relationship of gaze shifts between teachers and students indicated at least two types of sequential patterns: students that looked at the teacher shifted their gaze toward the objects that the teacher looked, and students spontaneously looked at other students that were speaking, who were not previously gazed at by the teacher. These results indicate that the teacher's gaze shifts are cues for students to participate in the whole classroom lesson. Implications of these findings for classroom education and limitations in interpreting these results are discussed.

View full abstract

Using conversation analytic approaches, this study aims to demonstrate how turn-taking system in sign language conversations is achieved and organized by arranging participants' gazes. Previous sign language studies have evinced the multi-modal nature of sign language structure. However, approaches from interactive points of view have started and published research results just in the last decade. Following on these recent approaches, I analyzed participants' gaze shifts at turn-taking and discuss the orderliness of gaze shifts in sign language conversations. More specifically, I obtained the following two findings. First, the return of a speaker's gaze after to a hearer before pre-completion point will project a TRP. Second, if a simultaneous talk occurs, participants' gaze is used as an overlap resolution device. I hope that these findings will contribute to discussions about sign language interactions, and more generally various interactive behaviors in other kinds of human interactions.

View full abstract

Researchers have identified several functions of the pointing gesture, but they have not yet fully explored the phenomenon in which Japanese people point at each other during a conversation to refer to an absent object. In this study, conversations between 22 pairs of friends were observed and classified to investigate pointing behavior. The results revealed that a speaker often points at a partner to refer to an absent object, person, or event that the partner is familiar with. This phenomenon appears to be related to a feature of the Japanese language: the fact that it is possible to omit a subject and/or object.

View full abstract

We investigated the influence of conversational behavior on the estimation of relationship-building skill. We used a multi-channel approach focusing on speech content and hand gestures. First, in experimental conversations, each participant's relationship-building skill was rated by their partner. Next, viewing recordings of these conversations, independent raters rated participants' relationship-building skills. Then we examined the relationship between relationship-building skill rating and speech content (i.e. attempted answers) and hand gestures (i.e. illustrators). In relation to speech content, the results indicated that positive reactions, attempted answers, and questions had a positive effect on relationship-building skill as rated by the independent viewers. As for hand gestures, illustrators and object manipulators were seen to have a positive influence on partner ratings, while illustrators were seen to have a positive influence on independent ratings. Moreover, focusing on illustrators related to speech content (Ekman & Friesen, 1969), interaction between illustrators and questions seemed to have a negative influence on relationship-building skill in partner ratings. Results showed the importance of the multi-channel approach focusing on both nonverbal and verbal behaviors to estimate relationship-building skill.

View full abstract

The present study investigated the processes in which counselors elicit desirable changes from their clients in counseling sessions. A previous study (Nagaoka & Komori, 2009) recorded a counseling session by an expert counselor, which represents role playing, and analyzed client-counselor synchrony of body movement, beginning style of counselor's turn (e.g., utterances starting with back-channel expressions), and client's long pauses, to refer each other according to time series. The results indicated that a temporal change of body movement synchrony corresponded to the time series change of beginning style of counselor's turn, or to the occurrence of client's long pauses. On the basis of the results from the previous study, the present study focused on the points just before the changes in nonverbal behaviors, and analyzed the clinical psychological interpretation of the client's and the counselor's utterances and the introspective reports by the client and the counselor. The results indicated that changes in mental processes of the client and the counselor occurred coincident with changes in nonverbal behavior. The counselor's skills that elicit changes of their client's mental processes were discussed.

View full abstract

The turn-initial position is a crucial location for three interactional operations within conversations: (1) turn-taking organization, (2) sequence organization, and (3) topic-management and activity organization. This paper examines the function of elements that indicate this position, that is, turn-entry devices (Sacks, Schegloff, & Jefferson, 1974), focusing on the discourse particle "nanka." The turn-taking organizational function of "nanka," which is assumed to have minimal influence on sequence organization (2) and topic-management and activity organization (3), is explored by examining which speaker drops out when two or more speakers start talking at same time. The results of the study show that in the event of an overlap, the speaker using "nanka" drops out. Since "nanka" rarely is rarely used for the function of sequence organization or topic-management/activity organization, speakers can use it as a turn-entry device which can indicate a "pro-tem speakership" (Jefferson, 1983) to the listener at the turn-initial position.

View full abstract

This study aims to clarify what teachers of Japanese as a second language consider a "good essay" for the purpose of improving writing education for students of Japanese. Our study involves analysis of both the teachers' rating processes and the rating criteria. We conducted research on ranking. Two types of six essays were ranked separately by 10 experienced Japanese teachers using holistic schemes. Raters' think-aloud protocols were recorded and analyzed along with a questionnaire. The results indicated that "fulfillment of task," "clarity of claim," "originality of content," "objective support from broader perspectives," "text organization," "discourse development" and "expressiveness of Japanese" were the decisive factors. However, there appeared to be no common understanding on the priority of each factor among raters. For essays with a good balance of each factor, the assessment results were agreed upon. Some essays, however, despite having failed to fulfill the task, or lack of such balance, were ranked high in holistic schemes because "academism" (academic atmosphere) was perceived in the supporting ideas or throughout the essay, as well as a high level of expressiveness. The study confirmed the necessity of a more common understanding, including scoring rubrics and scoring procedures, for writing assessment among Japanese language educators.

View full abstract