In everyday communication, we exchange messages not only through speech but also by using multiple modalities such as gesture, gaze, facial expressions, and posture. In my lab, we investigate the nature of such multimodal communication and the mechanisms underlying its production and comprehension. In particular, we focus on the relationship between gesture and language, examining the processes involved in their production and understanding, as well as how these processes change over development. At present, our work centres on the following three main themes.
Integrated comprehension of speech and gesture
Children acquire language, sociality, and communicative competence within richly multimodal environments. Adults around them naturally accompany their speech with facial expressions, gestures, and gaze cues when interacting with children. When, and in what ways, do children begin to understand these multimodal cues provided by adults? To address this question, we experimentally investigate the processing mechanisms and developmental trajectory of children’s integrated comprehension of gesture and speech, using behavioural measures and physiological indicators such as EEG.
Cultural influences on the developmental pathways of gesture and language
Children develop communicative skills through interactions with adults and peers. In this process, they acquire not only what to express (content) but also how to express it (modality and style). Such learning processes are not uniform; rather, the nature of adult input, the patterns of interaction, and the communicative value assigned to specific modalities differ across cultures. Focusing on Japanese children, we observe how bodily behaviours used in communication—such as nodding, negation gestures, posture, and iconic gestures—are learned and acquired in various contexts. Ultimately, we aim to compare these observations with those from other cultural settings to elucidate developmental pathways that are characteristic of Japanese communication.
The relationship between aphasia and gesture
We also study communication in individuals with aphasia and explore multimodal approaches to language assessment and intervention. Although some individuals with aphasia experience difficulties in aspects of spoken language—such as speaking or listening—their communicative and cognitive abilities often remain intact. In such cases, gesture can serve as an important and relatively accessible medium for conveying information. In collaboration with speech–language therapists, we examine the relationship between aphasic symptoms and gesture patterns, and work towards developing multimodal assessment and therapeutic techniques.
In addition to these main themes, we also conduct applied research on the role of bodily movement in domains such as sport and musical performance.
Click to view the Scopus page. The data was downloaded from Scopus API in December 02, 2025, via http://api.elsevier.com and http://www.scopus.com .