Updated on 2022/10/01

写真a

 
COSENTINO, Sarah
 
Affiliation
Faculty of Science and Engineering, Global Center for Science and Engineering
Job title
Associate Professor(without tenure)
Profile
Sarah is a hands-on engineer and academic researcher. She started working as a freelance collaborator in an Electronics company during high-school, until earning her M.Sc. in Electronic engineering. Straight after her graduation she moved to Japan for a prestigious 1-year industrial internship program, which was extended to a full employee contract by the company for another 2 years. After 3 years in Japan, working for challenging projects in Electronics R&D, she decided it was time to change, applied and was selected for a scholarship program, and enrolled in a Ph.D. course in one of the top Japanese Universities. During her studies she collaborated with several other researchers across the globe, spending months in leading universities in U.S. and Europe. Her main interests are human physiology, human sensing, human communication, affective computing and human-machine interaction. She has hands-on experience in electronic design and assembly, and a wide researching experience in developing sensor systems for applications mostly related to human sensing, and human-robot interaction, authoring several publications on her specific work.

Research Institute

  • 2020
    -
    2022

    理工学術院総合研究所   兼任研究員

Degree

  • Waseda University   Ph.D. in Robotics

 

Research Areas

  • Intelligent robotics

  • Robotics and intelligent system

  • Mechanics and mechatronics

  • Human interface and interaction

Papers

  • Rhythmic body movement analysis for robot-based music therapy

    Y-H Ma, J-Y Lin, S. Cosentino, A. Takanishi

    2021 IEEE International Conference on Advanced Robotics and Its Social Impacts (ARSO)    2021.07  [Refereed]

    Authorship:Corresponding author

    DOI

  • The Italy-Japan Workshop: A History of Bilateral Cooperation, Pushing the Boundaries of Robotics

    Gabriele Trovato, Leonardo Ricotti, Cecilia Laschi, Massimiliano Zecca, Sarah Cosentino, Luca Bartolomeo, Shuji Hashimoto, Atsuo Takanishi, Paolo Dario

    IEEE Robotics and Automation Magazine    2021  [Refereed]

    DOI

  • Design and evaluation of a robot limb for table key playing on humanoid saxophonist robot

    Jia-Yeu LIN, Yuya NISHIO, Mao KAWAI, Kei MATSUKI, Salvatore SESSA, Sarah COSENTINO, Atsuo TAKANISHI

    2019 IEEE/SICE International Symposium on System Integration (SII)    2019.01  [Refereed]

    DOI

  • Development and Evaluation of a Quantitative Analysis System for Robot-assisted Musical Therapy

    J. Lin, Y. Ma, Y. Han, S. Cosentino, A. Takanishi

    Delphi - Interdisciplinary Review of Emerging Technologies   2 ( 1 ) 35 - 42  2019  [Refereed]  [Invited]

    DOI

  • Development of Performance System With Musical Dynamics Expression on Humanoid Saxophonist Robot.

    Jia-Yeu Lin, Mao Kawai, Yuya Nishio, Sarah Cosentino, Atsuo Takanishi

    IEEE Robotics Autom. Lett.   4 ( 2 ) 1684 - 1690  2019  [Refereed]

    DOI

  • Expressive humanoid robot for automatic accompaniment

    Guangyu Xia, Mao Kawai, Kei Matsuki, Mutian Fu, Sarah Cosentino, Gabriele Trovato, Roger Dannenberg, Salvatore Sessa, Atsuo Takanishi

    SMC 2016 - 13th Sound and Music Computing Conference, Proceedings     506 - 511  2019  [Refereed]

     View Summary

    We present a music-robotic system capable of performing an accompaniment for a musician and reacting to human performance with gestural and facial expression in real time. This work can be seen as a marriage between social robotics and computer accompaniment systems in order to create more musical, interactive, and engaging performances between humans and machines. We also conduct subjective evaluations on audiences to validate the joint effects of robot expression and automatic accompaniment. Our results show that robot embodiment and expression improve the subjective ratings on automatic accompaniment significantly. Counterintuitively, such improvement does not exist when the machine is performing a fixed sequence and the human musician simply follows the machine. As far as we know, this is the first interactive music performance between a human musician and a humanoid music robot with systematic subjective evaluation.

  • Group Emotion Recognition Strategies for Entertainment Robots

    Sarah Cosentino, Estelle I. S. Randria, Jia-Yeu Lin, Thomas Pellegrini, Salvatore Sessa, Atsuo Takanishi

    Proceedings of the 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems     813 - 818  2018.10  [Refereed]

  • A human-robot interface to improve facial expression recognition in subjects with Autism Spectrum Disorder

    Yean HAN, Yuya NISHIO, MA Yi-Hsiang, Chiaki OSHIYAMA, Jia-Yeu LIN, Atsuo TAKANISHI, Sarah COSENTINO

    2018 9th International Conference on Awareness Science and Technology (iCAST)    2018.09  [Refereed]

    DOI

  • A Synchronization Feedback System to Improve Interaction Correlation in Subjects With Autism Spectrum Disorder

    Yi-Hsiang, M. A, Han, Y, Lin, J. Y, Cosentino, S, Nishio, Y, Oshiyama, C, Takanishi, A

    International Conference on Awareness Science and Technology (iCAST)IEEE     285 - 290  2018.09  [Refereed]

  • Evaluation of a Sensor System for Detecting Humans Trapped under Rubble: A Pilot Study.

    Array,Array,Ritaro Kasai, Sarah Cosentino, Cimarelli Giacomo, Yasuaki Mochida, Hiroya Yamada, Michele Guarnieri, Atsuo Takanishi

    Sensors   18 ( 3 ) 852  2018  [Refereed]

  • Development of a low-cost smart home system using wireless environmental monitoring sensors for functionally independent elderly people.

    Di Zhang, Weisheng Kong, Ritaro Kasai, Z. Gu, Yukitoshi Minami Shiguematsu, Sarah Cosentino, Salvatore Sessa, Atsuo Takanishi

    2017 IEEE International Conference on Robotics and Biomimetics, ROBIO 2017, Macau, Macao, December 5-8, 2017     153 - 158  2017  [Refereed]

    DOI

  • Development of a Humorous Humanoid Robot Capable of Quick-and-Wide Arm Motion

    T. Kishi, S. Shimomura, H. Futaki, H. Yanagino, M. Yahara, S. Cosentino, T. Nozawa, K. Hashimoto, A. Takanishi

    IEEE Robotics and Automation Letters   1 ( 2 ) 1081 - 1088  2016.07  [Refereed]

     View Summary

    This letter describes the development of a humanoid arm with quick-and-wide motion capability for making humans laugh. Laughter is attracting research attention because it enhances health by treating or preventing mental diseases. However, laughter has not been used effectively in healthcare because the mechanism of laughter is complicated and is yet to be fully understood. The development of a robot capable of making humans laugh will clarify the mechanism how humans experience humor from stimuli. Nonverbal funny expressions have the potential to make humans laugh across cultural and linguistic differences. In particular, we focused on the exaggerated arm motion widely used in slapsticks and silent comedy films. In order to develop a humanoid robot that can perform this type of movement, the required specification was calculated from slapstick skits performed by human comedians. To meet the required specifications, new arms for the humanoid robot were developed with a novel mechanism that includes lightweight joints driven by a flexible shaft and joints with high output power driven by a twin-motor mechanism. The results of experimental evaluation show that the quick-and-wide motion performed by the developed hardware is effective at making humans laugh.

    DOI

  • Automatic Segmentation for One Leg Stance Test with Inertial Measurement Unit

    W. Kong, T. Kodama, S. Sessa, S. Cosentino, D. Magistro, R. Kawashima, A. Takanishi

    2016 IEEE/SICE INTERNATIONAL SYMPOSIUM ON SYSTEM INTEGRATION (SII)     307 - 312  2016  [Refereed]

     View Summary

    One Leg Stance (OLS), a test assessing postural stability, is popularly conducted both in clinic and community settings because it is inexpensive and time-efficient. However, the evaluation based on visual observation and manual time measurement with a stop-watch cannot provide quantitative and detailed parameters for longitudinal or cross-sectional studies. In recent years, to overcome these limitations, the use of Inertial Measurement Unit (IMU) as objective measurement analysis tools is becoming more and more popular. However, the greatest issue is that IMU data segmentation is still time-consuming and prone to errors, as the OLS segmentation is being done manually, off-line, on recorded data. In this paper we proposed a novel algorithm for the automatic segmentation of IMU data of the OLS test. The result showed that the correct rate of detection was over 90% which was close to the correct rate in manual segmentation. Compared to manual segmentation with video, besides being less time-consuming, the proposed algorithm closes the loop making the data acquisition and analysis completely automatic, thus can be integrated in self-assessment smart phone applications, allowing the continuous tracking of postural stability also outside clinics and health-care facilities.

    DOI

  • Comparison of Gait Event Detection from Shanks and Feet in Single-task and Multi-task Walking of Healthy Older Adults

    W. Kong, J. Lin, L. Waaning, S. Sessa, S. Cosentino, D. Magistro, M. Zecca, R. Kawashima, A. Takanishi

    2016 IEEE INTERNATIONAL CONFERENCE ON ROBOTICS AND BIOMIMETICS (ROBIO)     2063 - 2068  2016  [Refereed]

     View Summary

    Automatic and objective detection algorithms for gait events from MEMS Inertial Measurement Units data have been developed to overcome subjective inaccuracy in traditional visual observation. Their accuracy and sensitivity have been verified with healthy older adults, Parkinson's disease and spinal injured patients, using single-task gait exercises, where events are precise as the subject is focusing only on walking. Multi-task walking instead simulates a more realisitc and challenging scenario where subjects perform secondary cognitive task while walking, so it is a better benchmark. In this paper, we test two algorithms based on shank and foot angular velocity data in single-task, dual-task and multi-task walking. Results show that both algorithms fail when the subject slows extremely down or pauses due to high cognitive and attentional load, and, in particular, the first stride detection error rate of the foot-based algorithm increases. Stride time is accurate with both algorithms regardless of walking types, but the shank-based algorithm leads to an overestimation on the proportion of swing phase in one gait cycle. Increasing the number of cognitive tasks also causes this error with both algorithms.

    DOI

  • The Development of Intraoral Pressure Control System on Humanoid Saxophone Playing Robot

    Jia-Yeu Lin, Keisuke Yoshida, Kei Matsuki, Kazuki Takikawa, Sarah Cosentino, Salvatore Sessa, Atsuo Takanishi

    2016 IEEE INTERNATIONAL CONFERENCE ON ROBOTICS AND BIOMIMETICS (ROBIO)     1649 - 1654  2016  [Refereed]

     View Summary

    In this paper, we describe an intraoral pressure control system for the humanoid saxophone-playing robot WAS-4. We studied the model for air flow control in a wind instrument, and via a series of assessment tests to analyze the robot performance, we discovered that the control of the intraoral pressure of the robot is critical for continuous sound production. We then developed a control system to maintain the intraoral pressure stable during note sustaining and changing, preserving reed vibration and sound production. We assessed the effectiveness of the system via a final evaluation experiment.

    DOI

  • One DoF robotic hand that makes human laugh by tickling through rubbing underarm

    T. Kishi, T. Nozawa, A. Nibori, H. Futaki, Y. Miura, M. Shina, K. Matsuki, H. Yanagino, S. Cosentino, K. Hashimoto, A. Takanishi

    2016 IEEE/RSJ INTERNATIONAL CONFERENCE ON INTELLIGENT ROBOTS AND SYSTEMS (IROS 2016)     404 - 409  2016  [Refereed]

     View Summary

    This paper describes the development of one DoF robotic hand that makes human laugh by tickling through rubbing underarm. Laughter is attracting research attention because it enhances health by treating or preventing mental diseases. However, laughter has not been used effectively in healthcare because the mechanism of laughter is complicated and is yet to be fully understood. The development of a robot capable of making humans laugh is useful for clarifying the mechanism of laughter because the stimuli by the robot is quantitative and reproductive. Especially, tickling matches to this purpose because the relationship between stimuli and reaction is simpler compared to other techniques. Therefore, this research aimed to develop a robotic hand that can output quantitative and reproductive tickling stimuli for clarifying the mechanism of laughter. Rubbing underarm is selected as a target motion of robot because previous research suggested that this is the best way for making humans feel ticklish. In order to achieve the tickling motion by robots as humans, the required specifications were determined through experimental method. In order to develop a robot that achieves the required fingertip trajectory by simple mechanisms as much as possible, mechanism with crank and link driven by single motor was developed. The result of experimental evaluation shows that the developed robot could make humans laugh by its rubbing motion. In addition, the quantitative tickling motion by developed robotic hand was suggested to be effective for clarifying the mechanism of laughter.

    DOI

  • Development of subliminal persuasion system to improve the upper limb posture in laparoscopic training: a preliminary study

    Di Zhang, Salvatore Sessa, Weisheng Kong, Sarah Cosentino, Daniele Magistro, Hiroyuki Ishii, Massimiliano Zecca, Atsuo Takanishi

    INTERNATIONAL JOURNAL OF COMPUTER ASSISTED RADIOLOGY AND SURGERY   10 ( 11 ) 1863 - 1871  2015.11  [Refereed]

     View Summary

    Current training for laparoscopy focuses only on the enhancement of manual skill and does not give advice on improving trainees' posture. However, a poor posture can result in increased static muscle loading, faster fatigue, and impaired psychomotor task performance. In this paper, the authors propose a method, named subliminal persuasion, which gives the trainee real-time advice for correcting the upper limb posture during laparoscopic training like the expert but leads to a lower increment in the workload.
    A 9-axis inertial measurement unit was used to compute the upper limb posture, and a Detection Reaction Time device was developed and used to measure the workload. A monitor displayed not only images from laparoscope, but also a visual stimulus, a transparent red cross superimposed to the laparoscopic images, when the trainee had incorrect upper limb posture. One group was exposed, when their posture was not correct during training, to a short (about 33 ms) subliminal visual stimulus. The control group instead was exposed to longer (about 660 ms) supraliminal visual stimuli.
    We found that subliminal visual stimulation is a valid method to improve trainees' upper limb posture during laparoscopic training. Moreover, the additional workload required for subconscious processing of subliminal visual stimuli is less than the one required for supraliminal visual stimuli, which is processed instead at the conscious level.
    We propose subliminal persuasion as a method to give subconscious real-time stimuli to improve upper limb posture during laparoscopic training. Its effectiveness and efficiency were confirmed against supraliminal stimuli transmitted at the conscious level: Subliminal persuasion improved upper limb posture of trainees, with a smaller increase on the overall workload.

    DOI

  • Angular Sway Propagation in One Leg Stance and Quiet Stance with Inertial Measurement Units for Older Adults

    W. Kong, S. Sessa, D. Zhang, M. Zecca, S. Cosentino, H. Ishii, D. Magistro, H. Takeuchi, R. Kawashima, A. Takanishi

    2015 37TH ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY (EMBC)     6955 - 6958  2015  [Refereed]

     View Summary

    Postural stability degrades with age, threating the health and life quality of the older adults. One Leg Stance (OLS) is one of the standard and commonly adopted assessments for postural stability, and the postural sway in OLS has been demonstrated to be related with age. The propagation of postural sway between body segments could be a hint to the underlying mechanism of balance control. However, it is not yet fully understood. Therefore, the aim of this paper was to study the angular sways and their propagation of the head, trunk, and lower limb in healthy older adults. A cross-correlation of the normalized angular speeds was performed and the experiment with 68 older adults was conducted. The results showed that the head, hip and ankle joints affected the transfer of angular sway with a relatively lower correlation and longer latency.

    DOI

  • Objective Evaluation of Oral Presentation Skills Using Inertial Measurement Units

    Salvatore Sessa, Weisheng Kong, Di Zhang, Sarah Cosentino, Udara Manawadu, Motoji Kawasaki, George Thuruthel Thomas, Tomohiro Suzuki, Ryosuke Tsumura, Atsuo Takanishi

    2015 37TH ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY (EMBC)     3117 - 3120  2015  [Refereed]

     View Summary

    Oral presentation is considered as one of the most sought after skills by companies and professional organizations and program accreditation agencies. However, both learning process and evaluation of this skill are time demanding and complex tasks that need dedication and experience. Furthermore, the role of the instructor is fundamental during the presentation assessment. The instructor needs to consider several verbal and nonverbal communications cues sent in parallel and this kind of evaluation is often subjective. Even if there are oral presentation rubrics that try to standardize the evaluation, they are not an optimal solution because they do not provide the presenter a real-time feedback. In this paper, we describe a system for behavioral monitoring during presentations. We propose an ecological measurement system based on Inertial Measurement Units to evaluate objectively the presenter's posture through objective parameters. The system can be used to provide a real-time feedback to the presenters unobtrusively.

    DOI

  • Automatic discrimination of laughter using distributed sEMG

    S. Cosentino, S. Sessa, W. Kong, D. Zhang, A. Takanishi, N. Bianchi-Berthouze

    2015 INTERNATIONAL CONFERENCE ON AFFECTIVE COMPUTING AND INTELLIGENT INTERACTION (ACII)     691 - 697  2015  [Refereed]

     View Summary

    Laughter is a very interesting non-verbal human vocalization. It is classified as a semi voluntary behavior despite being a direct form of social interaction, and can be elicited by a variety of very different stimuli, both cognitive and physical. Automatic laughter detection, analysis and classification will boost progress in affective computing, leading to the development of more natural human-machine communication interfaces. Surface Electromyography (sEMG) on abdominal muscles or invasive EMG on the larynx show potential in this direction, but these kinds of EMG-based sensing systems cannot be used in ecological settings due to their size, lack of reusability and uncomfortable setup. For this reason, they cannot be easily used for natural detection and measurement of a volatile social behavior like laughter in a variety of different situations. We propose the use of miniaturized, wireless, dry-electrode sEMG sensors on the neck for the detection and analysis of laughter. Even if with this solution the activation of specific larynx muscles cannot be precisely measured, it is possible to detect different EMG patterns related to larynx function. In addition, integrating sEMG analysis on a multisensory compact system positioned on the neck would improve the overall robustness of the whole sensing system, enabling the synchronized measure of different characteristics of laughter, like vocal production, head movement or facial expression; being at the same time less intrusive, as the neck is normally more accessible than abdominal muscles. In this paper, we report laughter discrimination rate obtained with our system depending on different conditions.

    DOI

  • A Multisensory Non-Invasive System for Laughter Analysis

    Sarah Cosentino, Susanne Burger, Lara Martin, Florian Metze, Tatsuhiro Kishi, Kenji Hashimoto, Salvatore Sessa, Massimiliano Zecca, Atsuo Takanishi

    Proceedings of the 36th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC 2014)    2014.08  [Refereed]

  • Natural human-robot musical interaction: understanding the music conductor gestures by using the WB-4 inertial measurement system

    Sarah Cosentino, Klaus Petersen, Zhuohua Lin, Luca Bartolomeo, Salvatore Sessa, Massimiliano Zecca, Atsuo Takanishi

    ADVANCED ROBOTICS   28 ( 11 ) 781 - 792  2014.06  [Refereed]

     View Summary

    This paper presents an inertial measurement unit-based human gesture recognition system for a robot instrument player to understand the instructions dictated by an orchestra conductor and accordingly adapt its musical performance. It is an extension of our previous publications on natural human-robot musical interaction. With this system, the robot can understand the real-time variations in musical parameters dictated by the conductor's movements, adding expression to its performance while being synchronized with all the other human partner musicians. The enhanced interaction ability would obviously lead to an improvement of the overall live performance, but also allow the partner musicians, as well as the conductor, to better appreciate a joint musical performance, thanks to the complete naturalness of the interaction.

    DOI

  • Objective skill evaluation of endotracheal intubation using muscle contraction sensor

    A. Niibori, Y. Matsuoka, L. Bartolomeo, S. Cosentino, W. Kong, U. Imtiaz, D. Zhang, Y. Kasuya, M. Nagai, M. Ozaki, S. Sessa, H. Ishii, M. Zecca, A. Takanishi

    2014 IEEE INTERNATIONAL CONFERENCE ON ROBOTICS AND BIOMIMETICS IEEE-ROBIO 2014     1862 - 1867  2014  [Refereed]

     View Summary

    Endotracheal intubation (ETI) is a difficult technique and requires a great deal of practice to master. Research on the difference in movements between experts and novices performing the procedure has shown that experts perform movements more precisely than novices. Experts keep a fixed posture and use the upper arm muscles and wrist joints more effectively. These studies were conducted using optical motion capture systems and surface electromyography (sEMG), which are measurement systems that require a long setup time and expensive equipment. In this paper, we propose a novel method to measure the biomechanical performance of doctors during ETI using an innovative muscle contraction sensing device (MC sensor) and inertial measurement units (IMUs). We performed several experiments to measure the movements of both experts and novices performing ETI and then analysed and compared the obtained data. The results clearly showed that our system, comprising an MC sensor and IMUs, allows for an objective evaluation of ETI skills and highlighted the major differences between the movements of novices and experts.

    DOI

  • Development of a Comic Mark Based Expressive Robotic Head Adapted to Japanese Cultural Background

    T. Kishi, H. Futaki, G. Trovato, N. Endo, M. Destephe, S. Cosentino, K. Hashimoto, A. Takanishi

    2014 IEEE/RSJ INTERNATIONAL CONFERENCE ON INTELLIGENT ROBOTS AND SYSTEMS (IROS 2014)     2608 - 2613  2014  [Refereed]

     View Summary

    This paper describes the development of a robotic head that has cartoon facial expression ability with comic marks. For communicating with humans, robots should have expressive facial expression ability for indicating their inner state. Our previous research suggests that robots can express its emotion clearly if it performs facial expressions that are adapted to the cultural background of the communication partner. As a first step, we focus on making expressions for Japanese people. Comic mark is a unique and famous way of emotion expression in Japanese culture. First, we defined facial expressions by combining cartoon-like shape of the facial parts with high emotion recognition rates. Then we asked cartoonists to draw comic marks which they think are effective for emotion expression and find the effective comic marks as "Cross popping veins" for "Anger", "Tear mark" for "Sadness" and "Vertical lines" for "Fear". Finally we obtained a model expression which has sufficiently high emotion recognition rate from the combination of the facial expression and the comic marks. In order to achieve these expressions, we developed flexible full color LED display matrix module and mechanism that push and pull the sheet for expressing black lines. Results of experimental evaluation shows that the new robotic head has over 90% average emotion recognition rates for each of the six basic emotions. The results with non-Japanese subjects suggests that impression of emotion expression on robotic head changes depending on the cultural background. These findings encourage us in pursuing this concept of designing robots that display emotions that are adapted to cultural background of communication partner.

    DOI

  • Bipedal humanoid robot that makes humans laugh with use of the method of comedy and affects their psychological state actively

    T. Kishi, N. Endo, T. Nozawa, T. Otani, S. Cosentino, M. Zecca, K. Hashimoto, A. Takanishi

    2014 IEEE INTERNATIONAL CONFERENCE ON ROBOTICS AND AUTOMATION (ICRA)     1965 - 1970  2014  [Refereed]

     View Summary

    This paper describes the bipedal humanoid robot that makes human laugh with its whole body expression and affect human's psychological state. In order to realize "Social interaction" between human and robot, the robot has to affect human's psychological state actively. We focused on "laugh" because it can be thought as a typical example for researching "Social interaction". Looking through a Japanese comedy style called "manzai" or the art of conversation, we picked out several methods for making human laugh. Then we made several skits with the advice of comedians, and made the whole body humanoid robot perform them. Results of experimental evaluation with these skits shows that the robot's behavior made subjects laugh and change their psychological state seen as a decrease of "Depression" and "Anger".

    DOI

  • Development of new muscle contraction sensor to replace sEMG for using in muscles analysis fields

    D. Zhang, Y. Matsuoka, W. Kong, U. Imtiaz, L. Bartolomeo, S. Cosentino, M. Zecca, S. Sessa, H. Ishii, A. Takanishi

    2014 36TH ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY (EMBC)     6945 - 6948  2014  [Refereed]

     View Summary

    Nowadays, the technologies for detecting, processing and interpreting bioelectrical signals have improved tremendously. In particular, surface electromyography (sEMG) has gained momentum in a wide range of applications in various fields. However, sEMG sensing has several shortcomings, the most important being: measurements are heavily sensible to individual differences, sensors are difficult to position and very expensive. In this paper, the authors will present an innovative muscle contraction sensing device (MC sensor), aiming to replace sEMG sensing in the field of muscle movement analysis. Compared with sEMG, this sensor is easier to position, setup and use, less dependent from individual differences, and less expensive. Preliminary experiments, described in this paper, confirm that MC sensing is suitable for muscle contraction analysis, and compare the results of sEMG and MC sensor for the measurement of forearm muscle contraction.

    DOI

  • Development of a real-time IMU-based motion capture system for gait rehabilitation

    W. Kong, S. Sessa, S. Cosentino, M. Zecca, K. Saito, C. Wang, U. Imtiaz, Z. Lin, L. Bartolomeo, H. Ishii, T. Ikai, A. Takanishi

    2013 IEEE INTERNATIONAL CONFERENCE ON ROBOTICS AND BIOMIMETICS (ROBIO)     2100 - 2105  2013  [Refereed]

     View Summary

    A common problem among elderly people is the loss of motor ability. Rehabilitation exercises can help these people recover strength and maintain a good level of mobility. However, high costs and the need for special equipment make professional rehabilitation impractical for regular use in daily life, precluding elderly the possibility to perform focalized training at home. The idea of telerehabilitation is becoming more and more concrete with the rapid development of internet technology. Telerehabilitation would allow the user to perform exercises at home with online professional direction from the doctor. However, at the present state, the doctor cannot obtain real-time and quantitative data from the user, and this limits the training effectiveness. To overcome this problem, an extremely miniaturized, portable motion capture system, named WB-4R, has been developed. Calibration and real-time link orientation reconstruction are very important to improve the accuracy in real-time measurement. In this paper, using the positive results of preliminary experiments on lower limbs, the authors will show the feasibility of the method and confirm the effectiveness of the developed system.

    DOI

  • Impression Survey of the Emotion Expression Humanoid Robot with Mental Model based Dynamic Emotions

    T. Kishi, T. Kojima, N. Endo, M. Destephe, T. Otani, L. Jamone, P. Kryczka, G. Trovato, K. Hashimoto, S. Cosentino, A. Takanishi

    2013 IEEE INTERNATIONAL CONFERENCE ON ROBOTICS AND AUTOMATION (ICRA)     1663 - 1668  2013  [Refereed]

     View Summary

    This paper describes the implementation in a walking humanoid robot of a mental model, allowing the dynamical change of the emotional state of the robot based on external stimuli; the emotional state affects the robot decisions and behavior, and it is expressed with both facial and whole-body patterns. The mental model is applied to KOBIAN-R, a 65-DoFs whole body humanoid robot designed for human-robot interaction and emotion expression. To evaluate the importance of the proposed system in the framework of human-robot interaction and communication, we conducted a survey by showing videos of the robot behaviors to a group of 30 subjects. The results show that the integration of dynamical emotion expression and locomotion makes the humanoid robot more appealing to humans, as it is perceived as more "favorable" and "useful", and less "robot-like."

    DOI

  • Biomechanical Evaluation of the Phases during Simulated Endotracheal Intubation (ETI): Pilot Study on the Effect of Different Laryngoscopes

    L. Bartolomeo, Y. Noh, Y. Kasuya, M. Nagai, M. Zecca, S. Sessa, S. Cosentino, K. Saito, Z. Lin, H. Ishii, A. Takanishi

    2013 35TH ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY (EMBC)     4887 - 4890  2013  [Refereed]

     View Summary

    Endotracheal Intubation (ETI) is a common airway procedure used to connect the larynx and the lungs through a windpipe in patients under emergency situations. The process is carried out by a laryngoscope inserted into the mouth, used to help doctors in visualizing the glottis and inserting the tube. Currently, very few studies on objective evaluation of the biomechanics of the doctors during the procedure have been done. Additionally, these studies have been concentrated only on the overall performance analysis, without any segmentation, with a consequent loss of important information. In this paper, the authors present a preliminary study on a methodology to objectively evaluate and segment the biomechanical performance of doctors during the ETI, using surface electromyography and inertial measurement units. In particular, the validation has been performed by comparing three kinds of laryngoscopes involving an expert doctor. Finally, results are presented and commented.

    DOI

  • Use of an ultra-miniaturized IMU-based motion capture system for objective evaluation and assessment of walking skills

    M. Zecca, K. Saito, S. Sessa, L. Bartolomeo, Z. Lin, S. Cosentino, H. Ishii, T. Ikai, A. Takanishi

    2013 35TH ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY (EMBC)     4883 - 4886  2013  [Refereed]

     View Summary

    The increasing age of the world population is posing new challenges to our society, such as how to keep this aging population healthy and active despite of the age. In recent years, there has been a lot of interest for gait analysis for rehabilitation purposes as well as for performance assessment of this aging population. While current systems work well, they still have several limitations. Cost, need for specialized personnel, need to be used in a research center, and sporadic measurement prevent these systems from being widely used.
    The authors propose the use of extremely miniaturized, portable measurement systems, which can be worn by the users during their everyday life, and can monitor their gait over a long timespan. This paper presents the preliminary experiments with such a system.

    DOI

  • Human-humanoid robot social interaction: Laughter

    Sarah Cosentino, Tatsuhiro Kishi, Massimiliano Zecca, Salvatore Sessa, Luca Bartolomeo, Kenji Hashimoto, Takashi Nozawa, Atsuo Takanishi

    2013 IEEE INTERNATIONAL CONFERENCE ON ROBOTICS AND BIOMIMETICS (ROBIO)     1396 - 1401  2013  [Refereed]

     View Summary

    In this paper, we describe a human gesture recognition system developed to make a humanoid robot understand non-verbal human social behaviors, and we present the results of preliminary experiments to demonstrate the feasibility of the proposed method. In particular, we have focused on the detection and recognition of laughter, a very peculiar human social signal. In fact, although it is a direct form of social interaction, laughter is classified as semi voluntary action, can be elicited by several different stimuli, and it is strongly associated with positive emotion and physical well-being. The possibility of recognize, and further elicit laughter, will help the humanoid robot to interact in a more natural way with humans, to build positive relationships and thus be more socially integrated in the human society.

    DOI DOI2

  • Music conductor gesture recognition by using inertial measurement system for human-robot musical interaction

    Sarah Cosentino, Yoshihisa Sugita, Massimiliano Zecca, Salvatore Sessa, Zhuohua Lin, Klaus Petersen, Hiroyuki Ishii, Atsuo Takanishi

    2012 IEEE INTERNATIONAL CONFERENCE ON ROBOTICS AND BIOMIMETICS (ROBIO 2012)     30 - 35  2012  [Refereed]

    Authorship:Lead author, Corresponding author

     View Summary

    In this paper, we describe a human gesture recognition system developed to enable a robot instrument player to recognize the variations in tempo and in articulation dictated by a conductor's movements and accordingly adapt its performance. The enhanced interaction ability would allow the partner musicians, as well as the conductor, to better appreciate a joint musical performance, because of the complete naturalness of the interaction. In addition, the possibility for the robot to change its performance parameters according to the conductor directions, thus being synchronized with all the other human musicians, would lead to an improvement in the overall musical performance.

    DOI DOI2

  • Musical robots: towards a natural joint performance

    Sarah Cosentino, Yoshihisa Sugita, Massimiliano Zecca, Salvatore Sessa, Zhuohua Lin, Klaus Petersen, Luca Bartolomeo, Hiroyuki Ishii, Kohei Saito, Atsuo Takanishi

    2012 FIRST INTERNATIONAL CONFERENCE ON INNOVATIVE ENGINEERING SYSTEMS (ICIES)     19 - 24  2012  [Refereed]

    Authorship:Lead author, Corresponding author

     View Summary

    In this paper, we describe the mechanical design system implemented to enable a robot flute player to enhance the expressiveness of its performance, reproducing the different musical articulations with the available features of the instrument. In addition, the robot has been equipped with a Human Gesture Recognition system to recognize the real-time variations in tempo, dynamics, and articulation dictated by the conductor's movements. The possibility for the robot to change its performance parameters according to the conductor directions, thus adding expression to its performance while being synchronized with all the other human musicians, would lead to an improvement in the overall joint musical performance.

    DOI DOI2

  • System integration: Development of a global network communication protocol

    Sarah Cosentino, Massimiliano Zecca

    2010 IEEE/SICE International Symposium on System Integration: SI International 2010 - The 3rd Symposium on System Integration, SII 2010, Proceedings     194 - 199  2010  [Refereed]

    Authorship:Lead author, Corresponding author

     View Summary

    In the era of globalization and standardization, in the perspective of industrial system integration, the implementation of a network framework able to interconnect any type of device and supporting both real-time (RT) and time-uncritical communication is becoming a must. This paper proposes an original entirely software-based communication framework to support RT communication over standard Ethernet, and compares it with the most relevant commercially available RT protocols in term of global costs and performances. ©2010 IEEE.

    DOI

▼display all

Misc

  • Development of Training System to Improve Walking Effect-Evaluation of Training Prototype Based on Estimation of Muscle Activity by Using Biped Robot Model and Machine Learning Technique-

    COSENTINO Sarah, GU Zixi, AL MAAMARI Mazoon, 川上泰雄, 高西淳夫

    デサントスポーツ科学   41  2020

    J-GLOBAL

  • Musical Articulation System on Humanoid Saxophonist Robot

    Jia Yeu Lin, Mao Kawai, Kei Matsuki, Sarah Cosentino, Salvatore Sessa, Atsuo Takanishi, Atsuo Takanishi

    CISM International Centre for Mechanical Sciences, Courses and Lectures   584   392 - 399  2019.01

     View Summary

    © 2019, CISM International Centre for Mechanical Sciences. In this paper, a musical articulation system for the Waseda Anthropomorphic Saxophonist robot is proposed. First, the specifications to produce musical articulation with a saxophone are determined. Then, a tonguing mechanism and a fast air flow control valve are implemented on the robot along with an integrated feedback control system to perform articulation by synchronously stopping reed vibration and reducing the air flow. The effectiveness of the system is verified by conducting a performance comparison experiment between the robot and a professional musician. Results are briefly discussed and future works in the same direction are considered.

    DOI

  • Development of a low-cost smart home system using wireless environmental monitoring sensors for functionally independent elderly people

    D. Zhang, W. Kong, R. Kasai, Z. Gu, Y. Minami Shiguematsu, S. Cosentino, S. Sessa, A. Takanishi

    2017 IEEE International Conference on Robotics and Biomimetics, ROBIO 2017   2018-January   153 - 158  2018.03

     View Summary

    © 2017 IEEE. The number of older adults is increasing much faster than the other age groups in the world. Especially in Japan, more and more older adult are becoming functionally independent. The smart home are appearing which provides comfort and monitors the life of the residents. However, when thinking about healthy older adults, there are two important points to take into account for the smart home: providing comfort and monitoring the position inside the house of the older adults on their daily life. This can also help their families giving them peace of mind. In this study, we designed a smart home system which includes low-cost sensor parts and a finger robot. This system can be used to control home appliances, such as air conditioners to keep the residents in a comfortable environment. Without the need of additional sensors only by using CO2 sensors, this system can provide information such as the room the resident is in, or if he/she is at home or not. As a result, we show a working system and the CO2 sensor is able to judge the position of the resident, and we are even able to understand the movement path of the resident.

    DOI

  • Evaluation of a sensor system for detecting humans trapped under rubble: A pilot study

    Di Zhang, Salvatore Sessa, Ritaro Kasai, Sarah Cosentino, Cimarelli Giacomo, Yasuaki Mochida, Hiroya Yamada, Michele Guarnieri, Atsuo Takanishi

    Sensors (Switzerland)   18 ( 3 )  2018.03

     View Summary

    Rapid localization of injured survivors by rescue teams to prevent death is a major issue. In this paper, a sensor system for human rescue including three different types of sensors, a CO2 sensor, a thermal camera, and a microphone, is proposed. The performance of this system in detecting living victims under the rubble has been tested in a high-fidelity simulated disaster area. Results show that the CO2 sensor is useful to effectively reduce the possible concerned area, while the thermal camera can confirm the correct position of the victim. Moreover, it is believed that the use of microphones in connection with other sensors would be of great benefit for the detection of casualties. In this work, an algorithm to recognize voices or suspected human noise under rubble has also been developed and tested.

    DOI PubMed

  • Audience mood estimation for the Waseda Anthropomorphic Saxophonist 5 (WAS-5) using cloud cognitive services

    RANDRIA Estelle I. S, COSENTINO Sarah, LIN Jia‐Yeu, PELLEGRINI Thomas, SESSA Salvatore, TAKANISHI Atsuo

    日本ロボット学会学術講演会予稿集(CD-ROM)   35th   ROMBUNNO.1C1‐04  2017.09

    J-GLOBAL

  • Reliability of stride length estimation in self-pace and brisk walking with an inertial measurement unit on shank

    R. Kasai, T. Kodama, Z. Gu, D. Zhang, W. Kong, S. Cosentino, S. Sessa, Y. Kawakami, A. Takanishi

    2017 IEEE International Conference on Mechatronics and Automation, ICMA 2017     671 - 676  2017.08

     View Summary

    The use of Inertial Measurement Unit (IMU) for gait analysis is gaining popularity because of its advantages of low cost and non-limited workspace. In this context, researchers are focusing on methods for automated data analysis. For example, many algorithms for stride length estimation have been developed. These algorithms rely on event detection to compute gait parameters during walking and on orientation estimation for a more precise double integration of acceleration. However, at the present, there is not comparison between existing algorithms, and the applicability of each algorithm for different walking patterns is not clear. In this paper, we studied the effect on the stride length estimation using three different techniques of event detection and two techniques of orientation estimation, by using an IMU on the lateral side of shank above the ankle. In total 6 patterns of stride estimation algorithms were compared on different walking patterns of normal and brisk walking. We evaluated the techniques in terms of precision, accuracy, and shape of the histogram of the stride estimation error.

    DOI

  • 足首に装着する単一の慣性センサに基づく足の筋肉収縮強度推定手法の提案

    GU Z., 兒玉拓磨, 河西利太朗, ZHANG D., KONG W., COSENTINO S., SESSA S., 川上泰雄, 高西淳夫, 高西淳夫

    計測自動制御学会システムインテグレーション部門講演会(CD-ROM)   18th  2017

    J-GLOBAL

  • Development of a Humorous Humanoid Robot Capable of Quick-and-Wide Arm Motion

    T. Kishi, S. Shimomura, H. Futaki, H. Yanagino, M. Yahara, S. Cosentino, T. Nozawa, K. Hashimoto, A. Takanishi

    IEEE Robotics and Automation Letters   1 ( 2 ) 1081 - 1088  2016.07

     View Summary

    This letter describes the development of a humanoid arm with quick-and-wide motion capability for making humans laugh. Laughter is attracting research attention because it enhances health by treating or preventing mental diseases. However, laughter has not been used effectively in healthcare because the mechanism of laughter is complicated and is yet to be fully understood. The development of a robot capable of making humans laugh will clarify the mechanism how humans experience humor from stimuli. Nonverbal funny expressions have the potential to make humans laugh across cultural and linguistic differences. In particular, we focused on the exaggerated arm motion widely used in slapsticks and silent comedy films. In order to develop a humanoid robot that can perform this type of movement, the required specification was calculated from slapstick skits performed by human comedians. To meet the required specifications, new arms for the humanoid robot were developed with a novel mechanism that includes lightweight joints driven by a flexible shaft and joints with high output power driven by a twin-motor mechanism. The results of experimental evaluation show that the quick-and-wide motion performed by the developed hardware is effective at making humans laugh.

    DOI

  • The Development of Intraoral Pressure Control System on Humanoid Saxophone Playing Robot

    Jia-Yeu Lin, Keisuke Yoshida, Kei Matsuki, Kazuki Takikawa, Sarah Cosentino, Salvatore Sessa, Atsuo Takanishi

    2016 IEEE INTERNATIONAL CONFERENCE ON ROBOTICS AND BIOMIMETICS (ROBIO)   2016 ( ROBIO ) 1649 - 1654  2016

     View Summary

    In this paper, we describe an intraoral pressure control system for the humanoid saxophone-playing robot WAS-4. We studied the model for air flow control in a wind instrument, and via a series of assessment tests to analyze the robot performance, we discovered that the control of the intraoral pressure of the robot is critical for continuous sound production. We then developed a control system to maintain the intraoral pressure stable during note sustaining and changing, preserving reed vibration and sound production. We assessed the effectiveness of the system via a final evaluation experiment.

    DOI J-GLOBAL

  • Quantitative Laughter Detection, Measurement, and Classification - A Critical Survey

    Sarah Cosentino, Salvatore Sessa, Atsuo Takanishi

    IEEE Reviews in Biomedical Engineering   9   148 - 162  2016

     View Summary

    The study of human nonverbal social behaviors has taken a more quantitative and computational approach in recent years due to the development of smart interfaces and virtual agents or robots able to interact socially. One of the most interesting nonverbal social behaviors, producing a characteristic vocal signal, is laughing. Laughter is produced in several different situations: in response to external physical, cognitive, or emotional stimuli
    to negotiate social interactions
    and also, pathologically, as a consequence of neural damage. For this reason, laughter has attracted researchers from many disciplines. A consequence of this multidisciplinarity is the absence of a holistic vision of this complex behavior: the methods of analysis and classification of laughter, as well as the terminology used, are heterogeneous
    the findings sometimes contradictory and poorly documented. This survey aims at collecting and presenting objective measurement methods and results from a variety of different studies in different fields, to contribute to build a unified model and taxonomy of laughter. This could be successfully used for advances in several fields, from artificial intelligence and human-robot interaction to medicine and psychiatry.

    DOI PubMed J-GLOBAL

  • Comparison of Gait Event Detection from Shanks and Feet in Single-task and Multi-task Walking of Healthy Older Adults

    W. Kong, J. Lin, L. Waaning, S. Sessa, S. Cosentino, D. Magistro, M. Zecca, R. Kawashima, A. Takanishi

    2016 IEEE INTERNATIONAL CONFERENCE ON ROBOTICS AND BIOMIMETICS (ROBIO)     2063 - 2068  2016

     View Summary

    Automatic and objective detection algorithms for gait events from MEMS Inertial Measurement Units data have been developed to overcome subjective inaccuracy in traditional visual observation. Their accuracy and sensitivity have been verified with healthy older adults, Parkinson's disease and spinal injured patients, using single-task gait exercises, where events are precise as the subject is focusing only on walking. Multi-task walking instead simulates a more realisitc and challenging scenario where subjects perform secondary cognitive task while walking, so it is a better benchmark. In this paper, we test two algorithms based on shank and foot angular velocity data in single-task, dual-task and multi-task walking. Results show that both algorithms fail when the subject slows extremely down or pauses due to high cognitive and attentional load, and, in particular, the first stride detection error rate of the foot-based algorithm increases. Stride time is accurate with both algorithms regardless of walking types, but the shank-based algorithm leads to an overestimation on the proportion of swing phase in one gait cycle. Increasing the number of cognitive tasks also causes this error with both algorithms.

    DOI

  • Facial Expression Design for the Saxophone Player RobotWAS-4

    Kei Matsuki, Keisuke Yoshida, Salvatore Sessa, Sarah Cosentino, Keiko Kamiyama, Atsuo Takanishi

    ROMANSY 21 - ROBOT DESIGN, DYNAMICS AND CONTROL   569   259 - 266  2016

     View Summary

    The implementation of strategies for intuitive and natural communication between robots and humans, depending on the specific situation, is becoming more and more important. Intuitive and human-like communication patterns for human-robot cooperation will ease communication cognitive load and allow the user to better focus on the task at hand. This work presents a basic facial expression system for a humanoid robot saxophonist, allowing the robot to change its expression during a musical performance to both send specific cues for synchronization to a partner human player, and also add emotional emphasis to the performance, as human players naturally do.

    DOI

  • One DoF robotic hand that makes human laugh by tickling through rubbing underarm

    T. Kishi, T. Nozawa, A. Nibori, H. Futaki, Y. Miura, M. Shina, K. Matsuki, H. Yanagino, S. Cosentino, K. Hashimoto, A. Takanishi

    2016 IEEE/RSJ INTERNATIONAL CONFERENCE ON INTELLIGENT ROBOTS AND SYSTEMS (IROS 2016)   2016-November   404 - 409  2016

     View Summary

    This paper describes the development of one DoF robotic hand that makes human laugh by tickling through rubbing underarm. Laughter is attracting research attention because it enhances health by treating or preventing mental diseases. However, laughter has not been used effectively in healthcare because the mechanism of laughter is complicated and is yet to be fully understood. The development of a robot capable of making humans laugh is useful for clarifying the mechanism of laughter because the stimuli by the robot is quantitative and reproductive. Especially, tickling matches to this purpose because the relationship between stimuli and reaction is simpler compared to other techniques. Therefore, this research aimed to develop a robotic hand that can output quantitative and reproductive tickling stimuli for clarifying the mechanism of laughter. Rubbing underarm is selected as a target motion of robot because previous research suggested that this is the best way for making humans feel ticklish. In order to achieve the tickling motion by robots as humans, the required specifications were determined through experimental method. In order to develop a robot that achieves the required fingertip trajectory by simple mechanisms as much as possible, mechanism with crank and link driven by single motor was developed. The result of experimental evaluation shows that the developed robot could make humans laugh by its rubbing motion. In addition, the quantitative tickling motion by developed robotic hand was suggested to be effective for clarifying the mechanism of laughter.

    DOI

  • Automatic Segmentation for One Leg Stance Test with Inertial Measurement Unit

    W. Kong, T. Kodama, S. Sessa, S. Cosentino, D. Magistro, R. Kawashima, A. Takanishi

    2016 IEEE/SICE INTERNATIONAL SYMPOSIUM ON SYSTEM INTEGRATION (SII)     307 - 312  2016

     View Summary

    One Leg Stance (OLS), a test assessing postural stability, is popularly conducted both in clinic and community settings because it is inexpensive and time-efficient. However, the evaluation based on visual observation and manual time measurement with a stop-watch cannot provide quantitative and detailed parameters for longitudinal or cross-sectional studies. In recent years, to overcome these limitations, the use of Inertial Measurement Unit (IMU) as objective measurement analysis tools is becoming more and more popular. However, the greatest issue is that IMU data segmentation is still time-consuming and prone to errors, as the OLS segmentation is being done manually, off-line, on recorded data. In this paper we proposed a novel algorithm for the automatic segmentation of IMU data of the OLS test. The result showed that the correct rate of detection was over 90% which was close to the correct rate in manual segmentation. Compared to manual segmentation with video, besides being less time-consuming, the proposed algorithm closes the loop making the data acquisition and analysis completely automatic, thus can be integrated in self-assessment smart phone applications, allowing the continuous tracking of postural stability also outside clinics and health-care facilities.

    DOI

  • Development of subliminal persuasion system to improve the upper limb posture in laparoscopic training: a preliminary study

    Di Zhang, Salvatore Sessa, Weisheng Kong, Sarah Cosentino, Daniele Magistro, Hiroyuki Ishii, Massimiliano Zecca, Atsuo Takanishi

    INTERNATIONAL JOURNAL OF COMPUTER ASSISTED RADIOLOGY AND SURGERY   10 ( 11 ) 1863 - 1871  2015.11

     View Summary

    Current training for laparoscopy focuses only on the enhancement of manual skill and does not give advice on improving trainees' posture. However, a poor posture can result in increased static muscle loading, faster fatigue, and impaired psychomotor task performance. In this paper, the authors propose a method, named subliminal persuasion, which gives the trainee real-time advice for correcting the upper limb posture during laparoscopic training like the expert but leads to a lower increment in the workload.
    A 9-axis inertial measurement unit was used to compute the upper limb posture, and a Detection Reaction Time device was developed and used to measure the workload. A monitor displayed not only images from laparoscope, but also a visual stimulus, a transparent red cross superimposed to the laparoscopic images, when the trainee had incorrect upper limb posture. One group was exposed, when their posture was not correct during training, to a short (about 33 ms) subliminal visual stimulus. The control group instead was exposed to longer (about 660 ms) supraliminal visual stimuli.
    We found that subliminal visual stimulation is a valid method to improve trainees' upper limb posture during laparoscopic training. Moreover, the additional workload required for subconscious processing of subliminal visual stimuli is less than the one required for supraliminal visual stimuli, which is processed instead at the conscious level.
    We propose subliminal persuasion as a method to give subconscious real-time stimuli to improve upper limb posture during laparoscopic training. Its effectiveness and efficiency were confirmed against supraliminal stimuli transmitted at the conscious level: Subliminal persuasion improved upper limb posture of trainees, with a smaller increase on the overall workload.

    DOI PubMed

  • Objective Evaluation of Oral Presentation Skills Using Inertial Measurement Units

    Salvatore Sessa, Weisheng Kong, Di Zhang, Sarah Cosentino, Udara Manawadu, Motoji Kawasaki, George Thuruthel Thomas, Tomohiro Suzuki, Ryosuke Tsumura, Atsuo Takanishi

    2015 37TH ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY (EMBC)   2015-November   3117 - 3120  2015

     View Summary

    Oral presentation is considered as one of the most sought after skills by companies and professional organizations and program accreditation agencies. However, both learning process and evaluation of this skill are time demanding and complex tasks that need dedication and experience. Furthermore, the role of the instructor is fundamental during the presentation assessment. The instructor needs to consider several verbal and nonverbal communications cues sent in parallel and this kind of evaluation is often subjective. Even if there are oral presentation rubrics that try to standardize the evaluation, they are not an optimal solution because they do not provide the presenter a real-time feedback. In this paper, we describe a system for behavioral monitoring during presentations. We propose an ecological measurement system based on Inertial Measurement Units to evaluate objectively the presenter's posture through objective parameters. The system can be used to provide a real-time feedback to the presenters unobtrusively.

    DOI PubMed

  • Angular Sway Propagation in One Leg Stance and Quiet Stance with Inertial Measurement Units for Older Adults

    W. Kong, S. Sessa, D. Zhang, M. Zecca, S. Cosentino, H. Ishii, D. Magistro, H. Takeuchi, R. Kawashima, A. Takanishi

    2015 37TH ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY (EMBC)   2015-November   6955 - 6958  2015

     View Summary

    Postural stability degrades with age, threating the health and life quality of the older adults. One Leg Stance (OLS) is one of the standard and commonly adopted assessments for postural stability, and the postural sway in OLS has been demonstrated to be related with age. The propagation of postural sway between body segments could be a hint to the underlying mechanism of balance control. However, it is not yet fully understood. Therefore, the aim of this paper was to study the angular sways and their propagation of the head, trunk, and lower limb in healthy older adults. A cross-correlation of the normalized angular speeds was performed and the experiment with 68 older adults was conducted. The results showed that the head, hip and ankle joints affected the transfer of angular sway with a relatively lower correlation and longer latency.

    DOI PubMed

  • Automatic discrimination of laughter using distributed sEMG

    S. Cosentino, S. Sessa, W. Kong, D. Zhang, A. Takanishi, N. Bianchi-Berthouze

    2015 INTERNATIONAL CONFERENCE ON AFFECTIVE COMPUTING AND INTELLIGENT INTERACTION (ACII)     691 - 697  2015

     View Summary

    Laughter is a very interesting non-verbal human vocalization. It is classified as a semi voluntary behavior despite being a direct form of social interaction, and can be elicited by a variety of very different stimuli, both cognitive and physical. Automatic laughter detection, analysis and classification will boost progress in affective computing, leading to the development of more natural human-machine communication interfaces. Surface Electromyography (sEMG) on abdominal muscles or invasive EMG on the larynx show potential in this direction, but these kinds of EMG-based sensing systems cannot be used in ecological settings due to their size, lack of reusability and uncomfortable setup. For this reason, they cannot be easily used for natural detection and measurement of a volatile social behavior like laughter in a variety of different situations. We propose the use of miniaturized, wireless, dry-electrode sEMG sensors on the neck for the detection and analysis of laughter. Even if with this solution the activation of specific larynx muscles cannot be precisely measured, it is possible to detect different EMG patterns related to larynx function. In addition, integrating sEMG analysis on a multisensory compact system positioned on the neck would improve the overall robustness of the whole sensing system, enabling the synchronized measure of different characteristics of laughter, like vocal production, head movement or facial expression; being at the same time less intrusive, as the neck is normally more accessible than abdominal muscles. In this paper, we report laughter discrimination rate obtained with our system depending on different conditions.

    DOI

  • Natural human-robot musical interaction: understanding the music conductor gestures by using the WB-4 inertial measurement system

    Sarah Cosentino, Klaus Petersen, Zhuohua Lin, Luca Bartolomeo, Salvatore Sessa, Massimiliano Zecca, Atsuo Takanishi

    ADVANCED ROBOTICS   28 ( 11 ) 781 - 792  2014.06

     View Summary

    This paper presents an inertial measurement unit-based human gesture recognition system for a robot instrument player to understand the instructions dictated by an orchestra conductor and accordingly adapt its musical performance. It is an extension of our previous publications on natural human-robot musical interaction. With this system, the robot can understand the real-time variations in musical parameters dictated by the conductor's movements, adding expression to its performance while being synchronized with all the other human partner musicians. The enhanced interaction ability would obviously lead to an improvement of the overall live performance, but also allow the partner musicians, as well as the conductor, to better appreciate a joint musical performance, thanks to the complete naturalness of the interaction.

    DOI J-GLOBAL

  • Natural human-robot musical interaction: understanding the music conductor gestures by using the WB-4 inertial measurement system

    Sarah Cosentino, Klaus Petersen, Zhuohua Lin, Luca Bartolomeo, Salvatore Sessa, Massimiliano Zecca, Atsuo Takanishi

    ADVANCED ROBOTICS   28 ( 11 ) 781 - 792  2014.06

     View Summary

    This paper presents an inertial measurement unit-based human gesture recognition system for a robot instrument player to understand the instructions dictated by an orchestra conductor and accordingly adapt its musical performance. It is an extension of our previous publications on natural human-robot musical interaction. With this system, the robot can understand the real-time variations in musical parameters dictated by the conductor's movements, adding expression to its performance while being synchronized with all the other human partner musicians. The enhanced interaction ability would obviously lead to an improvement of the overall live performance, but also allow the partner musicians, as well as the conductor, to better appreciate a joint musical performance, thanks to the complete naturalness of the interaction.

    DOI

  • 2A1-L01 Cross-cultural Impression Survey of Japanese "Manga" Marks Based Robotic Facial Expressions(Communication Robot)

    Kishi Tatsuhiro, Futaki Hajime, Trovato Gabriele, Cosentino Sarah, Hashimoto Kenji, Takanishi Atsuo

    Robomech   2014 ( 0 ) _2A1 - L01_1-_2A1-L01_4  2014

     View Summary

    This paper describes a cross-cultural impression survey about the robotic facial expressions. For a fluent communication between humans and robots, robots should be able to express emotions with their face. Our previous research shows that robots should express facial expressions that adapt to the cultural background of their partners. First, we developed a robotic head that uses Japanese "Manga" marks in order to adapt to the Japanese cultural background. In this research, we conducted an impression survey of those robotic facial expressions with Japanese and Germans. The result shows that facial expressions are less easy to understand if they are evaluated by the people belong to the other cultural background. This result supports the importance of robots able to adapt their facial expressions according to the cultural background of their partners.

    CiNii

  • Development of a Comic Mark Based Expressive Robotic Head Adapted to Japanese Cultural Background

    T. Kishi, H. Futaki, G. Trovato, N. Endo, M. Destephe, S. Cosentino, K. Hashimoto, A. Takanishi

    2014 IEEE/RSJ INTERNATIONAL CONFERENCE ON INTELLIGENT ROBOTS AND SYSTEMS (IROS 2014)     2608 - 2613  2014

     View Summary

    This paper describes the development of a robotic head that has cartoon facial expression ability with comic marks. For communicating with humans, robots should have expressive facial expression ability for indicating their inner state. Our previous research suggests that robots can express its emotion clearly if it performs facial expressions that are adapted to the cultural background of the communication partner. As a first step, we focus on making expressions for Japanese people. Comic mark is a unique and famous way of emotion expression in Japanese culture. First, we defined facial expressions by combining cartoon-like shape of the facial parts with high emotion recognition rates. Then we asked cartoonists to draw comic marks which they think are effective for emotion expression and find the effective comic marks as "Cross popping veins" for "Anger", "Tear mark" for "Sadness" and "Vertical lines" for "Fear". Finally we obtained a model expression which has sufficiently high emotion recognition rate from the combination of the facial expression and the comic marks. In order to achieve these expressions, we developed flexible full color LED display matrix module and mechanism that push and pull the sheet for expressing black lines. Results of experimental evaluation shows that the new robotic head has over 90% average emotion recognition rates for each of the six basic emotions. The results with non-Japanese subjects suggests that impression of emotion expression on robotic head changes depending on the cultural background. These findings encourage us in pursuing this concept of designing robots that display emotions that are adapted to cultural background of communication partner.

    DOI

  • A robotic head that displays japanese ‘‘manga’’ marks

    Tatsuhiro Kishi, Hajime Futaki, Gabriele Trovato, Nobutsuna Endo, Matthieu Destephe, Sarah Cosentino, Kenji Hashimoto, Atsuo Takanishi

    Mechanisms and Machine Science   22   245 - 253  2014

     View Summary

    This paper describes the development of a robotic head with ability to display marks commonly used in ‘‘manga’’ (Japanese comics). To communicate with humans, robots should have an expressive facial expression ability for indicating its inner state. Our previous research suggests that, robots can express its emotion clearly if it perform facial expressions that can adapt with the cultural background of the communication partner. As a first step, we focus on making expressions for Japanese people. Manga mark is a unique and famous way of emotion expression in Japanese culture. In a previous preliminary experiment, we determined facial expressions for the robot KOBIAN-R with manga marks. Those expressions included four manga marks as ‘‘Cross popping veins’’ for ‘‘Anger’’, ‘‘Tear mark’’ for ‘‘Sadness’’, ‘‘Vertical lines’’ for ‘‘Fear’’ and ‘‘Wrinkle’’ for ‘‘Disgust’’. A new head that express these marks was developed. Flexible full color LED matrix display and mechanism for indicating black lines were implemented. Experimental evaluation shows that the new robotic head has over 90% average emotion recognition rates by 30 Japanese participants for each of the six emotions.

    DOI

  • Bipedal humanoid robot that makes humans laugh with use of the method of comedy and affects their psychological state actively

    T. Kishi, N. Endo, T. Nozawa, T. Otani, S. Cosentino, M. Zecca, K. Hashimoto, A. Takanishi

    2014 IEEE INTERNATIONAL CONFERENCE ON ROBOTICS AND AUTOMATION (ICRA)     1965 - 1970  2014

     View Summary

    This paper describes the bipedal humanoid robot that makes human laugh with its whole body expression and affect human's psychological state. In order to realize "Social interaction" between human and robot, the robot has to affect human's psychological state actively. We focused on "laugh" because it can be thought as a typical example for researching "Social interaction". Looking through a Japanese comedy style called "manzai" or the art of conversation, we picked out several methods for making human laugh. Then we made several skits with the advice of comedians, and made the whole body humanoid robot perform them. Results of experimental evaluation with these skits shows that the robot's behavior made subjects laugh and change their psychological state seen as a decrease of "Depression" and "Anger".

    DOI

  • Development of new muscle contraction sensor to replace sEMG for using in muscles analysis fields

    D. Zhang, Y. Matsuoka, W. Kong, U. Imtiaz, L. Bartolomeo, S. Cosentino, M. Zecca, S. Sessa, H. Ishii, A. Takanishi

    2014 36TH ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY (EMBC)     6945 - 6948  2014

     View Summary

    Nowadays, the technologies for detecting, processing and interpreting bioelectrical signals have improved tremendously. In particular, surface electromyography (sEMG) has gained momentum in a wide range of applications in various fields. However, sEMG sensing has several shortcomings, the most important being: measurements are heavily sensible to individual differences, sensors are difficult to position and very expensive. In this paper, the authors will present an innovative muscle contraction sensing device (MC sensor), aiming to replace sEMG sensing in the field of muscle movement analysis. Compared with sEMG, this sensor is easier to position, setup and use, less dependent from individual differences, and less expensive. Preliminary experiments, described in this paper, confirm that MC sensing is suitable for muscle contraction analysis, and compare the results of sEMG and MC sensor for the measurement of forearm muscle contraction.

    DOI PubMed

  • Balance analysis of one leg stance for older adults with Inertial Measurement Units

    W. Kong, S. Sessa, D. Magistro, D. Zhang, U. Imtiaz, S. Cosentino, M. Zecca, H. Ishii, H. Takeuchi, R. Kawashima, A. Takanishi

    2014 IEEE Healthcare Innovation Conference (HIC)     307 - 310  2014

     View Summary

    Decreasing balance ability in older adults increases the risk of falls. According to the literature, two balance strategies are revealed, ankle strategy and hip strategy. The aim of this paper is to study the postural stability of the one leg stance on the frontal plane and verify the strategy used in older adults. A hypothesis about two different patterns in hip strategy is made and experiments were conducted with 80 healthy elderly subjects. The posture is captured by Waseda Bioinstrumentation -4 Rev Inertial Measurement Unit (WB-4R IMU). The preliminary analysis shows the dominant pattern is moving the upper body in a lateral direction when losing balance on frontal plane.

    DOI

  • Objective skill evaluation of endotracheal intubation using muscle contraction sensor

    A. Niibori, Y. Matsuoka, L. Bartolomeo, S. Cosentino, W. Kong, U. Imtiaz, D. Zhang, Y. Kasuya, M. Nagai, M. Ozaki, S. Sessa, H. Ishii, M. Zecca, A. Takanishi

    2014 IEEE INTERNATIONAL CONFERENCE ON ROBOTICS AND BIOMIMETICS IEEE-ROBIO 2014     1862 - 1867  2014

     View Summary

    Endotracheal intubation (ETI) is a difficult technique and requires a great deal of practice to master. Research on the difference in movements between experts and novices performing the procedure has shown that experts perform movements more precisely than novices. Experts keep a fixed posture and use the upper arm muscles and wrist joints more effectively. These studies were conducted using optical motion capture systems and surface electromyography (sEMG), which are measurement systems that require a long setup time and expensive equipment. In this paper, we propose a novel method to measure the biomechanical performance of doctors during ETI using an innovative muscle contraction sensing device (MC sensor) and inertial measurement units (IMUs). We performed several experiments to measure the movements of both experts and novices performing ETI and then analysed and compared the obtained data. The results clearly showed that our system, comprising an MC sensor and IMUs, allows for an objective evaluation of ETI skills and highlighted the major differences between the movements of novices and experts.

    DOI

  • Human-humanoid robot social interaction: Laughter

    Sarah Cosentino, Tatsuhiro Kishi, Massimiliano Zecca, Salvatore Sessa, Luca Bartolomeo, Kenji Hashimoto, Takashi Nozawa, Atsuo Takanishi

    2013 IEEE INTERNATIONAL CONFERENCE ON ROBOTICS AND BIOMIMETICS (ROBIO)     1396 - 1401  2013

     View Summary

    In this paper, we describe a human gesture recognition system developed to make a humanoid robot understand non-verbal human social behaviors, and we present the results of preliminary experiments to demonstrate the feasibility of the proposed method. In particular, we have focused on the detection and recognition of laughter, a very peculiar human social signal. In fact, although it is a direct form of social interaction, laughter is classified as semi voluntary action, can be elicited by several different stimuli, and it is strongly associated with positive emotion and physical well-being. The possibility of recognize, and further elicit laughter, will help the humanoid robot to interact in a more natural way with humans, to build positive relationships and thus be more socially integrated in the human society.

    DOI

  • Human-humanoid robot social interaction: Laughter

    Sarah Cosentino, Tatsuhiro Kishi, Massimiliano Zecca, Salvatore Sessa, Luca Bartolomeo, Kenji Hashimoto, Takashi Nozawa, Atsuo Takanishi

    2013 IEEE INTERNATIONAL CONFERENCE ON ROBOTICS AND BIOMIMETICS (ROBIO)   31st   1396 - 1401  2013

     View Summary

    In this paper, we describe a human gesture recognition system developed to make a humanoid robot understand non-verbal human social behaviors, and we present the results of preliminary experiments to demonstrate the feasibility of the proposed method. In particular, we have focused on the detection and recognition of laughter, a very peculiar human social signal. In fact, although it is a direct form of social interaction, laughter is classified as semi voluntary action, can be elicited by several different stimuli, and it is strongly associated with positive emotion and physical well-being. The possibility of recognize, and further elicit laughter, will help the humanoid robot to interact in a more natural way with humans, to build positive relationships and thus be more socially integrated in the human society.

    J-GLOBAL

  • Development of a real-time IMU-based motion capture system for gait rehabilitation

    W. Kong, S. Sessa, S. Cosentino, M. Zecca, K. Saito, C. Wang, U. Imtiaz, Z. Lin, L. Bartolomeo, H. Ishii, T. Ikai, A. Takanishi

    2013 IEEE INTERNATIONAL CONFERENCE ON ROBOTICS AND BIOMIMETICS (ROBIO)     2100 - 2105  2013

     View Summary

    A common problem among elderly people is the loss of motor ability. Rehabilitation exercises can help these people recover strength and maintain a good level of mobility. However, high costs and the need for special equipment make professional rehabilitation impractical for regular use in daily life, precluding elderly the possibility to perform focalized training at home. The idea of telerehabilitation is becoming more and more concrete with the rapid development of internet technology. Telerehabilitation would allow the user to perform exercises at home with online professional direction from the doctor. However, at the present state, the doctor cannot obtain real-time and quantitative data from the user, and this limits the training effectiveness. To overcome this problem, an extremely miniaturized, portable motion capture system, named WB-4R, has been developed. Calibration and real-time link orientation reconstruction are very important to improve the accuracy in real-time measurement. In this paper, using the positive results of preliminary experiments on lower limbs, the authors will show the feasibility of the method and confirm the effectiveness of the developed system.

    DOI

  • Biomechanical Evaluation of the Phases during Simulated Endotracheal Intubation (ETI): Pilot Study on the Effect of Different Laryngoscopes

    L. Bartolomeo, Y. Noh, Y. Kasuya, M. Nagai, M. Zecca, S. Sessa, S. Cosentino, K. Saito, Z. Lin, H. Ishii, A. Takanishi

    2013 35TH ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY (EMBC)     4887 - 4890  2013

     View Summary

    Endotracheal Intubation (ETI) is a common airway procedure used to connect the larynx and the lungs through a windpipe in patients under emergency situations. The process is carried out by a laryngoscope inserted into the mouth, used to help doctors in visualizing the glottis and inserting the tube. Currently, very few studies on objective evaluation of the biomechanics of the doctors during the procedure have been done. Additionally, these studies have been concentrated only on the overall performance analysis, without any segmentation, with a consequent loss of important information. In this paper, the authors present a preliminary study on a methodology to objectively evaluate and segment the biomechanical performance of doctors during the ETI, using surface electromyography and inertial measurement units. In particular, the validation has been performed by comparing three kinds of laryngoscopes involving an expert doctor. Finally, results are presented and commented.

    DOI PubMed

  • Use of an ultra-miniaturized IMU-based motion capture system for objective evaluation and assessment of walking skills

    M. Zecca, K. Saito, S. Sessa, L. Bartolomeo, Z. Lin, S. Cosentino, H. Ishii, T. Ikai, A. Takanishi

    2013 35TH ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY (EMBC)     4883 - 4886  2013

     View Summary

    The increasing age of the world population is posing new challenges to our society, such as how to keep this aging population healthy and active despite of the age. In recent years, there has been a lot of interest for gait analysis for rehabilitation purposes as well as for performance assessment of this aging population. While current systems work well, they still have several limitations. Cost, need for specialized personnel, need to be used in a research center, and sporadic measurement prevent these systems from being widely used.
    The authors propose the use of extremely miniaturized, portable measurement systems, which can be worn by the users during their everyday life, and can monitor their gait over a long timespan. This paper presents the preliminary experiments with such a system.

    DOI PubMed

  • Impression Survey of the Emotion Expression Humanoid Robot with Mental Model based Dynamic Emotions

    T. Kishi, T. Kojima, N. Endo, M. Destephe, T. Otani, L. Jamone, P. Kryczka, G. Trovato, K. Hashimoto, S. Cosentino, A. Takanishi

    2013 IEEE INTERNATIONAL CONFERENCE ON ROBOTICS AND AUTOMATION (ICRA)     1663 - 1668  2013

     View Summary

    This paper describes the implementation in a walking humanoid robot of a mental model, allowing the dynamical change of the emotional state of the robot based on external stimuli; the emotional state affects the robot decisions and behavior, and it is expressed with both facial and whole-body patterns. The mental model is applied to KOBIAN-R, a 65-DoFs whole body humanoid robot designed for human-robot interaction and emotion expression. To evaluate the importance of the proposed system in the framework of human-robot interaction and communication, we conducted a survey by showing videos of the robot behaviors to a group of 30 subjects. The results show that the integration of dynamical emotion expression and locomotion makes the humanoid robot more appealing to humans, as it is perceived as more "favorable" and "useful", and less "robot-like."

    DOI

  • Walking assessment in the phase space by using ultra-miniaturized Inertial Measurement Units

    S. Sessa, K. Saito, M. Zecca, L. Bartolomeo, Z. Lin, S. Cosentino, H. Ishii, T. Ikai, A. Takanishi

    2013 IEEE International Conference on Mechatronics and Automation, IEEE ICMA 2013     902 - 907  2013

     View Summary

    Physical therapy helps patients to restore the use of the musculoskeletal and the nervous systems through the use of specifics techniques and exercises. The introduction of measurement systems for patient assessment may allow detection of initial stage of diseases, an objective severity assessment, and efficient delivery of drugs and therapies. In rehabilitation centers, sometimes there are specific devices and methodologies available for the locomotion assessment. However, the measurements are usually carried out in a short time slot and this could lead to an overestimation of the walking abilities. © 2013 IEEE.

    DOI

  • 笑い方策を利用した2足ヒューマノイドロボットによる人間の笑い誘発と心理状態への積極的な働きかけ

    岸竜弘, 岸竜弘, 遠藤信綱, 大谷拓也, COSENTINO Sarah, 野澤隆司, ZECCA Massimiliano, ZECCA Massimiliano, 橋本健二, 高西淳夫, 高西淳夫

    日本ロボット学会学術講演会予稿集(CD-ROM)   31st  2013

    J-GLOBAL

  • Motion recognition system for conductor and flutist robot interaction

    COSENTINO Sarah, SUGITA Yoshihisa, ZECCA Massimiliano, SESSA Salvatore, LIN Zhuohua, PETERSEN Klaus, ISHII Hiroyuki, TAKANISHI Atsuo

    日本ロボット学会学術講演会予稿集(CD-ROM)   30th   ROMBUNNO.4D3-6  2012.09

    J-GLOBAL

  • Assessment of walking quality by using Inertial Measurement Units

    K. Saito, M. Zecca, S. Sessa, Z. Lin, L. Bartolomeo, S. Cosentino, K. Petersen, H. Ishii, T. Ikai, A. Takanishi

    2012 FIRST INTERNATIONAL CONFERENCE ON INNOVATIVE ENGINEERING SYSTEMS (ICIES)     13 - 18  2012

     View Summary

    A device for the gait analysis during a long-distance walking is important for the correct assessment of patients during the rehabilitation. This device should be able to measure all gait parameters in a single unit. In addition, it is required that the measurement system is not spatially constrained. In our group, we have been developing a new wireless system, namely WB-4R, composed of Inertial Measurement Units, to be used in rehabilitation centers for gait analysis that is cheap, small, and relatively maintenance-free. This paper presents the results of a pilot study conducted with healthy subjects. The system was able to detect the gait phase and provide frequency analysis of the angular velocity and acceleration of the lower limbs.

    DOI

  • Performance analysis and solutions for network communications in industrial robotics systems

    ZECCA Massimiliano, COSENTINO Sarah

    日本ロボット学会学術講演会予稿集(CD-ROM)   28th   ROMBUNNO.3C1-1  2010

    J-GLOBAL

▼display all

Research Projects

  • Robot music therapy for effective social training of ASD children

    Japan Society for the Promotion of Science  Grants-in-Aid for Scientific Research Grant-in-Aid for Young Scientists (B)

    Project Year :

    2017.04
    -
    2020.03
     

    COSENTINO SARAH

Specific Research

  • Automatic behavior analysis

    2021   Mauricio MENDEZ MARTINEZ, Zixi GU, Jia-Yeu LIN

     View Summary

    This project stemmed from ideas presented in the past rejected kakenhi project Kiban B, but had a different core scientific question, and served as a base for starting research and acquire preliminary data on unobtrusive and ecological methods to continuously analyze human behavior in unstructured environment. The core idea was to create a modular, reconfigurable sensor algorithm which could be deployed on modular, reconfigurable hybrid (environmental and wearable) sensor systems, to detect the user's and automatic assess their behavior and vital parameters, detecting possible anomalies.Using the funds, a concept prototype of the sensor system has been designed.Due to the pandemic, the system development was severely delayed and experiments in real environment with external users were not possible. For this reason, we acquired all the necessary material and we are planning to continue the testing in the next year.

  • SMART – System for Multitask Analysis、 Rating and Trending

    2020   Jia-Yeu LIN, Zixi GU, Mauricio MENDEZ, Shengxu LIU, Shi FENG

     View Summary

    This project follows the idea presented in the rejected kakenhi project Kiban B, and serves as a base for starting research and acquire preliminary data on unobtrusive and ecological methods to continuously analyze Multi-Task (MT) mobility and assess cognitive activities in activities of daily living (ADL). The core idea is to create a sensorized environment which can monitor the users and automatic assess theirbehavior and vital parameters, detecting possible anomalies and triggering safety responses when needed, while respecting the user privacy at all times. Such system will prove very useful in ensuring safety and independence to senior citizens during ADL, drastically improving the users' Quality of Life (QoL). Due to the pandemic, the system development was severely delayed and experiments in real environment with external users were not possible. For this reason, we acquired all the necessary material and we are planning to continue the development in the next year. In particular, we will add a module for anonimized data upload to the cloud, and start monitoring experiments in the laboratory, to refine the sensor system and AI behavior data analysis module.

  • Development of gesture recognition system for therapy applications

    2019  

     View Summary

    This project stemmed from the selected kakenhi project Wakate B 17K18178, and served as a base for continuing research on robotic platforms for cognitive and social therapy applications. We developed two systems: a gesture recognition system to analyze human movement via real-time video data analysis, a system for movement input recognition, and a system for real-time video gaze direction recognition. These systems were integrated in two therapy session protocols for ASD children, for social training interaction robotic therapy using our WAS Waseda Anthropomorphic Saxophonist robot.From the analysis of body movement and gaze direction, as well as, if present, facial expression, the robot can recognize basic interactive behaviors and emotional patterns of the child, and adapt its performance consequently, following the suitable therapeutic protocol. In this way, also the effects of both active and receptive music therapy can be combined, as the robot can choose appropriate melodies and music tunes according to the current needs of the child under therapy.This supporting grant was useful in running more experiments and refining the prototype systems developed during Wakate B 17K18178.

  • Iconic facial expressions able robotic face for direct signaling in human-robot dynamic musical interaction

    2016   Salvatore Sessa

     View Summary

    After an extensive evaluation and recalibration of the needed hardware for the implementation of the proposed solution, we decided to move the focus of the facial interaction from action to recognition. We implemented a system that allows the robot, using the cloud computing and the Microsoft APIs, to recognize facial expression and emotional cues, and adapt its performance accordingly.This system would allow the robot not only to recognize specific facial expression cues from partner human players during a joint performance, but also and especially to recognize emotional expressions from the audience and respond in a tailored way.To do so, we had to implement the facial recognition software system, but also to improve the musical expressivity of the robot, by refining the hardware. We then redesigned the robot control system to produce better and more diverse sounds, allowing the robot to add specific emotional connotation to the performed piece.In the future, we are planning to redesign the entire robot hands, to be able to extend the register range of the performance and extend the possible musical emotional connotations.

 

Syllabus

▼display all

Teaching Experience

▼display all