In the field of 4-D space visualization, the information of 4-D space is obtained by projecting 4-D data onto 3-D space. Most of the previous research has been aimed at the recognition of 4-D space, whereas the target of the recognition has been limited to the geometrical information of 4-D objects in 4-D space or static spatial information without dynamics. Our research aims to develop a visualization system for providing the human experience of the physics-based environment in 4-D space. In this research, we mainly focus on collision detection and the behaviour of 4-D objects in 4-D space in order to construct the physics-based environment of 4-D space. Our contribution in this paper is the development of a collision detection algorithm for 4-D objects and a calculation method for physics based behaviour of 4-D objects. Our proposed collision detection algorithm is based on the intersection test of tetrahedrons in 4-D space, so that 4-D objects in our system is represented as tetrahedral meshes. The tetrahedron-based collision detection algorithm is performed by a combination of half-space tests with the use of 5-D homogeneous processing to enhance the calculation accuracy of the collision detection. Our proposed method calculates the behaviour of the 4-D objects after the collision by solving the motion equation based on the principle of physics. Consequently, the visualization system with the proposed algorithm allows us to observe the physics-based environment in 4-D space.

This paper reports the temperature and relative humidity dependence of self-propelled oil droplets. Self-propelled oil droplets locomote by internal Marangoni convection, which is caused by gradients in the interfacial tension due to surfactant and temperature gradients at the oil/water interface. We discover the external environment dependence of the self-propulsion of a droplet and discuss the mechanism of the property. Our study will serve as an innovative foundation for practical research, such as drug delivery systems.

<p>"Tears of wine" is a physical phenomenon that appears due to the result of a liquid flow along the glass surface against the gravity. In this study, we try to visualize the convection inside droplets in "tears of wine" by using the particle image velocimetry (PIV) method. We also examine the difference of convection flow which occurs when heat is applied to a part of a droplet. From the observation results, we assume that the temperature gradient can change the Marangoni flow in "tears of wine".</p>

<p>We study the motion control of a self-propelled oil droplet dropped into aqueous solution by using a rigid frame. It is well known that an oil droplet with fatty acid anhydride moves in random directions in aqueous solution. Although we have discovered that the moving direction could be controlled using a rigid boomerang-shaped frame, the mechanism of the autonomous motion in a particular direction is not understood theoretically. Since self-propelled droplets move with the effect of Marangoni convection caused by surface tension difference at the droplet surface, we focus on the directional behavior of the Marangoni convection and verify it using a SPH (Smoothed Particle Hydrodynamics) method, which is a meshfree fluid simulation.</p>

© Copyright © 2020 Hashimoto, Shigemune, Minaminosono, Maeda and Sawada. Self-folding technologies have been studied by many researchers for applications to various engineering fields. Most of the self-folding methods that use the physical properties of materials require complex preparation, and usually take time to complete. In order to solve these problems, we focus on the elasticity of a material, and propose a model for forming a 3D structure using its characteristics. Our proposed model achieves high-speed and high-precision self-folding with a simple structure, by attaching rigid frames to a stretchable elastomer. The self-folded structure is applied to introduce a self-assembled actuator by exploiting a dielectric elastomer actuator (DEA). We develop the self-assembled actuator driven with the voltage application by attaching stretchable electrodes on the both side of the elastomer. We attempt several experiments to investigate the basic characteristics of the actuator. We also propose an application of the self-assembled actuator as a gripper based on the experimental results. The gripper has three joints with the angle of 120°, and successfully grabs objects by switching the voltage.

© 2019 SPIE. For the visualization of a 4-D object, information in 4-D space is obtained by projecting a 4-D space onto a 3-D space. Previous researches mainly paid attention to the geometrical information of a single 4-D object in 4-D space. Our research aims to develop a visualization system for physical phenomena in 4-D space and to study the human spatial cognition of the 4-D space through the projected 3-D space. In this research, we focus on a collision of two 4-D objects in 4-D space and develop an algorithm for the collision detection between two 4-D objects to be displayed in 3-D space. In our developed algorithm, a 4-D object is defined as a mesh data of tetrahedrons, and a tetrahedron-based collision detection algorithm is performed by a combination of half-space tests with the use of 5-D homogeneous processing to enhance the calculation accuracy of the collision detection. The results of the collision detection between the 4-D objects are presented.

© 2019 The Royal Society of Chemistry. This paper introduces a novel directional control method of self-propelled oil droplets. Oil droplets locomote spontaneously with surfactant action. This self-propulsion is caused by Marangoni convection within the oil droplet due to differences in the surfactant concentration at the droplet surface. Recent studies have reported that self-propelled oil droplets change their locomotion style depending on their shapes. We confirm that spherical oil droplets move randomly, including straight motion, bending motion, and rotation. In particular, we discover that boomerang-shaped oil droplets exhibit only straight motion. In this study, we introduce an exoskeleton for the directional and velocity control of oil droplets. A droplet shaped as a boomerang by an exoskeleton locomotes in the direction from a concave region to a convex region. Through experimental studies, we found that the stability of the velocity and locomotion direction depended on the boomerang shape. Self-propelled oil droplets with exoskeletons were then applied to a transporting robot driven only by the energy obtained from chemical reactions. We demonstrate the robot pushes and transports an object floating on water.

© 2018 IEEE. In this paper, we propose a printed self-oscillatory mechanism inspired by an electric bell. We have proposed paper mechatronics to make a robot with printing methods. In the paper mechatronics, we also aim to fabricate a controller with printing methods. We attempt to obtain a control signal from a mechanism. An electric bell generates an oscillatory signal with a coupling mechanism between a coil and a magnet. We develop a printed electric bell on a paper with printing methods. We propose the design of the electric bell which can be fabricated on a sheet of paper. The coil and the wiring were printed on the paper, and the device was fabricated by folding the paper. We demonstrate that the device oscillates for 23 min by applying a 0.5 A of constant current. The design is scalable to be applied for microsystems with using printing methods.

The authors are developing a talking robot based on the physical model of human vocal organs in order to reproduce human speech mechanically. Humans use speech in daily communication, and each different country has a different language. The complex movements of vocal organs generate different vocal sounds. This study focuses on the performance of the talking robot to reproduce Vietnamese language, which is a tonal language and contains many unvoiced sounds and fricative sounds. The mechanical system of the talking robot is modified by adding an unvoiced sound source for fricative sound generation and a new flexible vocal cords. The control of the artificial vocal cords is adjusted to give the tonal effect. The robot is programmed to speak several Vietnamese phrases, and the output speech is evaluated by questionars.

Communication between machines and people is achieving dynamic relations. In fact, we can recognize this fact in a wide range of products like Google Home, Amazon Echo or Apple HomePod. They are all called voice chatbots. These chatbots, however, mainly respond to speech in English, while in Vietnam such chatbots are still not popular. This paper introduces a Vietnamese voice Chatbot, in which users can use Vietnamese to communicate with Chatbot through voice. Chatbot can complete tasks from user’s requests such as chatting with pre-written scripts; informing time and weather; searching on Google and Wikipedia or communicating and controlling other devices. In addition, the article introduces the emotion recognition from user voice for the chatbot. The project uses the Python programming language as well as the platforms built by Google.

In this paper, we propose a novel method for a humanoid robot to recover its standing posture from different fallen postures. In the proposed method, the robot estimates its posture using the ground-contacting information, which is sensed by the joint angle information and pressure sensors attached to the whole body. Then, the robot searches a motion sequence based on the estimated posture to stand up. The robot preserves every successful motion sequence into the tree structure to be reused in the next time. To verify the proposed algorithm, we conducted evaluation experiments by employing a real robot. The results show that the proposed method enables the robot to successfully search the motion sequence for standing up from the fallen posture.

© 2018 IEEE. Shape-memory alloys (SMAs) are metal which have the shape memory effects. By forming them to thin wires, they can shrink at a certain temperature; and then extend to original length after cooling below this temperature, thus shape-memory effect is observed. When pulse current is applied to a SMA wire, Joule heat generated inside makes the wire shrink about 5% of its length, and returns to the origin length when the pulse stops, which generate vibration in accordance with the frequency of the current. A novel designed actuator using the vibration of two SMA wires is proposed in this paper. The actuator has compact size and low energy consumption of milliwatts order. Tactile sensation is one of the important ways in human's five senses, which are used to acquire information from surrounding environment. In this paper, we are trying to transmit recognized information by translating it to tactile sensation perceived by skin around eyes. We divide the skin area around eyes into 4 zones, and design two tactile display prototypes. By installing multiple actuators in prototypes and applying pulse signals with different parameters, we conduct experiments to find optimal driving parameters to generate tactile sensation in designated skin areas. The higher-level perception in these area is tested by combining different parameters and stimulation patterns. Finally, information transmission is evaluated by the feedback from subjects and it turns out the proposed system performs well. In the final part we also propose potential applications of this system.

© 2018 IEEE. Soft robotic arms have gained popularity in the recent years because of their dexterity, robustness and safe interaction with humans. However, since these arms are subject to non-linear mechanics and are intrinsically under-actuated, their control still present many challenges. Octopus arms are one of the most popular biological models for soft robotics. It is known that the octopus reaching movement consists in two steps: (1) the rotation of the arm's base towards the target, and (2) the extension of the arm to reach the target. From a robotics point of view, the rotation of the base adds one additional degree of freedom to an already hyper-redundant system. Therefore, its role in the effectiveness of the control is ambiguous. In this work, we investigate the role of the base rotation for learning an effective reaching strategy. We conduct numerical experiments based on a mathematical model of the mechanics of the octopus arm in water and a simple neural network enabling to encode the control strategy through optimization learning. The network node corresponding to the base rotation is switched on or off for comparison. We test the reaching success rate with and without base rotation with targets in various positions. The results show that the addition of the base rotation can be highly beneficial or even detrimental, based on the position of the target. Nonetheless, globally the addition of base rotation affects the control strategy and expand the reachable regions.

© 2017 IEEE. Paper mechatronics has been proposed as a concept to fabricate robots with printing methods in our previous studies. Actuators fabricated by printing on a sheet material can be low-cost and lightweight. In this paper, we propose a swinging paper actuator driven by conduction Electrohydrodynamics (EHD). EHD is a phenomenon that generates flow by applying voltage to a dielectric liquid. The EHD flow is generated by the applied voltage in a wiring made by silver ink printed with an inkjet printer. The silver wiring was printed on a sheet of paper, and the paper with the wiring was folded manually to form its structure. We explored the possibility to manufacture EHD actuators by using the printing techniques developed in paper mechatronics. We expected to obtain high performance from the generated actuators due to the light weight of the paper printed structures. Finally, we succeeded in activating the swinging actuator by applying the high voltage to the paper electrode. We applied sine and square inputs, and investigated several characteristics. We found conditions to obtain the largest displacement in our situation. The swinging actuator proves a new concept to incorporate two alluring researches which are printed paper actuator and EHD phenomenon.

© 2017 IEEE. This paper presents the processing technique of hidden hypersurface removal for 4-D space visualization. When 4-D space and objects are visualized in 3-D space by perspective projection, if a 4-D scene includes multiple 4-D objects and intricate structures, visibility becomes a fundamental problem in 4-D space visualization. To address this problem and improve the perception of 4-D space, we extend Warnock&#039;s area-subdivision algorithm, which is used for hidden surface removal in the domain of 2-D projections of a 3-D scene, to the domain of 3-D projections of a 4-D scene. The visualization results show that our algorithm can display a 4-D scene with correct visibility, and suggest that the hidden hypersurface removal can help us understand the spatial relationship of the 4-D scene.

© 2017 IEEE. The authors have been developing a novel micro-vibration actuator using a shape-memory alloy (SMA) wire for the presentation of various tactile sensations to a human skin. An SMA wire has a unique characteristic to shrink to the lengthwise direction at a certain temperature according to the transition between the martensite and austenite phases. We have found a novel and characteristic effect that, by making the SMA into a fine wire with the diameter of 30-100 μm, it accepts a short pulse-current to instantly shrink and then return to the original length in accordance with the pulse frequency up to 300 Hz. The distinctive behavior of an SMA wire has been applied to present various tactile stimuli to human skin in this study.

The authors are developing a talking robot which is a mechanical vocalization system modeling the human articulatory system. The talking robot is constructed with mechanical parts that are made by referring to human vocal organs biologically and functionally. In this study, a newly redesigned artificial vocal cords is developed for the purpose of extending the speaking capability of the talking robot. The artificial vocal cords is developed using functional approach. A thin layer of rubber band with a width of 7mm is attached on a plastic body in a sealed chamber creates an artificial sound source. The fundamental frequency of the sound source vary from 80Hz to 250 Hz depending on the pressure to the rubber band and the curving shape of the rubber band. The curving shape of the rubber band is adjustable by an innovative design mechanism driven by a servo motor. The amount of air pressure applied to the rubber band is controlled by another servo motor. The experiments to verify the speak capability of the talking robot with this newly redesigned vocal cords is conducted by letting the robot generating five vowel sounds with different combinations of 20 levels of rubber band shape and 5 levels of air pressure input. The pitch of each sound is extracted to determine the effect of the rubber band shape and the pressure input on the output sound. The result shows that this newly redesigned vocal cords greatly increase the speaking capability of the talking robot, especially its singing performance.

The authors are developing a talking robot which is a mechanical vocalization system modeling the human articulatory system. The talking robot is constructed with mechanical parts that are made by referring to human vocal organs biologically and functionally. In this study, a newly redesign artificial vocal cord is developed for the purpose of extending the speaking capability of the talking robot. The artificial vocal cord is developed using functional approach. A thin layer of rubber band with a width of 7mm is attached on a plastic body in a sealed chamber creates an artificial sound source. The fundamental frequency of the sound source vary from 80Hz to 280 Hz depending on the pressure to the rubber band and the curving shape of the rubber band. The curving shape of the rubber band is adjustable by an innovative design mechanism driven by a servo motor. The amount of air pressure applied to the rubber band is controlled by another servo motor which is referred as volume motor in this study. The experiment to verify the speak capability of the talking robot with this newly redesigned vocal cord is conducted by letting the robot generating five vowel sounds with difference combination of 20 levels of rubber band shape and 10 levels of air pressure input. The pitch and volume of each sound is extracted to determine the effect of the rubber band shape and the pressure input on the output sound. The result shows that this newly redesigned vocal cord greatly increase the speaking capability of the talking robot, especially its singing performance.

The timing control is necessary for determining its duration, stress, and rhythm in human speech; however, little attention has been paid to these issues when building a speech synthesis system. We have developed a talking robot, which generates human-like vocal sounds. The cerebellum is an important part of human brain organ that has a significant role in the coordination, precision, and timing of motor responses. In this study, we develop a simplified cerebellum-like spiking neural network model to control the timing function for the talking robot. The model was designed using the System Generator software in Matlab, and the timing duration of trained speech was estimated using hardware cosimulated with a field programmable gate array board (FPGA). The timing information obtained from the co-simulation, together with the output motor vector, is sent to the talking robot controller to generate a sound with a short duration. The result indicates that this model can be used for short-range timing learning of the talking robot.

Purpose: Endoscope-assisted surgery has widely been adopted as a basic surgical procedure, with various training systems using virtual reality developed for this procedure. In the present study, a basic training system comprising virtual reality for the removal of submandibular glands under endoscope assistance was developed. The efficacy of the training system was verified in novice oral surgeons.
Material and methods: A virtual reality training system was developed using existing haptic devices. Virtual reality models were constructed from computed tomography data to ensure anatomical accuracy. Novice oral surgeons were trained using the developed virtual reality training system.
Results: The developed virtual reality training system included models of the submandibular gland and surrounding connective tissues and blood vessels entering the submandibular gland. Cutting or abrasion of the connective tissue and manipulations, such as elevation of blood vessels, were reproduced by the virtual reality system. A training program using the developed system was devised. Novice oral surgeons were trained in accordance with the devised training program.
Conclusions: Our virtual reality training system for endoscope-assisted removal of the submandibular gland is effective in the training of novice oral surgeons in endoscope-assisted surgery. (C) 2016 The Author(s). Published by Elsevier Ltd on behalf of European Association for Cranio-Maxillo-Facial Surgery.

Diabetes mellitus is a group of metabolic diseases that cause high blood sugar due to functional problems with the pancreas or metabolism. Diabetic patients have few subjective symptoms and may experience decreased sensation without being aware of it. The commonly performed tests for sensory disorders are qualitative in nature. The authors pay attention to the decline of the sensitivity of tactile sensations, and develop a non-invasive method to detect the level of tactile sensation using a novel micro-vibration actuator that employs shape-memory alloy wires. Previously, we performed a pilot study that applied the device to 15 diabetic patients and confirmed a significant reduction in the tactile sensation in diabetic patients when compared to healthy subjects. In this study, we focus on the asymptomatic development of decreased sensation associated with diabetes mellitus. The objectives are to examine diabetic patients who are unaware of abnormal or decreased sensation using the quantitative tactile sensation measurement device and to determine whether tactile sensation is decreased in patients compared to healthy controls. The finger method is used to measure the Tactile Sensation Threshold (TST) score of the index and middle fingers using the new device and the following three procedures: TST-1, TST-4, and TST-8. TST scores ranged from 1 to 30 were compared between the two groups. The TST scores were significantly higher for the diabetic patients (P &lt; 0.05). The TST scores for the left fingers of diabetic patients and healthy controls were 5.9 +/- 6.2 and 2.7 +/- 2.9 for TST-1, 15.3 +/- 7.0 and 8.7 +/- 6.4 for TST-4, and 19.3 +/- 7.8 and 12.7 +/- 9.1 for TST-8. Our data suggest that the use of the new quantitative tactile sensation measurement device enables the detection of decreased tactile sensation in diabetic patients who are unaware of abnormal or decreased sensation compared to controls.

A compact device for the presentation of tactile sensation is introduced in this chapter. By employing shape-memory alloy (SMA) wires, a novel tactile information transmission device is developed for displaying various tactile sensations. A tactile actuator consists of a thin SMA wire with the diameter of 50 μm, and generates micro-vibrations with various frequencies and amplitudes in accordance with a driving pulse signal. The actuator features compactness and low energy consumption, and is applied to tactile displays and tactile interaction systems.

Users are able to experience a high immersive feeling easily in a virtual environment with the widespread availability of human-machine interface devices such as head mounted displays and less expensive human motion sensors. The introduction of such electronic devices has led to more research being conducted in the area of the interaction of a human with robots and machine systems in virtual reality environment and a augmented reality space. Human-machine interface (HMI) researchers are looking for new ways to deal various information, and a variety of studies that present tactile experiences to users are being introduced by employing tactile actuators and devices. Tactile higher-level perception such as the phantom sensations and the apparent movement is known as tactile illusions, by which a user feels virtual sensation as being stimulated at different points on skin at the same time, and could be also effectively used for presenting better tactile sensations. In this study, we focus on the tactile perception of the apparent movement in a human body, and develop a system, by which a user has special tactile experience in an environment with a mobile robot to know its situation such as the moving speed and the roughness of the located ground and the wall as a feedback from the robot.

This study compares several acoustic features for developing an automatic vowel sequence reproduction system for a talking robot, which is a mechanical vocalization system modeling the human articulatory system. Matlab-based control system is used to analyze a recorded sound and drives the articulatory motors of the talking robot. A novel method based on short-time energy analysis is used to extract a human speech and translate into a sequence of sound elements for the sequence of vowels reproduction. Then, several phonemes detection methods including the direct cross-correlation analysis, the linear predictive coding (LPC) association, the partial correlation (PARCOR) coefficients analysis, and the formant frequencies comparison are applied to each sound element to give the corrected command for the talking robot to repeat the sound sequentially. Finally, experiments to compare these techniques and verify the working behavior of the robot are performed. The result of the tests indicates that the robot is able to repeat a sequence of vowels spoken by a human with a successful rate of more than 70% for the PARCOR analysis technique and the formant frequencies comparison technique. The greatest accuracy for repeating the sequence is given by the formant comparison method, while the direct cross-correlation method delivers the least accuracy.

The main focus of this study is to develop an automatic vowels sequence reproduction system for a talking robot, which is built to reproduce human voice based on the working behavior of human articulatory system. The control system using Matlab software to drive the articulatory motors of the talking robot. A sound analysis system is developed to record a sentence spoken by a human (mainly vowels sequence in Japanese language) and then analyze that sentence to give the corrected command packet for the talking robot to repeat it. Partial Correlation (PARCOR) coefficients analysis is used to detect the similar voice in the talking robot's database with the spoken voice; thus, the generated vocal sound of the talking robot will be the similar vowels sequence spoken by human. Finally, two tests to verify the working behavior of the robot are performed. The result of the tests indicate that the robot can be able to repeat a sequence of vowels spoken by human with an average successful rate more than 70%

In the recognition of an object situated in front of us, we firstly watch it and then try to touch it to feel the tactile and haptic sensation to be perceived by our hands. Various virtual reality and augmented reality systems have been introduced with the growing computational powers, however visual and auditory information related with a virtual object is mainly presented simultaneously to a user for enhancing the reality. If we consider the real situation of a user to try to recognize an object in the living environment, tactile and haptic information is assumed to be rather important for the better interaction with the object. In this study, we develop a haptotactile display that covers the movable range of a human arm in the tactual search for presenting tactile sensation of a virtual object. We design and construct a tactile pad to be fixed at the tip of a passively-driven haptic arm, and the tactile sensation and haptic force are simultaneously displayed to a user wearing the pad in the hand. The development of the hapto-tactile display for human tactile search will be described in this paper, together with an experiment to verify the validity of the system.

An estimated number of 420 million people in the world had diabetes mellitus in 2014, which had become quadruple since 1980, and the number is estimated to be 700 million by 2025. Diabetes mellitus is a group of metabolic diseases, which causes high blood sugar to a person, due to the functional problems of the pancreas or the metabolism. Patients of untreated diabetes would be damaged by the high blood sugar in vessels, and this starts to destroy capillary vessels to lower the sensitivity of tactile sensations, then effects to various organs and nerve systems. Diabetic peripheral neuropathy is one of the complications of diabetes mellitus, and the authors pay attention to the decline of the sensitivity of tactile sensations in the early stage of diabetes. By using a novel micro-vibration actuator that employs a shape-memory alloy wire, we develop a non-invasive screening device of the level of diabetes based on the perception of micro-vibration patterns. Experiments are conducted in a medical clinic, and the relation between the tactile stimuli and the medical diagnosis of diabetes are examined.

The author is developing a talking robot by reconstructing a human vocal system mechanically based on the physical model of human vocal organs. The robotic system consists of motor-controlled vocal organs such as vocal cords, a vocal tract and a nasal cavity to generate a natural voice imitating a human vocalization. By applying the technique of the mechanical construction and its adaptive control, the robot is able to autonomously reproduce a humanlike vocal articulation using its vocal organs. In vocalization, the vibration of vocal cords generates a source sound, and then the sound wave is led to a vocal tract, which works as a resonance filter to determine the spectrum envelope. For the autonomous acquisition of the robot's vocalization skills, an adaptive learning using an auditory feed-back control is introduced. In this manuscript, a human-like expressive speech production by the talking robot is presented. The construction of the talking robot and the autonomous acquisition of the vocal articulation are firstly introduced, and then the acquired control methods for producing human-like speech with various expressions will be described.

With the widespread availability of computers and mobile devices, we use them for daily communication by exchanging visual and verbal/non-verbal media. We obtain various information from multimedia devices, and such devices recently have a touch panel interface to allow a user to intuitively and interactively use them. A touch panel consists of a touch-sensitive panel and a graphical display, and accepts user's intuitive touch inputs together with visual and auditory feedback. With the employment of touch actions in such interface devices, tactile feedback will be expected, if we consider the manipulation of a real object. For presenting tactile feedback, some tactile actuators have been introduced recently, however these actuators require the reference from real objects to generate the appropriate tactile sensations, and the input data to be associated with the output tactile sensations should be precisely prepared in the design of such systems. This study introduces a tactile pen to work with a touch screen, together with an algorithm to automatically generate parameters from an object's picture for driving tactile actuators. A tactile presentation system with a visual display is constructed, and the validity of the texture sensations is verified by a user's experiment.

Information presentation through tactile sensation is actively studied these days, and various tactile displays using different actuators have been introduced by many researchers so far. It is, however, still hard to assume that information presentation through tactile medium would be suitably applied to products for general public. The authors have been developing micro-vibration actuators using shape-memory alloy (SMA) wires to display various tactile sensations, and in this paper an innovative tactile presentation technique for presenting tactile stroking sensations from a touch screen will be introduced. The cutaneous rabbit illusion is introduced firstly, together with its presentation by employing the SMA wire actuators. Then, the construction of a touch screen to present tactile sensation as a feedback against touch and flicking inputs of a user is described.

An innovative tactile pad that presents various tactile sensations synchronizing with moving pictures is presented in this paper. To display tactile/touch sensations from a pad, a tactile display panel is designed by employing 10 micro-vibration actuators that provide tactile-stimulation sources using shape memory alloy wires. With the employment of the human higher-level perception caused by appropriate micro-vibratory stimuli, the resolution of the designed tactile display panel increases more than three times, so that people perceive various tactile sensations with high resolution enough for the better reality. By employing a webcam to capture moving pictures, the tactile pad system automatically recognizes the texture information contained in the scene, which are associated with tactile information to be output from the tactile pad. In order to let people reasonably perceive the presented tactile sensations, the developed system automatically translates visual information to tactile sensation. For verifying the effectiveness of the developed vision-based tactile pad system, experiments are conducted and the results are presented.

In the vocal communication, we employ not only the speech and auditory ability but also the body actions such as gestures and gaze to react to the speech. For example, a human nods to a speaker for an agreement, and makes eye contact during the conversation. The purpose of this study is to realize a human-like auditory system using a robotic arm, which listens to a human voice like a human, and interacts with human speakers. In this paper, the imitation of human listening behavior for realizing the natural communication between a human and a robot by the robotic auditory system is introduced.

The authors have developed a talking robot by reconstructing a human vocal system mechanically based on the physical model of human vocal organs. The robotic system consists of motor-controlled vocal organs such as vocal cords, a vocal tract and a nasal cavity to generate a natural voice imitating a human vocalization. By applying the technique of the mechanical construction and its adaptive control, the robot is able to autonomously reproduce a human-like vocal articulation using its vocal organs. In vocalization, the vibration of vocal cords generates a source sound, and then the sound wave is led to a vocal tract, which works as a resonance filter to determine the spectrum envelope. For the autonomous acquisition of the robot's vocalization skills, an adaptive learning using an auditory feedback control is introduced. In this study, a human-like expressive speech production by the talking robot is introduced. A human generates speech by controlling their own vocal organs for changing the speech expressions such as the volume and the intonations. To realize the human-like expressive speech production, a control method for the mechanical organs has to be established. In the first part of the paper, the construction of the talking robot and the acquisition of the vocalization will be described, and then the control method for producing human-like speech with various expressions will be introduced.

Humans communicate with each other by using not only verbal media but also the five senses such as vision, audition, olfaction and tactile sensations. Various devices and systems have been introduced for supporting human communication, and most of them are based on visual and auditory media. For presenting tactile sensations, some tactile actuators have been introduced recently, however these actuators require real objects to generate the various tactile sensations and input data to be associated with output tactile sensations should be prepared manually by a user. This study introduces an algorithm to automatically generate parameters from an object's image for driving tactile actuators. A tactile presentation system is constructed, and the validity of the texture sensations is verified by a user's experiment. © 2013 IEEE.

Diabetes mellitus is a group of metabolic diseases which causes high blood sugar to a person, due to the functional problems of the pancreas or the metabolism. For a person with diabetes, glucose would not be used properly in the metabolic system, and cause health problems, such as polyphagia, polyuria and polydipsia, since glucose is the main source of energy for the body cells. Symptoms of untreated diabetes would be damaged by the high blood sugar in vessels, and this starts to destroy capillary vessels to lower the sensitivity of tactile sensations, then effects to various organs and nerve systems. The authors pay attention to the decline of the sensitivity of tactile sensations associated with diabetes, and develop a non-destructive screening method of the level of diabetes using a novel micro-vibration actuator that employs a shape-memory alloy wire. The actuators are arranged in an array, and various tactile stimuli are generated by just controlling the driving current signals. The tactile stimuli are presented to symptom's, index and middle fingers., and he/she responds to the stimuli by answering how they are perceived. We made an experiment in a medical clinic., and examined the relation between the Hb Al C values and tactile stimuli. © 2013 IEEE.

This paper introduces a support system for inputting electronic medical records based on the automatic voice recognition. The system is being constructed by the combination of an electronic medical record that has the ontology-based structure, and a voice recognition engine, and is expected to work in the actual environment. In our previous study, we found that environmental noises caused the failure of the voice recognition, and resulted in the low recognition rate to some subjects. In this study, a recognition experiment was conducted to examine the influence of an acoustic noise in a real situation. Environmental noises were recorded in a medical clinic, which were used in the voice recognition experiment. We found that the noise greater than 20 [dB] in the S/N ratio caused the failure of the voice recognition. In the future work, by employing a noise reduction technique in the voice input part, we further examine the recognition ability.

Mangosteen export generates large revenue
however, translucent mangosteens, which contain undesirable internal condition, result in the shipment rejection and decrease the reliability of the export. This research investigates a novel non-destructive classification approach based on acoustic frequency response to detect mangosteens containing translucent fleshes. The set of uniform-distributed multi-frequency acoustic signal is generated and passed through each mangosteen under the test. The frequency responses, describing a feature space, for all mangosteens are computed via the discrete Fourier transform. To prevent intensive computation, a linear optimization is adopted to select relevant frequency contents, creating a compact classifying feature vector. To solve the classification problem, two proposed acoustic-based classification techniques are studied, namely linear classifier (LC), and non-linear classifier (NLC) based on an artificial neural network. Then the results from both classifiers are compared against the results from the conventional water-floating (WF) approach. Against the experimental data, it is found that the average flesh classification accuracy of good mangoteens achieved from the LC and the NLC are about 61% and 74% respectively, while the WF yields an accuracy of about 69%. It is evident that the acoustic-based approach possesses the convincing accuracy for solving the problem of export-grade translucent mangosteen classification. In addition, the paper shows that a mangosteen's physical density can possibly provide intuitive information for better classification performance in the future research study. © 2012 IEEE.

A talking and singing robot which adaptively learns the vocalization skill by an auditory feedback learning is being developed. The fundamental frequency and the spectrum envelope determine the principal characteristics of a sound. The former is the characteristics of a source sound generated by a vibrating object, and the latter is operated by the work of the resonance effects. In vocalization, the vibration of vocal cords generates a source sound, and then the sound wave is led to a vocal tract, which works as a resonance filter to determine the spectrum envelope. The paper describes the construction of vocal cords and a vocal tract for the realization of a talking and singing robot, together with the control algorithm for the acquisition of singing performance by mimicking human vocalization and singing voices. Generated voices were evaluated by listening experiments.

In the research of augmented reality, many experimental systems have been introduced so far by presenting CG objects into a real environment with visual and auditory information, so that a user would interact with the CG objects and the environment. To present higher reality, not only visual and auditory information but also tactile and other stimuli should be presented to users to enhance the perception. We have paid attention to the human tactile sensations, and have been constructing an interactive system, which presents a tactile sensation in addition to visual and auditory sensation. The system is constructed by a head mounted display (HMD), audio speakers, a microphone, two tactile gloves and a PC. A user wares a HMD, and a CG character is superposed on his palm wearing the tactile glove. Stepping sounds of the character are output from the speakers, and various tactile stimuli are presented from the glove according to the motion of the character. The user is also able to interact with the character using the other hand by pushing and picking the object. This paper presents the gestural interaction with a virtual character employing a tactile glove, together with the evaluation of the tactile sensations by the users' experiments.

In this study, a new Braille display is constructed based on the vibration of a Shape Memory Alloy (SMA) wire. To present Braille information, a vibration actuator is used instead of conventional Braille dots. According to the temperature-dependent characteristic shrinkage and the vest ration to the initial length of an SMA wire, a vibration actuator which can be driven by several [Hz]-100 [Hz] pulse signals has been developed. A method for presenting Braille information is proposed by the application of the developed actuator to a Braille display. In this research, multiple actuators constructed by using metal pins (0.7 [mm] in diameter, 3 [mm] in length) and SMA wires (50 [um] in diameter, 3 [mm] in length), are placed as to form standard Braille. The actuators are vibrated by PWM signals with different frequencies and appropriate timings, and the effectiveness of the proposed method for Braille display is verified by experiments. From the experimental results, the highest recognition rate of 100 [%] was achieved under the conditions of 50 [Hz] vibration frequency and 500 [ms] time delay. This means the vibration patterns effectively stimulated the Meissner corpuscles, and helped the subjects properly discriminate Braille characters presented by the developed display. Good evaluations were received from the subjects who are visually-impaired. © 2012 by JSME.

A talking and singing robot which adaptively learns the vocalization skill by an auditory feedback learning is being developed. The fundamental frequency and the spectrum envelope determine the principal characteristics of a sound. The former is the characteristics of a source sound generated by a vibrating object, and the latter is operated by the work of the resonance effects. In vocalization, the vibration of vocal cords generates a source sound, and then the sound wave is led to a vocal tract, which works as a resonance filter to determine the spectrum envelope. The paper describes the construction of vocal cords and a vocal tract for the realization of a talking and singing robot, together with the control algorithm for the acquisition of singing performance by mimicking human vocalization and singing voices. Generated voices were evaluated by listening experiments.

A novel portable Braille display for mobile use is constructed by employing the micro-vibrations generated by small Shape Memory Alloy (SMA) wires. According to the characteristics of the shrinkage and expansion of a SMA wire formed into a diameter with 50 micro-meters, a vibration actuator which can be driven by several to 300 hertz pulse signals has been developed. To present Braille information for the blind, the vibration actuator is used instead of conventional Braille dots. A new method for presenting Braille information is proposed to apply the developed actuator into Braille display. The effectiveness of the proposed Braille display is verified by experimental results.

Feature extraction is a crucial step for many systems of face detection and facial expression recognition. In this paper, we present edge-based feature extraction for recognizing six different expressions, which are angry, fear, happy, neutral, sadness and surprise. Edge detection is performed by using Gabor wavelet and convolution filters. In this paper we propose two convolution kernels that are specific for the edge detection of facial components in two orientations. In this study, Principal Component Analysis (PCA) is used to reduce the features dimension. To validate the performance of our proposed feature extraction, the generated features are classified using Support Vector Machine. The experimental results demonstrated that the proposed feature extraction method could generate significant facial features and these features are able to be classified into each expression.

This paper presents the method of autonomous voice acquisition for a talking robot by applying a dual-SOM. We have so far developed the talking robot, which has mechanical organs as a human. By applying an auditory feedback control, the robot autonomously learns the vocalization skill. For the autonomous learning method, a Self Organizing Neural Network (SONN) by combining a Self-Organizing Map (SOM) with a Neural Network (NN) was employed. The SONN had 2-dimensional mapping space, which was used to locate phonetic features of voices generated by the robot. By choosing cells on the map, voice articulations were autonomously recreated. However, due to the spatial restriction of the map, the voice transition from one vocal sound to another was not always recreated properly. To solve the problems, a dual-SOM, the combination of a phonetic-SOM and a motor-SOM, both of which have 3 dimensional mapping spaces, is introduced. The structure of the dual-SOM is firstly described, and then acquired vocal sounds are evaluated, together with the analysis of the behavior of the SOM. © 2011 The Japan Society of Mechanical Engineers.

In this paper, an innovative vision based tactile display system is presented. Using web camera to get visual information, the developed system captures the shape of an object and transform it to tactile sensation. In order to let people sense the shape of the captured object, tactile display panel is designed with 25 pin-type actuators to provide stimulation sources by using shape memory alloys. By employing higher-level perception, tactile display panel can represent the shape of the captured object. The effectiveness of the developed vision based tactile display system is verified by experiments. © 2011 Intl Journal of Adv Mechatr.

Facial expression recognition has recently become an important research area, and many efforts have been made in facial feature extraction and its classification to improve face recognition systems. Most researchers adopt a posed facial expression database in their experiments, but in a real-life situation the facial expressions may not be very obvious. This article describes the extraction of the minimum number of Gabor wavelet parameters for the recognition of natural facial expressions. The objective of our research was to investigate the performance of a facial expression recognition system with a minimum number of features of the Gabor wavelet. In this research, principal component analysis (PCA) is employed to compress the Gabor features. We also discuss the selection of the minimum number of Gabor features that will perform the best in a recognition task employing a multiclass support vector machine (SVM) classifier. The performance of facial expression recognition using our approach is compared with those obtained previously by other researchers using other approaches. Experimental results showed that our proposed technique is successful in recognizing natural facial expressions by using a small number of Gabor features with an 81.7% recognition rate. In addition, we identify the relationship between the human vision and computer vision in recognizing natural facial expressions. © ISAROB 2011.

我々はこれまでに，形状記憶合金の細線を微小振動アクチュエータとして利用した触覚ディスプレイを開発し，素材の手触り感覚を呈示できることを見い出した．呈示触覚と実素材との比較実験を行った結果，触覚刺激のみによる呈示では個人差が大きく，同じ条件の刺激を呈示しても人によって異なる素材として認識される場合があることが分かった．人間は物体のテクスチャを触覚によって認識する際，視覚情報や体性感覚も有効に活用している．そこで，視覚ディスプレイ上に表示したテクスチャ画像に対応した触覚刺激を，マウス上に実装した薄型触覚ディスプレイによってユーザの手掌部に呈示するシステムを構築した．本システムは，複数のテクスチャ画像を視覚ディスプレイに表示し，ユーザがマウス操作によって各画像上でカーソルを移動させることにより，その速度に応じた触覚刺激を呈示して，能動的に物体の表面を撫でているかのような感覚を呈示することが可能である．システムの評価実験から，視覚情報が触覚認識に有意に影響を与えていることが示され，また触覚刺激と同時に適切な視覚情報を呈示することで，よりリアリティのある触覚感覚の呈示が可能となることが分かった．The authors have developed a tactile display using shape memory alloy wires, and constructed a presentation system of various texture sensations. However, different subjects perceive different sensations by one tactile stimulus generated by certain conditions. A human recognizes a texture of an object by referring not only to the tactile sensations, but also to the visual and other physical sensations. This paper introduces a texture presentation system, which consists of the tactile display and a visual display, and describes the evaluation of the system and effects of visual stimuli to texture sensations by several experiments.

The authors have developed a vocalization training system for the auditory impaired using a talking robot. The training system mainly consists of a talking robot that has mechanical organs like a human. With an adaptive learning strategy using an auditory feedback control, the robot autonomously learns the vocalization, and then reproduces the speech articulation from inputted sounds. In the previous system, the speech-learning algorithm of the robot was constructed by employing a self-organizing neural network (SONN), which consists of the combination of a self-organizing map (SOM) and a neural network (NN). However, improper maps were occasionally generated in the results of the speech articulation learning. To solve this problem, a new algorithm introducing two three-dimensional SOMs, called a dual-SOM, was employed for the autonomous learning of the robotic articulations. By applying the robot and its properties, we have constructed an interactive training system. The training is divided into two approaches
one is to use the talking robot to show the shape and the motion of the vocal organs, and the other is to use a topological map to present the difference of phonetic features of a trainee's voices. In this study, first, the construction of the training system is described together with the autonomous learning of robotic vocalization using the dual-SOM algorithm, and then the analysis of the speech training progress is presented based on the phonetic features and the mechanical vocal articulations. © 2011 by Walter de Gruyter · Berlin · New York.

A talking robot which adaptively learns the vocalization skill by an auditory feedback learning is being developed. The talking robot is constructed by the mechanical vocal systems, which respectively correspond to human organs. In vocalization, the vibration of vocal cords generates a source sound, and then the sound wave is led to a vocal tract, which works as a resonance filter to determine the spectrum envelope. The talking robot autonomously learns the vocalization by applying auditory feedback. In this study, a human-like expressive speech production by the talking robot is introduced. A human generates speech by controlling their own vocal organs for changing the speech expressions such as the volume and the intonations. To realize the human-like expressive speech production, a control method for the mechanical organs has to be established. In the first part of the paper, the construction of the talking robot and the acquisition of the vocalization will be described, and then the control method for producing human-like speech with various expressions will be introduced.

In the human communication, auditory and speech functions play an important role for recognizing someone to talk to and understanding the conversations. Various vocal sounds are generated by the complex movements of speaker's vocal organs, and this mechanism contributes to generate different voices that include speech expressions and speaker's individuality. This paper introduces several approaches to speech and auditory systems for introducing flexible interactions between humans and systems, which have been constructed by the authors.

In order to provide a portable Braille display for the blind and visually impaired, a novel compact device is developed by using shape memory alloy (SMA) wires. In details, according to the ability of transition between Martensite phase and the Austenite phase, SMA can expand and shrink depending on temperature. Using movement of SMA under different temperature, six compact SMA actuators which are settled in ring-like bandage and fixed on human fingers, are vibrated to stimulate human tactile sensation. Depending on optimal displacement, these actuators can present six-dot Braille and make the blind and visually impaired get information from the device. Experimental results are shown to verify the effectiveness of the developed portable Braille display.

A touch screen is an electronic visual display that can detect the presence and location of a touch within the display area surface. The term generally refers to stressing the display area of the device with a finger of a hand. A common type of touch screen only receives a user's finger stress as an input. However, the user himself is not able to feel the presence of the object displayed on the screen such as a displayed button image on the screen. This paper introduces the development of button repulsive sensations displayed by the utilization of micro-vibrations generated from a shape memory alloy wire on a touch screen interface as a tactile display actuator. © 2011 IEEE.

In the research field of augmented reality, many experimental systems have been introduced so far to provide natural interactions with a virtual character and a surrounded environment. To introduce a high level interaction, not only visual and auditory information but also other human sensations such as tactile sensations and somatosensory information should be presented to a user. In this study, we pay attention to the tactile sensation in the manipulation of a virtual character, and develop TactoGlove for presenting tactile sensations to a user. The virtual environment is manipulated by intuitive gestures, and tactile sensations corresponding with the gestural actions are presented to the user through TactoGove to establish multimodal interactions. Four subjects tried the system, and significant evaluations were obtained in the interaction between a virtual character and a user. © 2011 IEEE.

Facial expressions are an important channel of nonverbal communication. Currently, many facial expression analysis or recognition systems have been proposed. In this paper, a study of dimension reduction of Gabor features from different facial expressions is presented. Principle Component Analysis (PCA) is used as dimension reduction method. The experiment is conducted by using samples of face image for eight subjects. There are six facial expressions; anger, fear, happy, neutral, sadness and surprise are used in this study. In this experiment, we use different poses and head postures of each subject. Experiment results demonstrated the reduced dimensions of Gabor features could be effectively used in the next processing for recognizing facial expressions. © 2010 ISAROB.

A human has various sensory perceptions, and effectively uses them in communication. Auditory and visual functions especially play an important role for recognizing someone to talk to and understanding the conversation. In vocal communication, we are able to detect the position of a source sound in 3D space, extract a particular sound from mixed sounds, and recognize who is talking. In addition, we are able to detect a particular person by recognizing body features and individual gestures. By realizing this mechanism using a computer, new applications will be presented, which are utilized in the flexible and intuitive communication with humans. The authors are working for the identification of a particular person using microphones and a USB camera. The paper describes the development of an information fusion system and how to deal with multiple data obtained by different sensors.

A talking and singing robot which adaptively learns the vocalization skill by an auditory feedback learning is being developed. The fundamental frequency and the spectrum envelope determine the principal characteristics of a sound. The former is the characteristic of a source sound generated by a vibrating object, and the latter is operated by the work of the resonance effects. In vocalization, the vibration of vocal cords generates a source sound, and then the sound wave is led to a vocal tract, which works as a resonance filter to determine the spectrum envelope. The paper describes the construction of vocal cords and a vocal tract for the realization of a talking and singing robot, together with the control algorithm for the acquisition of singing performance by mimicking human vocalization and singing voices. Generated voices were evaluated by listening experiments.

A talking and singing robot which adaptively learns the vocalization skill by an auditory feedback learning is being developed. The fundamental frequency and the spectrum envelope determine the principal characteristics of a sound. The former is the characteristic of a source sound generated by a vibrating object, and the latter is operated by the work of the resonance effects. In vocalization, the vibration of vocal cords generates a source sound, and then the sound wave is led to a vocal tract, which works as a resonance filter to determine the spectrum envelope. The paper describes the construction of vocal cords and a vocal tract for the realization of a talking and singing robot, together with the control algorithm for the acquisition of singing performance by mimicking human vocalization and singing voices. Generated voices were evaluated by listening experiments.

In the study of virtual reality and mixed reality, the developments of an input device and user interface are important for the interaction with virtual objects and environment. However, a mouse or a joystick is commonly used in present systems. If we could manipulate a virtual object intuitively by using natural gestures without any sensors or devices, the system would contribute to present reality. In this paper, we introduce a technique to let a user move his hand in front of a camera to manipulate a virtual object, and the gestural movements are recognized in real-time to establish an interactive communication with computational systems. © 2009 SICE.

Shape-memory alloys (SMAs) are the metals which have the shape-memory effect. An alloy has peculiar temperature Tp, and the shape memory effect is observed when the body temperature is cooled to below Tp. The effect has been widely applied in different fields such as robotic muscles, medical surgeries, actuators and elastic wires. The SMA also has a unique characteristic to shrink at a certain temperature. The authors found a novel and interesting effect that, by making the SMA into a fine wire, it accepts a pulse-signal to generate a vibration in accordance with the pulse frequency of the signal, and have developed a vibration actuator for presenting tactile stimuli. By coupling the devices as a pair, a tactile display was constructed for presenting the phantom sensation and the apparent movement of the tactility, to transmit quite novel tactile sensations to a user. The information transmission by the device was tested by subjects, and was evaluated by questionnaires. The apparent movement was especially well perceived by users as a sensation of something running across fingers and a palm, or as being tapped by something, according to the well-determined signals given to the display. Several users reported that they perceived a novel rubbing sensation given by the apparent movement, and we further experimented the presentation of the sensation in detail to be used as a tactile display for the information transmission.

A novel micro-vibration actuator using a shape-memory alloy (SMA) wire is being developed for the presentation of tactile sensations to a human skin. In this study, the SMA wire is arranged two dimensionally to construct a compact Braille display for the blind. The display is driven by electric current pulse for mobile use to present Braille information. The display was evaluated in a school for the blind, and the results are presented in the paper.

The use of human motions for the interaction between humans and computers is becoming an attractive alternative to verbal media, especially through the visual interpretation of the human body motion. In particular, hand gestures are used as non-verbal media for the humans to communicate with machines that pertain to the use of the human gestures to interact with them. This paper introduces a 3D motion measurement of the human upper body for the purpose of the gesture recognition, which is based on the probability distribution of arm trajectories. In this study, by examining the characteristics of the arm trajectories given by a signer, motion features are selected and classified by using a fuzzy technique. Experimental results show that the use of the features extracted from arm trajectories effectively works on the recognition of dynamic gestures of a human, and gives a good performance to classify various gesture patterns.

本稿では，形状記憶合金糸を振動アクチュエータとして利用した新しい触覚ディスプレイを提案し，各アクチュエータを駆動するパルス信号を，その発生確率密度によって制御して触覚感覚を呈示する手法について述べる．著者らは糸状に加工した形状記憶合金が温度により伸縮する特性を利用し，数Hz～数100 Hzのパルス信号によって微小振動を発生する小型アクチュエータを開発し，これを触覚呈示に用いる手法を提案してきた．本研究では，複数個の形状記憶合金糸のアクチュエータを面状に配置した触覚ディスプレイの構築を行った．本アクチュエータは，パルス信号を与えることによってその周期に同期して微小振動を発生するが，ランダム発生パルスによって各アクチュエータを駆動することにより，様々な触覚感覚を呈示できることが分かった．さらに，パルス信号の発生確率密度を時間的に変化させることにより，面のテクスチャや，物体をなぞった感覚を生成できることを明らかにした．本稿では，まずこれらの触覚感覚の呈示手法について述べ，実験による本ディスプレイの有効性の評価について述べる．The paper introduces the development of a compact tactile display employing novel micro-vibration actuators using a shape-memory alloy, and describes the presentation of various tactile sensations to a human skin by the control of pulse-signal probability density given to the vibration actuators. The authors have paid attention to the super-elasticity characteristics of a shape-memory alloy formed into a thread, and have so far developed a thin tactile device by driving each actuator with pulse current signal for generating vibrations. In this study, we found novel tactile sensations presented by driving actuators with randomly generated pulse signals. This paper describes the generation of touch feelings such as texture information of objects and a rubbing sensation by controlling the signal probability density, which was evaluated by the users&#039; experiments.

A novel micro-vibration actuator using a shape-memory alloy for the presentation of various tactile sensations to a human skin is introduced in this paper. The authors paid attention to the characteristics of a shape-memory alloy formed into a thread, which changes its length according to its temperature, and developed a vibration-generating actuator electrically driven by periodic signals generated by current control circuits. For the tactile information transmission to a human body, the human higher-level perception such as the phantom sensation and the apparent movement of the tactility is employed in consideration of presenting novel tactile sensations. By coupling the actuators as a pair, an information transmission is realized for presenting novel tactile sensations to a user. The information transmission was tested by changing various conditions of electric signals to drive actuators, and was evaluated by questionnaires. Several users reported that they perceived a novel rubbing sensation given by the apparent movement, and we further experimented the presentation of the sensation in detail to be used as a tactile display for the information transmission through our body.

A human is able to exchange information smoothly using voice under different situations such as a noisy environment in a crowd and with the existence of multiple speakers. We are able to detect the position of a source sound in three-dimensional space, extract a particular sound from mixed sounds, and recognize who is talking. By realizing this mechanism with a computer, new applications will be presented for recording a sound with high quality by reducing noise, presenting a clarified sound, and realizing microphone-free speech recognition by extracting particular sounds. The paper will introduce real-time detection and identification of a particular speaker in a noisy environment using a microphone array based on the location of a speaker and the individual voice characteristics. The study will be applied to develop an adaptive auditory system for a mobile robot which collaborates with a factory worker. (c) 2007 Wiley Periodicals, Inc.

The use of human motions for the interaction between humans and computers is becoming an attractive alternative, especially through the visual interpretation of the human body motion. In particular, hand gesture is used as a non-verbal media for the humans to communicate with machines that pertains to the use of human gesture to interact with them. Recently, many studies for recognizing the human gesture have been reported, and most of them deal with the shape and motion of hands. This paper introduces dynamic gesture recognition based on the arm trajectory and fuzzy algorithm approach. In this study, by examining the characteristics of the human upper body motions of a signer, motion features are selected and classified by using the fuzzy technique. Experimental results show that the use of the features extracted from the upper body motion effectively works on the recognition of the dynamic gesture of a human, and gives a good performance to classify various gesture patterns.

A talking and singing robot which adaptively learns the vocalization skill by means of an auditory feedback learning algorithm is being developed. The robot consists of motor-controlled vocal organs such as vocal cords, a vocal tract and a nasal cavity to generate a natural voice imitating a human vocalization. In this study, the robot is applied to the training system of speech articulation for the hearing-impaired, because the robot is able to reproduce their vocalization and to teach them how it is to be improved to generate clear speech. The paper briefly introduces the mechanical construction of the robot and how it autonomously acquires the vocalization skill in the auditory feedback learning by listening to human speech. Then the training system is described, together with the evaluation of the speech training by auditory impaired people. Copyright (C) 2008 Hideyuki Sawada et al.

The paper introduces the development of a tactile device using a shape-memory alloy, and describes the information transmission by the higher-level perception such as the phantom sensation and the apparent movement of the tactility. The authors paid attention to the characteristic of a shape-memory alloy formed into a thread, which changes its length according to its temperature, and developed a vibration-generating actuator electrically driven by periodic signals generated by current control circuits, for the tactile information transmission. By coupling the devices as a pair, an information transmission system was constructed for presenting the phantom sensation and the apparent movement of the tactility, to transmit quite novel sensation to a user. The information transmission by the device was tested by subjects, and was evaluated by questionnaires. The apparent movement was especially well perceived by users as a sensation of something running across fingers or as being tapped by something, according to the well-determined signals given to the devices. Several users reported that they perceived a novel rubbing sensation given by the apparent movement, and we further experimented the presentation of the sensation in detail to be used as a tactile display for the information transmission.

In the vocal communication, we are able to detect the position of a source sound in 3D space, extract particular sound from mixed sounds, and recognize what the sound is. An ability that extracts and recognizes particular sound in noisy environment is called cocktail party effect. By realizing this mechanism with a computer, new applications will be presented to be utilized in the communication with humans. In this study, musical sounds are dealt for the tracking of sounds generated by musical instruments. Musical sounds have different features from human voice. For example, instrumental sounds have variety of spectra distributed in wide frequency range. In this study, a system that can deal with various kinds of sounds will be developed. For the estimation of musical pitch, this study focuses on musical sounds with different pitches generated from musical instruments, and a filtering technique using comb filters is being developed. A recorder, harmonica and keyboard-harmonica were selected as target musical instruments in the experiment and the result showed this technique was effective for the estimation of musical pitches.

Humans are able to communicate with each other by using not only verbal media but also the five senses such as vision, audition, olfaction and tactility. Our emotions and feelings are directly transmitted through non-verbal media, especially tactile sensation senses the physical information about an object or an environment. We have paid attention to the information transmission through the higher-psychological perception of tactility, and are now constructing a tactile transmission system for presenting stroking sensation. This paper introduces the development of a tactile display using a shape-memory alloy, and describes the transmission of stroking sensation by the higher-psychological perception such as the phantom sensation (PS) and the apparent movement (AM) of the tactility.

A talking and singing robot which adaptively learns the vocalization skill by an auditory feedback learning is being developed. The robot consists of motor-controlled vocal organs such as vocal cords, a vocal tract and a nasal cavity to generate a natural voice imitating a human vocalization. The robot is applied to the training system of speech articulation for the hearing-impaired people, because the robot is able to reproduce their vocalization and to teach them how it is improved to generate clear speech. The paper briefly introduces the mechanical construction of the robot and how it autonomously acquires the vocalization skill in the auditory feedback learning by listening to human speech. Then the training system is described, together with the evaluation of the speech training by auditory impaired people.

Measurement of human motion is widely required for various applications, and a significant part of this task is to identify motion in the process of human motion recognition. There are several application purposes of doing this research such as in surveillance, entertainment, medical treatment and traffic applications as user interfaces that require the recognition of different parts of human body to identify an action or a motion. The most challenging task in human motion recognition is to achieve the ability and reliability of a motion capture system for tracking and recognizing dynamic movements, because human body structure has many degrees of freedom. Many attempts for recognizing body actions have been reported so far, in which gestural motions have to be measured by some sensors first, and the obtained data are processed in a computer. This paper introduces the 3D motion analysis of human upper body using an optical motion capture system for the purpose of gesture recognition. In this study, the image processing technique to track optical markers attached at feature points of human body is introduced for constructing a human upper body model and estimating its three dimensional motion.

Full automation featured many problems, such as the difficulty of production changes, etc. Eventually, cell production, a human based production system, was introduced in the 1990s. The cell production System, however, features similar problems. Beginning in 2000. advanced cell production systems have been pursued, whereby production is accomplished through the harmonization of human operators and automatic machines. The aim of this study is to take the know-how that inspectors have unconsciously acquired on a daily basis and, transition the inspectors themselves to automated systems. To achieve this aim we established ROI macro commands and developed the ROI macro command system. About 80% of requests from manufacturing sites were fulfilled by using, those 8 types of ROI macro commands. Moreover, 488 sites used it oil an actual basis. As a result, we have concluded that the inspectors' know-how was passed on to external configuration inspections across a wide-range of fields.

The purpose of this study is to develop an information fusion system for the application of human-machine Communication. The information fusion system bundles and controls sensing units distributed in a space. One sensing unit consists of several sensors, whose data are given to a computer and are transformed into significant information. This information is fed to other sensing units, to be fused with other significant information. All the information are integrated in a main computer to recognize and identify a particular person. The main computer is responsible for managing all the information in a distributed fusion system. For a large and complex system, it may consist of several communities of sensors that are Controlled by several main computers to manage the different fusions of the systems. This paper introduces a methodology, oil how to group a several functions from different sensors that share a same environment for the purpose of recognizing a particular person.

Human is able to exchange information smoothly using voice under different situations such as noisy environment in a crowd and with the existence of plural speakers. We are able to detect the position of a source sound in 3D space, extract a particular sound from mixed sounds, and recognize who is talking. By realizing this mechanism with a computer, new applications will be presented for recording a sound with high quality by reducing noise, presenting a clarified sound, and realizing a microphone-free speech recognition by extracting particular sound. The paper will introduce a realtime detection and identification of particular speaker in noisy environment using a microphone array based on the location of a speaker and the individual voice characteristics. The study will be applied to develop an adaptive auditory system of a mobile robot which collaborates with a factory worker.

Human is able to exchange information smoothly using voice under different situations such as noisy environment in a crowd and with the existence of plural speakers. We are able to detect the position of a source sound in 3D space, extract a particular sound from mixed sounds, and recognize who is talking. By realizing this mechanism with a computer, new applications will be presented for recording a sound with high quality by reducing noise, presenting a clarified sound, and realizing a microphone-free speech recognition by extracting particular sound. The paper will introduce a realtime detection and identification of particular speaker in noisy environment using a microphone array based on the location of a speaker and the individual voice characteristics. The study will be applied to develop an adaptive auditory system of a mobile robot which collaborates with a factory worker.

This paper introduces the development of a tactile device using a shape-memory alloy and describes the information transmission by the higher psychological perception, such as the phantom sensation and the apparent movement of the tactility. The authors paid attention to the characteristic of a shape-memory alloy formed into a thread, which changes its length according to its body temperature, and developed a vibration-generating actuator electrically driven by periodic signals generated by current control circuits, for the tactile information transmissioa The size of the actuator is quite compact, and the energy consumption is only 20mW. By coupling the actuators as a pair, an information-transmission system was constructed for presenting the apparent movement of the tactility to transmit a quite novel sensation to a user. Based on the preliminary experiment, the parameters for the tactile information transmission were examined. Then the information transmission by the device was tested by 10 subjects and evaluated by questionnaires. The apparent movement was especially well perceived by users as a sensation of a small object running on the skin surface or as being tapped by something, according to the well-determined signals given to the actuators. Several users reported that they perceived a novel rubbing sensation given by the AM, and we further experimented the presentation of the sensation in detail to be used as a sensory-aids tactile display for handicapped and elderly people. © 2006, by Walter de Gruyter GmbH &amp
Co. All rights reserved.

A talking and singing robot which adaptively learns the vocalization skill by an auditory feedback learning is being developed. In vocalization, the vibration of vocal cords generates a source sound, and then the sound wave is led to a vocal tract, which works as a resonance filter to determine the spectrum envelope. The robot consists of motor-controlled vocal organs such as vocal cords, a vocal tract and a nasal cavity to generate a natural voice imitating a human vocalization. The paper briefly introduces the construction of vocal cords and vocal tract for the realization of the talking robot, and then describes how the robot autonomously acquires the vocalization skill in the auditory feedback learning by listening to human talking and singing voices. The acquired voices were evaluated by listening experiments.

Voice and sounds are the primary media employed for the human communication. Human is able to exchange information smoothly using voice under different situations such as noisy environment in a crowd and with the existence of plural speakers. We are surrounded by various sounds, and are able to detect the position of a source sound in 31) space, extract a particular sound from mixed sounds, and recognize what makes the sounds. The paper will introduce a realtime detection and identification of particular sound among plural sound sources using a microphone array based on the location of a speaker and the tonal characteristics. The algorithm is applied to an adaptive auditory system of a robotic arm for the interaction with humans.

This paper presents a digital filtering algorithm that clarifies dysphonic speech with the speaker&#039;s individuality preserved. The study deals with the clarification of esophageal speech and the speech of patients with cerebral palsy. The filtering ability is evaluated by listening experiments. Over 20,000 patients currently suffer from laryngeal cancer in Japan, and the only treatment for the terminal symptoms requires the removal of the larynx, including vocal cords. The authors are developing a clarification filtering algorithm of esophageal speech, and the primal algorithm of software clarification and its effectiveness was reported in the previous ICDVRAT. Several algorithms for the clarification have been newly developed and implemented and are being evaluated by questionnaires. The algorithms were extended and applied for the clarification of the speech by the patients of cerebral palsy. Copyright © Freund Publishing House Limited.

This article introduces a new device driven by emotional human twiddling gestures, and its application as a musical instrument. In traditional musical instruments, the relationship between the action and the generated sound is determined by the physical structures of the instruments. Although various computerized musical instruments have been actively developed and reported, the development of a human-computer interface is not sufficient for the translation of emotional feelings into sounds. We have so far paid attention to emotional feelings presented in gestures and music, and have developed a gesture acquisition device that measures grasping forces caused by user's twiddling actions by using compact pressure sensors and acceleration sensors. For the association of gesture patterns with sound parameters, a SOM algorithm is applied. The approach is considered to provide not only a new sound generation technique in the music scene, but also a flexible human-computer interface.

In order to improve the link between an operator and its machine, some human oriented communication systems are now using natural languages like speech or gesture. The goal of this paper is to present a gesture recognition system based on the fusion of measurements issued from different kind of sources. It is necessary to have some sensors that are able to capture at least the position and the orientation of the hand such as Dataglove and a video camera. Datagloge gives a measure of the hand posture and a video camera gives a measure of the general arm gesture which represents the physical and spatial properties of the gesture, and based on the 2D skeleton representation of the arm. The measurements used are partially complementary and partially redundant. The application is distributed on intelligent cooperating sensors. The paper presents the measurement of the hand and the arm gestures, the fusion processes, and the implementation solution.

A talking and singing robot which has equivalent mechanical organs to human vocal system is being developed based on a mechatronics technology under a feedback control. While various ways of vocal sound production have been actively studied so far, a mechanical construction of the vocal system is considered to advantageously realize natural vocalization with its fluid dynamics. Motors are employed for the manipulation of the mechanical system. The robot adaptively learns the relations between motor control parameters and the generated vocal sounds using an auditory feedback learning with neural networks, and sings a song by mimicking a human vocalization. This paper presents the construction of the talking robot and its singing performance, together with the adaptive control for the pitch and phoneme learning. The robot generates vowel and consonant sounds of different pitches by dynamically controlling the vocal cords, vocal tract and nasal cavity.

This paper presents a digital filtering algorithm that clarifies dysphonic speech with the speaker's individuality preserved. The study deals with the clarification of esophageal speech and the speech of patients with cerebral palsy. The filtering ability is evaluated by listening experiments. Over 20,000 patients currently suffer from laryngeal cancer in Japan, and the only treatment for the terminal symptoms requires the removal of the larynx, including vocal cords. The authors are developing a clarification filtering algorithm of esophageal speech, and the primal algorithm of software clarification and its effectiveness was reported in the previous ICDVRAT. Several algorithms for the clarification have been newly developed and implemented and are being evaluated by questionnaires. The algorithms were extended and applied for the clarification of the speech by the patients of cerebral palsy. © 2005, by Walter de Gruyter GmbH &amp
Co. All rights reserved.

A mechanical model of the human vocal system is I being developed based on a mechatronics technology under a feedback control. While various ways of vocal sound production have been actively studied so far, a mechanical construction of the vocal system is considered to advantageously realize natural vocalization with its fluid dynamics. Motors are employed for the manipulation of the mechanical system. The system is able to adaptively learn the relations between motor control parameters and the produced vocal sounds using an auditory feedback with neural networks, by mimicking a human vocalization. This paper presents the construction of a talking robot and its singing performance, together with the adaptive control for the pitch learning. The talking robot generates vowel and consonant sounds of different pitches by dynamically controlling the vocal cords, vocal tract and nasal cavity.

Nowadays agile market is in common, and the fundamental technology supporting next-generation production system requires further development of machine and information technologies to establish “human technology” and a bridging of these technologies together. IMS-HUTOP project proposes a new product life cycle that respects the human nature of individuals, and establishes the elemental technologies necessary for acquiring, modelling and evaluating various human factors in an effort to achieve the HUTOP cycle. In this paper we propose a human centred KANSEI manufacturing system, which has been proposed in the IMS-HUTOP project with 5 work packages. © 2004, The Institute of Electrical Engineers of Japan. All rights reserved.

A system to detect a car by measuring an automobile sound is Introduced. The time difference between the signals taken from plural microphones is measured by using a specially-designed microphone array. The sound source location is estimated based on the calculation of the cross-correlation, and a particular car is identified. The measurement was tested la different noisy environments far the practical use, and fair results were obtained.

Human is able to exchange information smoothly using voice under different situations such as noisy environment in a crowd and with the existence of plural speakers. We are able to detect the position of a source sound in 3D space, extract a particular sound from mixed sounds, and recognize who is talking. By realizing this mechanism with a computer, new applications will be presented for recording a sound with high quality by reducing noise, presenting a clarified sound. and realizing a microphone-free speech recognition by extracting particular sound. The paper will introduce a realtime detection and identification of particular speaker in noisy environment using a microphone array based on the location of a speaker and the individual voice characteristics. The study will be applied to develop an adaptive auditory system of a mobile robot which collaborates with a factory worker.

A talking and singing robot is being developed based on the mechanical model of the human vocal system by using mechatronics and feedback control technologies. While various ways of vocal sound production have been actively studied so far, a mechanical construction of the vocal system is considered to advantageously realize natural vocalization with its fluid dynamics. Motors are employed for the manipulation of the mechanical system. The robot is able to adaptively learn the relations between motor control parameters and the produced vocal sounds using an auditory feedback with neural networks, by mimicking a human vocalization. This paper presents the construction of the robot and its taking and singing performance, together with the adaptive control for the pitch and phoneme learning. The robot generates vowel and consonant sounds of different pitches by dynamically controlling the vocal cords, vocal tract and nasal cavity.

A display device called intuition-driven monitor will be introduced. The device has been developed by combining a LCD, a flexible arm suspending the display, angle sensors and a CCD camera, and is directly manipulated by a user with his direct operations by hands, eye-gaze and facial expressions. A user is able to explore virtual space and manipulate virtual environment with gestures and the sense of haptics.

Voices are used as primary media in human communication. A human voice is a sound generated by the complex movements of the vocal organs, and is the most important media employed for the communication in the day life to logical discussions. A human is able to detect the position of a source sound in 3D space by perceiving the time difference reaching to the both ears. Furthermore we can selectively listen to an objective voice in the crowds. This paper presents a realtime detection and identification of a particular person among plural speakers using a microphone array. The system identifies the position of a particular speaker, and enhances the voice signal selectively.

The gesture recognition needs to take into account very the fusion of hand shape different information. It needs toper and motion information. This paper presents a solution using a fuzzy sensor approach for the aggregation of such measurement information. The motion recognition is based on a measurement represented by numerical fuzzy subsets, and the shape recognition is based on measurements represented by lexical fuzzy subsets.

A display device called intuition-driven monitor will be introduced. The device has been developed by combining a LCD, a flexible arm suspending the display, angle sensors and a CCD camera, and is directly manipulated by a user with his direct operations by hands, eye-gaze and facial expressions. A user is able to explore virtual space and manipulate virtual environment with gestures and the sense of haptics.

A mechanical model of a human vocal system is being developed based on mechatronics technology. Although various ways of vocal sound production have been actively studied, mechanical construction is considered to advantageously realize natural vocalization with its fluid dynamics. The mechanical vocal system has several motors to manipulate the vocal tract and the vocal cords. It became possible to learn the relations between motor positions and the produced vocal sounds by an auditory feedback, and produce Japanese five vowels (a, i, u, e, o) by mimicking a human speech. In addition, the mechanical model could produce some consonant sounds by attaching a nasal cavity with the dynamic control. This paper introduces an adaptive learning algorithm for the mimicry of human vocalization, and presents a listening experiment of generated sounds for the evaluation.

In the human to human communication, we are able to communicate with each other by using not only verbal media but also the five senses. With the widespread availability of computers, the media which directly affect our emotions will be effectively utilized together with the audio and images. In this paper a display device called intuition-driven monitor will be introduced together with two application systems, in which a user is able to directly manipulate a virtual 3D object with the haptic senses.

Humans are able to communicate with each other by using not only verbal media but also the five senses. A display device called intuition-driven monitor has been developed by combining a LCD display, flexible arm, angle sensors and a CCD camera. The display is manipulated by a user with his direct operations by hands and gaze. The joint angles are measured by angle sensors, and the three dimensional position and posture of the display are calculated in real time by the computation of the system geometry. A user is able to. explore virtual space and manipulate virtual environment with the sense of haptics.

We are developing a mechanical model of a human vocal system based on a mechatronics technology. Although various ways of vocal sound production have been actively studied so far, a mechanical construction is considered to advantageously realize natural vocalization with its fluid dynamics. In the voice generation, the analysis of the behaviors of the vocal cords and the vocal tract are required to be employed in the mechanical system. Furthermore, the fluid mechanics is less stable to make the control difficult. Several motors are employed to manipulate the mechanical vocal system. A neural network works to establish the relations between motor positions and the produced vocal sounds by an auditory feedback in the learning phase. In the speech performance, the mechanical system is able to utter while vocal pitches and phonemes are adaptively controlled by an auditory feedback control. This paper presents the construction of a vocal tract part and its adaptive control by the neural networks.

This paper describes two humanoid robots developed in the Humanoid Robotics Institute, Waseda University. Hadaly-2 is intended to realize information interaction with humans by integrating environmental recognition with vision, conversation capability (voice recognition, voice synthesis), and gesture behaviors. It also possesses physical interaction functions for direct contact with humans and behaviors that are gentle and safe for humans. WABIAN is a robot with a complete human configuration that is capable of walking on two legs and carrying things as with humans. Furthermore, it has functions for information interactions suite for uses at home.

This paper introduces a mechanical model of human vocal system to realize human-like speech under an auditory feedback control. While various ways of vocal sound production have been actively studied so far, we are constructing a phonetic machine having a vocal chord and a vocal tract with an auditory system. By imitating the human learning procedure of voice acquisition, a mechanical construction of a human vocal system is considered to generate natural voice so that it can be advantageously applied to a singing voice production.

This paper introduces a mechanical model of human vocal system to realize human-like speech under an auditory feedback control. While various ways of vocal sound production have been actively studied so far, we are constructing a phonetic machine having a vocal chord and a vocal tract with an auditory system. By imitating the human learning procedure of voice acquisition, a mechanical construction of a human vocal system is considered to generate natural voice so that it can be advantageously applied to a singing voice production.

Various ways of vocal sound production are being actively studied. We are constructing a phonetic machine with a vocal chord and a vocal tract based on mechatronics technology. Mechanical construction of a human vocal system is considered to generate natural voice so that it can be advantageously applied to singing voice production. In voice generation, analysis and mechanical realization of the behaviors of the vocal chords and vocal tract are required. Furthermore. the fluid mechanical system is less stable, thus making control more difficult. Several motors are employed to manipulate the mechanical vocal system. Mappings between motor positions and the produced vocal sounds are automatically established in the learning phase. In the singing performance, the system is able to sing while vocal pitches and phonemes are adaptively controlled by an auditory feedback process. This paper presents the latest mechanisms of our mechanical vocal system together with adaptive tuning algorithms of the physical mechanism with an auditory system.

Various ways of vocal sound production are being actively studied. We are constructing a phonetic machine with a vocal chord and a vocal tract based on mechatronics technology. Mechanical construction of a human vocal system is considered to generate natural voice so that it can be advantageously applied to singing voice production. In voice generation, analysis and mechanical realization of the behaviors of the vocal chords and vocal tract are required. Furthermore. the fluid mechanical system is less stable, thus making control more difficult. Several motors are employed to manipulate the mechanical vocal system. Mappings between motor positions and the produced vocal sounds are automatically established in the learning phase. In the singing performance, the system is able to sing while vocal pitches and phonemes are adaptively controlled by an auditory feedback process. This paper presents the latest mechanisms of our mechanical vocal system together with adaptive tuning algorithms of the physical mechanism with an auditory system.

The importance of human-machine interface is widely acknowledged in accordance with the increase of the computer powers. In human to human communication, we communicate with each other by using not only verbal media but also non-verbal media. Especially information transmitted through the five senses is able to affect our emotions and feelings directly making for smooth communication. This paper introduces the development of a new haptic device driven by hand gestures. Since the device is molded with silicon-rubber with the softness of human skin, a user is able to make hand gestures with his &#039;sense of force&#039; by the transforming manipulations. The device is able not only to sense user&#039;s twiddling gestures but also to display reactions against the hand gestures by vibration patterns. As an experimental trial of a tele-communication system, the device was employed for terminal devices, and questionnaires were conducted to evaluate the device in the haptic communication.

In the fully customer-oriented design system, design candidates should be evaluated if they reflect appropriately the ambiguous preferences and demands intended by customers and designers or not. The researches in the total human-oriented design system have come up with prototype systems that have functions capable not only of providing an environment for products, but for displaying and evaluating products as well. There are many researches in this field. Here, the technologies to extract the personal room environment models and to control the display system by users&#039; gesture have been introduced.

A future system for flexible manufacturing must be a sensor-complex for multi-modality with high intelligence, since such system should work with human who has a variety of expression ways and acceptance channels. This paper describes three topics from our recent researches on gesture and haptic interfaces which can he introduced for human-friendly interface in designer-consumer cooperative production system. The first one is a gesture information acquisition method by using image processing technique. The second one is another gesture recognition system by employing accelerometers to measure the forces applied to the body. A new haptic-communication device is introduced as the third topic of the paper It is able not only to detect grasping force and the device orientation but also to display the tactile sensation using the vibration motors to realize human-machine and human-human haptic communications.

Gesture plays an important role in our daily life as nonverbal media for emotional human communication. And the gesticulation is a necessary method of communication among the people who are hearing impaired as sign language. The arm motion and the hand figure are considered to be the primitive information in understanding gestures. This paper proposes a gesture recognition algorithm based on motion primitives and hand figure primitives extracted from acceleration sensors, position sensors and datagloves, to understand human dynamic movements and hand postures. The sensor integration method realized the robust gesture recognition in spite of the simple algorithm. The algorithm is applied to a sign-language recognition system for the realization of new human-machine interface through gestures. We also propose a sign language database based on the motion primitives and hand figure primitives.

ジェスチャと音響, 音声は, 人間同士のコミュニケーションにおいて深いかかわりをもっている.音響の合成にはこれまでさまざまな手法が提案され実用化されてきたが, 合成のためのパラメータの扱いの煩雑さや, 波形計算時間に起因するレスポンスの悪さなどの問題点も見られる.またジェスチャは音楽演奏や音楽パフォーマンスにおいて特によく利用されており, これまでにも人間の動作により音楽演奏を制御するシステムの試みも多く見られる.ここではハンドジェスチャに着目し, ハンドジェスチャセンシングデバイスの開発とそれを用いた音響生成システムについて報告する.音響の生成には, 音源と共鳴フィルタ特性が大きくかかわっているが, これらを自由なジェスチャによってコントロールすることが可能である.ジェスチャの計測量と音響生成パラメータの対応付けには, 自己組織化マップを用いている.提案したジェスチャセンシングデバイスは, 音響生成, 制御への応用ばかりではなく, 柔軟で感性的なマン・マシンインタフェースとしての利用が期待できる.

Although algorithmic sound syntheses, such as a granular synthesis and a physical model based synthesis, have been studied actively and has come in practical use, the development of human-machine interface is not sufficient for the translation of emotional feelings into sound. This paper presents two sound generation systems in which sounds are generated by the direct manipulations of interface devices. A pair of Datagloves and a specially designed equipment which measures grasping forces are employed in the systems. The approach is considered to provide a new sound generation techniques in music scene together with the flexible man-machine interface.

It is understood that nonverbal means of communication are more important than verbal communication in the transmission of intention or emotion. Gestures using body or hand are typical nonverbal means of communication. Human intention or emotion appears to be expressed to a greater extent by the force applied to the body than by the position of the hand. The authors noted that the force acting during motion can be sensed as an acceleration and therefore attempted recognition of gestures with a three-dimensional acceleration sensor. In the experiments using this system, the changes of the acceleration, the rotational force, and the directional distribution of the acceleration are determined from the sequential three-dimensional acceleration data, as features of the motion. By examining the extent of matching to standard patterns, ten kinds of gestures can be discriminated with a nearly 100 percent recognition rate. A real-time music control system is also constructed by applying the proposed method. Compared to the conventional methods using image processing or the data gloves, in the new system the delay in tempo detection is shorter and the mechanism is simpler. The system gives a clue to the design of flexible and sensitive man-machine interfaces based on gesture. © 1997 Scripta Technica, Inc.

意志や感情の伝達においては, 言語以上に非言語的手段によるところが大きいと言われる. 身振り手振りなどのジェスチャーはその代表的なものである. ジェスチャーにおいて人間の意志や感情は, 手などの位置よりもむしろ身体に加えられる力に顕著に現れると考えられる. ここでは運動中に働く力は加速度によって検出することができることに注目し, 3次元加速度センサによるジェスチャー認識を試みた. 試作システムでの実験では, 一連の3次元加速度データから運動の特徴量として, 加速度の変化, 回転力, 加速方向の分布などを求め, 標準パターンとのマッチングを行うことによって10種類程度のジェスチャーをほぼ100%の識別率で認識できることがわかった. また, 提案手法を適用して試作したジェスチャーによる実時間音楽制御システムについても述べる. 試作システムは, 従来の画像処理やデータグローブを用いた場合に比べて, テンポ検出の遅れも少なく構成も簡単であり, ジェスチャーによる柔軟で感性的なマン・マシンインタフェースの実現への手掛りが得られた.