In the field of 4-D space visualization, the information of 4-D space is obtained by projecting 4-D data onto 3-D space. Most of the previous research has been aimed at the recognition of 4-D space, whereas the target of the recognition has been limited to the geometrical information of 4-D objects in 4-D space or static spatial information without dynamics. Our research aims to develop a visualization system for providing the human experience of the physics-based environment in 4-D space. In this research, we mainly focus on collision detection and the behaviour of 4-D objects in 4-D space in order to construct the physics-based environment of 4-D space. Our contribution in this paper is the development of a collision detection algorithm for 4-D objects and a calculation method for physics based behaviour of 4-D objects. Our proposed collision detection algorithm is based on the intersection test of tetrahedrons in 4-D space, so that 4-D objects in our system is represented as tetrahedral meshes. The tetrahedron-based collision detection algorithm is performed by a combination of half-space tests with the use of 5-D homogeneous processing to enhance the calculation accuracy of the collision detection. Our proposed method calculates the behaviour of the 4-D objects after the collision by solving the motion equation based on the principle of physics. Consequently, the visualization system with the proposed algorithm allows us to observe the physics-based environment in 4-D space.

This paper reports the temperature and relative humidity dependence of self-propelled oil droplets. Self-propelled oil droplets locomote by internal Marangoni convection, which is caused by gradients in the interfacial tension due to surfactant and temperature gradients at the oil/water interface. We discover the external environment dependence of the self-propulsion of a droplet and discuss the mechanism of the property. Our study will serve as an innovative foundation for practical research, such as drug delivery systems.

<p>"Tears of wine" is a physical phenomenon that appears due to the result of a liquid flow along the glass surface against the gravity. In this study, we try to visualize the convection inside droplets in "tears of wine" by using the particle image velocimetry (PIV) method. We also examine the difference of convection flow which occurs when heat is applied to a part of a droplet. From the observation results, we assume that the temperature gradient can change the Marangoni flow in "tears of wine".</p>

<p>We study the motion control of a self-propelled oil droplet dropped into aqueous solution by using a rigid frame. It is well known that an oil droplet with fatty acid anhydride moves in random directions in aqueous solution. Although we have discovered that the moving direction could be controlled using a rigid boomerang-shaped frame, the mechanism of the autonomous motion in a particular direction is not understood theoretically. Since self-propelled droplets move with the effect of Marangoni convection caused by surface tension difference at the droplet surface, we focus on the directional behavior of the Marangoni convection and verify it using a SPH (Smoothed Particle Hydrodynamics) method, which is a meshfree fluid simulation.</p>

© Copyright © 2020 Hashimoto, Shigemune, Minaminosono, Maeda and Sawada. Self-folding technologies have been studied by many researchers for applications to various engineering fields. Most of the self-folding methods that use the physical properties of materials require complex preparation, and usually take time to complete. In order to solve these problems, we focus on the elasticity of a material, and propose a model for forming a 3D structure using its characteristics. Our proposed model achieves high-speed and high-precision self-folding with a simple structure, by attaching rigid frames to a stretchable elastomer. The self-folded structure is applied to introduce a self-assembled actuator by exploiting a dielectric elastomer actuator (DEA). We develop the self-assembled actuator driven with the voltage application by attaching stretchable electrodes on the both side of the elastomer. We attempt several experiments to investigate the basic characteristics of the actuator. We also propose an application of the self-assembled actuator as a gripper based on the experimental results. The gripper has three joints with the angle of 120°, and successfully grabs objects by switching the voltage.

© 2019 SPIE. For the visualization of a 4-D object, information in 4-D space is obtained by projecting a 4-D space onto a 3-D space. Previous researches mainly paid attention to the geometrical information of a single 4-D object in 4-D space. Our research aims to develop a visualization system for physical phenomena in 4-D space and to study the human spatial cognition of the 4-D space through the projected 3-D space. In this research, we focus on a collision of two 4-D objects in 4-D space and develop an algorithm for the collision detection between two 4-D objects to be displayed in 3-D space. In our developed algorithm, a 4-D object is defined as a mesh data of tetrahedrons, and a tetrahedron-based collision detection algorithm is performed by a combination of half-space tests with the use of 5-D homogeneous processing to enhance the calculation accuracy of the collision detection. The results of the collision detection between the 4-D objects are presented.

© 2019 The Royal Society of Chemistry. This paper introduces a novel directional control method of self-propelled oil droplets. Oil droplets locomote spontaneously with surfactant action. This self-propulsion is caused by Marangoni convection within the oil droplet due to differences in the surfactant concentration at the droplet surface. Recent studies have reported that self-propelled oil droplets change their locomotion style depending on their shapes. We confirm that spherical oil droplets move randomly, including straight motion, bending motion, and rotation. In particular, we discover that boomerang-shaped oil droplets exhibit only straight motion. In this study, we introduce an exoskeleton for the directional and velocity control of oil droplets. A droplet shaped as a boomerang by an exoskeleton locomotes in the direction from a concave region to a convex region. Through experimental studies, we found that the stability of the velocity and locomotion direction depended on the boomerang shape. Self-propelled oil droplets with exoskeletons were then applied to a transporting robot driven only by the energy obtained from chemical reactions. We demonstrate the robot pushes and transports an object floating on water.

© 2018 IEEE. In this paper, we propose a printed self-oscillatory mechanism inspired by an electric bell. We have proposed paper mechatronics to make a robot with printing methods. In the paper mechatronics, we also aim to fabricate a controller with printing methods. We attempt to obtain a control signal from a mechanism. An electric bell generates an oscillatory signal with a coupling mechanism between a coil and a magnet. We develop a printed electric bell on a paper with printing methods. We propose the design of the electric bell which can be fabricated on a sheet of paper. The coil and the wiring were printed on the paper, and the device was fabricated by folding the paper. We demonstrate that the device oscillates for 23 min by applying a 0.5 A of constant current. The design is scalable to be applied for microsystems with using printing methods.

The authors are developing a talking robot based on the physical model of human vocal organs in order to reproduce human speech mechanically. Humans use speech in daily communication, and each different country has a different language. The complex movements of vocal organs generate different vocal sounds. This study focuses on the performance of the talking robot to reproduce Vietnamese language, which is a tonal language and contains many unvoiced sounds and fricative sounds. The mechanical system of the talking robot is modified by adding an unvoiced sound source for fricative sound generation and a new flexible vocal cords. The control of the artificial vocal cords is adjusted to give the tonal effect. The robot is programmed to speak several Vietnamese phrases, and the output speech is evaluated by questionars.

Communication between machines and people is achieving dynamic relations. In fact, we can recognize this fact in a wide range of products like Google Home, Amazon Echo or Apple HomePod. They are all called voice chatbots. These chatbots, however, mainly respond to speech in English, while in Vietnam such chatbots are still not popular. This paper introduces a Vietnamese voice Chatbot, in which users can use Vietnamese to communicate with Chatbot through voice. Chatbot can complete tasks from user’s requests such as chatting with pre-written scripts; informing time and weather; searching on Google and Wikipedia or communicating and controlling other devices. In addition, the article introduces the emotion recognition from user voice for the chatbot. The project uses the Python programming language as well as the platforms built by Google.

In this paper, we propose a novel method for a humanoid robot to recover its standing posture from different fallen postures. In the proposed method, the robot estimates its posture using the ground-contacting information, which is sensed by the joint angle information and pressure sensors attached to the whole body. Then, the robot searches a motion sequence based on the estimated posture to stand up. The robot preserves every successful motion sequence into the tree structure to be reused in the next time. To verify the proposed algorithm, we conducted evaluation experiments by employing a real robot. The results show that the proposed method enables the robot to successfully search the motion sequence for standing up from the fallen posture.

© 2018 IEEE. Shape-memory alloys (SMAs) are metal which have the shape memory effects. By forming them to thin wires, they can shrink at a certain temperature; and then extend to original length after cooling below this temperature, thus shape-memory effect is observed. When pulse current is applied to a SMA wire, Joule heat generated inside makes the wire shrink about 5% of its length, and returns to the origin length when the pulse stops, which generate vibration in accordance with the frequency of the current. A novel designed actuator using the vibration of two SMA wires is proposed in this paper. The actuator has compact size and low energy consumption of milliwatts order. Tactile sensation is one of the important ways in human's five senses, which are used to acquire information from surrounding environment. In this paper, we are trying to transmit recognized information by translating it to tactile sensation perceived by skin around eyes. We divide the skin area around eyes into 4 zones, and design two tactile display prototypes. By installing multiple actuators in prototypes and applying pulse signals with different parameters, we conduct experiments to find optimal driving parameters to generate tactile sensation in designated skin areas. The higher-level perception in these area is tested by combining different parameters and stimulation patterns. Finally, information transmission is evaluated by the feedback from subjects and it turns out the proposed system performs well. In the final part we also propose potential applications of this system.

© 2018 IEEE. Soft robotic arms have gained popularity in the recent years because of their dexterity, robustness and safe interaction with humans. However, since these arms are subject to non-linear mechanics and are intrinsically under-actuated, their control still present many challenges. Octopus arms are one of the most popular biological models for soft robotics. It is known that the octopus reaching movement consists in two steps: (1) the rotation of the arm's base towards the target, and (2) the extension of the arm to reach the target. From a robotics point of view, the rotation of the base adds one additional degree of freedom to an already hyper-redundant system. Therefore, its role in the effectiveness of the control is ambiguous. In this work, we investigate the role of the base rotation for learning an effective reaching strategy. We conduct numerical experiments based on a mathematical model of the mechanics of the octopus arm in water and a simple neural network enabling to encode the control strategy through optimization learning. The network node corresponding to the base rotation is switched on or off for comparison. We test the reaching success rate with and without base rotation with targets in various positions. The results show that the addition of the base rotation can be highly beneficial or even detrimental, based on the position of the target. Nonetheless, globally the addition of base rotation affects the control strategy and expand the reachable regions.

© 2017 IEEE. Paper mechatronics has been proposed as a concept to fabricate robots with printing methods in our previous studies. Actuators fabricated by printing on a sheet material can be low-cost and lightweight. In this paper, we propose a swinging paper actuator driven by conduction Electrohydrodynamics (EHD). EHD is a phenomenon that generates flow by applying voltage to a dielectric liquid. The EHD flow is generated by the applied voltage in a wiring made by silver ink printed with an inkjet printer. The silver wiring was printed on a sheet of paper, and the paper with the wiring was folded manually to form its structure. We explored the possibility to manufacture EHD actuators by using the printing techniques developed in paper mechatronics. We expected to obtain high performance from the generated actuators due to the light weight of the paper printed structures. Finally, we succeeded in activating the swinging actuator by applying the high voltage to the paper electrode. We applied sine and square inputs, and investigated several characteristics. We found conditions to obtain the largest displacement in our situation. The swinging actuator proves a new concept to incorporate two alluring researches which are printed paper actuator and EHD phenomenon.

© 2017 IEEE. This paper presents the processing technique of hidden hypersurface removal for 4-D space visualization. When 4-D space and objects are visualized in 3-D space by perspective projection, if a 4-D scene includes multiple 4-D objects and intricate structures, visibility becomes a fundamental problem in 4-D space visualization. To address this problem and improve the perception of 4-D space, we extend Warnock&#039;s area-subdivision algorithm, which is used for hidden surface removal in the domain of 2-D projections of a 3-D scene, to the domain of 3-D projections of a 4-D scene. The visualization results show that our algorithm can display a 4-D scene with correct visibility, and suggest that the hidden hypersurface removal can help us understand the spatial relationship of the 4-D scene.

© 2017 IEEE. The authors have been developing a novel micro-vibration actuator using a shape-memory alloy (SMA) wire for the presentation of various tactile sensations to a human skin. An SMA wire has a unique characteristic to shrink to the lengthwise direction at a certain temperature according to the transition between the martensite and austenite phases. We have found a novel and characteristic effect that, by making the SMA into a fine wire with the diameter of 30-100 μm, it accepts a short pulse-current to instantly shrink and then return to the original length in accordance with the pulse frequency up to 300 Hz. The distinctive behavior of an SMA wire has been applied to present various tactile stimuli to human skin in this study.

The authors are developing a talking robot which is a mechanical vocalization system modeling the human articulatory system. The talking robot is constructed with mechanical parts that are made by referring to human vocal organs biologically and functionally. In this study, a newly redesigned artificial vocal cords is developed for the purpose of extending the speaking capability of the talking robot. The artificial vocal cords is developed using functional approach. A thin layer of rubber band with a width of 7mm is attached on a plastic body in a sealed chamber creates an artificial sound source. The fundamental frequency of the sound source vary from 80Hz to 250 Hz depending on the pressure to the rubber band and the curving shape of the rubber band. The curving shape of the rubber band is adjustable by an innovative design mechanism driven by a servo motor. The amount of air pressure applied to the rubber band is controlled by another servo motor. The experiments to verify the speak capability of the talking robot with this newly redesigned vocal cords is conducted by letting the robot generating five vowel sounds with different combinations of 20 levels of rubber band shape and 5 levels of air pressure input. The pitch of each sound is extracted to determine the effect of the rubber band shape and the pressure input on the output sound. The result shows that this newly redesigned vocal cords greatly increase the speaking capability of the talking robot, especially its singing performance.

The authors are developing a talking robot which is a mechanical vocalization system modeling the human articulatory system. The talking robot is constructed with mechanical parts that are made by referring to human vocal organs biologically and functionally. In this study, a newly redesign artificial vocal cord is developed for the purpose of extending the speaking capability of the talking robot. The artificial vocal cord is developed using functional approach. A thin layer of rubber band with a width of 7mm is attached on a plastic body in a sealed chamber creates an artificial sound source. The fundamental frequency of the sound source vary from 80Hz to 280 Hz depending on the pressure to the rubber band and the curving shape of the rubber band. The curving shape of the rubber band is adjustable by an innovative design mechanism driven by a servo motor. The amount of air pressure applied to the rubber band is controlled by another servo motor which is referred as volume motor in this study. The experiment to verify the speak capability of the talking robot with this newly redesigned vocal cord is conducted by letting the robot generating five vowel sounds with difference combination of 20 levels of rubber band shape and 10 levels of air pressure input. The pitch and volume of each sound is extracted to determine the effect of the rubber band shape and the pressure input on the output sound. The result shows that this newly redesigned vocal cord greatly increase the speaking capability of the talking robot, especially its singing performance.

The timing control is necessary for determining its duration, stress, and rhythm in human speech; however, little attention has been paid to these issues when building a speech synthesis system. We have developed a talking robot, which generates human-like vocal sounds. The cerebellum is an important part of human brain organ that has a significant role in the coordination, precision, and timing of motor responses. In this study, we develop a simplified cerebellum-like spiking neural network model to control the timing function for the talking robot. The model was designed using the System Generator software in Matlab, and the timing duration of trained speech was estimated using hardware cosimulated with a field programmable gate array board (FPGA). The timing information obtained from the co-simulation, together with the output motor vector, is sent to the talking robot controller to generate a sound with a short duration. The result indicates that this model can be used for short-range timing learning of the talking robot.

Purpose: Endoscope-assisted surgery has widely been adopted as a basic surgical procedure, with various training systems using virtual reality developed for this procedure. In the present study, a basic training system comprising virtual reality for the removal of submandibular glands under endoscope assistance was developed. The efficacy of the training system was verified in novice oral surgeons.
Material and methods: A virtual reality training system was developed using existing haptic devices. Virtual reality models were constructed from computed tomography data to ensure anatomical accuracy. Novice oral surgeons were trained using the developed virtual reality training system.
Results: The developed virtual reality training system included models of the submandibular gland and surrounding connective tissues and blood vessels entering the submandibular gland. Cutting or abrasion of the connective tissue and manipulations, such as elevation of blood vessels, were reproduced by the virtual reality system. A training program using the developed system was devised. Novice oral surgeons were trained in accordance with the devised training program.
Conclusions: Our virtual reality training system for endoscope-assisted removal of the submandibular gland is effective in the training of novice oral surgeons in endoscope-assisted surgery. (C) 2016 The Author(s). Published by Elsevier Ltd on behalf of European Association for Cranio-Maxillo-Facial Surgery.

Diabetes mellitus is a group of metabolic diseases that cause high blood sugar due to functional problems with the pancreas or metabolism. Diabetic patients have few subjective symptoms and may experience decreased sensation without being aware of it. The commonly performed tests for sensory disorders are qualitative in nature. The authors pay attention to the decline of the sensitivity of tactile sensations, and develop a non-invasive method to detect the level of tactile sensation using a novel micro-vibration actuator that employs shape-memory alloy wires. Previously, we performed a pilot study that applied the device to 15 diabetic patients and confirmed a significant reduction in the tactile sensation in diabetic patients when compared to healthy subjects. In this study, we focus on the asymptomatic development of decreased sensation associated with diabetes mellitus. The objectives are to examine diabetic patients who are unaware of abnormal or decreased sensation using the quantitative tactile sensation measurement device and to determine whether tactile sensation is decreased in patients compared to healthy controls. The finger method is used to measure the Tactile Sensation Threshold (TST) score of the index and middle fingers using the new device and the following three procedures: TST-1, TST-4, and TST-8. TST scores ranged from 1 to 30 were compared between the two groups. The TST scores were significantly higher for the diabetic patients (P &lt; 0.05). The TST scores for the left fingers of diabetic patients and healthy controls were 5.9 +/- 6.2 and 2.7 +/- 2.9 for TST-1, 15.3 +/- 7.0 and 8.7 +/- 6.4 for TST-4, and 19.3 +/- 7.8 and 12.7 +/- 9.1 for TST-8. Our data suggest that the use of the new quantitative tactile sensation measurement device enables the detection of decreased tactile sensation in diabetic patients who are unaware of abnormal or decreased sensation compared to controls.

A compact device for the presentation of tactile sensation is introduced in this chapter. By employing shape-memory alloy (SMA) wires, a novel tactile information transmission device is developed for displaying various tactile sensations. A tactile actuator consists of a thin SMA wire with the diameter of 50 μm, and generates micro-vibrations with various frequencies and amplitudes in accordance with a driving pulse signal. The actuator features compactness and low energy consumption, and is applied to tactile displays and tactile interaction systems.

Users are able to experience a high immersive feeling easily in a virtual environment with the widespread availability of human-machine interface devices such as head mounted displays and less expensive human motion sensors. The introduction of such electronic devices has led to more research being conducted in the area of the interaction of a human with robots and machine systems in virtual reality environment and a augmented reality space. Human-machine interface (HMI) researchers are looking for new ways to deal various information, and a variety of studies that present tactile experiences to users are being introduced by employing tactile actuators and devices. Tactile higher-level perception such as the phantom sensations and the apparent movement is known as tactile illusions, by which a user feels virtual sensation as being stimulated at different points on skin at the same time, and could be also effectively used for presenting better tactile sensations. In this study, we focus on the tactile perception of the apparent movement in a human body, and develop a system, by which a user has special tactile experience in an environment with a mobile robot to know its situation such as the moving speed and the roughness of the located ground and the wall as a feedback from the robot.

This study compares several acoustic features for developing an automatic vowel sequence reproduction system for a talking robot, which is a mechanical vocalization system modeling the human articulatory system. Matlab-based control system is used to analyze a recorded sound and drives the articulatory motors of the talking robot. A novel method based on short-time energy analysis is used to extract a human speech and translate into a sequence of sound elements for the sequence of vowels reproduction. Then, several phonemes detection methods including the direct cross-correlation analysis, the linear predictive coding (LPC) association, the partial correlation (PARCOR) coefficients analysis, and the formant frequencies comparison are applied to each sound element to give the corrected command for the talking robot to repeat the sound sequentially. Finally, experiments to compare these techniques and verify the working behavior of the robot are performed. The result of the tests indicates that the robot is able to repeat a sequence of vowels spoken by a human with a successful rate of more than 70% for the PARCOR analysis technique and the formant frequencies comparison technique. The greatest accuracy for repeating the sequence is given by the formant comparison method, while the direct cross-correlation method delivers the least accuracy.

The main focus of this study is to develop an automatic vowels sequence reproduction system for a talking robot, which is built to reproduce human voice based on the working behavior of human articulatory system. The control system using Matlab software to drive the articulatory motors of the talking robot. A sound analysis system is developed to record a sentence spoken by a human (mainly vowels sequence in Japanese language) and then analyze that sentence to give the corrected command packet for the talking robot to repeat it. Partial Correlation (PARCOR) coefficients analysis is used to detect the similar voice in the talking robot's database with the spoken voice; thus, the generated vocal sound of the talking robot will be the similar vowels sequence spoken by human. Finally, two tests to verify the working behavior of the robot are performed. The result of the tests indicate that the robot can be able to repeat a sequence of vowels spoken by human with an average successful rate more than 70%

In the recognition of an object situated in front of us, we firstly watch it and then try to touch it to feel the tactile and haptic sensation to be perceived by our hands. Various virtual reality and augmented reality systems have been introduced with the growing computational powers, however visual and auditory information related with a virtual object is mainly presented simultaneously to a user for enhancing the reality. If we consider the real situation of a user to try to recognize an object in the living environment, tactile and haptic information is assumed to be rather important for the better interaction with the object. In this study, we develop a haptotactile display that covers the movable range of a human arm in the tactual search for presenting tactile sensation of a virtual object. We design and construct a tactile pad to be fixed at the tip of a passively-driven haptic arm, and the tactile sensation and haptic force are simultaneously displayed to a user wearing the pad in the hand. The development of the hapto-tactile display for human tactile search will be described in this paper, together with an experiment to verify the validity of the system.

An estimated number of 420 million people in the world had diabetes mellitus in 2014, which had become quadruple since 1980, and the number is estimated to be 700 million by 2025. Diabetes mellitus is a group of metabolic diseases, which causes high blood sugar to a person, due to the functional problems of the pancreas or the metabolism. Patients of untreated diabetes would be damaged by the high blood sugar in vessels, and this starts to destroy capillary vessels to lower the sensitivity of tactile sensations, then effects to various organs and nerve systems. Diabetic peripheral neuropathy is one of the complications of diabetes mellitus, and the authors pay attention to the decline of the sensitivity of tactile sensations in the early stage of diabetes. By using a novel micro-vibration actuator that employs a shape-memory alloy wire, we develop a non-invasive screening device of the level of diabetes based on the perception of micro-vibration patterns. Experiments are conducted in a medical clinic, and the relation between the tactile stimuli and the medical diagnosis of diabetes are examined.

The author is developing a talking robot by reconstructing a human vocal system mechanically based on the physical model of human vocal organs. The robotic system consists of motor-controlled vocal organs such as vocal cords, a vocal tract and a nasal cavity to generate a natural voice imitating a human vocalization. By applying the technique of the mechanical construction and its adaptive control, the robot is able to autonomously reproduce a humanlike vocal articulation using its vocal organs. In vocalization, the vibration of vocal cords generates a source sound, and then the sound wave is led to a vocal tract, which works as a resonance filter to determine the spectrum envelope. For the autonomous acquisition of the robot's vocalization skills, an adaptive learning using an auditory feed-back control is introduced. In this manuscript, a human-like expressive speech production by the talking robot is presented. The construction of the talking robot and the autonomous acquisition of the vocal articulation are firstly introduced, and then the acquired control methods for producing human-like speech with various expressions will be described.

With the widespread availability of computers and mobile devices, we use them for daily communication by exchanging visual and verbal/non-verbal media. We obtain various information from multimedia devices, and such devices recently have a touch panel interface to allow a user to intuitively and interactively use them. A touch panel consists of a touch-sensitive panel and a graphical display, and accepts user's intuitive touch inputs together with visual and auditory feedback. With the employment of touch actions in such interface devices, tactile feedback will be expected, if we consider the manipulation of a real object. For presenting tactile feedback, some tactile actuators have been introduced recently, however these actuators require the reference from real objects to generate the appropriate tactile sensations, and the input data to be associated with the output tactile sensations should be precisely prepared in the design of such systems. This study introduces a tactile pen to work with a touch screen, together with an algorithm to automatically generate parameters from an object's picture for driving tactile actuators. A tactile presentation system with a visual display is constructed, and the validity of the texture sensations is verified by a user's experiment.

Information presentation through tactile sensation is actively studied these days, and various tactile displays using different actuators have been introduced by many researchers so far. It is, however, still hard to assume that information presentation through tactile medium would be suitably applied to products for general public. The authors have been developing micro-vibration actuators using shape-memory alloy (SMA) wires to display various tactile sensations, and in this paper an innovative tactile presentation technique for presenting tactile stroking sensations from a touch screen will be introduced. The cutaneous rabbit illusion is introduced firstly, together with its presentation by employing the SMA wire actuators. Then, the construction of a touch screen to present tactile sensation as a feedback against touch and flicking inputs of a user is described.

An innovative tactile pad that presents various tactile sensations synchronizing with moving pictures is presented in this paper. To display tactile/touch sensations from a pad, a tactile display panel is designed by employing 10 micro-vibration actuators that provide tactile-stimulation sources using shape memory alloy wires. With the employment of the human higher-level perception caused by appropriate micro-vibratory stimuli, the resolution of the designed tactile display panel increases more than three times, so that people perceive various tactile sensations with high resolution enough for the better reality. By employing a webcam to capture moving pictures, the tactile pad system automatically recognizes the texture information contained in the scene, which are associated with tactile information to be output from the tactile pad. In order to let people reasonably perceive the presented tactile sensations, the developed system automatically translates visual information to tactile sensation. For verifying the effectiveness of the developed vision-based tactile pad system, experiments are conducted and the results are presented.

In the vocal communication, we employ not only the speech and auditory ability but also the body actions such as gestures and gaze to react to the speech. For example, a human nods to a speaker for an agreement, and makes eye contact during the conversation. The purpose of this study is to realize a human-like auditory system using a robotic arm, which listens to a human voice like a human, and interacts with human speakers. In this paper, the imitation of human listening behavior for realizing the natural communication between a human and a robot by the robotic auditory system is introduced.

The authors have developed a talking robot by reconstructing a human vocal system mechanically based on the physical model of human vocal organs. The robotic system consists of motor-controlled vocal organs such as vocal cords, a vocal tract and a nasal cavity to generate a natural voice imitating a human vocalization. By applying the technique of the mechanical construction and its adaptive control, the robot is able to autonomously reproduce a human-like vocal articulation using its vocal organs. In vocalization, the vibration of vocal cords generates a source sound, and then the sound wave is led to a vocal tract, which works as a resonance filter to determine the spectrum envelope. For the autonomous acquisition of the robot's vocalization skills, an adaptive learning using an auditory feedback control is introduced. In this study, a human-like expressive speech production by the talking robot is introduced. A human generates speech by controlling their own vocal organs for changing the speech expressions such as the volume and the intonations. To realize the human-like expressive speech production, a control method for the mechanical organs has to be established. In the first part of the paper, the construction of the talking robot and the acquisition of the vocalization will be described, and then the control method for producing human-like speech with various expressions will be introduced.

Humans communicate with each other by using not only verbal media but also the five senses such as vision, audition, olfaction and tactile sensations. Various devices and systems have been introduced for supporting human communication, and most of them are based on visual and auditory media. For presenting tactile sensations, some tactile actuators have been introduced recently, however these actuators require real objects to generate the various tactile sensations and input data to be associated with output tactile sensations should be prepared manually by a user. This study introduces an algorithm to automatically generate parameters from an object's image for driving tactile actuators. A tactile presentation system is constructed, and the validity of the texture sensations is verified by a user's experiment. © 2013 IEEE.

Diabetes mellitus is a group of metabolic diseases which causes high blood sugar to a person, due to the functional problems of the pancreas or the metabolism. For a person with diabetes, glucose would not be used properly in the metabolic system, and cause health problems, such as polyphagia, polyuria and polydipsia, since glucose is the main source of energy for the body cells. Symptoms of untreated diabetes would be damaged by the high blood sugar in vessels, and this starts to destroy capillary vessels to lower the sensitivity of tactile sensations, then effects to various organs and nerve systems. The authors pay attention to the decline of the sensitivity of tactile sensations associated with diabetes, and develop a non-destructive screening method of the level of diabetes using a novel micro-vibration actuator that employs a shape-memory alloy wire. The actuators are arranged in an array, and various tactile stimuli are generated by just controlling the driving current signals. The tactile stimuli are presented to symptom's, index and middle fingers., and he/she responds to the stimuli by answering how they are perceived. We made an experiment in a medical clinic., and examined the relation between the Hb Al C values and tactile stimuli. © 2013 IEEE.

This paper introduces a support system for inputting electronic medical records based on the automatic voice recognition. The system is being constructed by the combination of an electronic medical record that has the ontology-based structure, and a voice recognition engine, and is expected to work in the actual environment. In our previous study, we found that environmental noises caused the failure of the voice recognition, and resulted in the low recognition rate to some subjects. In this study, a recognition experiment was conducted to examine the influence of an acoustic noise in a real situation. Environmental noises were recorded in a medical clinic, which were used in the voice recognition experiment. We found that the noise greater than 20 [dB] in the S/N ratio caused the failure of the voice recognition. In the future work, by employing a noise reduction technique in the voice input part, we further examine the recognition ability.

Mangosteen export generates large revenue
however, translucent mangosteens, which contain undesirable internal condition, result in the shipment rejection and decrease the reliability of the export. This research investigates a novel non-destructive classification approach based on acoustic frequency response to detect mangosteens containing translucent fleshes. The set of uniform-distributed multi-frequency acoustic signal is generated and passed through each mangosteen under the test. The frequency responses, describing a feature space, for all mangosteens are computed via the discrete Fourier transform. To prevent intensive computation, a linear optimization is adopted to select relevant frequency contents, creating a compact classifying feature vector. To solve the classification problem, two proposed acoustic-based classification techniques are studied, namely linear classifier (LC), and non-linear classifier (NLC) based on an artificial neural network. Then the results from both classifiers are compared against the results from the conventional water-floating (WF) approach. Against the experimental data, it is found that the average flesh classification accuracy of good mangoteens achieved from the LC and the NLC are about 61% and 74% respectively, while the WF yields an accuracy of about 69%. It is evident that the acoustic-based approach possesses the convincing accuracy for solving the problem of export-grade translucent mangosteen classification. In addition, the paper shows that a mangosteen's physical density can possibly provide intuitive information for better classification performance in the future research study. © 2012 IEEE.

A talking and singing robot which adaptively learns the vocalization skill by an auditory feedback learning is being developed. The fundamental frequency and the spectrum envelope determine the principal characteristics of a sound. The former is the characteristics of a source sound generated by a vibrating object, and the latter is operated by the work of the resonance effects. In vocalization, the vibration of vocal cords generates a source sound, and then the sound wave is led to a vocal tract, which works as a resonance filter to determine the spectrum envelope. The paper describes the construction of vocal cords and a vocal tract for the realization of a talking and singing robot, together with the control algorithm for the acquisition of singing performance by mimicking human vocalization and singing voices. Generated voices were evaluated by listening experiments.

In the research of augmented reality, many experimental systems have been introduced so far by presenting CG objects into a real environment with visual and auditory information, so that a user would interact with the CG objects and the environment. To present higher reality, not only visual and auditory information but also tactile and other stimuli should be presented to users to enhance the perception. We have paid attention to the human tactile sensations, and have been constructing an interactive system, which presents a tactile sensation in addition to visual and auditory sensation. The system is constructed by a head mounted display (HMD), audio speakers, a microphone, two tactile gloves and a PC. A user wares a HMD, and a CG character is superposed on his palm wearing the tactile glove. Stepping sounds of the character are output from the speakers, and various tactile stimuli are presented from the glove according to the motion of the character. The user is also able to interact with the character using the other hand by pushing and picking the object. This paper presents the gestural interaction with a virtual character employing a tactile glove, together with the evaluation of the tactile sensations by the users' experiments.

In this study, a new Braille display is constructed based on the vibration of a Shape Memory Alloy (SMA) wire. To present Braille information, a vibration actuator is used instead of conventional Braille dots. According to the temperature-dependent characteristic shrinkage and the vest ration to the initial length of an SMA wire, a vibration actuator which can be driven by several [Hz]-100 [Hz] pulse signals has been developed. A method for presenting Braille information is proposed by the application of the developed actuator to a Braille display. In this research, multiple actuators constructed by using metal pins (0.7 [mm] in diameter, 3 [mm] in length) and SMA wires (50 [um] in diameter, 3 [mm] in length), are placed as to form standard Braille. The actuators are vibrated by PWM signals with different frequencies and appropriate timings, and the effectiveness of the proposed method for Braille display is verified by experiments. From the experimental results, the highest recognition rate of 100 [%] was achieved under the conditions of 50 [Hz] vibration frequency and 500 [ms] time delay. This means the vibration patterns effectively stimulated the Meissner corpuscles, and helped the subjects properly discriminate Braille characters presented by the developed display. Good evaluations were received from the subjects who are visually-impaired. © 2012 by JSME.

A talking and singing robot which adaptively learns the vocalization skill by an auditory feedback learning is being developed. The fundamental frequency and the spectrum envelope determine the principal characteristics of a sound. The former is the characteristics of a source sound generated by a vibrating object, and the latter is operated by the work of the resonance effects. In vocalization, the vibration of vocal cords generates a source sound, and then the sound wave is led to a vocal tract, which works as a resonance filter to determine the spectrum envelope. The paper describes the construction of vocal cords and a vocal tract for the realization of a talking and singing robot, together with the control algorithm for the acquisition of singing performance by mimicking human vocalization and singing voices. Generated voices were evaluated by listening experiments.

A novel portable Braille display for mobile use is constructed by employing the micro-vibrations generated by small Shape Memory Alloy (SMA) wires. According to the characteristics of the shrinkage and expansion of a SMA wire formed into a diameter with 50 micro-meters, a vibration actuator which can be driven by several to 300 hertz pulse signals has been developed. To present Braille information for the blind, the vibration actuator is used instead of conventional Braille dots. A new method for presenting Braille information is proposed to apply the developed actuator into Braille display. The effectiveness of the proposed Braille display is verified by experimental results.

Feature extraction is a crucial step for many systems of face detection and facial expression recognition. In this paper, we present edge-based feature extraction for recognizing six different expressions, which are angry, fear, happy, neutral, sadness and surprise. Edge detection is performed by using Gabor wavelet and convolution filters. In this paper we propose two convolution kernels that are specific for the edge detection of facial components in two orientations. In this study, Principal Component Analysis (PCA) is used to reduce the features dimension. To validate the performance of our proposed feature extraction, the generated features are classified using Support Vector Machine. The experimental results demonstrated that the proposed feature extraction method could generate significant facial features and these features are able to be classified into each expression.

This paper presents the method of autonomous voice acquisition for a talking robot by applying a dual-SOM. We have so far developed the talking robot, which has mechanical organs as a human. By applying an auditory feedback control, the robot autonomously learns the vocalization skill. For the autonomous learning method, a Self Organizing Neural Network (SONN) by combining a Self-Organizing Map (SOM) with a Neural Network (NN) was employed. The SONN had 2-dimensional mapping space, which was used to locate phonetic features of voices generated by the robot. By choosing cells on the map, voice articulations were autonomously recreated. However, due to the spatial restriction of the map, the voice transition from one vocal sound to another was not always recreated properly. To solve the problems, a dual-SOM, the combination of a phonetic-SOM and a motor-SOM, both of which have 3 dimensional mapping spaces, is introduced. The structure of the dual-SOM is firstly described, and then acquired vocal sounds are evaluated, together with the analysis of the behavior of the SOM. © 2011 The Japan Society of Mechanical Engineers.

In this paper, an innovative vision based tactile display system is presented. Using web camera to get visual information, the developed system captures the shape of an object and transform it to tactile sensation. In order to let people sense the shape of the captured object, tactile display panel is designed with 25 pin-type actuators to provide stimulation sources by using shape memory alloys. By employing higher-level perception, tactile display panel can represent the shape of the captured object. The effectiveness of the developed vision based tactile display system is verified by experiments. © 2011 Intl Journal of Adv Mechatr.

Facial expression recognition has recently become an important research area, and many efforts have been made in facial feature extraction and its classification to improve face recognition systems. Most researchers adopt a posed facial expression database in their experiments, but in a real-life situation the facial expressions may not be very obvious. This article describes the extraction of the minimum number of Gabor wavelet parameters for the recognition of natural facial expressions. The objective of our research was to investigate the performance of a facial expression recognition system with a minimum number of features of the Gabor wavelet. In this research, principal component analysis (PCA) is employed to compress the Gabor features. We also discuss the selection of the minimum number of Gabor features that will perform the best in a recognition task employing a multiclass support vector machine (SVM) classifier. The performance of facial expression recognition using our approach is compared with those obtained previously by other researchers using other approaches. Experimental results showed that our proposed technique is successful in recognizing natural facial expressions by using a small number of Gabor features with an 81.7% recognition rate. In addition, we identify the relationship between the human vision and computer vision in recognizing natural facial expressions. © ISAROB 2011.

我々はこれまでに，形状記憶合金の細線を微小振動アクチュエータとして利用した触覚ディスプレイを開発し，素材の手触り感覚を呈示できることを見い出した．呈示触覚と実素材との比較実験を行った結果，触覚刺激のみによる呈示では個人差が大きく，同じ条件の刺激を呈示しても人によって異なる素材として認識される場合があることが分かった．人間は物体のテクスチャを触覚によって認識する際，視覚情報や体性感覚も有効に活用している．そこで，視覚ディスプレイ上に表示したテクスチャ画像に対応した触覚刺激を，マウス上に実装した薄型触覚ディスプレイによってユーザの手掌部に呈示するシステムを構築した．本システムは，複数のテクスチャ画像を視覚ディスプレイに表示し，ユーザがマウス操作によって各画像上でカーソルを移動させることにより，その速度に応じた触覚刺激を呈示して，能動的に物体の表面を撫でているかのような感覚を呈示することが可能である．システムの評価実験から，視覚情報が触覚認識に有意に影響を与えていることが示され，また触覚刺激と同時に適切な視覚情報を呈示することで，よりリアリティのある触覚感覚の呈示が可能となることが分かった．The authors have developed a tactile display using shape memory alloy wires, and constructed a presentation system of various texture sensations. However, different subjects perceive different sensations by one tactile stimulus generated by certain conditions. A human recognizes a texture of an object by referring not only to the tactile sensations, but also to the visual and other physical sensations. This paper introduces a texture presentation system, which consists of the tactile display and a visual display, and describes the evaluation of the system and effects of visual stimuli to texture sensations by several experiments.

The authors have developed a vocalization training system for the auditory impaired using a talking robot. The training system mainly consists of a talking robot that has mechanical organs like a human. With an adaptive learning strategy using an auditory feedback control, the robot autonomously learns the vocalization, and then reproduces the speech articulation from inputted sounds. In the previous system, the speech-learning algorithm of the robot was constructed by employing a self-organizing neural network (SONN), which consists of the combination of a self-organizing map (SOM) and a neural network (NN). However, improper maps were occasionally generated in the results of the speech articulation learning. To solve this problem, a new algorithm introducing two three-dimensional SOMs, called a dual-SOM, was employed for the autonomous learning of the robotic articulations. By applying the robot and its properties, we have constructed an interactive training system. The training is divided into two approaches
one is to use the talking robot to show the shape and the motion of the vocal organs, and the other is to use a topological map to present the difference of phonetic features of a trainee's voices. In this study, first, the construction of the training system is described together with the autonomous learning of robotic vocalization using the dual-SOM algorithm, and then the analysis of the speech training progress is presented based on the phonetic features and the mechanical vocal articulations. © 2011 by Walter de Gruyter · Berlin · New York.

A talking robot which adaptively learns the vocalization skill by an auditory feedback learning is being developed. The talking robot is constructed by the mechanical vocal systems, which respectively correspond to human organs. In vocalization, the vibration of vocal cords generates a source sound, and then the sound wave is led to a vocal tract, which works as a resonance filter to determine the spectrum envelope. The talking robot autonomously learns the vocalization by applying auditory feedback. In this study, a human-like expressive speech production by the talking robot is introduced. A human generates speech by controlling their own vocal organs for changing the speech expressions such as the volume and the intonations. To realize the human-like expressive speech production, a control method for the mechanical organs has to be established. In the first part of the paper, the construction of the talking robot and the acquisition of the vocalization will be described, and then the control method for producing human-like speech with various expressions will be introduced.

In the human communication, auditory and speech functions play an important role for recognizing someone to talk to and understanding the conversations. Various vocal sounds are generated by the complex movements of speaker's vocal organs, and this mechanism contributes to generate different voices that include speech expressions and speaker's individuality. This paper introduces several approaches to speech and auditory systems for introducing flexible interactions between humans and systems, which have been constructed by the authors.

In order to provide a portable Braille display for the blind and visually impaired, a novel compact device is developed by using shape memory alloy (SMA) wires. In details, according to the ability of transition between Martensite phase and the Austenite phase, SMA can expand and shrink depending on temperature. Using movement of SMA under different temperature, six compact SMA actuators which are settled in ring-like bandage and fixed on human fingers, are vibrated to stimulate human tactile sensation. Depending on optimal displacement, these actuators can present six-dot Braille and make the blind and visually impaired get information from the device. Experimental results are shown to verify the effectiveness of the developed portable Braille display.

A touch screen is an electronic visual display that can detect the presence and location of a touch within the display area surface. The term generally refers to stressing the display area of the device with a finger of a hand. A common type of touch screen only receives a user's finger stress as an input. However, the user himself is not able to feel the presence of the object displayed on the screen such as a displayed button image on the screen. This paper introduces the development of button repulsive sensations displayed by the utilization of micro-vibrations generated from a shape memory alloy wire on a touch screen interface as a tactile display actuator. © 2011 IEEE.

In the research field of augmented reality, many experimental systems have been introduced so far to provide natural interactions with a virtual character and a surrounded environment. To introduce a high level interaction, not only visual and auditory information but also other human sensations such as tactile sensations and somatosensory information should be presented to a user. In this study, we pay attention to the tactile sensation in the manipulation of a virtual character, and develop TactoGlove for presenting tactile sensations to a user. The virtual environment is manipulated by intuitive gestures, and tactile sensations corresponding with the gestural actions are presented to the user through TactoGove to establish multimodal interactions. Four subjects tried the system, and significant evaluations were obtained in the interaction between a virtual character and a user. © 2011 IEEE.

Facial expressions are an important channel of nonverbal communication. Currently, many facial expression analysis or recognition systems have been proposed. In this paper, a study of dimension reduction of Gabor features from different facial expressions is presented. Principle Component Analysis (PCA) is used as dimension reduction method. The experiment is conducted by using samples of face image for eight subjects. There are six facial expressions; anger, fear, happy, neutral, sadness and surprise are used in this study. In this experiment, we use different poses and head postures of each subject. Experiment results demonstrated the reduced dimensions of Gabor features could be effectively used in the next processing for recognizing facial expressions. © 2010 ISAROB.

A human has various sensory perceptions, and effectively uses them in communication. Auditory and visual functions especially play an important role for recognizing someone to talk to and understanding the conversation. In vocal communication, we are able to detect the position of a source sound in 3D space, extract a particular sound from mixed sounds, and recognize who is talking. In addition, we are able to detect a particular person by recognizing body features and individual gestures. By realizing this mechanism using a computer, new applications will be presented, which are utilized in the flexible and intuitive communication with humans. The authors are working for the identification of a particular person using microphones and a USB camera. The paper describes the development of an information fusion system and how to deal with multiple data obtained by different sensors.

A talking and singing robot which adaptively learns the vocalization skill by an auditory feedback learning is being developed. The fundamental frequency and the spectrum envelope determine the principal characteristics of a sound. The former is the characteristic of a source sound generated by a vibrating object, and the latter is operated by the work of the resonance effects. In vocalization, the vibration of vocal cords generates a source sound, and then the sound wave is led to a vocal tract, which works as a resonance filter to determine the spectrum envelope. The paper describes the construction of vocal cords and a vocal tract for the realization of a talking and singing robot, together with the control algorithm for the acquisition of singing performance by mimicking human vocalization and singing voices. Generated voices were evaluated by listening experiments.

A talking and singing robot which adaptively learns the vocalization skill by an auditory feedback learning is being developed. The fundamental frequency and the spectrum envelope determine the principal characteristics of a sound. The former is the characteristic of a source sound generated by a vibrating object, and the latter is operated by the work of the resonance effects. In vocalization, the vibration of vocal cords generates a source sound, and then the sound wave is led to a vocal tract, which works as a resonance filter to determine the spectrum envelope. The paper describes the construction of vocal cords and a vocal tract for the realization of a talking and singing robot, together with the control algorithm for the acquisition of singing performance by mimicking human vocalization and singing voices. Generated voices were evaluated by listening experiments.

In the study of virtual reality and mixed reality, the developments of an input device and user interface are important for the interaction with virtual objects and environment. However, a mouse or a joystick is commonly used in present systems. If we could manipulate a virtual object intuitively by using natural gestures without any sensors or devices, the system would contribute to present reality. In this paper, we introduce a technique to let a user move his hand in front of a camera to manipulate a virtual object, and the gestural movements are recognized in real-time to establish an interactive communication with computational systems. © 2009 SICE.

Shape-memory alloys (SMAs) are the metals which have the shape-memory effect. An alloy has peculiar temperature Tp, and the shape memory effect is observed when the body temperature is cooled to below Tp. The effect has been widely applied in different fields such as robotic muscles, medical surgeries, actuators and elastic wires. The SMA also has a unique characteristic to shrink at a certain temperature. The authors found a novel and interesting effect that, by making the SMA into a fine wire, it accepts a pulse-signal to generate a vibration in accordance with the pulse frequency of the signal, and have developed a vibration actuator for presenting tactile stimuli. By coupling the devices as a pair, a tactile display was constructed for presenting the phantom sensation and the apparent movement of the tactility, to transmit quite novel tactile sensations to a user. The information transmission by the device was tested by subjects, and was evaluated by questionnaires. The apparent movement was especially well perceived by users as a sensation of something running across fingers and a palm, or as being tapped by something, according to the well-determined signals given to the display. Several users reported that they perceived a novel rubbing sensation given by the apparent movement, and we further experimented the presentation of the sensation in detail to be used as a tactile display for the information transmission.

A novel micro-vibration actuator using a shape-memory alloy (SMA) wire is being developed for the presentation of tactile sensations to a human skin. In this study, the SMA wire is arranged two dimensionally to construct a compact Braille display for the blind. The display is driven by electric current pulse for mobile use to present Braille information. The display was evaluated in a school for the blind, and the results are presented in the paper.

The use of human motions for the interaction between humans and computers is becoming an attractive alternative to verbal media, especially through the visual interpretation of the human body motion. In particular, hand gestures are used as non-verbal media for the humans to communicate with machines that pertain to the use of the human gestures to interact with them. This paper introduces a 3D motion measurement of the human upper body for the purpose of the gesture recognition, which is based on the probability distribution of arm trajectories. In this study, by examining the characteristics of the arm trajectories given by a signer, motion features are selected and classified by using a fuzzy technique. Experimental results show that the use of the features extracted from arm trajectories effectively works on the recognition of dynamic gestures of a human, and gives a good performance to classify various gesture patterns.

本稿では，形状記憶合金糸を振動アクチュエータとして利用した新しい触覚ディスプレイを提案し，各アクチュエータを駆動するパルス信号を，その発生確率密度によって制御して触覚感覚を呈示する手法について述べる．著者らは糸状に加工した形状記憶合金が温度により伸縮する特性を利用し，数Hz～数100 Hzのパルス信号によって微小振動を発生する小型アクチュエータを開発し，これを触覚呈示に用いる手法を提案してきた．本研究では，複数個の形状記憶合金糸のアクチュエータを面状に配置した触覚ディスプレイの構築を行った．本アクチュエータは，パルス信号を与えることによってその周期に同期して微小振動を発生するが，ランダム発生パルスによって各アクチュエータを駆動することにより，様々な触覚感覚を呈示できることが分かった．さらに，パルス信号の発生確率密度を時間的に変化させることにより，面のテクスチャや，物体をなぞった感覚を生成できることを明らかにした．本稿では，まずこれらの触覚感覚の呈示手法について述べ，実験による本ディスプレイの有効性の評価について述べる．The paper introduces the development of a compact tactile display employing novel micro-vibration actuators using a shape-memory alloy, and describes the presentation of various tactile sensations to a human skin by the control of pulse-signal probability density given to the vibration actuators. The authors have paid attention to the super-elasticity characteristics of a shape-memory alloy formed into a thread, and have so far developed a thin tactile device by driving each actuator with pulse current signal for generating vibrations. In this study, we found novel tactile sensations presented by driving actuators with randomly generated pulse signals. This paper describes the generation of touch feelings such as texture information of objects and a rubbing sensation by controlling the signal probability density, which was evaluated by the users&#039; experiments.

A novel micro-vibration actuator using a shape-memory alloy for the presentation of various tactile sensations to a human skin is introduced in this paper. The authors paid attention to the characteristics of a shape-memory alloy formed into a thread, which changes its length according to its temperature, and developed a vibration-generating actuator electrically driven by periodic signals generated by current control circuits. For the tactile information transmission to a human body, the human higher-level perception such as the phantom sensation and the apparent movement of the tactility is employed in consideration of presenting novel tactile sensations. By coupling the actuators as a pair, an information transmission is realized for presenting novel tactile sensations to a user. The information transmission was tested by changing various conditions of electric signals to drive actuators, and was evaluated by questionnaires. Several users reported that they perceived a novel rubbing sensation given by the apparent movement, and we further experimented the presentation of the sensation in detail to be used as a tactile display for the information transmission through our body.

A human is able to exchange information smoothly using voice under different situations such as a noisy environment in a crowd and with the existence of multiple speakers. We are able to detect the position of a source sound in three-dimensional space, extract a particular sound from mixed sounds, and recognize who is talking. By realizing this mechanism with a computer, new applications will be presented for recording a sound with high quality by reducing noise, presenting a clarified sound, and realizing microphone-free speech recognition by extracting particular sounds. The paper will introduce real-time detection and identification of a particular speaker in a noisy environment using a microphone array based on the location of a speaker and the individual voice characteristics. The study will be applied to develop an adaptive auditory system for a mobile robot which collaborates with a factory worker. (c) 2007 Wiley Periodicals, Inc.

The use of human motions for the interaction between humans and computers is becoming an attractive alternative, especially through the visual interpretation of the human body motion. In particular, hand gesture is used as a non-verbal media for the humans to communicate with machines that pertains to the use of human gesture to interact with them. Recently, many studies for recognizing the human gesture have been reported, and most of them deal with the shape and motion of hands. This paper introduces dynamic gesture recognition based on the arm trajectory and fuzzy algorithm approach. In this study, by examining the characteristics of the human upper body motions of a signer, motion features are selected and classified by using the fuzzy technique. Experimental results show that the use of the features extracted from the upper body motion effectively works on the recognition of the dynamic gesture of a human, and gives a good performance to classify various gesture patterns.

A talking and singing robot which adaptively learns the vocalization skill by means of an auditory feedback learning algorithm is being developed. The robot consists of motor-controlled vocal organs such as vocal cords, a vocal tract and a nasal cavity to generate a natural voice imitating a human vocalization. In this study, the robot is applied to the training system of speech articulation for the hearing-impaired, because the robot is able to reproduce their vocalization and to teach them how it is to be improved to generate clear speech. The paper briefly introduces the mechanical construction of the robot and how it autonomously acquires the vocalization skill in the auditory feedback learning by listening to human speech. Then the training system is described, together with the evaluation of the speech training by auditory impaired people. Copyright (C) 2008 Hideyuki Sawada et al.

The paper introduces the development of a tactile device using a shape-memory alloy, and describes the information transmission by the higher-level perception such as the phantom sensation and the apparent movement of the tactility. The authors paid attention to the characteristic of a shape-memory alloy formed into a thread, which changes its length according to its temperature, and developed a vibration-generating actuator electrically driven by periodic signals generated by current control circuits, for the tactile information transmission. By coupling the devices as a pair, an information transmission system was constructed for presenting the phantom sensation and the apparent movement of the tactility, to transmit quite novel sensation to a user. The information transmission by the device was tested by subjects, and was evaluated by questionnaires. The apparent movement was especially well perceived by users as a sensation of something running across fingers or as being tapped by something, according to the well-determined signals given to the devices. Several users reported that they perceived a novel rubbing sensation given by the apparent movement, and we further experimented the presentation of the sensation in detail to be used as a tactile display for the information transmission.

In the vocal communication, we are able to detect the position of a source sound in 3D space, extract particular sound from mixed sounds, and recognize what the sound is. An ability that extracts and recognizes particular sound in noisy environment is called cocktail party effect. By realizing this mechanism with a computer, new applications will be presented to be utilized in the communication with humans. In this study, musical sounds are dealt for the tracking of sounds generated by musical instruments. Musical sounds have different features from human voice. For example, instrumental sounds have variety of spectra distributed in wide frequency range. In this study, a system that can deal with various kinds of sounds will be developed. For the estimation of musical pitch, this study focuses on musical sounds with different pitches generated from musical instruments, and a filtering technique using comb filters is being developed. A recorder, harmonica and keyboard-harmonica were selected as target musical instruments in the experiment and the result showed this technique was effective for the estimation of musical pitches.

Humans are able to communicate with each other by using not only verbal media but also the five senses such as vision, audition, olfaction and tactility. Our emotions and feelings are directly transmitted through non-verbal media, especially tactile sensation senses the physical information about an object or an environment. We have paid attention to the information transmission through the higher-psychological perception of tactility, and are now constructing a tactile transmission system for presenting stroking sensation. This paper introduces the development of a tactile display using a shape-memory alloy, and describes the transmission of stroking sensation by the higher-psychological perception such as the phantom sensation (PS) and the apparent movement (AM) of the tactility.

A talking and singing robot which adaptively learns the vocalization skill by an auditory feedback learning is being developed. The robot consists of motor-controlled vocal organs such as vocal cords, a vocal tract and a nasal cavity to generate a natural voice imitating a human vocalization. The robot is applied to the training system of speech articulation for the hearing-impaired people, because the robot is able to reproduce their vocalization and to teach them how it is improved to generate clear speech. The paper briefly introduces the mechanical construction of the robot and how it autonomously acquires the vocalization skill in the auditory feedback learning by listening to human speech. Then the training system is described, together with the evaluation of the speech training by auditory impaired people.

Measurement of human motion is widely required for various applications, and a significant part of this task is to identify motion in the process of human motion recognition. There are several application purposes of doing this research such as in surveillance, entertainment, medical treatment and traffic applications as user interfaces that require the recognition of different parts of human body to identify an action or a motion. The most challenging task in human motion recognition is to achieve the ability and reliability of a motion capture system for tracking and recognizing dynamic movements, because human body structure has many degrees of freedom. Many attempts for recognizing body actions have been reported so far, in which gestural motions have to be measured by some sensors first, and the obtained data are processed in a computer. This paper introduces the 3D motion analysis of human upper body using an optical motion capture system for the purpose of gesture recognition. In this study, the image processing technique to track optical markers attached at feature points of human body is introduced for constructing a human upper body model and estimating its three dimensional motion.

Full automation featured many problems, such as the difficulty of production changes, etc. Eventually, cell production, a human based production system, was introduced in the 1990s. The cell production System, however, features similar problems. Beginning in 2000. advanced cell production systems have been pursued, whereby production is accomplished through the harmonization of human operators and automatic machines. The aim of this study is to take the know-how that inspectors have unconsciously acquired on a daily basis and, transition the inspectors themselves to automated systems. To achieve this aim we established ROI macro commands and developed the ROI macro command system. About 80% of requests from manufacturing sites were fulfilled by using, those 8 types of ROI macro commands. Moreover, 488 sites used it oil an actual basis. As a result, we have concluded that the inspectors' know-how was passed on to external configuration inspections across a wide-range of fields.

The purpose of this study is to develop an information fusion system for the application of human-machine Communication. The information fusion system bundles and controls sensing units distributed in a space. One sensing unit consists of several sensors, whose data are given to a computer and are transformed into significant information. This information is fed to other sensing units, to be fused with other significant information. All the information are integrated in a main computer to recognize and identify a particular person. The main computer is responsible for managing all the information in a distributed fusion system. For a large and complex system, it may consist of several communities of sensors that are Controlled by several main computers to manage the different fusions of the systems. This paper introduces a methodology, oil how to group a several functions from different sensors that share a same environment for the purpose of recognizing a particular person.

Human is able to exchange information smoothly using voice under different situations such as noisy environment in a crowd and with the existence of plural speakers. We are able to detect the position of a source sound in 3D space, extract a particular sound from mixed sounds, and recognize who is talking. By realizing this mechanism with a computer, new applications will be presented for recording a sound with high quality by reducing noise, presenting a clarified sound, and realizing a microphone-free speech recognition by extracting particular sound. The paper will introduce a realtime detection and identification of particular speaker in noisy environment using a microphone array based on the location of a speaker and the individual voice characteristics. The study will be applied to develop an adaptive auditory system of a mobile robot which collaborates with a factory worker.

Human is able to exchange information smoothly using voice under different situations such as noisy environment in a crowd and with the existence of plural speakers. We are able to detect the position of a source sound in 3D space, extract a particular sound from mixed sounds, and recognize who is talking. By realizing this mechanism with a computer, new applications will be presented for recording a sound with high quality by reducing noise, presenting a clarified sound, and realizing a microphone-free speech recognition by extracting particular sound. The paper will introduce a realtime detection and identification of particular speaker in noisy environment using a microphone array based on the location of a speaker and the individual voice characteristics. The study will be applied to develop an adaptive auditory system of a mobile robot which collaborates with a factory worker.

This paper introduces the development of a tactile device using a shape-memory alloy and describes the information transmission by the higher psychological perception, such as the phantom sensation and the apparent movement of the tactility. The authors paid attention to the characteristic of a shape-memory alloy formed into a thread, which changes its length according to its body temperature, and developed a vibration-generating actuator electrically driven by periodic signals generated by current control circuits, for the tactile information transmissioa The size of the actuator is quite compact, and the energy consumption is only 20mW. By coupling the actuators as a pair, an information-transmission system was constructed for presenting the apparent movement of the tactility to transmit a quite novel sensation to a user. Based on the preliminary experiment, the parameters for the tactile information transmission were examined. Then the information transmission by the device was tested by 10 subjects and evaluated by questionnaires. The apparent movement was especially well perceived by users as a sensation of a small object running on the skin surface or as being tapped by something, according to the well-determined signals given to the actuators. Several users reported that they perceived a novel rubbing sensation given by the AM, and we further experimented the presentation of the sensation in detail to be used as a sensory-aids tactile display for handicapped and elderly people. © 2006, by Walter de Gruyter GmbH &amp
Co. All rights reserved.

A talking and singing robot which adaptively learns the vocalization skill by an auditory feedback learning is being developed. In vocalization, the vibration of vocal cords generates a source sound, and then the sound wave is led to a vocal tract, which works as a resonance filter to determine the spectrum envelope. The robot consists of motor-controlled vocal organs such as vocal cords, a vocal tract and a nasal cavity to generate a natural voice imitating a human vocalization. The paper briefly introduces the construction of vocal cords and vocal tract for the realization of the talking robot, and then describes how the robot autonomously acquires the vocalization skill in the auditory feedback learning by listening to human talking and singing voices. The acquired voices were evaluated by listening experiments.

Voice and sounds are the primary media employed for the human communication. Human is able to exchange information smoothly using voice under different situations such as noisy environment in a crowd and with the existence of plural speakers. We are surrounded by various sounds, and are able to detect the position of a source sound in 31) space, extract a particular sound from mixed sounds, and recognize what makes the sounds. The paper will introduce a realtime detection and identification of particular sound among plural sound sources using a microphone array based on the location of a speaker and the tonal characteristics. The algorithm is applied to an adaptive auditory system of a robotic arm for the interaction with humans.

This paper presents a digital filtering algorithm that clarifies dysphonic speech with the speaker&#039;s individuality preserved. The study deals with the clarification of esophageal speech and the speech of patients with cerebral palsy. The filtering ability is evaluated by listening experiments. Over 20,000 patients currently suffer from laryngeal cancer in Japan, and the only treatment for the terminal symptoms requires the removal of the larynx, including vocal cords. The authors are developing a clarification filtering algorithm of esophageal speech, and the primal algorithm of software clarification and its effectiveness was reported in the previous ICDVRAT. Several algorithms for the clarification have been newly developed and implemented and are being evaluated by questionnaires. The algorithms were extended and applied for the clarification of the speech by the patients of cerebral palsy. Copyright © Freund Publishing House Limited.

This article introduces a new device driven by emotional human twiddling gestures, and its application as a musical instrument. In traditional musical instruments, the relationship between the action and the generated sound is determined by the physical structures of the instruments. Although various computerized musical instruments have been actively developed and reported, the development of a human-computer interface is not sufficient for the translation of emotional feelings into sounds. We have so far paid attention to emotional feelings presented in gestures and music, and have developed a gesture acquisition device that measures grasping forces caused by user's twiddling actions by using compact pressure sensors and acceleration sensors. For the association of gesture patterns with sound parameters, a SOM algorithm is applied. The approach is considered to provide not only a new sound generation technique in the music scene, but also a flexible human-computer interface.

In order to improve the link between an operator and its machine, some human oriented communication systems are now using natural languages like speech or gesture. The goal of this paper is to present a gesture recognition system based on the fusion of measurements issued from different kind of sources. It is necessary to have some sensors that are able to capture at least the position and the orientation of the hand such as Dataglove and a video camera. Datagloge gives a measure of the hand posture and a video camera gives a measure of the general arm gesture which represents the physical and spatial properties of the gesture, and based on the 2D skeleton representation of the arm. The measurements used are partially complementary and partially redundant. The application is distributed on intelligent cooperating sensors. The paper presents the measurement of the hand and the arm gestures, the fusion processes, and the implementation solution.

A talking and singing robot which has equivalent mechanical organs to human vocal system is being developed based on a mechatronics technology under a feedback control. While various ways of vocal sound production have been actively studied so far, a mechanical construction of the vocal system is considered to advantageously realize natural vocalization with its fluid dynamics. Motors are employed for the manipulation of the mechanical system. The robot adaptively learns the relations between motor control parameters and the generated vocal sounds using an auditory feedback learning with neural networks, and sings a song by mimicking a human vocalization. This paper presents the construction of the talking robot and its singing performance, together with the adaptive control for the pitch and phoneme learning. The robot generates vowel and consonant sounds of different pitches by dynamically controlling the vocal cords, vocal tract and nasal cavity.

This paper presents a digital filtering algorithm that clarifies dysphonic speech with the speaker's individuality preserved. The study deals with the clarification of esophageal speech and the speech of patients with cerebral palsy. The filtering ability is evaluated by listening experiments. Over 20,000 patients currently suffer from laryngeal cancer in Japan, and the only treatment for the terminal symptoms requires the removal of the larynx, including vocal cords. The authors are developing a clarification filtering algorithm of esophageal speech, and the primal algorithm of software clarification and its effectiveness was reported in the previous ICDVRAT. Several algorithms for the clarification have been newly developed and implemented and are being evaluated by questionnaires. The algorithms were extended and applied for the clarification of the speech by the patients of cerebral palsy. © 2005, by Walter de Gruyter GmbH &amp
Co. All rights reserved.

A mechanical model of the human vocal system is I being developed based on a mechatronics technology under a feedback control. While various ways of vocal sound production have been actively studied so far, a mechanical construction of the vocal system is considered to advantageously realize natural vocalization with its fluid dynamics. Motors are employed for the manipulation of the mechanical system. The system is able to adaptively learn the relations between motor control parameters and the produced vocal sounds using an auditory feedback with neural networks, by mimicking a human vocalization. This paper presents the construction of a talking robot and its singing performance, together with the adaptive control for the pitch learning. The talking robot generates vowel and consonant sounds of different pitches by dynamically controlling the vocal cords, vocal tract and nasal cavity.

Nowadays agile market is in common, and the fundamental technology supporting next-generation production system requires further development of machine and information technologies to establish “human technology” and a bridging of these technologies together. IMS-HUTOP project proposes a new product life cycle that respects the human nature of individuals, and establishes the elemental technologies necessary for acquiring, modelling and evaluating various human factors in an effort to achieve the HUTOP cycle. In this paper we propose a human centred KANSEI manufacturing system, which has been proposed in the IMS-HUTOP project with 5 work packages. © 2004, The Institute of Electrical Engineers of Japan. All rights reserved.

A system to detect a car by measuring an automobile sound is Introduced. The time difference between the signals taken from plural microphones is measured by using a specially-designed microphone array. The sound source location is estimated based on the calculation of the cross-correlation, and a particular car is identified. The measurement was tested la different noisy environments far the practical use, and fair results were obtained.

Human is able to exchange information smoothly using voice under different situations such as noisy environment in a crowd and with the existence of plural speakers. We are able to detect the position of a source sound in 3D space, extract a particular sound from mixed sounds, and recognize who is talking. By realizing this mechanism with a computer, new applications will be presented for recording a sound with high quality by reducing noise, presenting a clarified sound. and realizing a microphone-free speech recognition by extracting particular sound. The paper will introduce a realtime detection and identification of particular speaker in noisy environment using a microphone array based on the location of a speaker and the individual voice characteristics. The study will be applied to develop an adaptive auditory system of a mobile robot which collaborates with a factory worker.

A talking and singing robot is being developed based on the mechanical model of the human vocal system by using mechatronics and feedback control technologies. While various ways of vocal sound production have been actively studied so far, a mechanical construction of the vocal system is considered to advantageously realize natural vocalization with its fluid dynamics. Motors are employed for the manipulation of the mechanical system. The robot is able to adaptively learn the relations between motor control parameters and the produced vocal sounds using an auditory feedback with neural networks, by mimicking a human vocalization. This paper presents the construction of the robot and its taking and singing performance, together with the adaptive control for the pitch and phoneme learning. The robot generates vowel and consonant sounds of different pitches by dynamically controlling the vocal cords, vocal tract and nasal cavity.

A display device called intuition-driven monitor will be introduced. The device has been developed by combining a LCD, a flexible arm suspending the display, angle sensors and a CCD camera, and is directly manipulated by a user with his direct operations by hands, eye-gaze and facial expressions. A user is able to explore virtual space and manipulate virtual environment with gestures and the sense of haptics.

Voices are used as primary media in human communication. A human voice is a sound generated by the complex movements of the vocal organs, and is the most important media employed for the communication in the day life to logical discussions. A human is able to detect the position of a source sound in 3D space by perceiving the time difference reaching to the both ears. Furthermore we can selectively listen to an objective voice in the crowds. This paper presents a realtime detection and identification of a particular person among plural speakers using a microphone array. The system identifies the position of a particular speaker, and enhances the voice signal selectively.

The gesture recognition needs to take into account very the fusion of hand shape different information. It needs toper and motion information. This paper presents a solution using a fuzzy sensor approach for the aggregation of such measurement information. The motion recognition is based on a measurement represented by numerical fuzzy subsets, and the shape recognition is based on measurements represented by lexical fuzzy subsets.

A display device called intuition-driven monitor will be introduced. The device has been developed by combining a LCD, a flexible arm suspending the display, angle sensors and a CCD camera, and is directly manipulated by a user with his direct operations by hands, eye-gaze and facial expressions. A user is able to explore virtual space and manipulate virtual environment with gestures and the sense of haptics.

A mechanical model of a human vocal system is being developed based on mechatronics technology. Although various ways of vocal sound production have been actively studied, mechanical construction is considered to advantageously realize natural vocalization with its fluid dynamics. The mechanical vocal system has several motors to manipulate the vocal tract and the vocal cords. It became possible to learn the relations between motor positions and the produced vocal sounds by an auditory feedback, and produce Japanese five vowels (a, i, u, e, o) by mimicking a human speech. In addition, the mechanical model could produce some consonant sounds by attaching a nasal cavity with the dynamic control. This paper introduces an adaptive learning algorithm for the mimicry of human vocalization, and presents a listening experiment of generated sounds for the evaluation.

In the human to human communication, we are able to communicate with each other by using not only verbal media but also the five senses. With the widespread availability of computers, the media which directly affect our emotions will be effectively utilized together with the audio and images. In this paper a display device called intuition-driven monitor will be introduced together with two application systems, in which a user is able to directly manipulate a virtual 3D object with the haptic senses.

Humans are able to communicate with each other by using not only verbal media but also the five senses. A display device called intuition-driven monitor has been developed by combining a LCD display, flexible arm, angle sensors and a CCD camera. The display is manipulated by a user with his direct operations by hands and gaze. The joint angles are measured by angle sensors, and the three dimensional position and posture of the display are calculated in real time by the computation of the system geometry. A user is able to. explore virtual space and manipulate virtual environment with the sense of haptics.

We are developing a mechanical model of a human vocal system based on a mechatronics technology. Although various ways of vocal sound production have been actively studied so far, a mechanical construction is considered to advantageously realize natural vocalization with its fluid dynamics. In the voice generation, the analysis of the behaviors of the vocal cords and the vocal tract are required to be employed in the mechanical system. Furthermore, the fluid mechanics is less stable to make the control difficult. Several motors are employed to manipulate the mechanical vocal system. A neural network works to establish the relations between motor positions and the produced vocal sounds by an auditory feedback in the learning phase. In the speech performance, the mechanical system is able to utter while vocal pitches and phonemes are adaptively controlled by an auditory feedback control. This paper presents the construction of a vocal tract part and its adaptive control by the neural networks.

This paper describes two humanoid robots developed in the Humanoid Robotics Institute, Waseda University. Hadaly-2 is intended to realize information interaction with humans by integrating environmental recognition with vision, conversation capability (voice recognition, voice synthesis), and gesture behaviors. It also possesses physical interaction functions for direct contact with humans and behaviors that are gentle and safe for humans. WABIAN is a robot with a complete human configuration that is capable of walking on two legs and carrying things as with humans. Furthermore, it has functions for information interactions suite for uses at home.

This paper introduces a mechanical model of human vocal system to realize human-like speech under an auditory feedback control. While various ways of vocal sound production have been actively studied so far, we are constructing a phonetic machine having a vocal chord and a vocal tract with an auditory system. By imitating the human learning procedure of voice acquisition, a mechanical construction of a human vocal system is considered to generate natural voice so that it can be advantageously applied to a singing voice production.

This paper introduces a mechanical model of human vocal system to realize human-like speech under an auditory feedback control. While various ways of vocal sound production have been actively studied so far, we are constructing a phonetic machine having a vocal chord and a vocal tract with an auditory system. By imitating the human learning procedure of voice acquisition, a mechanical construction of a human vocal system is considered to generate natural voice so that it can be advantageously applied to a singing voice production.

Various ways of vocal sound production are being actively studied. We are constructing a phonetic machine with a vocal chord and a vocal tract based on mechatronics technology. Mechanical construction of a human vocal system is considered to generate natural voice so that it can be advantageously applied to singing voice production. In voice generation, analysis and mechanical realization of the behaviors of the vocal chords and vocal tract are required. Furthermore. the fluid mechanical system is less stable, thus making control more difficult. Several motors are employed to manipulate the mechanical vocal system. Mappings between motor positions and the produced vocal sounds are automatically established in the learning phase. In the singing performance, the system is able to sing while vocal pitches and phonemes are adaptively controlled by an auditory feedback process. This paper presents the latest mechanisms of our mechanical vocal system together with adaptive tuning algorithms of the physical mechanism with an auditory system.

Various ways of vocal sound production are being actively studied. We are constructing a phonetic machine with a vocal chord and a vocal tract based on mechatronics technology. Mechanical construction of a human vocal system is considered to generate natural voice so that it can be advantageously applied to singing voice production. In voice generation, analysis and mechanical realization of the behaviors of the vocal chords and vocal tract are required. Furthermore. the fluid mechanical system is less stable, thus making control more difficult. Several motors are employed to manipulate the mechanical vocal system. Mappings between motor positions and the produced vocal sounds are automatically established in the learning phase. In the singing performance, the system is able to sing while vocal pitches and phonemes are adaptively controlled by an auditory feedback process. This paper presents the latest mechanisms of our mechanical vocal system together with adaptive tuning algorithms of the physical mechanism with an auditory system.

The importance of human-machine interface is widely acknowledged in accordance with the increase of the computer powers. In human to human communication, we communicate with each other by using not only verbal media but also non-verbal media. Especially information transmitted through the five senses is able to affect our emotions and feelings directly making for smooth communication. This paper introduces the development of a new haptic device driven by hand gestures. Since the device is molded with silicon-rubber with the softness of human skin, a user is able to make hand gestures with his &#039;sense of force&#039; by the transforming manipulations. The device is able not only to sense user&#039;s twiddling gestures but also to display reactions against the hand gestures by vibration patterns. As an experimental trial of a tele-communication system, the device was employed for terminal devices, and questionnaires were conducted to evaluate the device in the haptic communication.

In the fully customer-oriented design system, design candidates should be evaluated if they reflect appropriately the ambiguous preferences and demands intended by customers and designers or not. The researches in the total human-oriented design system have come up with prototype systems that have functions capable not only of providing an environment for products, but for displaying and evaluating products as well. There are many researches in this field. Here, the technologies to extract the personal room environment models and to control the display system by users&#039; gesture have been introduced.

A future system for flexible manufacturing must be a sensor-complex for multi-modality with high intelligence, since such system should work with human who has a variety of expression ways and acceptance channels. This paper describes three topics from our recent researches on gesture and haptic interfaces which can he introduced for human-friendly interface in designer-consumer cooperative production system. The first one is a gesture information acquisition method by using image processing technique. The second one is another gesture recognition system by employing accelerometers to measure the forces applied to the body. A new haptic-communication device is introduced as the third topic of the paper It is able not only to detect grasping force and the device orientation but also to display the tactile sensation using the vibration motors to realize human-machine and human-human haptic communications.

Gesture plays an important role in our daily life as nonverbal media for emotional human communication. And the gesticulation is a necessary method of communication among the people who are hearing impaired as sign language. The arm motion and the hand figure are considered to be the primitive information in understanding gestures. This paper proposes a gesture recognition algorithm based on motion primitives and hand figure primitives extracted from acceleration sensors, position sensors and datagloves, to understand human dynamic movements and hand postures. The sensor integration method realized the robust gesture recognition in spite of the simple algorithm. The algorithm is applied to a sign-language recognition system for the realization of new human-machine interface through gestures. We also propose a sign language database based on the motion primitives and hand figure primitives.

ジェスチャと音響, 音声は, 人間同士のコミュニケーションにおいて深いかかわりをもっている.音響の合成にはこれまでさまざまな手法が提案され実用化されてきたが, 合成のためのパラメータの扱いの煩雑さや, 波形計算時間に起因するレスポンスの悪さなどの問題点も見られる.またジェスチャは音楽演奏や音楽パフォーマンスにおいて特によく利用されており, これまでにも人間の動作により音楽演奏を制御するシステムの試みも多く見られる.ここではハンドジェスチャに着目し, ハンドジェスチャセンシングデバイスの開発とそれを用いた音響生成システムについて報告する.音響の生成には, 音源と共鳴フィルタ特性が大きくかかわっているが, これらを自由なジェスチャによってコントロールすることが可能である.ジェスチャの計測量と音響生成パラメータの対応付けには, 自己組織化マップを用いている.提案したジェスチャセンシングデバイスは, 音響生成, 制御への応用ばかりではなく, 柔軟で感性的なマン・マシンインタフェースとしての利用が期待できる.

Although algorithmic sound syntheses, such as a granular synthesis and a physical model based synthesis, have been studied actively and has come in practical use, the development of human-machine interface is not sufficient for the translation of emotional feelings into sound. This paper presents two sound generation systems in which sounds are generated by the direct manipulations of interface devices. A pair of Datagloves and a specially designed equipment which measures grasping forces are employed in the systems. The approach is considered to provide a new sound generation techniques in music scene together with the flexible man-machine interface.

It is understood that nonverbal means of communication are more important than verbal communication in the transmission of intention or emotion. Gestures using body or hand are typical nonverbal means of communication. Human intention or emotion appears to be expressed to a greater extent by the force applied to the body than by the position of the hand. The authors noted that the force acting during motion can be sensed as an acceleration and therefore attempted recognition of gestures with a three-dimensional acceleration sensor. In the experiments using this system, the changes of the acceleration, the rotational force, and the directional distribution of the acceleration are determined from the sequential three-dimensional acceleration data, as features of the motion. By examining the extent of matching to standard patterns, ten kinds of gestures can be discriminated with a nearly 100 percent recognition rate. A real-time music control system is also constructed by applying the proposed method. Compared to the conventional methods using image processing or the data gloves, in the new system the delay in tempo detection is shorter and the mechanism is simpler. The system gives a clue to the design of flexible and sensitive man-machine interfaces based on gesture. © 1997 Scripta Technica, Inc.

意志や感情の伝達においては, 言語以上に非言語的手段によるところが大きいと言われる. 身振り手振りなどのジェスチャーはその代表的なものである. ジェスチャーにおいて人間の意志や感情は, 手などの位置よりもむしろ身体に加えられる力に顕著に現れると考えられる. ここでは運動中に働く力は加速度によって検出することができることに注目し, 3次元加速度センサによるジェスチャー認識を試みた. 試作システムでの実験では, 一連の3次元加速度データから運動の特徴量として, 加速度の変化, 回転力, 加速方向の分布などを求め, 標準パターンとのマッチングを行うことによって10種類程度のジェスチャーをほぼ100%の識別率で認識できることがわかった. また, 提案手法を適用して試作したジェスチャーによる実時間音楽制御システムについても述べる. 試作システムは, 従来の画像処理やデータグローブを用いた場合に比べて, テンポ検出の遅れも少なく構成も簡単であり, ジェスチャーによる柔軟で感性的なマン・マシンインタフェースの実現への手掛りが得られた.

<p>We focused on the high elasticity of elastomer, and proposed a model for forming a three-dimensional structure using its characteristics. Furthermore, by exploiting elastomer as a dielectric elastomer actuator (DEA), we developed prototypes of a self-assembled actuator which can be driven by voltage application. We attempted several experiments to investigate basic characteristics of the actuator. Finally, we propose applications of the proposed actuator to the robots based on the experimental results.</p>

人間は接触面の微小な凹凸を認識して触覚を得ているが，この機能を工学的に再現することは未だ難しい．そこで我々は，人間の皮膚構造が持つ形態計算に着目した. 人間の皮膚は，硬さの異なる3層の組織からなり，応答性の異なる4種の受容体と相互作用する構造を持つ．この皮膚構造をバネ定数の異なるバネの連結によってモデル化する．冗長な力学モデルでは入力が力学系内に保持され，次の入力に影響を与えることが知られている．この履歴情報を利用することで，皮膚の力学モデルが時系列データにおける学習性能に寄与することを確認した．実験結果をもとに，皮膚構造が触覚のセンシングにおいて果たす役割と，その情報処理能力について議論する．

近年，様々なアクチュエータを応用した触覚提示システムが提案されている．これまで我々は，システムの小型化や省電力化を目的として，糸状形状記憶合金を用いた触覚提示システムを構築してきた．本研究では，触覚提示システムのユーザビリティを向上するために，ワイヤレス電力伝送を新たに取り入れる。非接触での電力伝送の手法として，伝送距離の長い磁界共振方式を採用する．この方式ではコイル間の距離や位置関係の変化に伴い，共振周波数が変化するため，電力の送受電特性も変化する．本稿ではワイヤレス触覚提示システムのプロトタイプと，その性能評価並びに送受電特性について報告する．

形状記憶合金（Shape memory alloy： SMA）は，応力に応じて結晶構造が変化する性質を有し，これにより通常の金属とは異なる電気抵抗変化が生じる．この性質を利用し，振動を応力としてSMAに伝えることにより，電気抵抗変化による振動の計測が可能となる．しかし，SMAワイヤの微小な応力に対する電気抵抗変化の特性については，未だ詳しい研究はなされていない．本研究では，音の微小振動をSMAワイヤに伝えることで，新しい音響センシングシステムを構築した．本稿では，SMAワイヤへの音響入力条件を変化させて周波数応答を計測することで，音響センシング特性の評価と考察を行う．

様々なセンサを用いて，高い精度で人体動作の検出を行う手法が提案されてきた．しかしながら，これらセンサの多くは，大規模な光学システムや複雑な電子回路を要する．動作検出システムがより日常生活の中に溶け込むためには，シンプルなセンシングシステムが必要となる．我々は，簡易な回路構成で身体動作の検出が可能なシステムの構築を目的として，人体と物体との接触帯電現象に着目した．人体に直接電極を貼り付け，帯電物体周りで動作を行うことによって変化する人体電位を測定する．測定した人体電位を学習信号としてサポートベクターマシンにかけることで，初期位置や速度の異なる6種類の腕部動作を良好に識別することに成功した．

<p>We have developed a wheel robot which locally transforms its diameter to improve mobility of a wheel on uneven terrain. Our wheel robot locomotes on a flat road with high mobility of a wheel and on rough terrains by changing its diameters. We developed a variable diameter wheel as a prototype of the wheel robot and investigated its availability. In this paper、the wheel robot did not have an axle motor, however the robot could locomote by utilizing expansion and contraction of diameters with moving a center of gravity. We tested its mobility through the two experiments.</p>

<p>In this research, we focus on a self-propelled phenomenon of an oil droplet due to surface activation reaction. It is known that the behavior of a self-propelled oil droplet varies depending on its shape. When an oil droplet is spherical shape, the droplet shows stochastic behavior such as curvilinear motion and direction change, and when the oil droplet is boomerang shape, the droplet moves steadily in linear locomotion. Therefore, we focused on the shape of the oil droplet running straightly, and attempt the direction control by deforming the oil droplet into the boomerang shape using the exoskeleton frame. Furthermore, we exploit a method to transport an overwater object.</p>

A novel micro-vibration actuator using a shape-memory alloy (SMA) for the presentation of various tactile sensations to human skin will be introduced in this paper. A tactile actuator consists of a thin SMA wire with the diameter of 50μm, and generates micro-vibrations with various frequencies and amplitudes synchronizing with a driving pulse signal. By coupling the actuators as a pair, an information transmission system is created for presenting higher-level tactile perceptions such as the phantom sensation and the apparent movement to transmit novel tactile sensations to human skin. The apparent movement was especially well perceived by users as a sensation of something running across their fingers, or as being tapped by an object, according to well-determined signals given to the devices. The actuator features compactness and low energy consumption, and is applied to tactile displays and tactile interaction systems.

The paper introduces the development of a tactile device using a shape-memory alloy, and describes the information transmission by the higher-level perception such as the phantom sensation and the apparent movement of the tactility. The authors paid attention to the characteristic of a shape-memory alloy formed into a thread, which changes its length according to its temperature, and developed a vibration-generating actuator electrically driven by periodic signals generated by current control circuits, for the tactile information transmission. By coupling the devices as a pair, an information transmission system was constructed for presenting the phantom sensation and the apparent movement of the tactility, to transmit quite novel sensation to a user. The information transmission by the device was tested by subjects, and was evaluated by questionnaires. The apparent movement was especially well perceived by users as a sensation of something running across fingers or as being tapped by something, according to the well-determined signals given to the devices. Several users reported that they perceived a novel rubbing sensation given by the apparent movement, and we further experimented the presentation of the sensation in detail to be used as a tactile display for the information transmission.

A talking and singing robot which adaptively learns the vocalization skill by an auditory feedback learning is being developed. In vocalization, the vibration of vocal cords generates a source sound, and then the sound wave is led to a vocal tract, which works as a resonance filter to determine the spectrum envelope. The robot consists of motor-controlled vocal organs such as vocal cords, a vocal tract and a nasal cavity to generate a natural voice imitating a human vocalization. An adaptive learning algorithm using a neural network is introduced for the achievement of the autonomous acquisition of vocalization skill. In the learning phase, the robot vocalizes 150 patterns of voices by ramdomly giving the motor-control parameters, and the generated voice characteristics and the corresponding motor-control parameters are associated with each other by using a self-ornanizing neural network. After the learning, the neural network is connected in series into the vocal tract model. By inputting sound parameters of desired sounds to the NN, the corresponding form of the vocal tract is obtained to vocalize desired sounds. The robot is applied to the training system of speech articulation for the auditory impaired people, because the robot is able to reproduce their vocalization and to teach how they are improved to generate clear speech. The paper briefly introduces the mechanical construction of the robot and how it autonomously acquires the vocalization skill in the auditory feedback learning by listening to human speech. Then the training system is described, together with the evaluation of the speech training by auditory impaired people.

A talking and singing robot which has been constructed by referring to the human vocal system will be introduced in this paper. Although various ways of vocal sound production have been actively studied so far, mechanical construction of human vocal system is considered to advantageously realize natural vocalization with its fluid dynamics. The robot has several motors to manipulate the vocal tract and the vocal cords. The robot autonomously learns the relations between motor control parameters and the generated vocal sounds using an auditory feedback learning with neural networks, and sings a song by mimicking a human vocalization. This paper presents the construction of the talking robot and its singing performance, together with the adaptive control for the pitch and phoneme learning.

The paper introduces the development of a tactile device using a shape-memory alloy, and describes the information transmission by the higher-level perception such as the apparent movement and phantom sensation of the tactility. The authors paid attention to the characteristic of a shape-memory alloy formed into a thread, which changes its length according to its temperature, and developed a vibration-generating actuator electrically driven by periodic signals generated by current control circuits, for the tactile information transmission. By coupling the devices as a pair, an information transmission system was constructed for presenting the apparent movement of the tactility, to transmit quite novel sensation to a user. The information transmission by the device was tested by 10 subjects, and evaluated by questionnaires. The apparent movement was especially well perceived by users as a sensation of something running across fingers or as being tapped by something, according to the difference of signals given to the devices. Several users reported the rubbing sensation given by the AM, and we further experimented the presentation of the sensation in detail.

Gestures play one of the most important rolls for the expression of intention such as emotion and feelings in nonverbal human communication. When human gives an intention to a gesture, it could be obtained as the difference of force. The force applied to the body is measured as the acceleration derived from gestural trajectory. Therefore, the intention of gesture is considered to appear in the velocity and acceleration. This paper presents the extraction of intention from gestures based on the resampling algorithm applied to the gestural trajectory. By comparing the temporal trajectory with the resampled trajectory, an intention given to the gesture could be extracted and assessed. The proposed method was applied to the evaluation of intentions given to Japanese sign-language words.

Gestures play one of the most important rolls for the expression of intention such as emotion and feelings in nonverbal human communication. When human gives an intention to a gesture, it could be obtained as the difference of force. The force applied to the body is measured as the acceleration derived from gestural trajectory. Therefore, the intention of gesture is considered to appear in the velocity and acceleration. This paper presents the extraction of intention from gestures based on the resampling algorithm applied to the gestural trajectory. By comparing the temporal trajectory with the resampled trajectory, an intention given to the gesture could be extracted and assessed. The proposed method was applied to the evaluation of intentions given to Japanese sign-language words.

A talking and singing robot which has been constructed by referring to the human vocal system will be introduced in this paper. Although various ways of vocal sound production have been actively studied so far, mechanical construction of human vocal system is considered to advantageously realize natural vocalization with its fluid dynamics. The robot has several motors to manipulate the vocal tract and the vocal cords. It became possible to adaptively learn the relations between motor positions and the produced vocal sounds by an auditory feedback, and produce Japanese five vowels and consonants by mimicking a human speech. In addition, the robot generates vowel and consonant sounds of different pitches by dynamically controlling the vocal cords, vocal tract and nasal cavity using its auditory system.

Humans communicate with each other by using not only verbal media but also five senses effectively using their body parts. This paper describes the recognition of gaze and facial parts for the construction of the face interface. The extraction of individuality based on the facial parts motion is also discussed. The face interface is considered to be effectively and easily used by elderly persons and physically challenged who have difficulties in using the conventional interface devices.

Although almost all the animals have voices, only human beings are able to employ speech to make verbal communication. The vocal sound is produced by the relevant operations of the vocal organs such as a lung, trachea, vocal cords, vocal tract, tongue and muscles. If any part of the vocal organs is injured or disabled, we may be involved in the impediment in the vocalization and, in case of the worst, we may lose our voices. The paper presents a digital filtering algorithm which clarifies dysphonic speech with the speaker's individuality preserved. The study deals with the clarification of esophageal speech and the speech of patients with cerebral palsy, and the filtering ability is being evaluated by listening experiments.

Human is required to perform a complex operation of physical apparatus such as lung, trachea, vocal cord, vocal tract, tongue and muscle in producing a particular sound. In this research, we are constructing a model of human's phonetic system that is equipped with air pump as a substitute for human's lung, artifical vocal cord and artificial vocal tract, that is expected to produce natural biological sound. The aim of the model is to mimic a particular sound spectrum that is given as an input. For controlling the model, we apply a neural network that is trained on input-output relation of the individual model, rather than calculating PARCOR coefficients.

Recently, there are many researches for sign language which is the most important means of communication for a person with auditory disability. The principal aim of these research is recognition and synthesis of sign language, and the system which can search and store sign words is necessarily required. The sign language databases are mostly consist of motion pictures and index words, which makes it difficult that we search meaning from gesture. This paper describes the multimodal database which have information obtained from not only movie but also acceleration sensor, position sensor and dataglove. We are constructing a bi-directtional data retrieval system to use key words and gestures.

We are constructing a phonetic machine having a vocal chord and a vocal tract based on mechatronics technology, and have so far developed a pitch generation part as a subsystem for melody synthesis to sing in humming. In the pitch generation, the analysis and mechanical simulation of the behavior of the vocal chords are required. The fluid mechanical system is, however, less stable to make the control difficult. This paper presents a singing instrument system together with an adaptive tuning algorithm of the physical mechanism using an auditory feedback. The mechanical method is considered to be promising to generate more natural voice than algorithmic sound synthesis methods.

公開番号:特開2005-115813

公開番号:特開2007-033397

公開番号:特開2007-048268 / 登録番号:特許4291830

公開番号:特開2008-262478 / 登録番号:特許4395572

登録番号:特許5118777号

平成25年度までに舌部分切除術を表現したバーチャルリアリティ(VR)シミュレーションのプロトタイプソフトウェアを開発した。これまでにどの分野においても作製されたことのないVR舌モデルが作られたことにより、以前は実際の手術において経験と訓練を積むことしかできなかった舌部分切除術において、模擬訓練を行うことが可能となった。舌モデルは一辺25個のバーチャルボールで表現された立方体の範囲内で表現されている。ボールとボールの結合に関する力(引張、摩擦など)の値を変化させることにより、舌の硬さや癒着度などの性状を細かく表現することが可能である。この技術により、VR舌の表面または内部に硬度の異なる組織、すなわち悪性腫瘍(癌)を表現することが可能となった。範囲や硬さを調節することで、舌癌の進展度を変化させ、診断や切除における難易度を設定した。今後、口腔外科医がこのシミュレーションを使用して訓練を行った場合、口腔外科医の技術力にどのような進化が生ずるか、また舌癌手術経験者と未経験者における技術力変化量の違いなどについて評価を行う予定としていた。
また、現時点において手術に用いるツール(摂子やメス、電気メス)について、VR上での形態表現が不十分であったため、ソフトウェア制作を依頼している日本バイナリー株式会社に対し、バージョンアップの依頼を行っている。今後も、性状だけでなく、視覚的にもより現実的なシミュレーションを目指し、評価を行う予定である。

We developed a speech recognition system based on clinical ontology. The system is a custom version made from free speech recognition software "Julius". Some clinical ontologies (e.g. gastro-intestinal endoscopy findings, abdominal ultrasonography findings) are prepared for the system. The system has a function to import ontology and can recognize sentences with a dictionary made from imported ontology. The system is also able to fill fields of database because he has knowledge of clinical context from the ontologies. The feature will be useful to record vital signs in clinical scene.

In this study, a vocalization training system for the auditory-impaired was constructed by applying a talking robot, which has mechanical organs like a human. With an adaptive learning strategy using an auditory feedback control, the robot autonomously learned the human vocalizations, and then reproduced the speech articulations from inputted vocal sounds. By analyzing the acquired relations between vocal articulations and the generated vocal sounds stored in the robot, unclear vocal sounds given by auditory-impaired people were classified, and then particular features were extracted, which were associated with vocal articulations reproduced by the robot. The robot was employed in the speech training for auditory-impaired, and the training results verified the effectiveness of the interactive training employing the robot.

人間は言葉を有効に利用して、コミュニケーションをおこなっている。これは音声が特別な道具なしに情報を伝え合うことができる、最も容易な手段であるためと考えられる。人間の発声器官は主に、肺、声帯、声道、舌、口蓋とそれらを動かす筋肉などから成り、互いに適当な位置や形状を形成することにより言葉が生成される。最近のマン・マシンインタフェース技術には、人間らしい音声による情報の提示や、音声による入力デバイスが不可欠な要素となっている。本研究では、発話動作をおこなう器官をエアポンプ、人工声帯、声道共鳴管などを用いて機械的に構成し、計算機による聴覚フィードバック制御によって自らが音声を獲得、生成することができる発話ロボットの開発をおこなった。更に、聴覚障碍や発話障碍を持った患者が、対話的に発声訓練できるシステムの構築を目指した。
本年度は、前年度から構築してきた人間の発声器官に対応する各機械部および制御機構の見直しを図った。シリコーンゴムで成形した共鳴管の形状とその制御機構を見直すことにより、共鳴特性が向上した。また二枚振動声帯とその起振機構を開発し、人間と同様の声帯振動を実現した。発話ロボットから生成される音声をマイクロフォンから入力し、音響解析部で解析を行う聴覚フィードバック機構に、自己組織化ニューラルネットワーク(SONN)のオンライン学習を適用した。これにより、人間が目標となる音声を与えることにより、機械系が自ら音声および発話動作を獲得できる適応学習が可能となった。またピッチを適応的に学習、獲得することにより、発声練習から歌声を生成することも可能とした。
更に、聴覚障碍、発話障碍患者のための発声訓練システムの構築をおこなった。発話障碍者の不明瞭な発声からその発話動作を再現し、これを明瞭化する発話動作を提示するシステムを構築した。今後、実証実験をおこないシステムの有効性を検証していくと共に、新しい対話型発話訓練装置の実用化を目指していく。

喉頭摘出患者の食道発声音声、老化や脳性麻痺にともなう不明瞭音声を、発声原理に基づいて明瞭化するフィルタを構成することが本研究の目的である。最終年度は、これら不明瞭音声の特徴抽出、および音声の個人性を保持した明瞭化フィルタの構築をおこなった。更にここで得られた音声の特徴と発声器官の動作の関係をもとに、発声された音声の特徴から口内の動きを推定して提示することにより、発声障碍患者が自律的に発声訓練をおこなえるシステムに関する研究をおこなった。
喉頭発声音声との比較・分析から、食道発声音声については、雑音性が高い、基本周波数が低い、倍音構造が不明瞭、スペクトル包絡が平坦、発声が不安定であるといった特徴がみられた。一方の脳性麻痺患者の音声は、基本周波数が高く(男女共に300〜400Hz)、パワースペクトルのエネルギーが2kHzまでの低域に集中しており、フォルマントが不明瞭かつ独特の共鳴特性を持ち、更に発声が不安定であることがわかった。これらをもとに発声原理に基づいたリアルタイム明瞭化フィルタを構成した。
マイクからの入力音声に対してまず母音/子音の判別を行う。母音であれば基本周波数を抽出し、ピッチを喉頭発声音声の帯域へシフトした上で、櫛形フィルタにより倍音構造を明瞭化する。一方、子音に対しては振幅の増幅をおこなう。更に時間軸上では、子音の立ち上がりおよび母音発声時の定常状態を安定化させるための包絡処理をおこない、出力音声を生成する。
本手法の有効性を調べるため、アンケートによる聴取実験を行った。フィルタリング前後の音声の対を合計12対作成し、ランダムな順でスピーカーから呈示した。成人健聴者に対し、11の評価項目について5段階評価でアンケートに答えてもらった結果、本提案フィルタの有効性が示された。
今後は、基本周波数の高精度抽出と音声の自然性、個人性保持に関する検討を更に進めるとともに、発声障碍音声フィルタリング機器の実用化を目指していく。

人間の発声器官は主に、肺、声帯、声道、舌、口蓋とそれらを動かす筋肉などから成り、互いに適当な位置や形状を形成することにより音声が生成される。本研究は、人間と同等の発声器官を機械系によって構成し、聴覚フィードバックと自己学習によって人間の音声学習過程を再現し、また人間らしい発声手法を獲得させることを目的としてきた。
人間の発声機構は大きく分けて、声帯振動による音源の生成と、声道部共鳴によるホルマントの付加という二つの働きによって構成されている。主として顎や舌の非定常な動作によって引き起こされる変化から子音が生成され、母音は定常的な声道形状を形成することによって生成される。この発声機構を、エアコンプレッサ、圧力調節弁、流量調節弁、人工声帯、調音用共鳴管、マイクロホン、音声解析用計算機から成る機械系モデルによって構築した。7つのモータ(流量操作:1、声帯操作:1、共鳴管操作:5)によって機械系をダイナミックに制御することにより、聴覚フィードバックを用いた学習によって人間のように機械系が自ら音声を獲得できることを示した。
特に本年は、音声から発声動作を逆推定する処理にニューラルネットワーク(NN)を適用した。学習時において、NNはシステムが生成した音響の音響パラメータを入力とし、そのときの制御パラメータを教師信号とすることで、ある音響を生成するために必要な声道の形状を学習する。学習後、NNを声道モデルに対し直列につなぐ。NNに生成したい音響の音響パラメータを入力し、出力された制御パラメータから求められる声道の形状に変化させることによって、望む音響を生成することが可能となった。今後は、機械モデルによる音声獲得と人間の学習機構の解明を通して、新しいヒューマンインタフェース実現に向けての検討をおこなっていく。

A novel micro-vibration actuator using a shape-memory alloy (SMA) for the presentation of various tactile sensations to human skin will be introduced in this paper. A tactile actuator consists of a thin SMA wire with the diameter of 50μm, and generates micro-vibrations with various frequencies and amplitudes synchronizing with a driving pulse signal. By coupling the actuators as a pair, an information transmission system is created for presenting higher-level tactile perceptions such as the phantom sensation and the apparent movement to transmit novel tactile sensations to human skin. The apparent movement was especially well perceived by users as a sensation of something running across their fingers, or as being tapped by an object, according to well-determined signals given to the devices. The actuator features compactness and low energy consumption, and is applied to tactile displays and tactile interaction systems.

Humans are able to communicate with each other by using not only verbal media but also the five senses such as vision, audition, olfaction and tactility. Our emotions and feelings are directly transmitted through non-verbal media, especially tactile sensation senses the physical information about an object or an environment. We have paid attention to the information transmission through the higher-psychological perception of tactility, and are now constructing a tactile transmission system for presenting stroking sensation. This paper introduces the development of a tactile display using a shape-memory alloy, and describes the transmission of stroking sensation by the higher-psychological perception such as the phantom sensation (PS) and the apparent movement (AM) of the tactility. © 2007 SICE.

A talking and singing robot which adaptively learns the vocalization skill by an auditory feedback learning is being developed. In vocalization, the vibration of vocal cords generates a source sound, and then the sound wave is led to a vocal tract, which works as a resonance filter to determine the spectrum envelope. The robot consists of motor-controlled vocal organs such as vocal cords, a vocal tract and a nasal cavity to generate a natural voice imitating a human vocalization. An adaptive learning algorithm using a neural network is introduced for the achievement of the autonomous acquisition of vocalization skill. In the learning phase, the robot vocalizes 150 patterns of voices by ramdomly giving the motor-control parameters, and the generated voice characteristics and the corresponding motor-control parameters are associated with each other by using a self-ornanizing neural network. After the learning, the neural network is connected in series into the vocal tract model. By inputting sound parameters of desired sounds to the NN, the corresponding form of the vocal tract is obtained to vocalize desired sounds. The robot is applied to the training system of speech articulation for the auditory impaired people, because the robot is able to reproduce their vocalization and to teach how they are improved to generate clear speech. The paper briefly introduces the mechanical construction of the robot and how it autonomously acquires the vocalization skill in the auditory feedback learning by listening to human speech. Then the training system is described, together with the evaluation of the speech training by auditory impaired people.

A talking and singing robot which adaptively learns the vocalization skill by an auditory feedback learning is being developed. In vocalization, the vibration of vocal cords generates a source sound, and then the sound wave is led to a vocal tract, which works as a resonance filter to determine the spectrum envelope. The robot consists of motor-controlled vocal organs such as vocal cords, a vocal tract and a nasal cavity to generate a natural voice imitating a human vocalization. An adaptive learning algorithm using a neural network is introduced for the achievement of the autonomous acquisition of vocalization skill. In the learning phase, the robot vocalizes 150 patterns of voices by ramdomly giving the motor-control parameters, and the generated voice characteristics and the corresponding motor-control parameters are associated with each other by using a self-ornanizing neural network. After the learning, the neural network is connected in series into the vocal tract model. By inputting sound parameters of desired sounds to the NN, the corresponding form of the vocal tract is obtained to vocalize desired sounds. The robot is applied to the training system of speech articulation for the auditory impaired people, because the robot is able to reproduce their vocalization and to teach how they are improved to generate clear speech. The paper briefly introduces the mechanical construction of the robot and how it autonomously acquires the vocalization skill in the auditory feedback learning by listening to human speech. Then the training system is described, together with the evaluation of the speech training by auditory impaired people.

In the vocal communication, we are able to detect the position of a source sound in 3D space, extract particular sound from mixed sounds, and recognize what the sound is. An ability that extracts and recognizes particular sound in noisy environment is called cocktail party effect. By realizing this mechanism with a computer, new applications will be presented to be utilized in the communication with humans. In this study, musical sounds are dealt for the tracking of sounds generated by musical instruments. Musical sounds have different features from human voice. For example, instrumental sounds have variety of spectra distributed in wide frequency range. In this study, a system that can deal with various kinds of sounds will be developed. For the estimation of musical pitch, this study focuses on musical sounds with different pitches generated from musical instruments, and a filtering technique using comb filters is being developed. A recorder, harmonica and keyboard-harmonica were selected as target musical instruments in the experiment and the result showed this technique was effective for the estimation of musical pitches. © 2007 SICE.

A talking and singing robot which has been constructed by referring to the human vocal system will be introduced in this paper. Although various ways of vocal sound production have been actively studied so far, mechanical construction of human vocal system is considered to advantageously realize natural vocalization with its fluid dynamics. The robot has several motors to manipulate the vocal tract and the vocal cords. The robot autonomously learns the relations between motor control parameters and the generated vocal sounds using an auditory feedback learning with neural networks, and sings a song by mimicking a human vocalization. This paper presents the construction of the talking robot and its singing performance, together with the adaptive control for the pitch and phoneme learning.

The paper introduces the development of a tactile device using a shape-memory alloy, and describes the information transmission by the higher-level perception such as the apparent movement and phantom sensation of the tactility. The authors paid attention to the characteristic of a shape-memory alloy formed into a thread, which changes its length according to its temperature, and developed a vibration-generating actuator electrically driven by periodic signals generated by current control circuits, for the tactile information transmission. By coupling the devices as a pair, an information transmission system was constructed for presenting the apparent movement of the tactility, to transmit quite novel sensation to a user. The information transmission by the device was tested by 10 subjects, and evaluated by questionnaires. The apparent movement was especially well perceived by users as a sensation of something running across fingers or as being tapped by something, according to the difference of signals given to the devices. Several users reported the rubbing sensation given by the AM, and we further experimented the presentation of the sensation in detail.

Gestures play one of the most important rolls for the expression of intention such as emotion and feelings in nonverbal human communication. When human gives an intention to a gesture, it could be obtained as the difference of force. The force applied to the body is measured as the acceleration derived from gestural trajectory. Therefore, the intention of gesture is considered to appear in the velocity and acceleration. This paper presents the extraction of intention from gestures based on the resampling algorithm applied to the gestural trajectory. By comparing the temporal trajectory with the resampled trajectory, an intention given to the gesture could be extracted and assessed. The proposed method was applied to the evaluation of intentions given to Japanese sign-language words.

The paper introduces the development of a tactile device using a shape-memory alloy, and describes the information transmission by the higher-level perception such as the apparent movement and phantom sensation of the tactility. The authors paid attention to the characteristic of a shape-memory alloy formed into a thread, which changes its length according to its temperature, and developed a vibration-generating actuator electrically driven by periodic signals generated by current control circuits, for the tactile information transmission. By coupling the devices as a pair, an information transmission system was constructed for presenting the apparent movement of the tactility, to transmit quite novel sensation to a user. The information transmission by the device was tested by 10 subjects, and evaluated by questionnaires. The apparent movement was especially well perceived by users as a sensation of something running across fingers or as being tapped by something, according to the difference of signals given to the devices. Several users reported the rubbing sensation given by the AM, and we further experimented the presentation of the sensation in detail.

Gestures play one of the most important rolls for the expression of intention such as emotion and feelings in nonverbal human communication. When human gives an intention to a gesture, it could be obtained as the difference of force. The force applied to the body is measured as the acceleration derived from gestural trajectory. Therefore, the intention of gesture is considered to appear in the velocity and acceleration. This paper presents the extraction of intention from gestures based on the resampling algorithm applied to the gestural trajectory. By comparing the temporal trajectory with the resampled trajectory, an intention given to the gesture could be extracted and assessed. The proposed method was applied to the evaluation of intentions given to Japanese sign-language words.

A talking and singing robot which has been constructed by referring to the human vocal system will be introduced in this paper. Although various ways of vocal sound production have been actively studied so far, mechanical construction of human vocal system is considered to advantageously realize natural vocalization with its fluid dynamics. The robot has several motors to manipulate the vocal tract and the vocal cords. It became possible to adaptively learn the relations between motor positions and the produced vocal sounds by an auditory feedback, and produce Japanese five vowels and consonants by mimicking a human speech. In addition, the robot generates vowel and consonant sounds of different pitches by dynamically controlling the vocal cords, vocal tract and nasal cavity using its auditory system.

Humans communicate with each other by using not only verbal media but also five senses effectively using their body parts. This paper describes the recognition of gaze and facial parts for the construction of the face interface. The extraction of individuality based on the facial parts motion is also discussed. The face interface is considered to be effectively and easily used by elderly persons and physically challenged who have difficulties in using the conventional interface devices.

Although almost all the animals have voices, only human beings are able to employ speech to make verbal communication. The vocal sound is produced by the relevant operations of the vocal organs such as a lung, trachea, vocal cords, vocal tract, tongue and muscles. If any part of the vocal organs is injured or disabled, we may be involved in the impediment in the vocalization and, in case of the worst, we may lose our voices. The paper presents a digital filtering algorithm which clarifies dysphonic speech with the speaker's individuality preserved. The study deals with the clarification of esophageal speech and the speech of patients with cerebral palsy, and the filtering ability is being evaluated by listening experiments.

近年の画像処理技術の発達により、様々な画像や仮想空間を計算機で作り出すことができる。しかし既存のマウスとGUIによるインタフェースでは、直接的、直感的な操作が困難であるという問題点があげられる。本研究では、3段のアームと5つの角度センサによって保持されたモニタデバイスを制作し、角度センサからの入力情報を利用することでモニタへの力覚操作によって計算機を扱うことのできるインタフェースを開発した。またモニタ上部にカメラを装着してユーザの視線を認識することにより、視線と力覚によりモニタ内の仮想オブジェクトあるいは仮想空間を操作できるシステムを構築したので報告する。In the human to human communication, we are able to communicate with each other by using not only verbal media but also the five senses. With the widespread availability of computers, the media which directly affect our emotions will be effectively utilized together with the audio and images. In this paper a monitor device driven by gaze and haptic manipulation will be introduced together with two application systems, in which a user is able to directly manipulate a virtual 3D object with the gaze and haptic senses.

In the human to human communication, we are able to communicate with each other by using not only verbal media but also the five senses. With the widespread availability of computers, the media which directly affect our emotions will be effectively utilized together with the audio and images. In this paper a monitor device driven by gaze and haptic manipulation will be introduced together with two application systems, in which a user is able to directly manipulate a virtual 3D object with the gaze and haptic senses.

Over 20,000 patients are currently suffered from laryngeal cancer in Japan, and the only treatment for the terminal symptoms is to remove the larynx including vocal cords, which means that he loses his voice. Esophageal speech is a method of speech production using an esophagus. In the speech air is inhaled and caught in the upper esophagus instead of being swallowed, and then the released air generates the esophagus vibration to produce a "belch-like" sound that can be shaped into speech. This method, however, has difficulties to be mastered (over 5 years practice is ordinary required), the voice is able to keep his individuality since the speech is generated by his own vocal organs. This paper introduces a software filtering algorithm which clarifies esophageal speech with his individuality preserved, together with user's evaluations by questionnaires.

現在, VR技術の発達により樣々な仮想空間を計算機内で作り出すことができる。しかし, 既存のインタフェースではVR空間への直感的な操作が困難であるという問題点があげられる。本研究では, アームと5つの角度センサによって保持されたモニタデバイスを作成し, ユーザの力覚による直接的でかつ直感的操作が可能なインタフェースデバイスを開発した。またモニタの位置, 姿勢データを実時間で計算可能な点に注目し, モニタの力覚動作に基づくジェスチャ認識アルゴリズムを提案する。

Human is required to perform a complex operation of physical apparatus such as lung, trachea, vocal cord, vocal tract, tongue and muscle in producing a particular sound. In this research, we are constructing a model of human's phonetic system that is equipped with air pump as a substitute for human's lung, artifical vocal cord and artificial vocal tract, that is expected to produce natural biological sound. The aim of the model is to mimic a particular sound spectrum that is given as an input. For controlling the model, we apply a neural network that is trained on input-output relation of the individual model, rather than calculating PARCOR coefficients.

Recently, there are many researches for sign language which is the most important means of communication for a person with auditory disability. The principal aim of these research is recognition and synthesis of sign language, and the system which can search and store sign words is necessarily required. The sign language databases are mostly consist of motion pictures and index words, which makes it difficult that we search meaning from gesture. This paper describes the multimodal database which have information obtained from not only movie but also acceleration sensor, position sensor and dataglove. We are constructing a bi-directtional data retrieval system to use key words and gestures.

We are constructing a phonetic machine having a vocal chord and a vocal tract based on mechatronics technology, and have so far developed a pitch generation part as a subsystem for melody synthesis to sing in humming. In the pitch generation, the analysis and mechanical simulation of the behavior of the vocal chords are required. The fluid mechanical system is, however, less stable to make the control difficult. This paper presents a singing instrument system together with an adaptive tuning algorithm of the physical mechanism using an auditory feedback. The mechanical method is considered to be promising to generate more natural voice than algorithmic sound synthesis methods.

計算機を用いた音声合成はアルゴリズムによる手法が主流となっているが、その合成音は自然な歌声の生成に適当なものとは言えない。一方、人間の発声機構を物理的に構成することにより、より人間らしい音声を生成できると考えられる。また、このような機械系に聴覚フィードバックをつけて計算機によりダイナミカルに制御することによって、人間が発声技術を獲得したり、或いは練習によって声色をまねる過程をシミュレートすることも可能であろう。我々は現在、計算機制御された力学系による歌声生成システムの製作を試みている。音声生成に必要なピッチ、ホルマント等を生成するためのパラメータと機械系の動作を対応付け、計算機で実時間制御することにより自然な歌声を生成することが目的である。本報告では、声帯部に人工声帯を用いて声の高さ及び音量を適応的に制御することにより、ハミングの歌声を生成するシステムについて述べる。

我々の日常生活における情報伝達、中でも意志や感情の伝達においては、言語以上に非言語的手段(ノンバーバルコミュニケーション)によるところが大きいと言われる。何気なく用いているジェスチャーが機械によって理解できれば、伝達者の感情や情緒などの微妙な表現にも対応できる自然な入力デバイスが実現できる。ジェスチャー情報は障害者用のコミュニケーションエイドに用いられている一方で、音楽にも良く利用されている。これは楽器が、動作から音響への感性的な変換器であるためである。筆者らもデータグローブや画像処理を応用して動作によって音楽を制御する試みを行ってきたが、システムが複雑で高価になるという問題があった。本研究ではジェスチャーの中でも特に、「手振り」の加速度センサによる認識を取り上げた。手振りにおいて、人間の意志や感情は、その空間的な位置よりも、手に加えられる力に、より顕著に現れると考えられる。力の測定法は種々あるが、運動中に働く力は加速度によって検出することができる。加速度センサには構造が簡単であるという利点もある。また、身体にセンサあるいはマーカーなどを取り付けることによって得られる軌跡から力を算出する従来の方法では、時間的遅れが問題となる。ここでは、3次元加速度センサを用いた手振りの認識と、実時間サウンド制御システムの試作について報告する。

コンピュータを用いた実時間音楽制御システムの開発は従来、機械が人間の過去の動作を解析して次の動作を予想し決めていくという、あくまでも人間への機械追従形モデルとして進められてきた。しかし人間と機械が実時間で同期演奏を行う場合、機械の動作に対する人間の反応を考慮に入れる必要がある。筆者らはすでに人間も機械に影響せ受けるという相互作用を考慮した自動伴奏システムについて報告したが、そこではテンポ(速度)の追従を主として扱ってきた。本論では、さらに位相(位置)の追従も陽に含めた一般的なモデルを考え、音楽演奏における人間と機械の協調動作について検討した。

筆者らはこれまでに、計算機を用いた墨字楽譜・点字楽譜の自動認識、相互変換システムの構築、さらに実用化実験などを行ってきた。今回、初心者のための点字楽譜の学習ツールとして、楽譜記号と点字楽譜記号との個々の記号対応を、計算機を用いて容易に学べる学習支援システムを作成したので報告する。

近年、楽譜の自動作成システムが実用化されてきている。楽譜は、本来、曲を綴るものであるが、一方では、見た目に美しく且つ演奏し易いことが要求される。楽譜の作成には、種々、微妙な配慮が必要であるが、基本的には楽譜記号、五線幅あるいは五線長などは譜面の大きさにより一定の書式があり、また五線上の音符の上下の位置、即ち音高は既に曲により与えられているので、残された大きな自由度は音符の横方向の配置である。ここで、最も基本となる原則は、「音符間隔は音長に比例し、同種音符から成る場合は等間隔とする」というものである。しかし、その等間隔の長さも異種音符相互の音符間隔及び音符とそれ以外の記号との間隔も特別な規定はない。この不確定な部分を規定する一つの法則を見出すため、楽譜を、音符をはじめとする楽譜記号で構成される一つのポテンシャル場で表現する。この目的は、楽譜の自動作成への応用と同時に、楽譜の演奏のし易さ見た目の美しさという音楽・芸術のもつ感性的な面と論理的なポテンシャル場との対応を考えることにある.ここでは、簡略化のため1パートからなる単純な楽譜を対象とする。

形状記憶合金(SMA)の相変態は熱の移動によって引き起こされるため、これまでは高速動作が要求されるようなアクチュエーションには不向きであると考えられていた。申請者らはこれまでの研究で、SMAワイヤにパルス電流を流すと、その周波数に同期して300Hzを超える周波数まで、十分に触知が可能な振動が発生することを見だした。本研究では、その高速伸縮動作と時間応答を精密に測定し、物理シミュレーションと共にその物性的特性を考察した。更に、SMAの変態動作のダイナミクスならびに変態量を正確に制御する手法を提案し、これを人の皮膚に対して様々な触覚感覚を提示するデバイスおよびシステムへと応用した。

申請者らはこれまでに、形状記憶合金(ShapeMemory Alloy: SMA)の細線がパルス電流により高速に振動を起こす現象を発見した。本研究では、SMAワイヤの微小振動を高速度カメラで撮影、観測することにより、振動形態のダイナミクスの解析をおこなった。また、パルス電流の周波数、デューティ比、振幅などを制御することにより、様々な振動と触覚刺激が得られることを見出した。超小型・低消費電力駆動が可能な微小振動アクチュエータとしての特性の調査をおこない、微小振動を利用して手指に様々な触覚感覚を提示可能な触覚ディスプレイを構築した。更に、新しい触覚インタフェース装置として実装と評価をおこなった。

　ジェスチャは音声とともに、人間どうしのコミュニケーションにおいて重要な役割を果たしている。特に言語では表せない感情や感覚の表現には、身振り手振りが多様に用いられている。ジェスチャにおける人間の意志や感情は、手腕部などの位置よりもむしろ身体に加えられる力に顕著に現れることに着目し、腕部運動の計測には小型加速度センサを用いた。さらにハンドジェスチャセンシングデバイスGraspComの開発を行った。これらをもとに、日本語手話認識システムの構築および、力覚による双方向コミュニケーションシステムの試作を行い、新しいヒューマン・マシンインタフェースについて検討した。　身体のダイナミックな動きを計測するために小型３次元加速度センサを、手指の静的な位置や形状の測定には位置センサ及びデータグローブを用いることにより、運動特徴に基づいた、手話の実時間認識システムを構築した。約3000単語ある日本語手話の多くは、いくつかの単純なジェスチャーの組み合わせで表すことができる。そこで１１個の動作方向を基本動作として取り上げ、各手話単語を基本動作の列として記述し、これをＨＭＭ（隠れマルコフモデル）を用いて参照することにより認識を行った。自己紹介や日常生活でよく使われる３０程度の単語に対し、９０％以上の認識率を得た。　また、力覚入出力デバイスの開発を行った。本デバイスは人体の皮膚感覚を実現するためにシリコンで整形され、デバイスの変形量の測定には内圧測定センサを、姿勢の計測には小型加速度センサを用いている。更に、ハンドジェスチャ入力に対してリアクションを提示するために、小型振動モータを内蔵している。把持力にともなうデバイスの変形を、内部に埋め込んだ各センサにより計測すると同時に、人間が「もっともらしい」と感じるリアクションを力覚出力として生成することが可能である。GraspComをターミナルデバイスに用いることにより、遠隔地におかれた人どうしの力覚による双方向コミュニケーションシステムを試作し、アンケート結果から好意的な評価を得ることができた。　今後は、手話動作及びハンドジェスチャに含まれるより細かい感性的表現の解析をおこなっていくと共に、ジェスチャー入出力に基づく新しいマン・マシンインタフェースへの可能性を検討して行きたい。