Recently, Japan is reaching a super aging society. The staying alone elderly person tend to be less likely to talk with another, to have no one to rely when in trouble. The dialogue agent is expected to eliminate the social isolation of such as those elderly person living alone. One of roles of dialogue systems which users talk to by non-task-oriented is to share mental states. When users can not accept paralinguistic information that dialogue systems response to users' expressed mental state, users' dialogue continuance desire is decreasing. This study aims to solve such a problem of a failure of dialogue. Assuming a dialogue between the user and the system, we prepared speech stimulation connected the fixed users' utterance and the system response then conducted a listening evaluation experiment. For the system response, we used general purpose backchannel "Soudesuka". Through the analysis, the acceptance range of the system response diversity focusing on paralinguistic information was investigated. About the acceptance range, measurable correlation was seen in valence state of users' utterance and the system response. From discussion of research result, it leads to the proposal of the guideline for making the feature of the paralinguistic information of the system response to the users' utterance acceptable.

Linguistic alignment refers to the use of similar words to a conversational partner. In this research, we analyzed feature values of linguistic alignment in human-human non-task-oriented dialogue. Based on the obtained feature values, we then verified the influence on users by dialogue robots with linguistic alignment taking users' attributions into account. We conducted a control experiment using Wizard of Oz method for 38 subjects. The experimental conditions were divided into low-frequency, medium-frequency, and high-frequency according to the frequency of linguistic alignment during one dialogue. For each frequency condition, an experimental group (linguistic alignment) and a control group (backchannel) were set up. After each experiment, the subjects were required to answer three questions relating to empathy and desire of continuing dialogue using five-point Likert scale. As a result, the influence on users' empathy and desire of continuing dialogue by dialogue robots with linguistic alignment is investigated. Furthermore, it is suggested that the more negative the user's attitude toward the robot, the higher the effect of linguistic alignment.

本稿では,日本のポピュラー音楽のメロディ聴取時における,音楽ジャンル印象を考慮した自動編曲手法の提案を行う.従来の編曲手法と異なり,音楽ジャンル印象に基づいた編曲を行う点,複数の音楽ジャンルが混在したメロディの編曲を行う点から,多様な編曲結果を得ることを目標としている.日本人の被験者を対象とした印象評価実験を行い,メロディの特徴から音楽ジャンル印象の推定を行うモデルの構築を行った.また,モデルの構築結果を考慮し,ユーザが入力したメロディに対し,音楽ジャンル印象を任意の値に調節したメロディへ編曲を行うことができる,メロディ自動編曲システムの開発を行った.

We propose a humor utterance generation method that is compatible with dialogue systems, to increase "desire of continuing dialogue". A dialogue system retrieves leading-item: noun pairs from Twitter as knowledge and attempts to select the most humorous reply using word similarity, which reveals that incongruity can be explained by the incongruity-resolution model. We consider the differences among individuals, and confirm the validity of the proposed method. Ex-perimental results indicate that high-incongruity replies are significantly effective against low-incongruity replies with a limited condition.

The characteristics of the spoken language used to address infants have been eagerly studied as a part of the language acquisition research. Because of the uncontrollability factor with regard to the infants, the features and roles of infantdirected speech were tried to be revealed by the comparison of speech directed toward infants and that toward other listeners. However, they share few characteristics with infants, while infants have many characteristics which may derive the features of IDS. In this study, to solve this problem, we will introduce a new approach that replaces the infant with an infant-like robot which is designed to control its motions and to imitate its appearance very similar to a real infant. We have now recorded both infant-and infantlike robot-directed speech and are constructing both corpora. Analysis of these corpora is expected to contribute to the studies of infant-directed speech. In this paper, we discuss the contents of this approach and the outline of the corpora.

In this study, we propose humor generate method for Non-task-oriented Dialogue System using Twitter. We have been aiming to establish the design of dialogue systems with desire of continuing interaction by analyzing the factors to feel, "want to chat with next time". We confirmed dealing humor is valid for desire of continuing interaction. In this paper, we proposes a method which dialogue system can automatically generate humor with knowledge, extract from Twitter as Modifier-Noun pair and Value-Predicate clauses pair. And in Evaluation experiment, We confirmed proposal method can generate humor.

We aimed at improvement of desire of continuing interaction with a spoken dialogue system through the three cases of construction of spoken dialogue system. System utterances with humor, control of speech rate of system utterances and estimation of user's personality based on user's utterances are effective for improvement of desire of continuing interaction.

In this study, we analyzed the change of manner of speech with the level of intimacy. For our experiments, we collected two kinds of dialogue data: the initial meeting dialogue data we called "low intimacy" and that after six months we called "high intimacy". In our experiments, subjects listened to the dialogue of three pairs of speakers, and evaluated their impressions of the manner of speech through a questionnaire. Analyzing the results, we extracted four significant factors: "Liveliness", "Pleasantness", "Fluency" and "Speed". Comparing the factor scores for the low intimacy dialogues with the high intimacy dialogues, we found similar results for different partners in the low intimacy dialogues, butdifferent factor scores for different partners in the high intimacy dialogues. In particular, the fluency score increased in the dialogues after six months.

In this study, we consider effects of an agent's grasping user features on the user's attachment to the agent. Recently, some previous studies of HAI have researched for about strategies that make users continue to use spoken dialogue systems long period of time. In this research, we suggest a system estimate user's degree of activeness from prosody and accumulate that with a correct label decided from user response as training data for following estimation. Our results show our system performs more stable estimate, and higher estimation accuracy makes users conscious more intense attachment.

<p>高齢者や障害者をユーザとして想定した場合，ユーザへの負荷が小さいシステムが必要となる．しかし，現状の多くのシステムでは自然な対話が実現できておらず，ユーザに負担を課している．本研究では，非母語・母語対話における話者交替のタイミングを分析し，非母語話者-母語話者間で話者交替の様相が異なることを示す．この結果は，母語の対話では，参与者が互いに負荷を軽減するように話者交替を行っているこが示唆された．</p>

In this research, we tried to construct an evaluation scale for singing voice of popular music. In this paper, we considered the effectiveness of the scale in amateur singing voice and the factor of evaluation for singing voice. As a result, we constructed the scale with 12 words and confirmed its reliability. And, we found a characteristic factor in singing voice which differs from speaking voice.

We propose the phoneme acquisition model that simulates human auditory system that perceives the temporal variations of the speech. Our model is designed to learn values of the acoustic characteristic of a continuous speech and to estimate the number and boundaries of the phoneme categories without using explicit instructions. A purpose of this study is to clarify the role of an Infant-Directed Speech (IDS), that has the high variability. We performed a learning experiment of an Adult Directed Speech (ADS) and IDS using our model and compared the results. As a result, in the accuracy rate of the voiceless stops, IDS is significantly higher than ADS. Accordingly, it is possible that IDS emphasized the acoustic features of unsteady phonemes, and to assist the acquisition.

<p>健常な乳児は生後一年以内に母国語の音韻を獲得する。乳児は周囲の成人の言語入力に基づいた学習を行っていると考えられるが、詳細なメカニズムは明らかになっていない。そこで我々では、自然な連続音声の入力と、ヒトの認知能力を模擬したニューラルネットワークモデルによる音韻体系範疇化のシミュレーションを行っている。本研究では、母音と子音を音声の動的特徴に基づいて同一のモデルで取り扱う手法について報告する。</p>

This paper describes a method of estimation of stress by speech analysis technology for evaluation of speech interface. First, we conducted an experiment of evaluating a speech interface and observed participants' action and speech with some troubles in use of the interface. Some participants seem to feel anger and anxiety in using the speech interface and we focus on 'anxiety' which derives from the problem of timing to speak or designing of system functions. Then, we collected utterances which were spoken in condition of 'anxiety' on the simulated environment of a speech interface. We made a decision tree from the collected utterances and confirmed the high accuracy of 'anxiety' estimation. Finally, we applied the tree to the utterances in the experiment of evaluating a speech interface. The accuracy was high (71.9-90.0%) and it was confirmed that our method of estimation of 'anxiety' by speech analysis technology is effective to find out problems on use of a speech interface.

The human-human interactions in a spoken dialogue seem to use not only linguistic information in the utterances but also some sorts of additional information supporting linguistic information. We call these sorts of additional information "para-linguistic information". In this paper, we present a recognition method of attitudes by prosodic information, and a recognition method of head gestures. In the former method, in order to recognize two attitudes, such as "positive" and "negative", FO pattern and phoneme alignment are introduced as features. In the latter method, in order to recognize three gestures, such as "nod", "tilt" and "shake", left-to-right HMM is introduced as the probabilistic model as well as optical flow is introduced as features. Experimental results show that these methods are sufficient to recognize user's attitude as para-linguistic information. Finally, we show a proto-type spoken dialogue system using para-linguistic information and how these sorts of information contribute the efficient conversation.

The human-human interactions in a spoken dialogue seem to use not only linguistic information in the utterances but also some sorts of additional information supporting linguistic information. We call these sorts of additional information "para-linguistic information". In this paper, we present a recognition method of attitudes by prosodic information, and a recognition method of head gestures. In the former method, in order to recognize two attitudes, such as "positive" and "negative", F0 pattern and phoneme alignment are introduced as features. In the latter method, in order to recognize three gestures, such as "nod", "tilt" and "shake", left-to-right HMM is introduced as the probabilistic model as well as optical flow is introduced as features. Experiment results show that these methods are sufficient to recognize user's attitude as para-linguistic information. Finally, we show a proto-type spoken dialogue system using para-linguistic information and how these sorts of information contribute the efficient conversation.

Outline of the X-JToBI intonation labeling scheme, the extended version of the J_ToBI, is presented. The main motive for the extension being consisted in its application for spontaneous speech, the following extensions were introduced: 1)Exact match between the time-stamp of tone labels and the timing of physical events, 2)Enlargement of the inventory of Boundary Pitch Movement, 3)Extension and ramification of the usage of Boundary Indices, and 4)Proposal for the labeling of filled-pause and non-lexical penult prominence.

This paper presents an intelligent system for information retrieval based on human-machine dialogue through spoken language with novel features such as use of key concepts, unknown word processing, dialogue management through user and system modeling, and automatic acquisition of knowledge to adapt the system to individual users. It then describes an experimental system for academic information retrieval constructed to implement these features and to demonstrate their feasibility.

In this research, we aim at the establishment of a method of controlling dialogues in accordence with the changes of performance in computation. So that the system takes the most suitable dialogue strategy according to performance in computation, it needs to calculate evaluation functions modeled with performance in computation in the inside all time. In this paper, we model dialogue control centered on 'system transparency' with computation time and accuracy of system's processes. Then, we simulate this model by controlling dialogue strategies. We also confirm effectiveness of the model by preliminary experiment using the prototype of spoken dialogue system.

In this research, we focus on two main topics about the model for use of non-verbal information. The first topic is the clarification of the difference of the model between using non-verbal information by human beings and using non-verbal information by robot. The other topic is the clarification of the difference of the model between the CG robot and the real robot. As a result, we clarified the strength of constraint and naturalness of various types of non-verbal information. We also confirmed that appropriate output timing of non-verbal information with the CG robot is the start of utterances. This is the same as the usage of non-verbal information in human-human dialogue. Moreover, pie confirmed non-verbal information made speaker-change more smoothly for the humanoid robot than in the case of the CG robot.

This paper describes controlling robot's gaze which has relation to smoothness of turn-taking in communication. We considered the role of gaze in dialogues between human beings and examined it by CG simulation and our humanoid Also we analyzed the features of gaze movement in dialogues by plural persons and confirmed that controlling gaze is efficient in confirmation of communication channel by implementing it on the humanoid.

In natural dialogue, time constraint influences on speaker's plan of utterance generation. We analyzed influence of time constraint in natural dialogue on the dialogue corpus. We tried to make them clear by measuring time of speaker's turn and analyzing tendency in some situations. As the result, it was indicated that influence of time constraint on hearer's point had appeared with participant-dependency. Also it was indicated that function of utterance, modality of dialogue, and positiveness of participant had influences on time constraint.

This study aims at investigating influences of humans’ personal characteristics on forming impression of agent through human-agent interaction. We conducted an experiment in which subjects have some interaction with an agent or a human and form impression toward them. The result showed that subjects who has no experiences of programming tend to evaluate an agent lower than a human. Also subjects of the “high emotional-warmth” group tend to evaluate an agent lower than a human.Also we proposed a humor utterance generation method which compatible with dialogue system, to increase desire of sustainability. Through the experiment, we confirmed validity of the method. From the result, we observed high-incongruity reply is significantly effective against the low-incongruity and random reply. Finally we confirmed generating humor utterances is effective for increase desire of sustainability in interaction with dialogue system

Our research clarifies the differences of styles between four modes of Japanese sentences. Each of these modes are combination of a pair of exclusive modes: spoken/written, dialogue/monologue. We developed a method for acquiring sentences of such modes within which issues and stories are controlled.Acquired data shows that on dialogue condition, the differences of styles between spoken and written sentences are smaller than those on monologue condition. These results imply that the traits of dialogue in which talker is prompted to make quick composition of sentence and to pay more attention to listener in front of him/her decrease spoken or written specific styles

This study aims at investigating influences of humans’ personal characteristics on forming impression of agent through human-agent interaction. We conducted an experiment in which subjects have some interaction with an agent or a human and form impression toward them. The result showed that subjects who has no experiences of programming tend to evaluate an agent lower than a human. Also subjects of the “high emotional-warmth” group tend to evaluate an agent lower than a human.Also we proposed a humor utterance generation method which compatible with dialogue system, to increase desire of sustainability. Through the experiment, we confirmed validity of the method. From the result, we observed high-incongruity reply is significantly effective against the low-incongruity and random reply. Finally we confirmed generating humor utterances is effective for increase desire of sustainability in interaction with dialogue system

The differences among person marking system of several Ainu dialects have been observed and a prominent difference between Saru and Chitose dialects, which have so far been regarded as very similar dialects, was found. General tendency or historical implication of Ainu person marking system was investigated upon this observation. Difference among kinship terms of Saru and Shizunai dialects was researched using the epic texts of these dialects. The Ainu audio database was experimentally constructed and records for the database have been accumulated. Tagging to the verbs in Ainu text records for the database, focusing upon the grammatical and semantic role of each morpheme, was attempted and evaluated

In the dialogue among mother tongue adult speakers, overlapping utterances are produced in transition-relevance places (TRP). This phenomenon seems to appear as the result of some capability which reduces the mental ‘burden’ of the dialogue in the mother tongue. We examined the age to acquire this capability regarding Japanese mother tongue speakers.The timing of the turn taking of the mother tongue and the non-mother tongue speakers was compared first. It was found that the non-mother tongue speaker could not maintain TRP. Next, the timing of turn taking between adults and five-year olds or six-year olds was examined. As for five-year olds, a difference between them and adults was present, but a difference between six-year olds and adults was absent. As a result, it found that the capability was already acquired by six

The goal of the present project was to investigate how the nature of infant-directed speech(IDS) differs from that of adult-directed speech(ADS), and what functions the properties of IDS may play for infants phonological acquisition. In particular, we focused on the vowel category acquisition. In acquiring vowel categories, based on the quality differences(i. e.,/a/,/i/,/u/,/e/,/o/), SOM models that were developed for ADS were able to learn similar categories from IDS as well. The distinction between short and long vowels, however, turned out to be particularly challenging. That was because the actual duration of the vowels varied widely independent of whether it is phonologically short or long, and we are continuing our research into what additional information will be necessary for a model to acquire the long-short vowel distinction

In this study, we examined a framework of communication systems that interact with users spontaneously so as to cope with practical dialogue environments. While conventional spoken dialogue systems aimed to efficiently achieve specific speech-dialogue tasks, the framework of spontaneous communication system was developed in order to evolve these spoken dialogue systems to one that spontaneously start and continue spoken dialogues.For this purpose, three studies were conducted : (i) understanding of dialogue environment, which aims advanced human recognition and spoken dialogue recognition using image and speech signal, (ii) spontaneous communication management model, which models how to start, continue and end spoken dialogues, and(iii) speech generation and motion expression technology, which is how to present intentions of the system by utterances or motions.In the study(i), human pose estimation using stereo camera was developed. Simultaneous adoption of information of space depth of images, and shapes and textures of either human bodies or clothes realized accurate estimation of human poses. In addition, estimation of utterance intention was studied. Using characteristics of end of sentences and word N-grams, more accurate utterance intention was achieved.In the study(ii), models for a robot to start communication with a human, to continue it, and to end it were developed. we developed mental-state of a conversational partner for these purposes.In the study a speech generation technique for laughter was developed. Based on acoustical analyses of human laughter syntheses of both laughter and laughter-speech were realized. Throughout these three studies, basic framework of spontaneous communication was established

The first half of the three-year research project was devoted for the derivation of new research data from the Corpus of Spontaneous Japanese (CSJ), including,1) Prosodic and morphological information of the CSJ-Core (about 44 hours) is reorganized for the study of prosodic variations. The new data is encoded as an XML document whose hierarchical structure reflects those of Japanese prosody. The most basic node of the new XML document corresponds to so-called 'accentual phrase' of the Japanese language. (Done by Kikuchi)2) The word-origin information (i.e., Native, Sino-Japanese, Borrowing, and mixture of these) is given to the total of forty thousand short-unit-word recorded in the CSJ (by Ogura).3) Phonetic database for the study of the variation of the velar nasal in Tokyo Japanese (by Maekawa and Hibiya).4) RDB containing the whole word-from variations observed in the whole CSJ (7.52 million SUW.) (by Maekawa).Based upon these data, we analyzed language variations recorded in the CSJ including,1) Devoicing of vowels (by Maekawa ＆ Kikuchi)2) Non-lexical lengthening of vowels (by Den)3) Moraic nasalization of particles (by Koiso)4) Variation of velar nasal (by Hibiya)5) Word-form variation of the whole CSJ (by Maekawa)6) Variation in the accentual-phrase-final rising intonation (by Maekawa ＆ Kikuchi)7) Variation of morphological features at the end of sentence (by Ogura)The results of these studies were presented in international and domestic conferences, and reprinted in the 252-page final report of the project

To get substantive outcomes of e-learning courses, it is necessary for e-learning system including learning management systems to facilitate learners learning. Also it is necessary for teachers, coaches, and supporting staffs to work respectively. Teachers have three types of work: design, management and evaluation of the courses. Designing the detailed course structure is the new and important part of work for the teachers. And then online coaches appear to have a greater part of work to support the teacher and to facilitate classroom activities. Coaches have three types of work: facilitating the classroom activities, making classroom atmosphere and standards, facilitating the discussion processes. Many kind of learning management systems are now available free or commercially. The minimum functions are video streaming, bulletin board system, and testing, but these functions should be carefully designed and become more usable to get more substantive learning outcomes. Talking about the future learning environments, learner will be able to access directly his/her own working spaces by opening web brouser

Based on the analysis of college lecture class and English class at elementary school by using Observational System for Instructional Analysis originally developed by Hough and Duncan, the followings were found. The OSIA analysis, starting with transcription of class activities, takes the form of matrix and timeline.1) Compared with college class, the level of interaction was significantly higher and there were more students' talk observed in an elementary English class2) More experienced Assistant Language Teacher(ALT, native speaker of English) created higher level of interaction between teacher and students by not just starting repetition practice but waiting for students' utterances after showing visual materials at an elementary English class. This created more energetic class activities with a higher sense of participation.3) Richer facial expression of teachers increased positive class atmosphere.4) More experienced ALT used short sentences and phrases elaborately and effectively as feedback each time students answered.5) Class proceeded effectively and smoothly when appropriate artifact was used for instruction

Annotation of spontaneous speech data is a difficult task, but the maintenance of large annotated spontaneous speech database and the information retrieval of such database is all the more difficult. We proposed a XML format that can represent nearly all annotation information of the Corpus of Spontaneous Japanese. CSJ is a world's largest spontaneous speech database with very rich annotation including transcription, POS information, clause boundary information, dependency-structure information, discourse-boundary information, segment label, intonation label, and so forth.Our XML format includes 10 layers (starting with "Talk" element and ending in "Phone" and "Tone" elements) arranged according to the structure of natural language. 208 attributes covers linguistic, paralinguistic, and non-linguistic annotation of the speech data as well as various disfluency phenomena. Also, there are some attributes that are introduced to represent the format of the transcription text.We have converted all 3302 talks of the CSJ (661 hours, over 7.5 million morphemes) into XML document, and used them for the data validation purposes.Information retrieval experiments were also conducted using the XML documents. It turned out that the use of XSLT language gave satisfactory performance. Information retrievals of modest complexity could be performed within 15 to 30 minutes when a PC of ordinary performance (3Ghz CPU with 2GB memory) was used.Lastly, we developed a simple GUI-based search tool that helps naive users to make XSLT query scripts. The software is written in Java language and runs under nearly all PC platforms.The XML documents and GUI search tool will be publicly available as a part of the CSJ in June 2004

今年度の成果は以下の通りである。a)対話のリズムと韻律制御前年度までの成果に基づいて、対話における話題境界の判別を題材に、韻律情報におけるアクセント句単位でのパラメータを用いて統計的なモデルを学習し、オープンデータに対しても人間と同程度の判別精度が得られることを確認した。(白井・菊池)自然な対話システムを構築する上で重要なシステム側の相槌生成と話者交替のタイミングの決定を、韻律情報と表層的言語情報を用いて行う方法を開発した。この決定法を、実際に天気予報を題材にした雑談対話システムに実装し、被験者がシステムと対話することにより主観的な評価を行い、有用性を確認した。(中川)b)対話音声理解応用対話音声における繰り返しの訂正発話に関する特徴の統計的な分析結果を踏まえ、フレーズ単位の韻律的特徴の併用と訂正発話検出への適用を評価した。また、これらと併せた頑健な対話音声理解のため、フィラーの韻律的な特徴分析・モデル化の検討を行った。(甲斐)c)対話音声合成応用語彙の韻律的有標性について程度の副詞を用い、生成・聴覚の両面から分析を行い、自然な会話音声生成のための韻律的強勢制御を実現した。また、統計的計算モデルによる話速制御モデルを作成し、会話音声にみられる局所話速の分析を進め、自由な話速の制御を可能とした。さらに、韻律制御パラメータが合成音声の自然性品質に及ぼす影響を調べた。(匂坂)d)対話システム上記の成果をまとめ,対話システムを実装した。特に,顔表情の認識・生成システム,声表情の認識・生成システムなどを前年度までに開発した対話プラットホーム上に統合し,パラ言語情報の授受を可能とするリズムある対話システムを構築した。(小林