This paper introduces a system that can estimate the deformation process of a deformed flat object (folded plane) and generate the input data for a robot with human-like dexterous hands and fingers to reproduce the same deformation of another similar object. The system is based on processing RGB data and depth data with three core techniques: a weighted graph clustering method for non-rigid point matching and clustering; a refined region growing method for plane detection on depth data based on an offset error defined by ourselves; and a novel sliding checking model to check the bending line and adjacent relationship between each pair of planes. Through some evaluation experiments, we show the improvement of the core techniques to conventional studies. By applying our approach to different deformed papers, the performance of the entire system is confirmed to have around 1.59 degrees of average angular error, which is similar to the smallest angular discrimination of human eyes. As a result, for the deformation of the flat object caused by folding, if our system can get at least one feature point cluster on each plane, it can get spatial information of each bending line and each plane with acceptable accuracy. The subject of this paper is a folded plane, but we will develop it into a robotic reproduction of general object deformation.

This paper introduces an object model and an interaction method for a simulated experience of pottery on a potter’s wheel. Firstly, we propose a layered cylinder model for a 3D object of the pottery on a potter’s wheel. Secondly, we set three kinds of deformation functions to form the object model from an initial state to a bowl shape: shaping the external surface, forming the inner shape (deepening the opening and widening the opening), and reducing the total height. Next, as for the interaction method between a user and the model, we prepare a simple but similar method for hand-finger operations on pottery on a potter’s wheel, in which the index finger movement takes care of the external surface and the total height, and the thumb movement makes the inner shape. Those are implemented in the three-dimensional aerial image interface (3DAII) developed in our laboratory to build a simulated experience system. We confirm the operation of the proposed object model (layered cylinder model) and the functions of the prepared interaction method (a simple but similar method to actual hand-finger operations) through a preliminary evaluation of participants. The participants were asked to make three kinds of bowl shapes (cylindrical, dome-shaped, and flat-type) and then they answered the survey (maneuverability, visibility, and satisfaction). All participants could make something like three kinds of bowl shapes in less than 30 min from their first touch.

In this paper, the integration of Leap Motion controller with RoboAnalyzer software has been reported. Leap Motion is a vision based device that tracks the motion of human hands, which was processed and used to control a virtual robot model in RoboAnalyzer, an established robot simulation software. For intuitive control, the robot model was copied and transformed to be placed at four different locations such that the user watches four different views in the same graphics environment. This novel method was used to avoid multiple windows or viewports and was observed to have a marginally better rendering rate. Several trials of picking up cylindrical objects (pegs) and moving them and placing in cylindrical holes were carried out and it was found that the manipulation was intuitive, even for a novice user.

In this research, we focus on finding a new method of human-robot interaction in industrial environment. A visionbased dynamic hand gestures recognition system has been proposed for robot arm picking task. 8 dynamic hand gestures are captured for this task with a 100fps high speed camera. Based on the LRCN model, we combine the MobileNets (V2) and LSTM for this task, the MobileNets (V2) for extracting the image features and recognize the gestures, then, Long Short-Term Memory (LSTM) architecture for interpreting the features across time steps. Around 100 samples are taken for each gesture for training at first, then, the samples are augmented to 200 samples per gesture by data augmentation. Result shows that the model is able to learn the gestures varying in duration and complexity and gestures can be recognized in 88ms with 90.62% accuracy in the experiment on our hand gesture dataset.

In this paper, we introduce the three-dimensional aerial image interface, 3DAII. This interface reconstructs and aerially projects a three-dimensional object image, which can be simultaneously observed from various viewpoints or by multiple users with the naked eye. A pyramid reflector is used to reconstruct the object image, and a pair of parabolic mirrors is used to aerially project the image. A user can directly manipulate the three-dimensional object image by superimposing a user’s hand-finger or a rod on the image. A motion capture sensor detects the user’s hand-finger that manipulates the projected image, and the system immediately exhibits some reaction such as deformation, displacement, and discoloration of the object image, including sound effects. A performance test is executed to confirm the functions of 3DAII. The execution time of the end-tip positioning of a robotic arm has been compared among four operating devices: touchscreen, gamepad, joystick, and 3DAII. The results exhibit the advantages of 3DAII; we can directly instruct the movement direction and movement speed of the end-tip of the robotic arm, using the three-dimensional Euclidean vector outputs of 3DAII in which we can intuitively make the end-tip of the robotic arm move in three-dimensional space. Therefore, 3DAII would be one important alternative to an intuitive spatial user interface, e.g., an operation device of aerial robots, a center console of automobiles, and a 3D modelling system. A survey has been conducted to evaluate comfort and fatigue based on ISO/TS 9241-411 and ease of learning and satisfaction based on the USE questionnaire. We have identified several challenges related to visibility, workspace, and sensory feedback to users that we would like to address in the future.

In the brand-wise random-ordered drinking PET
bottles picking task, the overlapping and viewing angle problem
makes a low accuracy of the brand recognition. In this paper,
we set the problem to increase the brand recognition accuracy
and try to find out how the overlapping rate infects on the
recognition accuracy. By using a stepping motor and transparent fixture, the training images were taken automatically
from the bottles under 360 degrees to simulate a picture taken
from viewing angle. After that, the images are augmented with
random cropping and rotating to simulate the overlapping
and rotation in a real application. By using the automatically
constructed dataset, the Inception V3, which was transferred
learning from ImageNet, is trained for brand recognition. By
generating a random mask with a specific overlapping rate
on the original image, the Inception V3 can give 80% accuracy
when 45% of the object in the image is visible or 86% accuracy
when the overlapping rate is lower than 30%.

In this research, six brands of soft drinks are decided to be picked up by a robot with a monocular Red Green Blue (RGB) camera. The drinking bottles need to be located and classified with brands before being picked up. The Mask Regional Convolutional Neural Network (R-CNN), a mask generation network improved from Faster R-CNN, is trained with common object in contest datasets to detect and generate the mask on the bottles in the image. The Inception v3 is selected for the brand classification task. Around 200 images are taken or found at first; then, the images are augmented to 1500 images per brands by using random cropping and perspective transform. The result shows that the masked image can be labeled with its brand name with at least 85% accuracy in the experiment.

This article presents a framework for planning sidewalk-wise paths in data-limited pedestrian environments by visually recognizing city blocks in 2D digital maps (e.g. Google Maps, OpenStreetMaps) using contour detection, then applying graph theory to infer a pedestrian path from start till finish. There are two main targeted problems; firstly, several locations around the world (e.g. suburban / rural areas) do not have recorded data on street crossings and pedestrian walkways. Secondly, the continuous process of recording maps (i.e. digital cartography) is, to our current knowledge, manual and not yet fully automated in practice. Both issues contribute towards a scaling problem in which it becomes time and effort consuming to continuously monitor and record such data on a global scale. As a result, the framework’s purpose is to produce path plans that do not depend on prerecorded (e.g. using SLAM) or data-rich pedestrian maps, thus facilitating navigation for mobile robots and people of visual impairments alike. The framework was able to produce pedestrian paths for most locations where data on sidewalks and street crossings were indeed limited, but still some challenges remain. In this article, the framework’s structure, output, and challenges are explained. Additionally, we mention some works in the literature on how to make use of such path plan effectively.

In this research, six brands of soft drinks are decided to be picked up by a robot with a monocular RGB camera. The drinking bottles need to be located and classified with brands before being picked up. A Mask R-CNN is pretrained with COCO datasets to detect and generate the mask on the bottles in the image. The Inception v3 is selected for the brand classification task. Around 200 images are taken, then, the images are augmented to 1500 images per brand by using random cropping and perspective transform. The results show that the masked image can be labeled with its brand name with at least 85% accuracy in the experiment.

This paper highlights camera orientation estimation accuracy and precision, as well as proposing a new camera calibration technique using a reinforcement learning method named PPO (Proximal Policy Optimization) in offline mode. The offline mode is used just for extracting the camera geometry parameters that are used for improving accuracy in real-time camera pose estimation techniques. We experiment and compare two popular techniques using 2D vision feedbacks and evaluate their accuracy beside other considerations related to real applications such as disturbance cases from surrounding environment and pose data stability. First, we use feature points detection ORB (Oriented FAST and Rotated BRIEF) and BF (Brute-Force) matcher to detect and match points in different frames, respectively. Second, we use FAST (Features from Accelerated Segment Test) corners and LK (Lucas–Kanade) optical flow methods to detect corners and track their flow in different frames. Those points and corners are then used for the pose estimation through optimization process with the: (a) calibration method of Zhang using chessboard pattern and (b) our proposed method using PPO. The results using our proposed calibration method show significant accuracy improvements and easier deployment for end-user compared to the pre-used methods.

In this research, we proposed and implemented a finger pointing detection system with a short-range type depth camera. The most widely used pointing device for a computer interface is a mouse, and as an alternative there is a depth sensor to detect hand and finger movement. However, in the present literature, since the user must operate the cursor at a relatively large distance from the detection device, the user needs to raise his arm and make wide movements, which is inconvenient over long periods of time. To solve such usability problem, we proposed a comfortable and easy to use method by narrowing the distance between the keyboard and the pointing device. Next, we compared various depth sensors and selected one that can recognize even small movements Additionally, we proposed a mapping method between the users perceived cursor position and the real one pointed by the index finger direction. Furthermore, we compared our pointing method with a mouse and touch pad for the usability, accuracy and working speed. The results showed that it has users have better performance on continuous operation of character input from the keyboard and cursor pointing.

In this paper, an integration of Leap Motion controller with RoboAnalyzer software is proposed. Leap Motion is an inexpensive sensor which has three Infrared (IR) emitters and two IR cameras and can track 10 fingers of the two human hands. The device sends the data to the computer connected to it through its controller software which can be accessed in a variety of programming languages. In the proposed setup, the position of the index finger of the right hand is tracked in a Visual C# Server Application and the coordinates are extracted accurately with respect
to a frame attached at the center of the device. The coordinates are then sent to Virtual Robot Module (client application) in which a Coordinate System (marker) is mapped with the input it receives. Based on the movement of the index finger, the marker moves in the Virtual Robot Module (VRM). When the marker moves closer to the end-effector of the robot, the Server application attaches the marker with the end-effector. Thereafter, any incremental Cartesian motion of the index finger is mapped to equivalent Cartesian motion of the robot endeffector. This is achieved by finding the inverse kinematics solution for the new pose of the robot EE and of the
eight solutions obtained through inverse kinematics, the solution that is closest to the current pose is selected as the appropriate solution for the new pose. The server application then updates the VRM with the new joint angles and accordingly the robot moves instantly on the software. The setup connected to a laptop running Leap Motion Controller and VRM is shown in Figure 2(a) and the workflow is illustrated in Figure 2(b).

This paper describes a path planning framework for processing 2D maps of given pedestrian locations to provide sidewalk paths and street crossing information. The intention is to allow mobile robot platforms to navigate in pe-destrian environments without previous knowledge and using only the current location and destination as inputs. Depending on location, current path planning solutions on known 2D maps (e.g. Google Maps and OpenStreetMaps) from both research and industry do not always provide explicit information on sidewalk paths and street crossings, which is a common problem in suburban/rural areas. The framework’s goal is to provide path planning by means of visual inference on 2D map images and search queries through downloadable map data. The re-sults have shown both success and challenges in estimating viable city block paths and street crossings.

The Aerial Projection of 3D Hologram Object (3DHO) which we have proposed, is a hand gesture-based control system with an interactive 3D hologram object floating in mid-air as the hand movement reference. This system mainly consists of the pyramid-shaped reflector which produces 3D hologram object, the parabolic mirror, and the leap motion controller for capturing hand gesture command. The user can control or interact with 3DHO by using their finger or a baton-shaped object. This paper is focusing on the evaluation of the 3DHO by comparing it to other 3D input devices (such as joystick with a slider, joystick without the slider, and gamepad) on five different positioning tasks. We also investigate the assessment of comfort, ease of learning, and user satisfaction by questionnaire survey. From the experimentation, we learn that 3DHO is not good to use in one-dimensional workspace task, but has a good performance in two-dimensional and three-dimensional workspace tasks. From questionnaire results, we found out that 3DHO is averagely comfortable but may cause some fatigues around arm and shoulder. It is also easy to learn but not satisfying enough to use.

This paper presents a newly-developed calligraphy-stroke learning support system. The system has the following functions: a) displaying a brushwork, trajectory, and handwriting, b) recording and playback of an expert's calligraphy-stroke, and c) teaching a learner a calligraphy-stroke. The system has the following features which shows our contribution: (1) It is simple and compact built up with a sensor and a projector so as to be easy to introduce to usual educational fields and practical leaning situations. (2) Three-dimensional calligraphy-stroke is instructed by presenting two-dimensional visual information. (3) A trajectory region is generated as continuous squares calculated using a brush model based on a brush position information measured by a sensor. (4) A handwriting is expressed by mapping a handwriting texture image according to an ink concentration and a brush handling state. The result of a trial experiment suggests the effectiveness of the learning support function about a letter form and a calligraphy-stroke.

This article focuses on the implementation details of a portable interactive device called Image-projective Desktop Varnamala Trainer. The device uses a projector to produce a virtual display on a flat surface. For enabling interaction, the information about a user's hand movement is obtained from a single two-dimensional scanning laser range finder in contrast with a camera sensor used in many earlier applications. A generalized calibration process to obtain exact transformation from projected screen coordinate system to sensor coordinate system is proposed in this article and implemented for enabling interaction. This permits production of large interactive displays with minimal cost. Additionally, it makes the entire system portable, that is, display can be produced on any planar surface like floor, tabletop, and so on. The calibration and its performance have been evaluated by varying screen sizes and the number of points used for calibration. The device was successfully calibrated for different screens. A novel learning-based methodology for predicting a user's behaviour was then realized to improve the system's performance. This has been experimentally evaluated, and the overall accuracy of prediction was about 96%. An application was then designed for this set-up to improve the learning of alphabets by the children through an interactive audiovisual feedback system. It uses a game-based methodology to help students learn in a fun way. Currently, it has bilingual (Hindi and English) user interface to enable learning of alphabets and elementary mathematics. A user survey was conducted after demonstrating it to school children. The survey results are very encouraging. Additionally, a study to ascertain the improvement in the learning outcome of the children was done. The results clearly indicate an improvement in the learning outcome of the children who used the device over those who did not.

This paper proposes the ORB-SHOT SLAM or OS-SLAM, which is a novel method of 3D loop closing for trajectory correction of RGB-D visual SLAM. We obtain point clouds from RGB-D sensors such as Kinect or Xtion, and we use 3D SHOT descriptors to describe the ORB corners. Then, we train an offline 3D vocabulary that contains more than 600,000 words by using two million 3D descriptors based on a large number of images from a public dataset provided by TUM. We convert new images to bag-of-visual-words (BoVW) vectors and push these vectors into an incremental database. We query the database for new images to detect the corresponding 3D loop candidates, and compute similarity scores between the new image and each corresponding 3D loop candidate. After detecting 2D loop closures using ORB-SLAM2 system, we accept those loop closures that are also included in the 3D loop candidates, and we assign them corresponding weights according to the scores stored previously. In the final graph-based optimization, we create edges with different weights for loop closures and correct the trajectory by solving a nonlinear least-squares optimization problem. We compare our results with several state-of-the-art systems such as ORB-SLAM2 and RGB-D SLAM by using the TUM public RGB-D dataset. We find that accurate loop closures and suitable weights reduce the error on trajectory estimation more effectively than other systems. The performance of ORB-SHOT SLAM is demonstrated by 3D reconstruction application.

© 2016 IEEE.In this paper we present an interactive aerial projection of 3D hologram objects by using the pyramid hologram and parabolic mirror system (for 3D hologram object reconstruction) and the Leap Motion sensor (as a finger movement detector of a user). This system not only can reconstruct and project the 3D hologram object in the mid-air, but also provides a way to interact with it by moving a user's finger. There are three main steps: the reconstruction of 3D object, the projection of 3D hologram object in the mid-air, and the interactive manipulation of 3D hologram object. The first step is realized by using pyramid hologram with LCD display. The second step is achieved with the parabolic mirror hologram. And Leap Motion sensor is used for the last step to detect user finger movement. This paper traces the design concept and confirms the system function of an interactive aerial projection of 3D hologram objects by a prototype demonstration.

The purpose of this study is to realize a near-field touch interface that is compact,flexible,and highly accurate. We applied a 3-dimensional image sensor (time-of-flight camera) to achieve the basic functions of conventional touch interfaces,such as clicking,dragging,and sliding,and we designed a complete projector-sensor system. Unlike conventional touch interfaces,such as those on tablet PCs,the system can sense the 3-dimensional positions of fingertips and 3- dimensional directions of fingers. Moreover,it does not require a real touch screen but instead utilizes a mobile projector for display. Nonetheless,the system is compact,with a working distance of as short as around 30 cm. Our methods solve the shadow and reflection problems of the time-of-flight camera and can provide robust detection results. Tests have shown that our approach has a high success rate (98.4%) on touch/hover detection and a small standard error (2.21 mm) on position detection on average for different participants,which is the best performance we have achieved. Some applications,such as the virtual keyboard and virtual joystick,are also realized based on the proposed projector-sensor system.

This paper shows a method of tracking feature points to update camera pose and generating a synchronous map for AR (Augmented Reality) system. Firstly we select the ORB (Oriented FAST and Rotated BRIEF) [1] detection algorithm to detect the feature points which have depth information to be markers, and we use the LK (Lucas-Kanade) optical flow [2] algorithm to track four of them. Then we compute the rotation and translation of the moving camera by relationship matrix between 2D image coordinate and 3D world coordinate, and then we update the camera pose. Last we generate the map, and we draw some AR objects on it. If the feature points are missing, we can compute the same world coordinate as the one before missing to recover tracking by using new corresponding 2D/3D feature points and camera poses at that time. There are three novelties of this study: an improved ORB detection, which can obtain depth information, a rapid update of camera pose, and tracking recovery. Referring to the PTAM (Parallel Tracking and Mapping) [3], we also divide the process into two parallel sub-processes: Detecting and Tracking (including recovery when necessary) the feature points and updating the camera pose is one thread. Generating the map and drawing some objects is another thread. This parallel method can save time for the AR system and make the process work in real-time.

© 2015 IEEE.This paper shows a novel method to extract hand Representative Point (RP) based on 2-dimensional laser range scanner for hand motion estimation on projecting system. Image-projecting Desktop Arm Trainer (IDAT) is a projecting system for hand-eye coordination training, in which a projector displays an exercise screen on the desktop, and a laser range scanner detects trainee's hand motion. To realize multi-user HMI and expand more entertainment functions in IDAT system, an Air Hockey application was developed in which the hand RP requires a high precision. To generate hand RP precisely, we proposed our method in two parts to solve the data error problem and changeable hand contour problem. In part one, a data modifier is proposed and a sub-experiment is carried out to establish a modifying function for correcting sensor original data. In part two, we proposed three RP algorithms and carried out an evaluation experiment to estimate the reliability of three algorithms under different conditions. From the result, we get the most reliable algorithm corresponding to different situations in which the error of hand RP is less than 9.6 mm.

© 2015 IEEE.This paper shows the simulating and displaying of Puck motion in virtual Air Hockey based on projective interface for multi-users HRI Image-projective Desktop Arm Trainer (IDAT) is an upper-limb rehabilitation system for hand-eye coordination training based on projective interface. To expand more entertainment functions and realize multi-user HRI, we want to develop a virtual Air Hockey application in IDAT. To develop this application, Puck motion should be simulated correctly and displayed smoothly on the screen. There are mainly 3 problems: Data updating rate problem, virtual collision calculation and individual's hand misrecognition problem. We proposed our method into three parts corresponding to different problems. In part 1, we used multiple timers with shared memory to deal with unsynchronized data updating rate problem. In part 2, an original physical engine is designed to calculate Puck's velocity and detect virtual collision. In part 3, to deal with the individual's hand misrecognition problem, a history-based hand owner recognition algorithm is implemented to distinguish users' hands.

This paper proposes a novel technology to detect the orientation of an image relying on its contour which is noised to varying degrees. For the image orientation detection, most methods regard to the landscape image and the image taken of a single object. In these cases, the contours of these images are supposed to be immune to the noise. This paper focuses on the the contour noised after image segmentation. A polar orientation descriptor Orientation Context is viewed as a feature to describe the coarse distribution of the contour points. This descriptor is verified to be independent of translation, isotropic scaling, and rotation transformation by theory and experiment. The relative orientation depends on the minimum distance Roulette Distance between the descriptor of a template image and that of a test image. The proposed method is capable of detecting the direction on the interval from 0 to 359 degrees which is wider than the former contour-based means (Distance Phase [1], from 0 to 179 degrees). What's more, the results of experiments show that not only the normal binary image (Noise-0, Accuracy-1: 84.8%) (defined later) achieves more accurate orientation but also the binary image with slight contour noise (Noise-1, Accuracy-1: 73.5%) could obtain more precise orientation compared to Distance Phase (Noise-0, Accuracy-1: 56.3%
Noise-1, Accuracy-1: 27.5%). Although the proposed method (O(op2)) takes more time to detect the orientation than Distance Phase (O(st)), it could be realized including the preprocessing in real time test with a frame rate of 30.

This paper showcases the development of a mobile robot integrated with Projectable Interactive Surface to facilitate its interaction with human users. The system was designed to interact with users of any physical attributes such as height, arm span etc. without re-calibrating it. The system was designed in such a way that there would be no need for the human to come in physical contact with the robot to give it instructions. This system uses a projector to render a virtual display on the ground allowing us to project large displays. Microsoft Kinect integrated in the systems performs a dual functionality of tracking the user movements along with mapping the surrounding environment. The gestures of the tracked user are interpreted and an audio visual signal is projected by the robot in response.

In this paper we present an interactive aerial projection of 3D hologram objects by using the pyramid hologram and parabolic mirror system (for 3D hologram object reconstruction) and the Leap Motion sensor (as a finger movement detector of a user). This system not only can reconstruct and project the 3D hologram object in the mid-air, but also provides a way to interact with it by moving a user's finger. There are three main steps: the reconstruction of 3D object, the projection of 3D hologram object in the mid-air, and the interactive manipulation of 3D hologram object. The first step is realized by using pyramid hologram with LCD display. The second step is achieved with the parabolic mirror hologram. And Leap Motion sensor is used for the last step to detect user finger movement. This paper traces the design concept and confirms the system function of an interactive aerial projection of 3D hologram objects by a prototype demonstration.

In this paper we present an interactive aerial projection of 3D hologram objects by using the pyramid hologram and parabolic mirror system (for 3D hologram object reconstruction) and the Leap Motion sensor (as a finger movement detector of a user). This system not only can reconstruct and project the 3D hologram object in the mid-air, but also provides a way to interact with it by moving a user's finger. There are three main steps: the reconstruction of 3D object, the projection of 3D hologram object in the mid-air, and the interactive manipulation of 3D hologram object. The first step is realized by using pyramid hologram with LCD display. The second step is achieved with the parabolic mirror hologram. And Leap Motion sensor is used for the last step to detect user finger movement. This paper traces the design concept and confirms the system function of an interactive aerial projection of 3D hologram objects by a prototype demonstration.

This paper showcases the development of a mobile robot integrated with Projectable Interactive Surface to facilitate its interaction with human users. The system was designed to interact with users of any physical attributes such as height, arm span etc. without re-calibrating it. The system was designed in such a way that there would be no need for the human to come in physical contact with the robot to give it instructions. This system uses a projector to render a virtual display on the ground allowing us to project large displays. Microsoft Kinect integrated in the systems performs a dual functionality of tracking the user movements along with mapping the surrounding environment. The gestures of the tracked user are interpreted and an audio visual signal is projected by the robot in response.

This paper showcases the development of a mobile robot integrated with Projectable Interactive Surface to facilitate its interaction with human users. The system was designed to interact with users of any physical attributes such as height, arm span etc. without re-calibrating it. The system was designed in such a way that there would be no need for the human to come in physical contact with the robot to give it instructions. This system uses a projector to render a virtual display on the ground allowing us to project large displays. Microsoft Kinect integrated in the systems performs a dual functionality of tracking the user movements along with mapping the surrounding environment. The gestures of the tracked user are interpreted and an audio visual signal is projected by the robot in response.

In this paper we present an interactive aerial projection of 3D hologram objects by using the pyramid hologram and parabolic mirror system (for 3D hologram object reconstruction) and the Leap Motion sensor (as a finger movement detector of a user). This system not only can reconstruct and project the 3D hologram object in the mid-air, but also provides a way to interact with it by moving a user's finger. There are three main steps: the reconstruction of 3D object, the projection of 3D hologram object in the mid-air, and the interactive manipulation of 3D hologram object. The first step is realized by using pyramid hologram with LCD display. The second step is achieved with the parabolic mirror hologram. And Leap Motion sensor is used for the last step to detect user finger movement. This paper traces the design concept and confirms the system function of an interactive aerial projection of 3D hologram objects by a prototype demonstration.

This paper showcases the development of a mobile robot integrated with Projectable Interactive Surface to facilitate its interaction with human users. The system was designed to interact with users of any physical attributes such as height, arm span etc. without re-calibrating it. The system was designed in such a way that there would be no need for the human to come in physical contact with the robot to give it instructions. This system uses a projector to render a virtual display on the ground allowing us to project large displays. Microsoft Kinect integrated in the systems performs a dual functionality of tracking the user movements along with mapping the surrounding environment. The gestures of the tracked user are interpreted and an audio visual signal is projected by the robot in response.

© 2015 IEEE.This paper shows the simulating and displaying of Puck motion in virtual Air Hockey based on projective interface for multi-users HRI Image-projective Desktop Arm Trainer (IDAT) is an upper-limb rehabilitation system for hand-eye coordination training based on projective interface. To expand more entertainment functions and realize multi-user HRI, we want to develop a virtual Air Hockey application in IDAT. To develop this application, Puck motion should be simulated correctly and displayed smoothly on the screen. There are mainly 3 problems: Data updating rate problem, virtual collision calculation and individual's hand misrecognition problem. We proposed our method into three parts corresponding to different problems. In part 1, we used multiple timers with shared memory to deal with unsynchronized data updating rate problem. In part 2, an original physical engine is designed to calculate Puck's velocity and detect virtual collision. In part 3, to deal with the individual's hand misrecognition problem, a history-based hand owner recognition algorithm is implemented to distinguish users' hands.

In this paper, a calligraphy learning support system is presented for supporting brushwork learning by using a projector. The system was designed to provide the three kinds of training according to the learner's ability as followings: copying training, tracing training, and combination of them. In order to instruct the three-dimensional brushwork such as the writing speed, pressure, and orientation of the brush, we proposed the instruction method by presenting the information to only brush tip. This method can be visualized a brush position and the orientation. In addition, the copying experiment was performed using the proposed method. As a result, the efficiency of the proposed method was examined through experiment.

This paper shows a novel method to extract hand Representative Point (RP) based on 2-dimensional laser range scanner for hand motion estimation on projecting system. Image-projecting Desktop Arm Trainer (IDAT) is a projecting system for hand-eye coordination training, in which a projector displays an exercise screen on the desktop, and a laser range scanner detects trainee's hand motion. To realize multi-user HMI and expand more entertainment functions in IDAT system, an Air Hockey application was developed in which the hand RP requires a high precision. To generate hand RP precisely, we proposed our method in two parts to solve the data error problem and changeable hand contour problem. In part one, a data modifier is proposed and a sub-experiment is carried out to establish a modifying function for correcting sensor original data. In part two, we proposed three RP algorithms and carried out an evaluation experiment to estimate the reliability of three algorithms under different conditions. From the result, we get the most reliable algorithm corresponding to different situations in which the error of hand RP is less than 9.6 mm.

This paper shows the simulating and displaying of Puck motion in virtual Air Hockey based on projective interface for multi-users HRI. Image-projective Desktop Arm Trainer (IDAT) is an upper-limb rehabilitation system for hand-eye coordination training based on projective interface. To expand more entertainment functions and realize multi-user HRI, we want to develop a virtual Air Hockey application in IDAT. To develop this application, Puck motion should be simulated correctly and displayed smoothly on the screen. There are mainly 3 problems: Data updating rate problem, virtual collision calculation and individual's hand misrecognition problem. We proposed our method into three parts corresponding to different problems. In part 1, we used multiple timers with shared memory to deal with unsynchronized data updating rate problem. In part 2, an original physical engine is designed to calculate Puck's velocity and detect virtual collision. In part 3, to deal with the individual's hand misrecognition problem, a history-based hand owner recognition algorithm is implemented to distinguish users' hands.

An Image-projective Desktop Varnamala Trainer (IDVT) called SAKSHAR has been designed to improve the learning by children through an interactive audio visual feedback system. This device uses a projector to render a virtual display, which permits production of large interactive displays with minimal cost. The user's hand is recognized with the help of a Microsoft Kinect Version 2. The entire system is portable, i.e., it can be projected on any planar surface. Since the Kinect does not give precise 3D coordinates of points for detecting a touch, a model of recognition of a touch purely based on the contact of the user's hand with the surface would not yield accurate results. We have instead modeled the touch action by using multiple points along the trajectory of the tracked point of the user's hand while hand makes contact with the surface. Fitting a curve through these points and analyzing the errors is used to make the detection of touch accurate.

This paper shows a novel method to extract hand Representative Point (RP) based on 2-dimensional laser range scanner for hand motion estimation on projecting system. Image-projecting Desktop Arm Trainer (IDAT) is a projecting system for hand-eye coordination training, in which a projector displays an exercise screen on the desktop, and a laser range scanner detects trainee's hand motion. To realize multi-user HMI and expand more entertainment functions in IDAT system, an Air Hockey application was developed in which the hand RP requires a high precision. To generate hand RP precisely, we proposed our method in two parts to solve the data error problem and changeable hand contour problem. In part one, a data modifier is proposed and a sub-experiment is carried out to establish a modifying function for correcting sensor original data. In part two, we proposed three RP algorithms and carried out an evaluation experiment to estimate the reliability of three algorithms under different conditions. From the result, we get the most reliable algorithm corresponding to different situations in which the error of hand RP is less than 9.6 mm.

This paper shows the simulating and displaying of Puck motion in virtual Air Hockey based on projective interface for multi-users HRI. Image-projective Desktop Arm Trainer (IDAT) is an upper-limb rehabilitation system for hand-eye coordination training based on projective interface. To expand more entertainment functions and realize multi-user HRI, we want to develop a virtual Air Hockey application in IDAT. To develop this application, Puck motion should be simulated correctly and displayed smoothly on the screen. There are mainly 3 problems: Data updating rate problem, virtual collision calculation and individual's hand misrecognition problem. We proposed our method into three parts corresponding to different problems. In part 1, we used multiple timers with shared memory to deal with unsynchronized data updating rate problem. In part 2, an original physical engine is designed to calculate Puck's velocity and detect virtual collision. In part 3, to deal with the individual's hand misrecognition problem, a history-based hand owner recognition algorithm is implemented to distinguish users' hands.

An Image-projective Desktop Varnamala Trainer (IDVT) called SAKSHAR has been designed to improve the learning by children through an interactive audio visual feedback system. This device uses a projector to render a virtual display, which permits production of large interactive displays with minimal cost. The user's hand is recognized with the help of a Microsoft Kinect Version 2. The entire system is portable, i.e., it can be projected on any planar surface. Since the Kinect does not give precise 3D coordinates of points for detecting a touch, a model of recognition of a touch purely based on the contact of the user's hand with the surface would not yield accurate results. We have instead modeled the touch action by using multiple points along the trajectory of the tracked point of the user's hand while hand makes contact with the surface. Fitting a curve through these points and analyzing the errors is used to make the detection of touch accurate.

This paper shows a novel method to extract hand Representative Point (RP) based on 2-dimensional laser range scanner for hand motion estimation on projecting system. Image-projecting Desktop Arm Trainer (IDAT) is a projecting system for hand-eye coordination training, in which a projector displays an exercise screen on the desktop, and a laser range scanner detects trainee's hand motion. To realize multi-user HMI and expand more entertainment functions in IDAT system, an Air Hockey application was developed in which the hand RP requires a high precision. To generate hand RP precisely, we proposed our method in two parts to solve the data error problem and changeable hand contour problem. In part one, a data modifier is proposed and a sub-experiment is carried out to establish a modifying function for correcting sensor original data. In part two, we proposed three RP algorithms and carried out an evaluation experiment to estimate the reliability of three algorithms under different conditions. From the result, we get the most reliable algorithm corresponding to different situations in which the error of hand RP is less than 9.6 mm.

An Image-projective Desktop Varnamala Trainer (IDVT) called SAKSHAR has been designed to improve the learning by children through an interactive audio visual feedback system. This device uses a projector to render a virtual display, which permits production of large interactive displays with minimal cost. The user's hand is recognized with the help of a Microsoft Kinect Version 2. The entire system is portable, i.e., it can be projected on any planar surface. Since the Kinect does not give precise 3D coordinates of points for detecting a touch, a model of recognition of a touch purely based on the contact of the user's hand with the surface would not yield accurate results. We have instead modeled the touch action by using multiple points along the trajectory of the tracked point of the user's hand while hand makes contact with the surface. Fitting a curve through these points and analyzing the errors is used to make the detection of touch accurate.

This paper shows a novel walking training system for foot-eye coordination. To design customizable trajectories for different users conveniently in walking training, a new system which can track and record the actual walking trajectories by a tutor and can use these trajectories for the walking training by a trainee is developed. We set the four items as its human-robot interaction design concept: feedback, synchronization, ingenuity and adaptability. A foot model is proposed to define the position and direction of a foot. The errors in the detection method used in the system are less than 40 mm in position and 15 deg in direction. On this basis, three parts are structured to achieve the system functions: Trajectory Designer, Trajectory Viewer and Mobile Walking Trainer. According to the experimental results,we have confirmed the systemworks as intended and designed such that the steps recorded in Trajectory Designer could be used successfully as the footmarks projected in Mobile Walking Trainer and foot-eye coordination training would be conducted smoothly.

Multi-user laser pointers system, in which more than single people could use the laser pointers concurrently, has promising applications, such as group discussion, appliances control for several handicapped persons, controlling the large displays by a few users and the like. Conventional methods employed in the above-mentioned applications are common mouse, gesture control utilizing the Kinect sensor or leap motion, including the single laser pointer. Common mouse and single laser pointer could not meet the requirements, which let a few users operate simultaneously. And gesture control is limited to a far distance. Compared to multi-user laser pointers, the majority research focuses on single laser pointer. However, multiple users would suffer the dilemma in which each one is not able to point at the target on the screen any time, at far distance, just because the laser pointer is controlled by only one person. The paper proposes a novel way to make it possible that 3 laser pointers are pointing at the screen without intervening between each other. Firstly, the foreground with all the dynamic spots is extracted. On the foreground, these spots will be searched all the time by grasping the contours of them. Hence, various information of the spots like the coordinate, pixel value, area and so on is obtained. Secondly, a square image containing the whole spot is referred to as the input of the designing back propagation neural network. The BPNN output indicates the category to which the laser pointer belongs. An experiment verifies that it works well under certain light conditions (12-727lux) if green laser pointers are used.

© 2014, Fuji Technology Press. All rights received.This paper presents a novel arm-training system, known as the image-projecting desktop arm trainer (IDAT), which is aimed at hand-eye coordination training. The projector displays an exercise image on a desktop in front of a seated patient, and the scanning range finder measures the behavior of the patient as he/she performs the exercise. IDAT is non-invasive and does not constrain the patient. Its efficiency is based on the voluntary movements of the patient, although it offers neither the physical assistance nor tactile feedback of some conventional systems. Three kinds of training content have been developed: “mole-hitting,” “balloon-bursting,” and “fish-catching.” These games were designed for training hand-eye coordination in different directions. A patient and/or medical professional can set a suitable training level, that is, the training time, speed of movement of the objects, and number of objects to appear at any one time, based on the patient’s condition and ability. A questionnaire survey was carried out to evaluate IDAT-3, and the results showed that it was highly acclaimed in terms of user-friendliness, fun, and usefulness.

© 2014, Fuji Technology Press. All rights received.To meet the higher requirements of human-machine interface technology, a robot with human-following capability, a classic but significant problem, is discussed in this paper. We first propose a human detection method that uses only a single laser range scanner to detect the waist of the target person. Second, owing to the limited speed of a robot and the potential risk of obstructions, a new human-following algorithm is proposed. The speed and acceleration of a robot are adaptive to the human-walking speed and the distance between the human and robot. Finally, the performance of the proposed control system is successfully verified through a set of experimental results obtained using a two-wheelmobile robot working in a real environment under different scenarios.

This paper introduces the real-time fingers naming method then illustrates how hand languages could be recognized by hand gesture and relation between fingertips. Moreover, this paper provides information on how to calculate and name each finger then using arm for dynamically adjust and improve the stability while the hand is moving. Supported by the study of relation between fingertips, palms, and arms, this proposed method can recognize hand gesture and translate these signs into numbers according to the standard sign language. The approach using in this paper relies on the image depth and the RGB image to identify hands, arms and fingertips. Then, the relation between each part is used to recognize a finger name regardless of the different direction of movement. Also, this report describes how to implement the proposed method into the ASUS Xtion as a sensor.

Laser spot clicking technique is the term of remote interaction technique between human and computer using a laser pointer as a pointing device. This paper is focused on the performance test of two laser spot clicking techniques. An off-the-shelf laser pointer has a toggle switch to generate a laser spot, the presence (ON) or absence (OFF) of this spot and its combination are the candidates of the interaction technique. We conducted empirical study that compared remote pointing technique performed using combination of ON and OFF of the laser spot, ON-OFF and ON-OFF-ON, and using a desktop mouse as a baseline comparison. We present quantitative performance test based on Fitts' test using a one-direction tapping test in ISO/TS 9241-411 procedure; and assessment of comfort using a questionnaire. We hope this result give contribution to the interaction technique using laser pointer as a pointing device especially in selecting the appropriate clicking technique for real application. Our results suggest ON-OFF technique has positive advantages over ON-OFF-ON technique such as the throughput and comfort.

Robot human-following is an important part of interaction between robot and people. Generally, speed of mobile robots is limited and far slower than human beings naturally walking speed. In order to catch the human rapidly, in this paper, we introduce a control method which uses adaptive acceleration of robots speed. The speed of robots mainly depends on the human speed and distance. Also the robots speed acceleration is adaptive to the distance between human and robots. The proposed control is successfully verified through experiments. © 2013 IEEE.

Image-projective Desktop Arm Trainer IDAT is designed to improve the upper limb therapy effect. This paper is focused on its design concept. The most important structural feature of IDAT is that it is non-invasive and unconstrained. Setting on the desk apart from the trainee sitting makes no body contact. It works based on the trainee's voluntary movement, although it has neither physical assist nor tactile feedback on conventional systems. IDAT is developed based on the design concept that the direct interaction is critical in terms of the motivation increase. Instead of a joystick, a handle, a robotic arm and a display screen on conventional systems, IDAT uses the screen projected on a desktop and produces the visual reaction at the time on the spot where the trainee operates. It inspires the vivid, lively and actual feeling to the trainee. © 2013 IEEE.

This paper describes the application of step-on interface (SOI) to therapy. SOI consists of a projector and a sensor such as range scanner, and its special feature is using a projected screen as bidirectional interface through which the information is presented from a robot to a user and the user instructions are delivered to the robot. The human-friendly amusing mobile robot HFAMRO equipped with SOI on a mobile platform is a playing-tag-robot which can be used for gait training. The image-projective desktop arm trainer IDAT is designed set on a desk in front of a trainee sitting. This kind of system is adjustable and customized to each individual by setting parameters, using multimedia channels, or uploading the program. Therefore it could provide the motivation and accomplishment to a trainee and maintain his/her enthusiasm and interest.

This paper presents one of the interaction models between humans and machines using a camera, the projector, and the spot-light from a laser pointer device. A camera was attached on the top of the projector, and the projector directed a direction screen display on the wall, while the user pointed a laser pointer to the desired location on the direction screen display. It is confirmed that this system can handle some distortion conditions of the direction screen display, such as an oblique rectangle, horizontal trapezoid distortion, and vertical trapezoid distortion as well as some surface illuminance - 127, 425, 630, and 1100 lux; and the system is designed to be used for static and moving objects. The coordinates that were obtained from the distorted screen can be used to give commands to a specific machine, robot, and application.

Aim to improve upper limb rehabilitation effect, we design and develop an Image-projective Desktop Arm Trainer (IDAT). Compared with conventional therapy, IDAT provides a more effective and interesting training method. IDAT's goal is to maintain and improve patients' upper limb function by training their eye-hand coordination. We select step-on interface (SOI) as the input system which makes trainees can operate IDAT with hand directly. Trainees can make a customized training setting. It can provide motivation and accomplishment to trainees and maintain their enthusiasm and interest. So IDAT provides a much different human robot interaction with pervious upper limb rehabilitation robots that equip a joystick or controller to operate remotely. We propose this idea in 2007 and have applied SOI on some mobile robots. Now we apply it on IDAT to make a new way to upper limb rehabilitation.

We propose and implement a solution for applying an infrared radiation (IR) image sensor to step-on interface (SOI). SOI is a kind of natural human-robot interface what consists of a projector and a laser range scanner (LRG) sensor. And it enables any interactive touch applications on desktop or floor. We attempt to introduce an IR image sensor such as ASUS Xtion to SOI instead of LRG sensor. In this paper, we will describe the procedure that how to use the Xtion and detect touched point. We distinguish user's hand from background (surface) depends on depth data from ASUS Xtion, and detecting touching action when finger almost close to the background. The proposed processes involve IR depth image acquisition, seeking for hand and its contours by thresholding, recognition of touched areas and computing of theirs center position. The research enables ASUS Xtion be applied on SOI in a simple way. Moreover, this system can realize touch interaction on any surface.

A laser pointer can be used as an input interface in human-machine interaction. Such utilization, however, can be problematic and one of the main issues is the lack of good reliability in the laser spotlight detection. Another problem is how to interpret the user's movement of the spotlight into commands for the application. This paper proposes a method for a laser spotlight detection. The aim is to improve the practicality and reliability of the previous approaches. We use the maximum pixel value as a multiplier in determining the threshold. Maximum pixel value is obtained from environment brightness at a specified time. For the second problem we propose a simple interpretation of incidents that allows the user to use the application, with three main events: laser-move, hover, and single-click. There is no need for users and program to wait a specified time span to be able to interact with each other, and the user can directly give commands to the application after the single-click event. These approaches result in better reliability, easier operation of the application by the user, and allow opportunity for development of a system for rehabilitative, recreation, and input interface devices in the future.

This paper presents the informative motion study to make a human-coexistence robot useful. First the usage and the design and marketing of a human-coexistence robot are considered. Next, the informative motion study that we are tackling to improve the human-coexistence robot's personal affinity is explained advocating the informative kinesics for human-machine system. As an example of application deployment, study on continuous movement (usual movement) and preliminary operation (prior operation) is shown.

This paper presents the study results on the handover movement informing a receiver of the weight load as an example of the informative motion for the humansynergetic robot. To design and generate the movement depending on the weight load, the human movement is measured and analyzed, and four items are selected as the parameter to vary - the distance between target point and transferred point (in front-back direction), the distance between highest point and transferred point (in vertical direction), the elbow rotation angle, and the waist joint angle. The fitted curve of the parameter variation depending on the weigh load is obtained from the tendency of the subjects' movement data. The movement data for an arbitrary weight load is generated processing the standard data at 0 kg of weight load so that each parameter follows the fitted curve. From the questionnaire survey, although it is difficult for a receiver to estimate the exact weight load, he may distinguish the heavy weight load from the light weight load so that the package will be received safely and certainly.

This paper presents the study results on the handover movement informing a receiver of the weight load as an example of the informative motion for the humansynergetic robot. To design and generate the movement depending on the weight load, the human movement is measured and analyzed, and four items are selected as the parameter to vary - the distance between target point and transferred point (in front-back direction), the distance between highest point and transferred point (in vertical direction), the elbow rotation angle, and the waist joint angle. The fitted curve of the parameter variation depending on the weigh load is obtained from the tendency of the subjects' movement data. The movement data for an arbitrary weight load is generated processing the standard data at 0 kg of weight load so that each parameter follows the fitted curve. From the questionnaire survey, although it is difficult for a receiver to estimate the exact weight load, he may distinguish the heavy weight load from the light weight load so that the package will be received safely and certainly.

This paper presents the study results on the handover movement informing a receiver of the weight load as an example of the informative motion for the humansynergetic robot. To design and generate the movement depending on the weight load, the human movement is measured and analyzed, and four items are selected as the parameter to vary - the distance between target point and transferred point (in front-back direction), the distance between highest point and transferred point (in vertical direction), the elbow rotation angle, and the waist joint angle. The fitted curve of the parameter variation depending on the weigh load is obtained from the tendency of the subjects' movement data. The movement data for an arbitrary weight load is generated processing the standard data at 0 kg of weight load so that each parameter follows the fitted curve. From the questionnaire survey, although it is difficult for a receiver to estimate the exact weight load, he may distinguish the heavy weight load from the light weight load so that the package will be received safely and certainly.

The friendly amusing mobile (FAM) function in which a robot and a user can interact through a motion is proposed applying the mobile robot step-on interface (SOI) in which the user can direct robot movement or operation by stepping or pointing on the button which shows the contents to desire on the operation screen projected on a floor. HFAMRO-2 mobile robot is developed to realize playing a light tag and three applications-animal tail stepping, bomb fuse stamping, and footprint stepping-are produced as a trial after the design policy.

To expand use of the mobile robot Step-On Interface
(SOI), originally targeting maintenance, training, and recovery of human physical and cognitive functions, we introduce a “Truly-Tender-Tailed” (T3, pronounced tee-cube) tag-playing robot as a “Friendly Amusing Mobile” (FAM) function. Displaying a previously prepared bitmap (BMP) image and speeding up display make it easy to design button placement and other screen parameters using a painting software package. The BMP-image scope matrix simplifies step detection and recognition and the motion trajectory design editor facilitates robot behavior design.

This video shows the basic functions of HFAMRO-2 equipped with the step-on interface (SOI). In the SOI the projected screen is used as a bilateral interface. It not only presents information from the equipment to the user but also delivers the instructions from the user to the equipment. HFAMRO is intended to represent the concept based on which robots interact with users. It assumes, for example, the ability to play `tag' - in this case, playing tag with light, similar to `shadow' tag. The HFAMRO-2 mobile robot, developed to study the SOI's application with mobility, has two sets of the SOI consisting of a projector and a range scanner on a mobile platform. The projector displays a direction screen on a travel surface and the two-dimensional range scanner detects and measures the user's stepping to specify the selected button.

This paper introduces a tag-playing robot as a "Friendly Amusing Mobile" (FAM) function to expand the mobile robot Step-On Interface (SOI) use and to promote the human-robot interaction through a motion, targeting maintenance, training, and recovery of human physical and cognitive functions in the elderly, physically challenged, injured, etc. Displaying a previously prepared bitmap (BMP) image becomes the display update rate faster and makes it easy to design button placement and other screen parameters using a painting software package. The scope matrix generated from the BMP image simplifies the step detection and recognition, and the motion trajectory design editor facilitates the robot behavior design.

This paper discusses discrimination and implementation of emotions on a zoomorphic robot aiming at designing emotional robot movements and improving robot friendliness. We consider four emotions; joy, anger, sadness, and fear. Two opposite viewpoints, performer and observer, are considered to acquire emotional movement data for analysis. The performer subjects produce emotional movements and movements that are easy to be recognized as expressions of the intended emotion are selected among the measured movements by the observer subjects. Discriminating the emotion embedded in a movement is tried using feature values based on the Laban movement analysis (LMA) and the principal component analysis (PCA). By the discrimination using PCA, the resultant rates of the correct discrimination are about 70% for all four emotions. The features of emotional movements are presumed from the coefficients of the discrimination function obtained in the emotion discrimination using PCA. Emotions are implemented by converting a setup for basic movements by employing the design principle based on the movement features. The result of the verification experiment suggests that four emotional movements are divided into two groups; the joy and anger group and the sadness and fear group. The emotional movements of joy or anger are dynamic, large-scale and frequent being relatively easy to interpret the intended emotion, while the emotional movements of sadness or fear are static, small-scale and little making difficult to understand the feature of movements.

In studying dynamic remodeling of environmental mapping around a mobile robot operated remotely while data measured by the robot range sensor is sent from the robot to the operator, we introduce the Line & Hollow method and the Cell & Hollow method for environmental mapping. Results for the three types of environmental situation clarifies features and effectiveness of our approach. In Line & Hollow method, an isosceles triangle is set based on the range data. The base line is pulled to express obstacle shape and the inside is hollowed out to express vacant space. In Cell & Hollow method, the cell value corresponding to the range data is incremented, and an obstacle is assumed to be exist if the cell value exceeds the ascending threshold. The cell value is decremented on the line between the cell that measured data indicates and the cell located at the sensor, and the obstacle is deleted if the value drops below the descending threshold. We confirmed that environmental mapping for either reflects a dynamic environmental change.

To improve HFAMRO-1 mobile robot maneuverability and safety, we added a step-on interface (SOI) to direct robotic or mechatronic tasks and operations (HFAMRO: “human-friendly amusing” mobile robot). To do so, a projector displays a direction screen on a floor or other surface, and an operator specifies a button showing the selected movement by stepping or pointing. We modified the direction screen so that among buttons displayed on two lines, stepped-on buttons directing movement are displayed only on the lower line. We also shortened retention time and had selected movement executed only when the foot was removed from the stepped-on button. The robot has 2 SOIs and multiple projection screens, and can be controlled from either direction for the same function. We synchronized direction and preliminary-announcement screens to inform passersby which way the robot would move. Using range scanner data, the robot distinguishes feet from other objects based on size and autonomous movement fusion control to avoid obstacles is implemented.

This paper shows the result of the characteristics measurement of a two-dimensional active range scanner, URG made by Hokuyo Automatic Co., Ltd. which was released in 2004 and is spreading widely as an external sensor for mobile robot. The following items were clarified from the characteristics measurement of URG-X002S in various conditions. (1) In the case that the object has a gloss surface or a black surface, the error rate (the frequency that the scanner judges to be impossible to measure and an error code is output) rises when the oblique angle of object becomes large and also the distance to object becomes long. (2) In the case that the object has a white surface or a rough surface, not only the error rate is zero but also the margin of error becomes dozens of millimeters and the varying is small, if the oblique angle is smaller than 60 deg and the distance is shorter than 1 m. (3) The lateral error is negligibly small if the distance to detect is shorter than 1 m. Moreover it shows the result of the examination to apply the range scanner in the Step-On Interface (SOI), in which the scanner is used for detection and measurement of the stepping of an operator. Based on the measured results, we designed the judgment method of the stepping, the installation position of the scanner, and the placement of buttons in the direction screen to apply the range scanner to the SOI for operation of a mobile robot.© Takafumi Matsumaru
Licensee Bentham Open.

This paper presents the result of the study on handover movement informing a receiver of the weight load as an example of the informative motion of a human-coexistence type robot (human-friendly robot). Movement of human body when performing handover task to a person was measured and analyzed first, and the features of variance of movement depending on the weight load were clarified. Then, the questionnaire survey was carried out to verify whether people could read out the difference in weight load from the variance of movement, in which the movement was reproduced using a simulator. It may be said that a receiver can judge whether a package is heavy or not, although it is difficult to estimate the exact weight load. Accordingly, if the variance of the noted points is included in the design of the handover movement of a humanoid robot, people as a receiver will be able to estimate the weight load to make easy to receive a package safely and certainly.

This paper presents the result of trials of the discrimination of emotion from movement and the addition of emotion in movement on a teddy bear robot, aiming at both expressing a robot's emotion by movement and improving a robot's personal affinity. We addressed four kinds of emotion joy, anger, sadness, and fear. In this research, two standpoints were considered a performer and an observer to establish the data of emotional movement used for analysis. The data of movement were collected as a performer's standpoint and they were sorted out in an observer's standpoint. In discriminating the emotion included in movement from the data of movement, both the method using Laban's feature quantity and the method using principal component analysis were tried. By the discrimination using principal component analysis, about 70% of rate of correct discrimination was obtained on all the four emotions. The feature of movement that each emotion could be interpreted was presumed from the coefficient of the discrimination function obtained in the discrimination using principal component analysis. Using the obtained feature the design principle of movement to add emotion into a basic movement was defined. From the verification experiment, it was suggested that the movement which people can interpret the intended emotion with relatively high probability would be produced about joy and anger. About fear and sadness, since the movement to express those emotions has small and little motions, it would be difficult to distinguish the feature and to produce clear emotional movement.

This paper proposes the step-on interface (SOI) to operate a mobile robot in which a projector projects and displays a direction screen on a floor and a user specifies a button showing the selected movement by stepping or pointing. The HFAMRO-1 mobile robot has been developed to demonstrate the SOI's potential (HFAMRO: "human-friendly amusing" mobile robot). The SOI of HFAMRO-1 consists of a projector and a range scanner on an omni-directional mobile platform. From operational comparison with other input interfaces, it is confirmed that we can direct the robot movement using our own foot. We had some students who does not specialize robotic systems try to operate HFAMRO-1 with their shoes, and all trial students could specify the button to operate the robot satisfactorily and everyone mastered the SOI immediately. © Matsumaru and Akai.

This paper presents the result of the experimental examination by "passing each other" and "positional prediction" in simulated interactive situation between people and mobile robot. We have developed four prototype robots based on four proposed methods for preliminarily announcing and indicating to people the speed and direction of upcoming movement of mobile robot moving on two-dimensional plane. We observed significant difference between when there was a preliminary-announcement and indication (PAI) function and when there was not even in each experiment. Therefore the effect of preliminary-announcement and indication of upcoming operation was declared. In addition the feature and effective usage of each type of preliminary-announcement and indication method were clarified. That is, the method of announcing state of operation just after the present is effective when a person has to judge to which direction he should get on immediately due to the feature that simple information can be quickly transmitted. The method of indicating operations from the present to some future time continuously is effective when a person wants to avoid contact or collision surely and correctly owing to the feature that complicated information can be accurately transmitted. We would like to verify the result in various conditions such as the case that traffic lines are obliquely crossed.

This paper discusses the mobile robot PMR-5 with the preliminary-announcement and display function which indicates the forthcoming operations to the people near the robot by using a projector. The projector is set on a mobile robot and a 2-dimensional frame is projected on a running surface. In the frame, not only the scheduled course but also the states of operation can be clearly announced as the information about movement. We examine the presentation of the states of operation such as stop or going back including the time information of the scheduled course on the developed robot. Scheduled course is expressed as the arrows considering the intelligibility at sight. Arrow expresses the direction of motion directly and the length of arrow can announce the speed of motion. Operation until 3-second-later is indicated and three arrows classified by color for each second are connected and displayed so these might show the changing of speed during 3-second period. The sign for spot revolution and the characters for stop and going back are also displayed. We exhibited the robot and about 200 visitors did the questionnaire evaluation. The average of 5-stage evaluation is 4.5 points and 3.9 points for the direction of motion and the speed of motion respectively. So we obtained the evaluation that it is intelligible in general.

We propose approaches and equipment for preliminarily
announcing and indicating to people the speed and direction of movement of mobile robots moving on a two-dimensional plane. We introduce the four approaches categorized into (1) announcing the state just after the present and (2) indicating operations from the present to some future time continuously. To realize the approaches, we use omni-directional display (PMR-2), flat-panel display (PMR-6), laser pointer (PMR-1), and projection equipment (PMR-5) for the announcement unit of protobots. The four protobots were exhibited at the 2005 International Robot Exhibition (iREX05). We had visitors answer questionnaires in a 5-stage evaluation. The projector robot PMR-5 received the highest evaluation score among the four. An examination of differences by gender and age suggested that some people prefer simple information, friendly expressions, and a minimum of information to be presented at one time.

This research aims to propose the method and equipment to preliminary-announce and indicate the surrounding people both the speed of motion and the direction of motion of the mobile robot that moves on a two-dimensional plane. This paper discusses the mobile robot PMR-6, in which the liquid crystal display (LCD) is set up on the mobile unit, and the state of operation at 1.5 s before the actual motion is indicated. The basis of the content to display is 'arrow' considering the intelligibility for people even at first sight. The speed of motion is expressed as the size (length and width) of the arrow and its color based on traffic signal. The direction of motion is described with the curved condition of the arrow. The characters of STOP are displayed in red in case of stop. The robot was exhibited to the 2005 International Robot Exhibition held in Tokyo. About 200 visitors answered to the questionnaires. The average of five-stage evaluation is 3.56 and 3.97 points on the speed and on the direction respectively, so the method and expression were evaluated comparatively intelligible. As for the gender, the females appreciated about the speed of motion than the males on the whole. Concerning the age, some of the younger age and the upper age admired highly about the direction of motion than the middle age.

This paper proposed the model to estimate the load on the lumber vertebra considering not only the value presumed from posture but also the increased amount by Valsalva maneuver of extrapolating from the vital capacity. The pressure force can be estimated to be reduced by about 30% by the effect of the abdominal pressure by using the proposed model as said so far. From the error of the presumed value of ground reaction force from an actual measurement, it is thought that the presumed accuracy of the lumbar vertebra load using the proposed model is smaller than 10%. Furthermore, two operations with extreme start-on posture were compared on transition of the compressive force and shear force on the lumbar vertebra. It was brought out that a start-on posture signficantly affects the maximum load on the lumber vertebra. The result suggests that optimal start-on posture may be between two postures.

This paper discusses the design and the basic characteristic of the mobile robot PMR-1 with the preliminaryannouncement and display function of the forthcoming operation (the direction of motion and the speed of motion) to the people around the robot by drawing a scheduled course on a running surface using light-ray. The laser pointer is used as a light source and the light from the laser pointer is reflected in a mirror. The light-ray is projected on a running surface and a scheduled course is drawn by rotating the reflector around the pan and the tilt axes. The preliminary-announcement and display unit of the developed mobile robot can indicate the operation until 3-second-later preliminarily, so the robot moves drawing the scheduled course from the present to 3-second-later. The experiment on coordination between the preliminary-announcement and the movement has been carried out, and we confirmed the correspondence of the announced course with the robot trajectory both in the case that the movement path is given beforehand and in the case that the robot is operated with manual input from a joystick in real-time. So we have validated the coordination algorithm between the preliminary-announcement and the real movement.

The aim of this research is to establish the quantitative evaluation method for lifting task and to clarify the safety posture and optimal motion. This paper firstly examines the operation from differenet three start-on postures with four items: the maximum compressive force and share force on L5/S1 joint, the energy efficiency index, and the degree of contribution of each joint. As a result it has been shown that the load not only to lumbar vertebra but to knee joint should be emphasized and the pose-C is an appropriate start-on posture in which the knee is flexed almost at right-angled and the upper body is raised. Moreover from the simulation using minimum jerk model we found a proper operation, but the actual operation has earlier timing to extend the lower body than presumed. So not only the criteria on the whole operation or the relative criteria but also the absolute criteria paying attention to a portion like knee joint moment seems necessary to study the optimal lifting motion.

This paper discusses the design method of the bodily shape and motion of a humanoid robot to raise not only the emotional but informative interpersonal-affinity of a humanoid robot. Concrete knowledge and a concrete opinion are classified into the movement prediction from its configuration and the movement prediction from continuous motion or preliminary motion, and they are discussed mentioning the application and usage. Specifically, the bodily shape and motion, which is easier to predict and understand the capability and performance and the following action and intention of a humanoid robot for the surrounding people who are looking at it, are considered.

The aim of this research is to establish the quantitative evaluation method for lifting task and to clarify the safety posture and optimal motion. This paper examines the motion from three kinds of start-on posture with the acceptance rate. Operation-A starts from the pose-A in which the knee is extended to the maximum. Operation-B starts from the pose-B in which the knee is flexed to the maximum and the upper body is raised as much as possible. Operation-C starts from the pose-C in which the knee is flexed at almost right-angled and the upper body is raised. The acceptance rate is the estimated population rate who can permit the joint moment during the lifting operation, based on the presumed moment and the coefficient of variation of the acceptable marginal moment on each joint. The compressive force on lumbar vertebrae computed from the L5/S1 joint moment at 85[%] of the acceptance rate (about Av.-1SD) has not been over the standard value of a previous research. So we have set 85[%] as the safety standard acceptance rate, then we settled judging-A (over 95[%], recommended), judging-B (85-95[%], should note), and judging-C (under 85[%], should modify). Although the ankle joint on operation-A and the knee joint on operation-B is judged as C rank, every joint on operation-C shows high acceptance rate. So the validity of operation-C has been clarified quantitatively.

This paper discusses the mobile robot PMR-5 with the preliminary- announcement and display function which indicates the forthcoming operations to the people near the robot by using a projector. The projector is set on a mobile robot and a 2-dimensional frame is projected on a running surface. In the frame, not only the scheduled course but also the states of operation can be clearly announced as the information about movement. We examine the presentation of the states of operation such as stop or going back including the time information of the scheduled course on the developed robot. Scheduled course is expressed as the arrows considering the intelligibility at sight. Arrow expresses the direction of motion directly and the length of arrow can announce the speed of motion. Operation until 3-second-later is indicated and three arrows classified by color for each second are connected and displayed so these might show the changing of speed during 3-second period. The sign for spot revolution and the characters for stop and going back are also displayed. We exhibited the robot and about 200 visitors did the questionnaire evaluation. The average of 5-stage evaluation is 3.9 points and 4.5 points for the direction of motion and the speed of motion respectively. So we obtained the evaluation that it is intelligible in general. ©2006 IEEE.

This paper discusses the design and the basic characteristic of the mobile robot PMR-1 with the preliminary-announcement and display function of the forthcoming operation (the direction of motion and the speed of motion) to the people around the robot by drawing a scheduled course on a running surface using light-ray. The laser pointer is used as a light source and the light from the laser pointer is reflected in a mirror. The light-ray is projected on a running surface and a scheduled course is drawn by rotating the reflector around the pan and the tilt axes. The preliminary-announcement and display unit of the developed mobile robot can indicate the operation until 3-second-later preliminarily, so the robot moves drawing the scheduled course from the present to 3-second-later. The experiment on coordination between the preliminary-announcement and the movement has been carried out, and we confirmed the correspondence of the announced course with the robot trajectory both in the case that the movement path is given beforehand and in the case that the robot is operated with manual input from a joystick in real-time. So we have validated the coordination algorithm between the preliminary-announcement and the real movement.

人工機械システムに対する恐怖感や違和感を排除するには,外見を人間型や動物型にするだけでなく,サイズと性能・機能の関係などにおいて,人間の常識,経験知・暗黙知に合致した形態・動作が必要だと考える.具体的には,移動ロボットやヒューマノイド・ロボットにおいて,その次の行動・意図が,それを見ている周囲の人間によって予測しやすい形態・動作を考えている.本稿は,まず人間機械系の情報動作学の応用展開における研究項目と手順を説明し,つぎにこれまでに入手した資料から得られている知見のいくつかを,(1)形態からの運動予測と(2)連続動作・予備動作からの運動予測に分類し,その応用・適用方法に言及しながら論じる.

This paper explains the PMR-2R ( prototype mobile robot - 2 revised), the mobile robot with the eyeball expression as the preliminary-announcement and display of the robot's following motion. Firstly, we indicate the importance of the preliminary-announcement and display function of the mobile robot's following motion for the informational affinity between human being and a robot, with explaining the conventional methods and the related works. We show the proposed four methods which are categorized into two types: one type which indicates a state just after the moment and the other type which displays from the present to some future time continuously. Then we introduce the PMR-2R, which has the omni-directional display, the magicball, on which the eyeball expresses the robot's following direction of motion and the speed of motion at the same time. From the evaluation experiment, we confirmed the efficiency of the eyeball expression to transfer the information. We also obtained the announcement at around one or two second before the actual motion may be appropriate. And finally we compare the four types of eyeball expression: the one-eyeball type, the two-eyeball type, the will-o'-the-wisp type, and the armor- helmet type. From the evaluation experiment, we have declared the importance to make the robot's front more intelligible especially to announce the robot's direction of motion.

This paper examines the combination control of the manual operation and the autonomous motion to improve the maneuverability and safety on teleoperation of mobile robot. The autonomous motion which works to support the manual operation processing information from simple range sensors on mobile robot is examined with using a computer simulation. Three types of autonomous motion, revolution (RV), following (FL), and slowdown (SD), are proposed and the way to be equipped on the system are examined. In revolution, robot turns autonomously when robot approaches some obstacle too much. In following, robot translates parallel keeping its orientation to go along the form of the obstacle. In slowdown, robot is restricted its translation speed according to the distance to the obstacle and the translation speed of the robot. Features of each autonomous motion is declared; transit time and mileage become shorter with the revolution or the following, and the contact between robot and obstacle is almost avoided with the slowdown. When the distance to some obstacle to apply some autonomous motion is adjusted depending on the translation speed of the mobile robot, too much addition of autonomous motion against operator's intention is reduced so the maneuverability can be improved. When the revolution or the following is incorporated with the slowdown, even though the transit time is prolonged, the number of near miss for the robot to some obstacles is reduced so the safety can be improved.

This paper examines the combination control of the manual operation and the autonomous motion on tele-operation of mobile robot. The autonomous motion that is suitable for the situation when the robot passes through a passage with bends is examined to improve the maneuverability with using a computer simulation. The situation that the manually operated robot contacts with the sidewall of a passage is investigated with the experiments. It is pointed out that the contact tends to occur around the entrance and exit of bends: a robot tends to contact with the inside near the entrance of the bend and the outside around the exit of the bend. The situation that the operator uses the autonomous motion of the mobile robot under the combination control is also investigated with the experiments. The operator tends to use the autonomous following (FL) near the entrance, and the autonomous revolution (RV) is effective to make the robot return to the center of the passage around the exit. From these situation analyses the selective-revolution/following (S-R/F) is developed in which the situation is estimated based on the direction of the manual operation of the operator and the direction of the obstacle from the robot then the autonomous revolution or the autonomous following is selected and applied according to the estimated situation. The new technique is equipped with the simulation system and it is confirmed that the autonomous revolution against the operator's intention is not applied based on the situation, so the maneuverability can be improved.

This paper examines the preliminary-announcement and display function of human-friendly robot's following action and intention, especially about the direction of motion and the speed of motion for mobile robot which moves on a 2-dimentional plane. We proposed 2 types of methods: the method indicating a state just after the moment (lamp and party-blowouts) and the method displaying from the present to some future time continouously (beam-of-light and projector). Simulation system has been developed to confirm the effectiveness of the preliminary-announcement and display. Effctiveness can be evaluated by the mobile robot chasing. The mobile robot moves about at a random speed in a random direction. Subject person moves the operation robot by using joystick with looking at the preliminary-announcement on the mobile robot. So the method to display (lamp/blowout/beam) and the timing to announce (0.5-3.0 [s] before the actual motion of the robot) are evaluated numerically with the position/direction gap. The data is processed not only as the average and the standard deviation but also with the two-way ANOVA and t-screening. It was examined on a translation and a rotation separately and then on a 2-dimentional plane.<br>We have found that the method displaying from the present to some future time continuously (beam) is easy to understand. But some amount of length of the displayed path in necessary, which means an appropriate timing depending on conditions. The optimal timing for indicating a state is almost 1.0-1.5 [s] before. If the time difference is too much long, the position/direction gap become large due to the poor memory and the operational mistake. If that is too short the operation will be late owing to the reaction delay. Moreover it seems that human being tends to understand some information with a transforming object than with a changing-color object, and continuous changing is easier to understand than distributed changing.

Human-friendly robot which supports and assists human being directly beside human being is expected as it has become an aging and decrease-in-the-birthrate society. In addition to the safety function to avoid dangers for human being, we think the informational affinity function is important such as to announce surrounding people the robot's following motion before beginning to move. A person can anticipate and guess other's motion considering human body's functional features and behavioral pattern as common sense. This paper discusses the preliminary-announcement and display function of mobile robot's following motion, the direction of motion and the speed of motion, by using a omni-directional display, magicball (R). We have been developing the mobile robot PMR-2 which tells its following action with eyeball expressions.

Human-friendly robot which supports and assists human being directly beside human being is expected as it has become an aging and decrease-in-the-birthrate society. In addition to the safety function to avoid dangers for human being, we think the informational affinity function is important such as to announce surrounding people the robot's following motion before beginning to move. A person can anticipate and guess other's motion considering human body's functional features and behavioral pattern as common sense. This paper discusses the preliminary-announcement and display function of mobile robot's following motion, the direction of motion and the speed of motion, by using a omni-directional display, magicball (R). This examines the four types of eyeball expression, one-eyeball type, two-eyeball type, will-o'-the wipe type, and armor-helmet type.

This paper examines the preliminary-announcement and display function of the robot's following action and intention, especially about the direction of motion and the speed of motion for mobile robot which moves on a 2-dimentional plane. Based on the result of the fundamental examination, the validity of the preliminary-announcement and display at around 1 second before the actual motion has been verifiedon a 2-dimentional plane. Moreover we have found that human being tends to understand the change of speed of the mobile robot more with some transforming object than with some changing-color object. Comparing two types of the proposed methods, the efficiency of the preliminary-announcement and display by indicating the continuous information as drawing with beam-light more than by displaying a state as lighting lamp or using blowouts have been declared. Moreover, we have found that the scheduled path needs some amount of length for a person to understand the mobile robot's following action.

Human-friendly robot which supports and assists human being directly beside human being is expected as it has become an aging and decrease-in-the-birthrate society. In addition to the safety function to avoid dangers for human being, we think the informational affinity function is important such as to announce surrounding people the robot's following motion before beginning to move. A person can anticipate and guess other's motion considering human body's functional features and behavioral pattern as common sense. This paper discusses the preliminary announcement and display function of mobile robot's following motion, the direction of motion and the speed of motion, by using a omni-directional display, magicball (R). This examines the four types of eyeball expression, one-eyeball type, two-eyeball type, will-o'-the wipe type, and armor-helmet type.

Human-friendly robot which supports and assists human being directly beside human being is expected as it has become an aging and decrease-in-the-birthrate society. In addition to the safety function to avoid dangers for human being, we think the informational affinity function is important such as to announce surrounding people the robot's following motion before beginning to move. A person can anticipate and guess other's motion considering human body's functional features and behavioral pattern as common sense. This paper discusses the preliminary announcement and display function of mobile robot's following motion, the direction of motion and the speed of motion, by using a onmi-directional display, magicball (R). We have been developing the mobile robot PMR-2 which tells its following action with eyeball expressions.

This paper discusses the preliminary-announcement and display function of the robot's following action and intention, especially about the direction of motion and the speed of motion for mobile robot which moves on a 2-dimentional plane. We started with a software simulation about the announcement method indicating a state just after the moment, with lighting lamps and with using blowouts (elastic arrow) on translation and rotation separately. As a result, the following three remarks have been obtained: Not only the direction of motion but also the speed of motion is effective to estimate the following translation. Not the rotation speed but the rotation angle (the target direction) is efficient to understand the following revolution. Around I-second before the actual motion is the optimal timing to recognize the mobile robot's following motion both on translation and rotation. Moreover we have verified the validity of the preliminary-announcement and display also for the general movement on a plane.

This paper examines the combination control in which remote operation is combined with autonomous behaviors with the aim to realize the remote operation of mobile robot which moves in human-coexisting environment. We considers the distance and direction to an obstacle and the speed of motion of the mobile robot for revolution, following, and slowdown, which we have proposed as the autonomous action element in combination control. Fuzzy reasoning and vector components are used. From the experiment by three subject persons, almost the same result has been obtained. When the distance and direction to an obstacle and the speed of motion of the mobile robot are considered, there doesn't seem to be a great difference in following, but mileage becomes shorter in revolution and transit time is reduced in slowdown.

This paper examined the combination control in which remote operation is combined with autonomous behaviours with the aim to realize the remote operation of mobile robot which moves in human-coexisting environment. In revolution, following, and slowdown which we had proposed as combination control the autonomous action element in consideration of the distance and direction to an obstacle and the speed of motion of the mobile robot at that time has been examined Fuzzy reasoning and vector component are used to achieve this consideration. Although there was no big difference in following, mileage becomes short in revolution, and transit time was shortened in slowdown. Furthermore in order to avoid a near miss with an obstacle completely, slowdown is incorporated with revolution or following. Verification experiment in software simulation has been carried out with the same subject persons. Although it takes a little longer transit time, other special bad influence was not appeared.

This paper describes the preliminary-announcement and display of action and intention of mobile robot which moves in human-coexisting environment. First, the affinity to realize a human-friendly robot is mentioned. Next, the usefulness of remote-operated human-friendly mobile robot is shown with explaining several applications. And it is declared that the informational affinity as the function to announce and display beforehand the following action and intention of robot towards human being is necessary with explaining conventional mobile machine and remote-operated mobile robot. Then the method using "blowout" is proposed to inform both speed of motion and direction of motion simultaneously. Copyright (C) 2001 IFAC.

This paper discusses the preliminary-announcement and display method of action and intention which is carried with a robot which moves in a human-coexisting environment, It is pointed out that the speed of motion and the direction of motion are important to announce the action of a mobile robot. The method by using blowout is proposed to tell surrounding people about these two kinds of information. Method to announce the speed of motion in translation is examined using a software simulator concerning the case that a robot is equipped with blowout, lamp, or no apparatus. Consequently, the effectiveness of the preliminary-announcement and display using blowout and the importance of information about the speed of motion are confirmed. Moreover method to announce the direction of motion in rotation is examined concerning the case to announce the target direction or the rotation speed. As a result, it is found that the target direction is efficient to announce in rotation of a mobile robot.

This paper proposes three action strategies for remote operation of mobile robot in human coexistence environment as an application of network robotics towards human coexistence robot. Navigation of mobile robots from remote site only in picture information is very difficult. To realize a human coexistence type robot is a challenging subject. So we proposes (1) the combination control method with remote operation and autonomous motion with priority on the remote operation, (2) the method of extending the task-based data exchange which we have been proposed, and (3) the method of sharing information between coexisting human and mobile robot by preliminary announcement display of action intention of mobile robot.

This paper proposes the task based data exchange for teleoperation systems through communication network as an efficient transmission method of data between an operation device and a remote robot. On the task-based data exchange, the more important information according to the contents and conditions of the task which the robot performs is given priority to transmit, for example, by altering the contents of the transmitted data. We have built the experimental system in which the master arm in Tsukuba and the slave arm in Kawasaki are connected through N-ISDN and the standard techniques are utilized, such as TCP/IP, socket, JPEG, etc. A series of experimental task has been effectively carried out by the task based data exchange, that is, the crank operation which consists of grasp and revolution. The communication network with capacity limitation was used effectively and the high maneuverability in real-time with bilateral servo control has been realized. Then the effectiveness of the task based data exchange has been confirmed.

This paper proposes the task-based data exchange for teleoperation systems through a communication network as an efficient method of transmitting data between an operation device and a remote robot. Oil the task-based data exchange, the more important information according to the contents and conditions of the task which the robot performs is given transmission priority, for example, by altering the contents of the transmitted data. We have built an experimental system iii which the master arm in Tsukuba and the slave arm in Kawasaki, separated by about 100 km, are connected through N-ISDN and the standard techniques are utilized, such as TCP/IP, socket, JPEG, etc. A series of experimental tasks has been effectively carried out by the task-based data exchange, that is, crank operation which consists of grasping and revolution. The communication network with capacity limitations was used effectively, and the high maneuverability in real-time with bilateral servo control has been realized. The effectiveness of the task-based data exchange have been confirmed.

A design concept of TOMMS, TOshiba Modular Manipulator system, has been already proposed to achieve a modular manipulator system which can be assembled into any desired configuration to provide adaptability to tasks, using as few kinds and types of modules as possible, without special handling such as modification of control software. To realize the concept, we developed a constitution and configuration recognition method of the assembled manipulator using electric resistance which is simple, practical and reliable. Moreover, to actualize the system which can offer the best suitable manipulator constitution and configuration for the desired task, we developed a workability judgment method considering the degeneracy of degreeds of freedom (d. o. f.) of the manipulator and the conditions of the desired. These methods were applied to the trial system TOMMS-1 and their efficiency and practicality were confirmed.

The TOshiba Modular Manipulator System, TOMMS, consists of joint modules, link modules, and a control unit with a joystick. As to the trial manufacturing, a manipulator with 3 d. o. f. is assembled using three joint modules and optional link modules into any desired configuration and shape, for instance, a horizontal type and a vertical type. The assembled manipulator is connected to the control unit, and the position of the end tip of the manipulator is controlled using the joystick without special handling. There is only one type of joint module and link module. There are three input ports and two output ports on the joint module. The distance between the fore side and the back side of the link module is adjustable. The Jacobian matrix is applied to the control software. Control experiments were carried out and the efficiency of the design concept of TOMMS for mechanical hardware and control software was confirmed.

This paper describes a control method for manipulators which work by pressing the end-effector to the workpiece under constant force (e.g. grinding and cleaning) and also positioning the end-tip to everywhere on the workpiece using the operation device. Based on ergonomics, "the operator coordinate system" is introduced which is determined from the glance line to the workpiece and the both eyes of the operator. Further, "the corresponding-to-operational-motion type control method" is proposed, on which method the direction of motion of the operation device and the reactive direction of motion of the end-effector are corresponded in the operator coordinate system. Especially for the workpiece with wave shape, "the corresponding-to-objective-shape type control method" is designed, on which method the winding line and the valley line of the wave are recognized during the work and the directions of motion of the operation device are corresponded to these lines. These methods have been applied to the remote control system including the joystick and the lightweight manipulator, so the efficiency of these methods have been confirmed.

A design concept of TOMMS, TOshiba Modular Manipulator System, has been already proposed to achieve a modular manipulator system which can be assembled into any desired constitution and configuration to provide adaptability to tasks, using as few kinds and types of modules as possible, without special handling such as modification of control software. To realize the concept, we developed a constitution and configuration recognition method of the assembled manipulator using electric resistance which is simple, practical and reliable. Moreover, we developed a workability judgment method considering the degeneracy of degrees of freedom (d.o.f.) of the manipulator and the conditions of the desired task These methods were applied to the trial system TOMMS-1 and their efficiency and practicality were confirmed.

This paper describes the development of Windshield Cleaning Robot System : WSC. This system is intended to be used for Boeing 747, commonly called jumbo jet, parked at airports prior to service. The objects to be cleaned are spots on windshields caused by collision with dusts, insects, and birds during takeoff and landing. The intention of this new system is that one operator would perform the whole work in 10 minutes. So the system consists of the manipulator (the arm and the cleaning device), the installation unit, the control unit, and the operation unit. A position and force control method is applied to this system. The target position of the arm tip is modified using the signals from the force sensor and the joystick. In accordance with this control method, the pressure force is kept constant and the tip is moved so as to follow the shape of the windshields. The various safety features provided include interference limit to limit the area of movement. System experiments were carried out and the effectivity applying lightweight manipulator with long arms to this work was confirmed.

This research aims at realizing the robot throw-over movement from which a user is easy to predict the landing distance of the object when the robot throws it to a user by implementing the human movement characteristics. This paper shows the movement data classification after the measurement and analysis of human movement and the representative data selection for the evaluation experiment. Then it presents the experimental examination of the movement changing depending on the landing distance.

This paper shows evaluation results of the touch screen interface for mobile robot remote operation. Button function, joystick function, and route designation function are developed with an environmental map in the Line & Hollow or the Cell & Hollow method. Features and efficiency are confirmed according to the experimental results in which a mobile robot is remotely operated using the touch screen interface with the developed function to pass through a straight passage or slalom course.

With the touch panel type interface, the operator cannot only look environmental information around the robot but also control the remote robot and camera by using the button on the touch-screen. In addition the operator can direct the robot to move along the same course as drawing on the environmental map on the touch-screen. Both touch screen and joystick interfaces are experimented on a developed mobile robot remote operation system and discussed on maneuverability and operating accuracy.

We have been researching on the remote operated robot in which it is difficult for operator to understand the situation around the robot only by the camera image for remote control. The adequate security and maneuverability cannot be secured, and danger to cause accidents increases. As the solution, we have proposed the methods to construct the environmental map to present the operator the situation around the robot. The performance trial of two range scanners was also executed and reported.

We propose new input method of operation. Instructions to robot are given by stepping on a part of operation screen that is projected on running surface by projector equipped on the robot. This time, we designed the human-friendly amusing mobile (HFAM) function that is new usage of the robot equipped with the SOI. There is a scenario that the robot and a child will be able to play together. In this paper, the outline of the HFAM function and the technology are discussed.

We propose new input method of operation. Instructions to robot are given by stepping on a part of operation screen that is projected on running surface by projector equipped on the robot. This paper examines the layout of instruction buttons on operation screen, and the discrimination of obstacles from stepped foot to apply autonomous movement and the path search for obstacle avoidance are also discussed.

This research aims at realizing the robot throw-over movement which a person is easy to predict the distance to the targeted position of the object when throwing from a robot to a person by implementing the movement characteristics by people. This paper shows the experimental examination of points to change in the movement depending on the distance after the measurement and analysis. In addition we selected the movement data by a subject for a future evaluation experiment.

While it has become aged and decrease-in-birthrate society recently, a human-friendly robot which can support and assist human being directly is expected. We think it is important for mobile robot to have not only the safety function to avoid contact or collision with human being but also the function to preliminary-announce surrounding people the forthcoming motion of the robot before beginning to move. This paper reports on the result of the side-by-side test on the voice and the display.

The deliverer of handover task operates so that the weight load is informed to the receiver. This study is aimed to clarify the characteristic of handover movement by people and to implement it to humanoid robot to make it perform naturally. Movement of people when performing handover task to a person was measured and analyzed. It may be said that a receiver can judge whether the load is heavy or not. The design method of handover movement depending on weight load is discussed and experimented.

When a person does a handover task, the movement of operator contains some information about feature of the load even without oral remarks and the receiver picks up the meaning and provides against the load then receives the load safely and certainly. This research aims to achieve a natural handover task from robot to human by designing the robot body movement. This paper shows the result of analyzing the human's handover task as an earlier stage.

This research aims to achieve a smooth handover task from robot to person without making the person feel some stress and bringing discomfort. We captured the handover task between two persons and analyzed the feature of human movement of the task in order to use it to design the robot's movement. This paper shows the developed simulation software of humanoid robot's movement. Then it presents the result of the experiment about the weight estimation from the handover movement.

This paper discusses the touch screen as user interface for teleoperation system of mobile robot in order to improve the maneuverability. Button function, joystick function, and route designation are developed with Line&Hollow method as environmental map. Features and efficiency are confirmed on the experiments in which the mobile robot is remotely operated using the touchscreen interface with the developed functions to pass through a passage with orthogonal cranks.

We are planning to develop HFAMRO-2 which can play with people using the step-on interface. Therefore we would like to improve the mobility performance and to replace the range sensor from HFAMRO-1. This paper shows the modified design of mobile unit which will be able to double the speed of movement. And it also shows the comparison between new range sensor and old one based on the results of performance measurements. The new sensor can detect and measure the stepped foot more precisely.

We propose new input method of operation. Instructions to robot are given by stepping on a part of operation screen that is projected on running surface by projector equipped on the robot. This paper examines the layout of instruction buttons on operation screen. And additional functions, the preliminary-announcement and indication of forthcoming movement and the discrimination of obstacles from stepped foot to apply autonomous movement and mouse cursor operation, are also discussed.

Recently it has become an aging and decrease-in-the-birthrate society then human-friendly robot which can support and assist human being directly is expected. We think it is important for a mobile robot to have not only the safety function to avoid a collision with human being but also the function to announce surrounding people the following motion of a robot before beginning to move. This paper discusses preliminary-announcement function for mobile robot with a projection. We developed the PMR-5R and proposed the expression with a projection. We have confirmed its effect by some experimental results.

Recently it has become an aging and decrease-in-the-birthrate society then human-friendly robot which can support and assist human being directly is expected. We think it is important for a mobile robot to have not only the safety function to avoid a collision with human being but also the function to announce surrounding people the following motion of a robot before beginning to move. This paper discusses preliminary-announcement function for mobile robot with a laser-pointer. We developed the PMR-1 and proposed the expression with a laser-pointer. We have confirmed its effect by some experimental results.

Recently it has become an aging and decrease-in-the-birthrate society then human-friendly robot which can support and assist human being directly is expected. We think it is important for a mobile robot to have not only the safety function to avoid a collision with human being but also the function to announce surrounding people the following motion of a robot before beginning to move. This paper discusses preliminary-announcement function for mobile robot with a projection. We developed the PMR-2R and proposed the expression with an omni-directional display. We have confirmed its effect by some experimental results.

This paper examines the optimal motion on lifting task to prevent the lumbago during the movement. We settled three different posture to start the motion. We adopted four evaluation criteria to examine : pressure force to lumber vertebra, shearing force to lumber vertebra, total efficiency degree, and contribution rate on each joint.

Human-Friendly Robots is becoming popular. Human-Friendly Robot lives/works in the same space as human. We think the function to tell surrounding people the robot's following action is important as well as a safety function. We report the preliminary-announcement device of the mobile robot which draws a scheduled path on a ground with beam-light by reflector driving unit using stepping motor.

本研究は,"遠隔体験・鑑賞システム"を想定した『人間共存型移動ロボットの遠隔操作システム』の実現のための技術課題として,"操作する人間"とロボットの間における安全性や操作性に着目し,自律動作と遠隔操作の長所を効果的に取り入れ,それぞれの欠点を補うような融合制御手法を検討している.いわゆるshared controlにおけるcomputer controlの具体的な動作を提案・評価するものである.移動ロボットに搭載したカメラからの画像を見ながら,操作者がジョイスティックを使ってロボットを操作する.自律動作として,旋回動作,倣い動作,減速動作の3つを検討している.自律動作の組合せの方法,および,環境認識/状況判断の手法について,ソフトウェア・シミュレーションでの検討を継続した.減速動作を基本として,旋回動作や倣い動作を複合化する手法により,障害物とのニアミスの回数を半減でき,安全性が向上することを確認した.状況に適した自律動作に関しては,通路走行時の曲がり角の通り抜けについて,どのような状況で自律動作を積極的に利用するかを解明し,どのような自律動作が使いやすいか,を検討した.操作者のジョイスティック操作とロボットの距離センサ情報から,周囲の環境とロボットの状況を分類し,旋回動作と倣い動作を選択的に付加する手法により,操作者に違和感ない動作を実現し,操作性が向上できることを確認した.ハードウェアでは,オムニホイール(富士製作所)による4輪駆動機構を有する全方向移動機構を開発中であり,また前方約170度の範囲における対象物との距離を走査できる距離センサ(北陽電気PB9)を使用できるように準備している.これらを組み合わせた移動ロボットを早急に完成させたい.今後は,操作者への情報提示・フィードバック手法を検討するとともに,統合ハードウェア・システムを用いた提案手法の検証実験を行ないたい

The purpose of this study is make it possible for ALS (amyotrophic lateral sclerosis) patients to communicate with surrounding people and to operate remote human-friendly mobile robots by inputting characters and selecting menu options using residual function of the patients, a light of sight.The HMD (head mounted display) has eye-gaze detection function. It is attached to one eye and characterized that it can detect the exact eye gaze point on the HMD screen in spite of deviation of the eye and the HMD. Image from a PC is presented on the screen. At the same time, irradiation near-infrared light of an LED for the eye, and grabbing and processing the eye image and then detecting the centers of the pupil and the corneal reflection produced by the LED light source. The eye gaze point is calculated by the relative position of these centers. In fiscal 2005, eye gaze detection was difficult in the trial product because ghost images of the light source fell on the eye image. So in fiscal 2006, the reason of the ghost images was investigated. By changing light source arrangement, the problem was solved. However, its total size was bigger than expected. Moreover, due to their dispersion the signal of eye gaze points was smoothed. However, the characteristics which can compensate eye gaze points against deviation of the eye and the HMD is demonstrated.The remote control experiment was carried out in which the developed HMD was used as an I/O interface in a remote operation system of mobile robot. The robot has omni-directional mobile mechanism with the camera to take pictures in remote site and the range sensor to acquire distance information. The operation screen shown by HMD is low resolution, 640 by 480. However, we developed the operation interface including five functions as a minimum function to operate the robot remotely - a remote environmental map, the live images, the button to operate robot, the button to operate camera system, and the display of camera direction. When cursor was placed for two seconds on the robot operation button the operation which the button indicates was made to be performed considering safety. Moreover taking care of the characteristic of detecting the direction of gaze, the stop buttons are arranged in the four corners of operation screen. It has been confirmed that both the mobile robot and the camera system could be operated from operation experiment

This paper discusses the prospect of mobile robots’ navigational ability in pedestrian settings (e.g. sidewalks and street crossings), under the condition of not utilizing feature-rich maps that are provided before-hand (e.g. from SLAM-based algorithms). The main motivation is to mimic the human way of interpreting 2D maps (e.g. widely available Google Maps), which would lead to negate the need for pre-mapping a given location. The paper starts by summarizing previous literature regarding robotic navigation in pedestrian settings, leading up to outcomes of our own research. We aim to present results in path planning and real-world scene interpretation, then finally address remaining problems and future prospects.

Computer mice and keyboards are most widely used to work with personal computers as pointing and typing devices respectively. As alternatives to a computer mouse, there are other types of pointing devices that use depth sensor or camera for pointing detection. However, current implementations are uncomfortable for usage because a user must raise his arm(s) to control pointing, as well as having a relatively long distance from the sensing device. To solve those usability problems, we propose a pointing device that can narrow the distance within which users can comfortably move their wrists between the keyboard and the pointing device itself. For this system, we compared between various depth sensors’ performance and eventually chose the Intel Realsense sensor for our system’s use. Additionally, we performed a comparative study involving our proposed system’s performance and that of other conventional input devices. Although the total time to complete experimental tasks using our system was longer than by using other conventional input devices, our proposed system has the fastest time when switching between pointing and typing (i.e. moving hand from mouse to keyboard).

The three-dimensional aerial holographic image interface (3DAHII) is a system for aerial projection of varying 3D objects. The projected objects can be viewed from multiple directions without the use of special devices such as eyeglasses, gas or vapor for projection, rotating parts, and so on. In this research, we propose a system to capture a video of a human body from different perspectives in real-time, and project the result using 3DAHII. Because of realtime capturing and projection, users can interact and communicate directly. To realize this goal, we used a smaller prototype system in which four cameras capture a video of a physical object inside a frame, and the video result will be remotely projected in real-time. The smaller prototype is developed for evaluation and confirmation of the proposed system’s operation.

Computer mice and keyboards are most widely used to work with personal computers as pointing and typing devices respectively. As alternatives to a computer mouse, there are other types of pointing devices that use depth sensor or camera for pointing detection. owever, current implementations are uncomfortable for usage because a user must raise his arm(s) to control pointing, as well as having a relatively long distance from the sensing device. To solve those usability roblems, we propose a pointing device that can narrow the distance within which users can comfortably move their wrists between the keyboard and the pointing device itself. For this system, we compared between various depth sensors’ performance and eventually chose the Intel Realsense sensor for our system’s use. Additionally, we performed a comparative study involving our proposed system’s performance and that of other conventional input devices. Although the total time to complete experimental tasks using our system was longer than by using other conventional input devices, our proposed system has the fastest time when switching between pointing and typing (i.e. moving hand from mouse to keyboard). Key Words: Hand gesture recognition, Motion capture camera, Pointing device

We as human learn specific operations that typically belong to the object by touching and with sensory feedback data, we form particular skills of object handling such as with which hand pose to grasp the object, how much force to use, and which hand movement to make. As human development in babyhood, we all start learning how to handle objects by hands with repetitive interactions, from which we can learn our own skill on behaviors we could do with the object and how we handle it as well. By equipping a touch-oriented robot hand with tactile sensors and by using the collected sensory feedback data, our research is targeted at forming the underlying link between a particular object itself and the specific corresponding operations acted on the object which are helpful to feasibly accomplish a task ordered by human. Besides, a classification of those operations on the object is presented, from which optimal ones that best suit the human task requirements are determined. Specifically, in this paper we propose a machine-learning based approach in combination with an evolutionary method to help progressively build up the hand-based object cognitive intelligence. Overall, the scheme proposed exploits and reveals the robot-hand potential touch-based abilities/intelligence of object interaction and manipulation apart from the existing visually built-up cognition.

This paper describes a path planning system for processing 2D color maps of a given location to provide sidewalk paths, street crossing landmarks, and instructions for a robot platform or user to navigate, using only the current location and destination as inputs. For the navigation of robots and especially disabled people in outdoor pedestrian environments, path planning that explicitly takes into account sidewalks and street crossings is of great importance. Current path planning solutions on known 2D maps (e.g. Google Maps) from both research and industry do not always provide explicit information on sidewalk paths and street crossings, which is a common problem in suburban/rural areas. The path planner test results are shown for location of our campus.

This paper describes a path planning system for processing 2D color maps of a given location to provide sidewalk paths, street crossing landmarks, and instructions for a robot platform or user to navigate, using only the current location and destination as inputs. For the navigation of robots and especially disabled people in outdoor pedestrian environments, path planning that explicitly takes into account sidewalks and street crossings is of great importance. Current path planning solutions on known 2D maps (e.g. Google Maps) from both research and industry do not always provide explicit information on sidewalk paths and street crossings, which is a common problem in suburban/rural areas. The path planner test results are shown for location of our campus.

A Small Mobile Robot (SMR) moves inside another wider Frame Mobile Robot (FMR) and one of them stops at a time to be as a reference for the other using multi-laser pointers fixed in the FMR. These robots are designed to form a closed system that is to minimize accumulated errors occurred during the robot moves in which the robots’ positions are corrected regularly. There are several motivations that required high precision and accurate move of a mobile robot. For example; nowadays, different size of printers and plotters are available. Each printer/plotter has limited size of paper to print on. This limitation is a problem that can be solved if a mobile robot which moves precisely is made. However, can this robot perform printing task for unlimited size of area? What about accumulated errors that surely occur while the mobile robot moves? In what resolution can this robot print? The proposed system in this paper is a closed system that can collaborate between different robots in which an efficient method in localizing mobile robots is introduced using the advantage of straightening of laser waves transmissions in digitalize mode. The main two strengths of the research ar

A Small Mobile Robot (SMR) moves inside another wider Frame Mobile Robot (FMR) and one of them stops at a time to be as a reference for the other using multi-laser pointers fixed in the FMR. These robots are designed to form a closed system that is to minimize accumulated errors occurred during the robot moves in which the robots’ positions are corrected regularly. There are several motivations that required high precision and accurate move of a mobile robot. For example; nowadays, different size of printers and plotters are available. Each printer/plotter has limited size of paper to print on. This limitation is a problem that can be solved if a mobile robot which moves precisely is made. However, can this robot perform printing task for unlimited size of area? What about accumulated errors that surely occur while the mobile robot moves? In what resolution can this robot print? The proposed system in this paper is a closed system that can collaborate between different robots in which an efficient method in localizing mobile robots is introduced using the advantage of straightening of laser waves transmissions in digitalize mode. The main two strengths of the research are as follow: • Achieve precise and accurate mobile robot coordinates by minimizing accumulated errors • Manage mobile robots’ cooperation in localizing and performing different task at a certain time The robots’ movements are checked discretely by multi-laser pointers and corrected accordingly. This can contribute in achieving accurate and precise coordinates and thus, minimizing accumulated errors. As well as, the ability to collaborate between other robots can enhance some features of the system. For instance, it can accelerate the progress by adding other SMRs or even FRMs to finish a big job in less time. In this case, the system can divide different tasks between available robots to finish any assigned job earlier.

We have proposed a hand gestured based control system with an interactive 3D hologram object floating in mid-air as the hand movement reference, named The Aerial Projection of 3D Hologram Object (3DHO). The system consists of the hologram projector and a sensor to capture hand gesture. Leap motion is used for capturing the hand gesture command while it manipulates the 3DHO which produced by a pyramid-shaped reflector and two parabolic mirrors. We evaluate our 3DHO’s performance by comparing it to other 3D input devices such as joystick with slider, joystick without slider, and gamepad to do several pointing tasks. The assessment of comfort and user satisfaction also evaluated by using questionnaire survey. We found out that the 3DHO is good to use in three-dimensional workspace tasks. From the experiments, participants think our 3DHO is easy to learn but they felt some fatigue in their hands so 3DHO is not satisfying enough to use. The development by adding haptic feedback and make a wider 3DHO should be done for improving 3DHO performance.

Passenger comfort has been paramount in the design of suspension systems of high speed cars. The main objective of this paper is to reduce the vertical and pitch accelerations of a half car rail model. A rigid half-car high speed passenger vehicle with 10 degrees of freedom has been modelled to study the vertical and pitch accelerations. State space mathematical approach is used to model rail input that takes in track vibrations. An augmented track and the vehicle model is then designed. An active secondary suspension system based on the Linear Quadratic Gaussian (LQG) optimal control method is designed. Vehicle performances like vertical and pitch accelerations, front and rear suspension travel and control forces have been studied and compared with that of a passive system.

&lt;1&gt; Research and Development of Near-field Touch Interface using Time-of-flight CameraThe purpose of this study is to apply the 3-dimensional image sensor, time-of-flight camera, into a projector-sensor system, to achieve the basic functions (clicking, dragging and sliding) of conventional touch interface and expand it with some new functions such as finger direction detection. The research items are as followings: (1) Near-field hand extraction method, (2) High accuracy touch/hover detection, (3) Integrated and complete projector-sensor system, (4) Evaluation experiments.&lt;2&gt; Research and Development of Calligraphy-Brushwork Learning Support SystemA calligraphy learning support system was proposed for supporting brushwork learning by using a projector. The system was designed to provide the three kinds of training style according to the learner's ability: copying training, tracing training, and combination of both. In order to instruct the three-dimensional brushwork such as writing speed, pressure force, and orientation of the brush, we proposed the instruction method by presenting the information about the brush tip. This method visualizes the position, orientation and moving direction of the brush. A preliminary experiment on learning was performed and the efficiency of the proposed method was examined through the experimental results.

　壁面などに投射した操作パネルの画面を，ロボット・メカトロニクス機器の操作に利用する，投射画面インタフェースの高度・高機能化として，任意の面に投影する双方向インタフェース（ステップ・オン・インタフェースStep-on Interface, SOI）の汎用性の向上に関する研究開発を実施した．具体的には，プロジェクタで壁に投影した操作画面の操作ボタンを，レーザ・ポインタで特定して機器を動作させるシステムの実現を目指した．　本課題では，3つのレーザ・ポインタが互いに干渉することなく，ひとつの画面上で操作することができる，新しい方法を提案した．

　本課題では，複数の物体が同時に接触でき，接触する物体を識別する機能を持つ仮想タッチ・スクリーンの研究開発を実施した．具体的には，Xtionセンサによる測定データに基づき，接触／非接触センシングだけでなく，手指／道具の区別および接触速度の検出をする，仮想キーボード（鍵盤）および仮想シロフォン（木琴）を実現した．Xtionセンサを機器に実装する手法，各種アルゴリズム：(1)手指の認識，(2)接触の認識，(3)接触位置のマッピング，(4)手指／道具（マレット＝ばち）の区別，(5)接触速度の検出の手法，を検討した．これらの機能は，エデュテインメントとエンターテイメントの両方に役立つものだと考える．

画像投射式卓上型上肢訓練装置IDAT-3のアンケート調査による評価 （2014年04月01日）1. 目的 さまざまな年代の男女から次の3項目について評価を得ること．(1) 操作性（使いやすさ，使い勝手，直感性など）(2) 面白さ（楽しさ，充実感，達成感）(3) 有用性（有効性，実用性，利便性）また意見や所感を得ることで，現在のIDAT-3の欠点や不足を理解し，今後の性能の向上，新機能の開発の検討材料とすること．2. 方法と手順 Method and procedure 次の2つの展示会にIDAT-3を出展した．(1) 展示会1：北九州学術研究都市第13回産学連携フェア（http://fair.ksrp.or.jp/） 期日： 2013年10月23日(水)～10月25日(金) 10h00-17h00 場所：北九州学術研究都市　体育館 対象：専門家，職業人，学生，市民(2) 展示会2：2013国際ロボット展（http://www.nikkan.co.jp/eve/irex/） 期日：2013年11月06日(水)～11月09日(土) 10h00-17h00 場所：東京国際展示場　東1･2･3ホール 対象：専門家，職業人，学生，市民アンケートの実施手順は次のとおり：1) 研究開発の背景と目的，IDATの構成と機能，を来訪者に説明し，デモンストレーションする．2) 来訪者にトレーニングプログラム（モグラたたき，風船割り，さかな捕り）を体験してもらう．3) 好きなだけ試してもらったのちに，アンケートへ記入してもらう．アンケートの内容は次のとおり：1. Gender: □male □female2. Age: □-10s □20s □30s □40s □50s □60s □70s □80s- 3. Country: □Japan □Other ( )4. User-friendliness: (good)9・8・7・6・5(average)・4・3・2・1(bad) Comment: ( )5. Amusingness: (good)9・8・7・6・5(average)・4・3・2・1(bad) Comment: ( )6. Usefulness: (good)9・8・7・6・5(average)・4・3・2・1.(bad) Comment: ( )7. Feedback and opinions: ( )3つの評価項目によるIDAD-3の評価は，相対的な値ではなく主観的な絶対的な値でお願いした．合計7日間の展示より88通の回答を得た．性別では男性67名，女性21名．年齢別では10代以下40名，20代30名，30代4名，40代4名，50代以上6名となった．3. 評価値 3つの評価項目における平均値（標準偏差）は次のとおり：(1) 操作性：　10代以下7.25(1.80)，20代7.23(1.45)，30代7.38(1.32)，40代7.75(0.83)，50代9.00(0.00)，60代6.33(0.94)，70代7.00(0.00)，男性7.28(1.54)，女性7.29(1.72)，合計7.28(1.59)(2) 面白さ：　10代以下8.25(1.34)，20代7.93(1.18)，30代8.25(0.97)，40代6.75(1.09)，50代9.00(0.00)，60代5.67(0.94)，70代7.00(0.00)，男性7.88(1.37)，女性8.33(1.17)，合計7.99(1.34)(3) 有用性：　10代以下7.60(1.50)，20代7.67(1.45)，30代7.38(1.32)，40代8.00(0.71)，50代9.00(0.00)，60代8.33(0.94)，70代7.00(0.00)，男性7.60(1.51)，女性7.95(1.27)，合計7.67(1.46)すべての回答者による平均値は，3つの評価項目（操作性，面白さ，有用性）においてすべて，7と8の間にある． “操作性”が，3項目の中で最小の平均値（7.28）と最大の標準偏差（1.59）を得た． “面白さ”が，3項目の中で最大の平均値（7.99）と最小の標準偏差（1.34）を得た．女性の回答者が少ないため，男女間の結果の差異の検討はむずかしい．そこで以下では，年齢による結果の差異を検討する．3.1 操作性 年齢を若年（-10s），壮年（20s-40s），後年（50s-）の3群に分けてまとめた得点はそれぞれ7.25(1.80)，7.31(1.39)，7.33(1.37)となる．年齢群間での得点の有意な差は認められない．すなわち，どの年齢群でもほぼ同じ得点（7.25, 7.31, 7.33）で，IDATの操作感は年齢による影響は小さい．しかし他の2項目に比べてあまり高く評価されていない．問題点はあとで回答者からの意見に基づいて検討する．3.2 面白さ 年齢を若年（-10s），壮年（20s-40s），後年（50s-）の3群に分けてまとめた得点はそれぞれ8.25(1.34)，7.89(1.20)，7.00(1.63)となる．年齢群間での得点の有意な差は認められない．しかし，年齢が上がると面白さの得点が下がる傾向がある．若年者による得点は8.25と高く，興味を引くことができたと考えられる．ゲームのようなトレーニング内容により，若者の関心を得ることが容易だが，高齢者を引き付けるのはそれほど簡単ではない．3.3 有用性 年齢を若年（-10s），壮年（20s-40s），後年（50s-）の3群に分けてまとめた得点はそれぞれ7.60(1.50)，7.64(1.46)，8.33(0.94)となる．年齢群間での得点の有意な差は認められない．しかし，年齢が上がると有用性を評価する傾向がある．これは，このアンケート調査において回答者の一人一人にIDAT訓練のねらいである手と眼の協調訓練の重要性を説明しているからだと思われる．高齢者は身体および認知機能の低下にともなう危機意識をもっており，このIDAT訓練の意義を認識してもらいやすい．4. 意見や所感 意見や所感に寄せられたコメントを項目に分けて紹介し考察する．4.1 表示 ・もぐらは，平面的な絵が出てくるより立体的なのが出てきたら面白いかも．あとは壁に映してみるとか．・投影面が斜面のときはどうなるでしょう？リハビリテーション訓練装置の映像にどの程度の臨場感が必要かは，よく検討するべきである．一方，家庭用ゲーム機を応用対象としたリアルタイム３次元コンピュータグラフィックスを実現するLSIの性能は向上し続けている．コスト・パフォーマンスを考慮しながら，高精細で高品質の３次元画像を導入することも将来的にはあり得る．また本体が平面に垂直に設置されていれば，その面が傾いていてもその動作に問題はない．机の上でなく壁での実施も，機器構成の工夫により可能になる．4.2 反応感度 ・タッチ反応が少し遅い．・反応が鈍い．・反応速度を向上してほしい．・タイムラグが気になる．障害物を検知するための探索区域は，作業面から約1cm離れた平面上である．したがって，システムは面に触れていない手にも反応する．これは反応感度に対してよく働くはずであるが，確かに応答時間差を感じる．これには，ハードウェアとソフトウェアとの両面で原因が考えられる．まず信号伝達である：動く物体を検出するにはセンサ情報の伝達速度が遅い．センサデータを取得するサイクルタイムを短縮するために，コンピュータの処理能力を含めた見直しが必要である．次にプログラミングである：当たり判定の計算に時間がかかりすぎる．アルゴリズムを見直して最適化する必要がある．4.3 判定精度 ・当たり判定の精度を良くして欲しい．・連続的な当たり判定が遅いときがある．ある程度の反応感度を得るために，障害物の正確な大きさや形状を認識していない．手の大きさ（子供／大人，人種，など）だけでなく，手の状態（握った手，開いた手，など）やトイハンマの使用などにより，算出された障害物の位置が多少異なる．あらゆる状況により正確に対応するためには，訓練をはじめる前に訓練者の叩いた状態を計測して補正値を算出しておくなどの方法を考える必要がある．4.4. 小型化・低コスト化 ・小型化が楽しみにしている．・コストが高い．現在は研究開発のために汎用コンピュータを使っている．しかし商品化にはワンチップマイクロコンピュータを使用して本体と一体化することもできる．また，高価なレーザレンジファインダ（URG）を置き換えて同じ機能を提供する低価格のRGB-Dセンサについても別途検討している．4.5. 全体的な意見 多くの肯定的で励みになる回答を得た．・これなら楽しくリハビリが進むと思いました．・操作しやすい，いろんな人が楽しめる，ぜひまたやりたい．・リハビリに最も良い，楽しさを感じる，実用性あり．・未来の発展を期待しています．このように，多くの来場者は我々の研究開発の重要性と必要性を認め，高く評価してくれた．しかし同時に，改善の余地があることを私たちに知らせる貴重な意見を得ることができた．これらの提言に基づいて，さらにより良いシステムにするべく研究を続ける．

　本課題は，上体の運動および認知機能の維持・回復訓練を目的として，SOI（ステップ・オン・インタフェース：プロジェクタの投影画面を機器から人への情報提示だけでなく人から機器への指示入力としても利用する新たらしいインタフェース）とPCのみで構成する単純な可搬型機器を製作すると共に，訓練コンテンツとしてのゲーム・アプリケーションを開発するものである．　まず2011年度の前半は，SOIを二輪型移動機構の前後に2組搭載した移動ロボットHFAMRO-2上でアプリケーションの開発をすすめ，「風船割り（対象が上方に動く）」，「魚つかみ（対象が左右に動く）」，「もぐらたたき（対象がランダムな位置に出現）」について目処を立てていた．2011年度の後半からは，北九州ロボットフォーラムの事業である市内発ロボット創生プロジェクトにも採択され，（財）北九州産業学術推進機構を事務局として，産業医科大学リハビリテーション医学講座，九州産業大学工学部，リーフ（株），（株）コア九州カンパニーをメンバーとする共同研究としても実施できた．これにより，機器装置としてこれまでに，1号機（反射型：普及型プロジェクタを用いるが投影距離を稼ぎながら装置の高さを低く抑えるために反射鏡を用いるもの），2号機（直映型：比較的低い位置に設置した少し高価な短焦点型プロジェクタで机上に直接投影するもの），3号機（臨床試用プロトタイプ：最新型のプロジェクタを用いて収納性と可搬性を狙ったもの）を試作し，産業医科大学リハビリテーション科や特別養護老人ホームもじみ苑に持ち込んで意見をうかがう機会を持つこともできた．一方，画面デザインの追加・変更の容易性や詳細な調整を可能とするためにアプリケーション・プログラムのアルゴリズムと実現する手法を全面的に改めた新規開発を実施している，いまだに完全には動作しておらず，不十分な状態である．　2011年度末には産業医科大学と早稲田大学の双方における「人を対象とする研究に関する倫理審査委員会」の承認を得て，2012年度には，リハビリテーション用（医大リハ科）およびレクリエーション用（特養施設）としての臨床試用を開始することになっているため，引き続き何らかの助成と必要としている．