The article proposes a method for classifying human postures using an improved probabilistic neural network (PNN) with different distance measures and different probabilistic distribution functions (PDF). We found that the PNN with angular distance provides better accuracy, precision, and recall for the postures classification tasks than the PNN with conventional Euclidean distance. The k Nearest Neighbors (kNN) method gives slightly better prediction results than PNN, but our PNN is much faster. Such good computational performance is beneficial for posture recognition tasks that require real-time functions. An example is the needs of cobots or service robots. The article also proposes a method for selecting the distribution smoothing parameter ( ) using the sub-optimization process based on the improved Gray Wolf optimization (I-GWO) algorithm. It was found that the impact of PDF differences on the quality of the results can be reduced by choosing the best possible . In order to evaluate the developed method, selected human activities were recorded. The datasets were created using two different RGB-D systems located in two different laboratories.

This paper introduces a method that can represent the irregular deformations of a flat flexible object and reproduce the same deformations on a similar object which is in an initial state. The work was carried out in two stages: (a) analyze and estimate the deformation of the object in the target state by a deformation estimation method, (b) planning and execution by a real-time sensor feedback loop. The novelty of the proposal concerns detecting the major deformations and ignoring the undesired deformations due to surface wrinkling, surface roughness, or noisy depth data. Different experimental settings were applied to efficiently estimate the various deformations of the handkerchief and reproduce similar deformations by the six-degree-of-freedom robotic manipulator and RGB-D vision sensor. The subject of this work is a flat flexible object, but it can be extended for more general conditions.

Image super-resolution (ISR) technology aims to enhance resolution and improve image quality. It is widely applied to various real-world applications related to image processing, especially in medical images, while relatively little applied to anime image production. Furthermore, contemporary ISR tools are often based on convolutional neural networks (CNNs), while few methods attempt to use transformers that perform well in other advanced vision tasks. We propose a so-called anime image super-resolution (AISR) method based on the Swin Transformer in this work. The work was carried out in several stages. First, a shallow feature extraction approach was employed to facilitate the features map of the input image’s low-frequency information, which mainly approximates the distribution of detailed information in a spatial structure (shallow feature). Next, we applied deep feature extraction to extract the image semantic information (deep feature). Finally, the image reconstruction method combines shallow and deep features to upsample the feature size and performs sub-pixel convolution to obtain many feature map channels. The novelty of the proposal is the enhancement of the low-frequency information using a Gaussian filter and the introduction of different window sizes to replace the patch merging operations in the Swin Transformer. A high-quality anime dataset was constructed to curb the effects of the model robustness on the online regime. We trained our model on this dataset and tested the model quality. We implement anime image super-resolution tasks at different magnifications (2×, 4×, 8×). The results were compared numerically and graphically with those delivered by conventional convolutional neural network-based and transformer-based methods. We demonstrate the experiments numerically using standard peak signal-to-noise ratio (PSNR) and structural similarity (SSIM), respectively. The series of experiments and ablation study showcase that our proposal outperforms others.

Recently, the dangers associated with face generation technology have been attracting much attention in image processing and forensic science. The current face anti-spoofing methods based on Generative Adversarial Networks (GANs) suffer from defects such as overfitting and generalization problems. This paper proposes a new generation method using a one-class classification model to judge the authenticity of facial images for the purpose of realizing a method to generate a model that is as compatible as possible with other datasets and new data, rather than strongly depending on the dataset used for training. The method proposed in this paper has the following features: (a) we adopted various filter enhancement methods as basic pseudo-image generation methods for data enhancement; (b) an improved Multi-Channel Convolutional Neural Network (MCCNN) was adopted as the main network, making it possible to accept multiple preprocessed data individually, obtain feature maps, and extract attention maps; (c) as a first ingenuity in training the main network, we augmented the data using weakly supervised learning methods to add attention cropping and dropping to the data; (d) as a second ingenuity in training the main network, we trained it in two steps. In the first step, we used a binary classification loss function to ensure that known fake facial features generated by known GAN networks were filtered out. In the second step, we used a one-class classification loss function to deal with the various types of GAN networks or unknown fake face generation methods. We compared our proposed method with four recent methods. Our experiments demonstrate that the proposed method improves cross-domain detection efficiency while maintaining source-domain accuracy. These studies show one possible direction for improving the correct answer rate in judging facial image authenticity, thereby making a great contribution both academically and practically.

The purpose of this paper is to introduce and discuss the following two functions that are considered to be important in human‐coexistence robots and human‐symbiotic robots: the method of generating emotional movements, and the method of transmitting behavioral intentions. The generation of emotional movements is to design the bodily movements of robots so that humans can feel specific emotions. Specifically, the application of Laban movement analysis, the development from the circumplex model of affect, and the imitation of human movements are discussed. However, a general technique has not yet been established to modify any robot movement so that it contains a specific emotion. The transmission of behavioral intentions is about allowing the surrounding humans to understand the behavioral intentions of robots. Specifically, informative motions in arm manipulation and the transmission of the movement intentions of robots are discussed. In the former, the target position in the reaching motion, the physical characteristics in the handover motion, and the landing distance in the throwing motion are examined, but there are still few research cases. In the latter, no groundbreaking method has been proposed that is fundamentally different from earlier studies. Further research and development are expected in the near future.

Light detection and ranging (LiDAR) systems, also called laser radars, have a wide range of applications. This paper considers two problems in LiDAR data. The first problem is occlusion. A LiDAR acquires point clouds by scanning the surrounding environment with laser beams emitting from its center, and therefore an object behind another cannot be scanned. The second problem is sample-sparsity. LiDAR scanning is usually taken with a fixed angular step, consequently the sample points on an object surface at a long distance are sparse, and thus accurate boundary and detailed surface of the object cannot be obtained. To address the occlusion problem, we design and implement a novel three-dimensional (3D) LiDAR with a long-arm which is able to scan occluded objects from their flanks. To address the sample-sparsity problem, we propose an adaptive resolution scanning method which detects object and adjusts the angular step in realtime according to the density of points on the object being scanned. Experiments on our prototype system and scanning method verify their effectiveness in anti-occlusion and anti-sparsity as well as accuracy in measuring. The data and the codes are shared on the web site of https: //github.com/SCVision/LongarmLiDAR.

This paper introduces a system that can estimate the deformation process of a deformed flat object (folded plane) and generate the input data for a robot with human-like dexterous hands and fingers to reproduce the same deformation of another similar object. The system is based on processing RGB data and depth data with three core techniques: a weighted graph clustering method for non-rigid point matching and clustering; a refined region growing method for plane detection on depth data based on an offset error defined by ourselves; and a novel sliding checking model to check the bending line and adjacent relationship between each pair of planes. Through some evaluation experiments, we show the improvement of the core techniques to conventional studies. By applying our approach to different deformed papers, the performance of the entire system is confirmed to have around 1.59 degrees of average angular error, which is similar to the smallest angular discrimination of human eyes. As a result, for the deformation of the flat object caused by folding, if our system can get at least one feature point cluster on each plane, it can get spatial information of each bending line and each plane with acceptable accuracy. The subject of this paper is a folded plane, but we will develop it into a robotic reproduction of general object deformation.

This paper introduces an object model and an interaction method for a simulated experience of pottery on a potter’s wheel. Firstly, we propose a layered cylinder model for a 3D object of the pottery on a potter’s wheel. Secondly, we set three kinds of deformation functions to form the object model from an initial state to a bowl shape: shaping the external surface, forming the inner shape (deepening the opening and widening the opening), and reducing the total height. Next, as for the interaction method between a user and the model, we prepare a simple but similar method for hand-finger operations on pottery on a potter’s wheel, in which the index finger movement takes care of the external surface and the total height, and the thumb movement makes the inner shape. Those are implemented in the three-dimensional aerial image interface (3DAII) developed in our laboratory to build a simulated experience system. We confirm the operation of the proposed object model (layered cylinder model) and the functions of the prepared interaction method (a simple but similar method to actual hand-finger operations) through a preliminary evaluation of participants. The participants were asked to make three kinds of bowl shapes (cylindrical, dome-shaped, and flat-type) and then they answered the survey (maneuverability, visibility, and satisfaction). All participants could make something like three kinds of bowl shapes in less than 30 min from their first touch.

In this paper, the integration of Leap Motion controller with RoboAnalyzer software has been reported. Leap Motion is a vision based device that tracks the motion of human hands, which was processed and used to control a virtual robot model in RoboAnalyzer, an established robot simulation software. For intuitive control, the robot model was copied and transformed to be placed at four different locations such that the user watches four different views in the same graphics environment. This novel method was used to avoid multiple windows or viewports and was observed to have a marginally better rendering rate. Several trials of picking up cylindrical objects (pegs) and moving them and placing in cylindrical holes were carried out and it was found that the manipulation was intuitive, even for a novice user.

In this research, we focus on finding a new method of human-robot interaction in industrial environment. A visionbased dynamic hand gestures recognition system has been proposed for robot arm picking task. 8 dynamic hand gestures are captured for this task with a 100fps high speed camera. Based on the LRCN model, we combine the MobileNets (V2) and LSTM for this task, the MobileNets (V2) for extracting the image features and recognize the gestures, then, Long Short-Term Memory (LSTM) architecture for interpreting the features across time steps. Around 100 samples are taken for each gesture for training at first, then, the samples are augmented to 200 samples per gesture by data augmentation. Result shows that the model is able to learn the gestures varying in duration and complexity and gestures can be recognized in 88ms with 90.62% accuracy in the experiment on our hand gesture dataset.

In this paper, we introduce the three-dimensional aerial image interface, 3DAII. This interface reconstructs and aerially projects a three-dimensional object image, which can be simultaneously observed from various viewpoints or by multiple users with the naked eye. A pyramid reflector is used to reconstruct the object image, and a pair of parabolic mirrors is used to aerially project the image. A user can directly manipulate the three-dimensional object image by superimposing a user’s hand-finger or a rod on the image. A motion capture sensor detects the user’s hand-finger that manipulates the projected image, and the system immediately exhibits some reaction such as deformation, displacement, and discoloration of the object image, including sound effects. A performance test is executed to confirm the functions of 3DAII. The execution time of the end-tip positioning of a robotic arm has been compared among four operating devices: touchscreen, gamepad, joystick, and 3DAII. The results exhibit the advantages of 3DAII; we can directly instruct the movement direction and movement speed of the end-tip of the robotic arm, using the three-dimensional Euclidean vector outputs of 3DAII in which we can intuitively make the end-tip of the robotic arm move in three-dimensional space. Therefore, 3DAII would be one important alternative to an intuitive spatial user interface, e.g., an operation device of aerial robots, a center console of automobiles, and a 3D modelling system. A survey has been conducted to evaluate comfort and fatigue based on ISO/TS 9241-411 and ease of learning and satisfaction based on the USE questionnaire. We have identified several challenges related to visibility, workspace, and sensory feedback to users that we would like to address in the future.

In the brand-wise random-ordered drinking PET
bottles picking task, the overlapping and viewing angle problem
makes a low accuracy of the brand recognition. In this paper,
we set the problem to increase the brand recognition accuracy
and try to find out how the overlapping rate infects on the
recognition accuracy. By using a stepping motor and transparent fixture, the training images were taken automatically
from the bottles under 360 degrees to simulate a picture taken
from viewing angle. After that, the images are augmented with
random cropping and rotating to simulate the overlapping
and rotation in a real application. By using the automatically
constructed dataset, the Inception V3, which was transferred
learning from ImageNet, is trained for brand recognition. By
generating a random mask with a specific overlapping rate
on the original image, the Inception V3 can give 80% accuracy
when 45% of the object in the image is visible or 86% accuracy
when the overlapping rate is lower than 30%.

This article presents a framework for planning sidewalk-wise paths in data-limited pedestrian environments by visually recognizing city blocks in 2D digital maps (e.g. Google Maps, OpenStreetMaps) using contour detection, then applying graph theory to infer a pedestrian path from start till finish. There are two main targeted problems; firstly, several locations around the world (e.g. suburban / rural areas) do not have recorded data on street crossings and pedestrian walkways. Secondly, the continuous process of recording maps (i.e. digital cartography) is, to our current knowledge, manual and not yet fully automated in practice. Both issues contribute towards a scaling problem in which it becomes time and effort consuming to continuously monitor and record such data on a global scale. As a result, the framework’s purpose is to produce path plans that do not depend on prerecorded (e.g. using SLAM) or data-rich pedestrian maps, thus facilitating navigation for mobile robots and people of visual impairments alike. The framework was able to produce pedestrian paths for most locations where data on sidewalks and street crossings were indeed limited, but still some challenges remain. In this article, the framework’s structure, output, and challenges are explained. Additionally, we mention some works in the literature on how to make use of such path plan effectively.

In this research, six brands of soft drinks are decided to be picked up by a robot with a monocular Red Green Blue (RGB) camera. The drinking bottles need to be located and classified with brands before being picked up. The Mask Regional Convolutional Neural Network (R-CNN), a mask generation network improved from Faster R-CNN, is trained with common object in contest datasets to detect and generate the mask on the bottles in the image. The Inception v3 is selected for the brand classification task. Around 200 images are taken or found at first; then, the images are augmented to 1500 images per brands by using random cropping and perspective transform. The result shows that the masked image can be labeled with its brand name with at least 85% accuracy in the experiment.

In this research, six brands of soft drinks are decided to be picked up by a robot with a monocular RGB camera. The drinking bottles need to be located and classified with brands before being picked up. A Mask R-CNN is pretrained with COCO datasets to detect and generate the mask on the bottles in the image. The Inception v3 is selected for the brand classification task. Around 200 images are taken, then, the images are augmented to 1500 images per brand by using random cropping and perspective transform. The results show that the masked image can be labeled with its brand name with at least 85% accuracy in the experiment.

This paper highlights camera orientation estimation accuracy and precision, as well as proposing a new camera calibration technique using a reinforcement learning method named PPO (Proximal Policy Optimization) in offline mode. The offline mode is used just for extracting the camera geometry parameters that are used for improving accuracy in real-time camera pose estimation techniques. We experiment and compare two popular techniques using 2D vision feedbacks and evaluate their accuracy beside other considerations related to real applications such as disturbance cases from surrounding environment and pose data stability. First, we use feature points detection ORB (Oriented FAST and Rotated BRIEF) and BF (Brute-Force) matcher to detect and match points in different frames, respectively. Second, we use FAST (Features from Accelerated Segment Test) corners and LK (Lucas–Kanade) optical flow methods to detect corners and track their flow in different frames. Those points and corners are then used for the pose estimation through optimization process with the: (a) calibration method of Zhang using chessboard pattern and (b) our proposed method using PPO. The results using our proposed calibration method show significant accuracy improvements and easier deployment for end-user compared to the pre-used methods.

In this research, we proposed and implemented a finger pointing detection system with a short-range type depth camera. The most widely used pointing device for a computer interface is a mouse, and as an alternative there is a depth sensor to detect hand and finger movement. However, in the present literature, since the user must operate the cursor at a relatively large distance from the detection device, the user needs to raise his arm and make wide movements, which is inconvenient over long periods of time. To solve such usability problem, we proposed a comfortable and easy to use method by narrowing the distance between the keyboard and the pointing device. Next, we compared various depth sensors and selected one that can recognize even small movements Additionally, we proposed a mapping method between the users perceived cursor position and the real one pointed by the index finger direction. Furthermore, we compared our pointing method with a mouse and touch pad for the usability, accuracy and working speed. The results showed that it has users have better performance on continuous operation of character input from the keyboard and cursor pointing.

In this paper, an integration of Leap Motion controller with RoboAnalyzer software is proposed. Leap Motion is an inexpensive sensor which has three Infrared (IR) emitters and two IR cameras and can track 10 fingers of the two human hands. The device sends the data to the computer connected to it through its controller software which can be accessed in a variety of programming languages. In the proposed setup, the position of the index finger of the right hand is tracked in a Visual C# Server Application and the coordinates are extracted accurately with respect
to a frame attached at the center of the device. The coordinates are then sent to Virtual Robot Module (client application) in which a Coordinate System (marker) is mapped with the input it receives. Based on the movement of the index finger, the marker moves in the Virtual Robot Module (VRM). When the marker moves closer to the end-effector of the robot, the Server application attaches the marker with the end-effector. Thereafter, any incremental Cartesian motion of the index finger is mapped to equivalent Cartesian motion of the robot endeffector. This is achieved by finding the inverse kinematics solution for the new pose of the robot EE and of the
eight solutions obtained through inverse kinematics, the solution that is closest to the current pose is selected as the appropriate solution for the new pose. The server application then updates the VRM with the new joint angles and accordingly the robot moves instantly on the software. The setup connected to a laptop running Leap Motion Controller and VRM is shown in Figure 2(a) and the workflow is illustrated in Figure 2(b).

This paper describes a path planning framework for processing 2D maps of given pedestrian locations to provide sidewalk paths and street crossing information. The intention is to allow mobile robot platforms to navigate in pe-destrian environments without previous knowledge and using only the current location and destination as inputs. Depending on location, current path planning solutions on known 2D maps (e.g. Google Maps and OpenStreetMaps) from both research and industry do not always provide explicit information on sidewalk paths and street crossings, which is a common problem in suburban/rural areas. The framework’s goal is to provide path planning by means of visual inference on 2D map images and search queries through downloadable map data. The re-sults have shown both success and challenges in estimating viable city block paths and street crossings.

The Aerial Projection of 3D Hologram Object (3DHO) which we have proposed, is a hand gesture-based control system with an interactive 3D hologram object floating in mid-air as the hand movement reference. This system mainly consists of the pyramid-shaped reflector which produces 3D hologram object, the parabolic mirror, and the leap motion controller for capturing hand gesture command. The user can control or interact with 3DHO by using their finger or a baton-shaped object. This paper is focusing on the evaluation of the 3DHO by comparing it to other 3D input devices (such as joystick with a slider, joystick without the slider, and gamepad) on five different positioning tasks. We also investigate the assessment of comfort, ease of learning, and user satisfaction by questionnaire survey. From the experimentation, we learn that 3DHO is not good to use in one-dimensional workspace task, but has a good performance in two-dimensional and three-dimensional workspace tasks. From questionnaire results, we found out that 3DHO is averagely comfortable but may cause some fatigues around arm and shoulder. It is also easy to learn but not satisfying enough to use.

This paper presents a newly-developed calligraphy-stroke learning support system. The system has the following functions: a) displaying a brushwork, trajectory, and handwriting, b) recording and playback of an expert's calligraphy-stroke, and c) teaching a learner a calligraphy-stroke. The system has the following features which shows our contribution: (1) It is simple and compact built up with a sensor and a projector so as to be easy to introduce to usual educational fields and practical leaning situations. (2) Three-dimensional calligraphy-stroke is instructed by presenting two-dimensional visual information. (3) A trajectory region is generated as continuous squares calculated using a brush model based on a brush position information measured by a sensor. (4) A handwriting is expressed by mapping a handwriting texture image according to an ink concentration and a brush handling state. The result of a trial experiment suggests the effectiveness of the learning support function about a letter form and a calligraphy-stroke.

This article focuses on the implementation details of a portable interactive device called Image-projective Desktop Varnamala Trainer. The device uses a projector to produce a virtual display on a flat surface. For enabling interaction, the information about a user's hand movement is obtained from a single two-dimensional scanning laser range finder in contrast with a camera sensor used in many earlier applications. A generalized calibration process to obtain exact transformation from projected screen coordinate system to sensor coordinate system is proposed in this article and implemented for enabling interaction. This permits production of large interactive displays with minimal cost. Additionally, it makes the entire system portable, that is, display can be produced on any planar surface like floor, tabletop, and so on. The calibration and its performance have been evaluated by varying screen sizes and the number of points used for calibration. The device was successfully calibrated for different screens. A novel learning-based methodology for predicting a user's behaviour was then realized to improve the system's performance. This has been experimentally evaluated, and the overall accuracy of prediction was about 96%. An application was then designed for this set-up to improve the learning of alphabets by the children through an interactive audiovisual feedback system. It uses a game-based methodology to help students learn in a fun way. Currently, it has bilingual (Hindi and English) user interface to enable learning of alphabets and elementary mathematics. A user survey was conducted after demonstrating it to school children. The survey results are very encouraging. Additionally, a study to ascertain the improvement in the learning outcome of the children was done. The results clearly indicate an improvement in the learning outcome of the children who used the device over those who did not.

This paper proposes the ORB-SHOT SLAM or OS-SLAM, which is a novel method of 3D loop closing for trajectory correction of RGB-D visual SLAM. We obtain point clouds from RGB-D sensors such as Kinect or Xtion, and we use 3D SHOT descriptors to describe the ORB corners. Then, we train an offline 3D vocabulary that contains more than 600,000 words by using two million 3D descriptors based on a large number of images from a public dataset provided by TUM. We convert new images to bag-of-visual-words (BoVW) vectors and push these vectors into an incremental database. We query the database for new images to detect the corresponding 3D loop candidates, and compute similarity scores between the new image and each corresponding 3D loop candidate. After detecting 2D loop closures using ORB-SLAM2 system, we accept those loop closures that are also included in the 3D loop candidates, and we assign them corresponding weights according to the scores stored previously. In the final graph-based optimization, we create edges with different weights for loop closures and correct the trajectory by solving a nonlinear least-squares optimization problem. We compare our results with several state-of-the-art systems such as ORB-SLAM2 and RGB-D SLAM by using the TUM public RGB-D dataset. We find that accurate loop closures and suitable weights reduce the error on trajectory estimation more effectively than other systems. The performance of ORB-SHOT SLAM is demonstrated by 3D reconstruction application.

© 2016 IEEE.In this paper we present an interactive aerial projection of 3D hologram objects by using the pyramid hologram and parabolic mirror system (for 3D hologram object reconstruction) and the Leap Motion sensor (as a finger movement detector of a user). This system not only can reconstruct and project the 3D hologram object in the mid-air, but also provides a way to interact with it by moving a user's finger. There are three main steps: the reconstruction of 3D object, the projection of 3D hologram object in the mid-air, and the interactive manipulation of 3D hologram object. The first step is realized by using pyramid hologram with LCD display. The second step is achieved with the parabolic mirror hologram. And Leap Motion sensor is used for the last step to detect user finger movement. This paper traces the design concept and confirms the system function of an interactive aerial projection of 3D hologram objects by a prototype demonstration.

The purpose of this study is to realize a near-field touch interface that is compact,flexible,and highly accurate. We applied a 3-dimensional image sensor (time-of-flight camera) to achieve the basic functions of conventional touch interfaces,such as clicking,dragging,and sliding,and we designed a complete projector-sensor system. Unlike conventional touch interfaces,such as those on tablet PCs,the system can sense the 3-dimensional positions of fingertips and 3- dimensional directions of fingers. Moreover,it does not require a real touch screen but instead utilizes a mobile projector for display. Nonetheless,the system is compact,with a working distance of as short as around 30 cm. Our methods solve the shadow and reflection problems of the time-of-flight camera and can provide robust detection results. Tests have shown that our approach has a high success rate (98.4%) on touch/hover detection and a small standard error (2.21 mm) on position detection on average for different participants,which is the best performance we have achieved. Some applications,such as the virtual keyboard and virtual joystick,are also realized based on the proposed projector-sensor system.

This paper shows a method of tracking feature points to update camera pose and generating a synchronous map for AR (Augmented Reality) system. Firstly we select the ORB (Oriented FAST and Rotated BRIEF) [1] detection algorithm to detect the feature points which have depth information to be markers, and we use the LK (Lucas-Kanade) optical flow [2] algorithm to track four of them. Then we compute the rotation and translation of the moving camera by relationship matrix between 2D image coordinate and 3D world coordinate, and then we update the camera pose. Last we generate the map, and we draw some AR objects on it. If the feature points are missing, we can compute the same world coordinate as the one before missing to recover tracking by using new corresponding 2D/3D feature points and camera poses at that time. There are three novelties of this study: an improved ORB detection, which can obtain depth information, a rapid update of camera pose, and tracking recovery. Referring to the PTAM (Parallel Tracking and Mapping) [3], we also divide the process into two parallel sub-processes: Detecting and Tracking (including recovery when necessary) the feature points and updating the camera pose is one thread. Generating the map and drawing some objects is another thread. This parallel method can save time for the AR system and make the process work in real-time.

© 2015 IEEE.This paper shows a novel method to extract hand Representative Point (RP) based on 2-dimensional laser range scanner for hand motion estimation on projecting system. Image-projecting Desktop Arm Trainer (IDAT) is a projecting system for hand-eye coordination training, in which a projector displays an exercise screen on the desktop, and a laser range scanner detects trainee's hand motion. To realize multi-user HMI and expand more entertainment functions in IDAT system, an Air Hockey application was developed in which the hand RP requires a high precision. To generate hand RP precisely, we proposed our method in two parts to solve the data error problem and changeable hand contour problem. In part one, a data modifier is proposed and a sub-experiment is carried out to establish a modifying function for correcting sensor original data. In part two, we proposed three RP algorithms and carried out an evaluation experiment to estimate the reliability of three algorithms under different conditions. From the result, we get the most reliable algorithm corresponding to different situations in which the error of hand RP is less than 9.6 mm.

© 2015 IEEE.This paper shows the simulating and displaying of Puck motion in virtual Air Hockey based on projective interface for multi-users HRI Image-projective Desktop Arm Trainer (IDAT) is an upper-limb rehabilitation system for hand-eye coordination training based on projective interface. To expand more entertainment functions and realize multi-user HRI, we want to develop a virtual Air Hockey application in IDAT. To develop this application, Puck motion should be simulated correctly and displayed smoothly on the screen. There are mainly 3 problems: Data updating rate problem, virtual collision calculation and individual's hand misrecognition problem. We proposed our method into three parts corresponding to different problems. In part 1, we used multiple timers with shared memory to deal with unsynchronized data updating rate problem. In part 2, an original physical engine is designed to calculate Puck's velocity and detect virtual collision. In part 3, to deal with the individual's hand misrecognition problem, a history-based hand owner recognition algorithm is implemented to distinguish users' hands.

This paper proposes a novel technology to detect the orientation of an image relying on its contour which is noised to varying degrees. For the image orientation detection, most methods regard to the landscape image and the image taken of a single object. In these cases, the contours of these images are supposed to be immune to the noise. This paper focuses on the the contour noised after image segmentation. A polar orientation descriptor Orientation Context is viewed as a feature to describe the coarse distribution of the contour points. This descriptor is verified to be independent of translation, isotropic scaling, and rotation transformation by theory and experiment. The relative orientation depends on the minimum distance Roulette Distance between the descriptor of a template image and that of a test image. The proposed method is capable of detecting the direction on the interval from 0 to 359 degrees which is wider than the former contour-based means (Distance Phase [1], from 0 to 179 degrees). What's more, the results of experiments show that not only the normal binary image (Noise-0, Accuracy-1: 84.8%) (defined later) achieves more accurate orientation but also the binary image with slight contour noise (Noise-1, Accuracy-1: 73.5%) could obtain more precise orientation compared to Distance Phase (Noise-0, Accuracy-1: 56.3%
Noise-1, Accuracy-1: 27.5%). Although the proposed method (O(op2)) takes more time to detect the orientation than Distance Phase (O(st)), it could be realized including the preprocessing in real time test with a frame rate of 30.

This paper showcases the development of a mobile robot integrated with Projectable Interactive Surface to facilitate its interaction with human users. The system was designed to interact with users of any physical attributes such as height, arm span etc. without re-calibrating it. The system was designed in such a way that there would be no need for the human to come in physical contact with the robot to give it instructions. This system uses a projector to render a virtual display on the ground allowing us to project large displays. Microsoft Kinect integrated in the systems performs a dual functionality of tracking the user movements along with mapping the surrounding environment. The gestures of the tracked user are interpreted and an audio visual signal is projected by the robot in response.

In this paper we present an interactive aerial projection of 3D hologram objects by using the pyramid hologram and parabolic mirror system (for 3D hologram object reconstruction) and the Leap Motion sensor (as a finger movement detector of a user). This system not only can reconstruct and project the 3D hologram object in the mid-air, but also provides a way to interact with it by moving a user's finger. There are three main steps: the reconstruction of 3D object, the projection of 3D hologram object in the mid-air, and the interactive manipulation of 3D hologram object. The first step is realized by using pyramid hologram with LCD display. The second step is achieved with the parabolic mirror hologram. And Leap Motion sensor is used for the last step to detect user finger movement. This paper traces the design concept and confirms the system function of an interactive aerial projection of 3D hologram objects by a prototype demonstration.

In this paper we present an interactive aerial projection of 3D hologram objects by using the pyramid hologram and parabolic mirror system (for 3D hologram object reconstruction) and the Leap Motion sensor (as a finger movement detector of a user). This system not only can reconstruct and project the 3D hologram object in the mid-air, but also provides a way to interact with it by moving a user's finger. There are three main steps: the reconstruction of 3D object, the projection of 3D hologram object in the mid-air, and the interactive manipulation of 3D hologram object. The first step is realized by using pyramid hologram with LCD display. The second step is achieved with the parabolic mirror hologram. And Leap Motion sensor is used for the last step to detect user finger movement. This paper traces the design concept and confirms the system function of an interactive aerial projection of 3D hologram objects by a prototype demonstration.

This paper showcases the development of a mobile robot integrated with Projectable Interactive Surface to facilitate its interaction with human users. The system was designed to interact with users of any physical attributes such as height, arm span etc. without re-calibrating it. The system was designed in such a way that there would be no need for the human to come in physical contact with the robot to give it instructions. This system uses a projector to render a virtual display on the ground allowing us to project large displays. Microsoft Kinect integrated in the systems performs a dual functionality of tracking the user movements along with mapping the surrounding environment. The gestures of the tracked user are interpreted and an audio visual signal is projected by the robot in response.

In this paper we present an interactive aerial projection of 3D hologram objects by using the pyramid hologram and parabolic mirror system (for 3D hologram object reconstruction) and the Leap Motion sensor (as a finger movement detector of a user). This system not only can reconstruct and project the 3D hologram object in the mid-air, but also provides a way to interact with it by moving a user's finger. There are three main steps: the reconstruction of 3D object, the projection of 3D hologram object in the mid-air, and the interactive manipulation of 3D hologram object. The first step is realized by using pyramid hologram with LCD display. The second step is achieved with the parabolic mirror hologram. And Leap Motion sensor is used for the last step to detect user finger movement. This paper traces the design concept and confirms the system function of an interactive aerial projection of 3D hologram objects by a prototype demonstration.

This paper showcases the development of a mobile robot integrated with Projectable Interactive Surface to facilitate its interaction with human users. The system was designed to interact with users of any physical attributes such as height, arm span etc. without re-calibrating it. The system was designed in such a way that there would be no need for the human to come in physical contact with the robot to give it instructions. This system uses a projector to render a virtual display on the ground allowing us to project large displays. Microsoft Kinect integrated in the systems performs a dual functionality of tracking the user movements along with mapping the surrounding environment. The gestures of the tracked user are interpreted and an audio visual signal is projected by the robot in response.

© 2015 IEEE.This paper shows the simulating and displaying of Puck motion in virtual Air Hockey based on projective interface for multi-users HRI Image-projective Desktop Arm Trainer (IDAT) is an upper-limb rehabilitation system for hand-eye coordination training based on projective interface. To expand more entertainment functions and realize multi-user HRI, we want to develop a virtual Air Hockey application in IDAT. To develop this application, Puck motion should be simulated correctly and displayed smoothly on the screen. There are mainly 3 problems: Data updating rate problem, virtual collision calculation and individual's hand misrecognition problem. We proposed our method into three parts corresponding to different problems. In part 1, we used multiple timers with shared memory to deal with unsynchronized data updating rate problem. In part 2, an original physical engine is designed to calculate Puck's velocity and detect virtual collision. In part 3, to deal with the individual's hand misrecognition problem, a history-based hand owner recognition algorithm is implemented to distinguish users' hands.

In this paper, a calligraphy learning support system is presented for supporting brushwork learning by using a projector. The system was designed to provide the three kinds of training according to the learner's ability as followings: copying training, tracing training, and combination of them. In order to instruct the three-dimensional brushwork such as the writing speed, pressure, and orientation of the brush, we proposed the instruction method by presenting the information to only brush tip. This method can be visualized a brush position and the orientation. In addition, the copying experiment was performed using the proposed method. As a result, the efficiency of the proposed method was examined through experiment.

This paper shows a novel method to extract hand Representative Point (RP) based on 2-dimensional laser range scanner for hand motion estimation on projecting system. Image-projecting Desktop Arm Trainer (IDAT) is a projecting system for hand-eye coordination training, in which a projector displays an exercise screen on the desktop, and a laser range scanner detects trainee's hand motion. To realize multi-user HMI and expand more entertainment functions in IDAT system, an Air Hockey application was developed in which the hand RP requires a high precision. To generate hand RP precisely, we proposed our method in two parts to solve the data error problem and changeable hand contour problem. In part one, a data modifier is proposed and a sub-experiment is carried out to establish a modifying function for correcting sensor original data. In part two, we proposed three RP algorithms and carried out an evaluation experiment to estimate the reliability of three algorithms under different conditions. From the result, we get the most reliable algorithm corresponding to different situations in which the error of hand RP is less than 9.6 mm.

This paper shows the simulating and displaying of Puck motion in virtual Air Hockey based on projective interface for multi-users HRI. Image-projective Desktop Arm Trainer (IDAT) is an upper-limb rehabilitation system for hand-eye coordination training based on projective interface. To expand more entertainment functions and realize multi-user HRI, we want to develop a virtual Air Hockey application in IDAT. To develop this application, Puck motion should be simulated correctly and displayed smoothly on the screen. There are mainly 3 problems: Data updating rate problem, virtual collision calculation and individual's hand misrecognition problem. We proposed our method into three parts corresponding to different problems. In part 1, we used multiple timers with shared memory to deal with unsynchronized data updating rate problem. In part 2, an original physical engine is designed to calculate Puck's velocity and detect virtual collision. In part 3, to deal with the individual's hand misrecognition problem, a history-based hand owner recognition algorithm is implemented to distinguish users' hands.

An Image-projective Desktop Varnamala Trainer (IDVT) called SAKSHAR has been designed to improve the learning by children through an interactive audio visual feedback system. This device uses a projector to render a virtual display, which permits production of large interactive displays with minimal cost. The user's hand is recognized with the help of a Microsoft Kinect Version 2. The entire system is portable, i.e., it can be projected on any planar surface. Since the Kinect does not give precise 3D coordinates of points for detecting a touch, a model of recognition of a touch purely based on the contact of the user's hand with the surface would not yield accurate results. We have instead modeled the touch action by using multiple points along the trajectory of the tracked point of the user's hand while hand makes contact with the surface. Fitting a curve through these points and analyzing the errors is used to make the detection of touch accurate.

This paper shows a novel method to extract hand Representative Point (RP) based on 2-dimensional laser range scanner for hand motion estimation on projecting system. Image-projecting Desktop Arm Trainer (IDAT) is a projecting system for hand-eye coordination training, in which a projector displays an exercise screen on the desktop, and a laser range scanner detects trainee's hand motion. To realize multi-user HMI and expand more entertainment functions in IDAT system, an Air Hockey application was developed in which the hand RP requires a high precision. To generate hand RP precisely, we proposed our method in two parts to solve the data error problem and changeable hand contour problem. In part one, a data modifier is proposed and a sub-experiment is carried out to establish a modifying function for correcting sensor original data. In part two, we proposed three RP algorithms and carried out an evaluation experiment to estimate the reliability of three algorithms under different conditions. From the result, we get the most reliable algorithm corresponding to different situations in which the error of hand RP is less than 9.6 mm.

This paper shows the simulating and displaying of Puck motion in virtual Air Hockey based on projective interface for multi-users HRI. Image-projective Desktop Arm Trainer (IDAT) is an upper-limb rehabilitation system for hand-eye coordination training based on projective interface. To expand more entertainment functions and realize multi-user HRI, we want to develop a virtual Air Hockey application in IDAT. To develop this application, Puck motion should be simulated correctly and displayed smoothly on the screen. There are mainly 3 problems: Data updating rate problem, virtual collision calculation and individual's hand misrecognition problem. We proposed our method into three parts corresponding to different problems. In part 1, we used multiple timers with shared memory to deal with unsynchronized data updating rate problem. In part 2, an original physical engine is designed to calculate Puck's velocity and detect virtual collision. In part 3, to deal with the individual's hand misrecognition problem, a history-based hand owner recognition algorithm is implemented to distinguish users' hands.

An Image-projective Desktop Varnamala Trainer (IDVT) called SAKSHAR has been designed to improve the learning by children through an interactive audio visual feedback system. This device uses a projector to render a virtual display, which permits production of large interactive displays with minimal cost. The user's hand is recognized with the help of a Microsoft Kinect Version 2. The entire system is portable, i.e., it can be projected on any planar surface. Since the Kinect does not give precise 3D coordinates of points for detecting a touch, a model of recognition of a touch purely based on the contact of the user's hand with the surface would not yield accurate results. We have instead modeled the touch action by using multiple points along the trajectory of the tracked point of the user's hand while hand makes contact with the surface. Fitting a curve through these points and analyzing the errors is used to make the detection of touch accurate.

This paper shows a novel method to extract hand Representative Point (RP) based on 2-dimensional laser range scanner for hand motion estimation on projecting system. Image-projecting Desktop Arm Trainer (IDAT) is a projecting system for hand-eye coordination training, in which a projector displays an exercise screen on the desktop, and a laser range scanner detects trainee's hand motion. To realize multi-user HMI and expand more entertainment functions in IDAT system, an Air Hockey application was developed in which the hand RP requires a high precision. To generate hand RP precisely, we proposed our method in two parts to solve the data error problem and changeable hand contour problem. In part one, a data modifier is proposed and a sub-experiment is carried out to establish a modifying function for correcting sensor original data. In part two, we proposed three RP algorithms and carried out an evaluation experiment to estimate the reliability of three algorithms under different conditions. From the result, we get the most reliable algorithm corresponding to different situations in which the error of hand RP is less than 9.6 mm.

An Image-projective Desktop Varnamala Trainer (IDVT) called SAKSHAR has been designed to improve the learning by children through an interactive audio visual feedback system. This device uses a projector to render a virtual display, which permits production of large interactive displays with minimal cost. The user's hand is recognized with the help of a Microsoft Kinect Version 2. The entire system is portable, i.e., it can be projected on any planar surface. Since the Kinect does not give precise 3D coordinates of points for detecting a touch, a model of recognition of a touch purely based on the contact of the user's hand with the surface would not yield accurate results. We have instead modeled the touch action by using multiple points along the trajectory of the tracked point of the user's hand while hand makes contact with the surface. Fitting a curve through these points and analyzing the errors is used to make the detection of touch accurate.

© 2014, Fuji Technology Press. All rights received.This paper presents a novel arm-training system, known as the image-projecting desktop arm trainer (IDAT), which is aimed at hand-eye coordination training. The projector displays an exercise image on a desktop in front of a seated patient, and the scanning range finder measures the behavior of the patient as he/she performs the exercise. IDAT is non-invasive and does not constrain the patient. Its efficiency is based on the voluntary movements of the patient, although it offers neither the physical assistance nor tactile feedback of some conventional systems. Three kinds of training content have been developed: “mole-hitting,” “balloon-bursting,” and “fish-catching.” These games were designed for training hand-eye coordination in different directions. A patient and/or medical professional can set a suitable training level, that is, the training time, speed of movement of the objects, and number of objects to appear at any one time, based on the patient’s condition and ability. A questionnaire survey was carried out to evaluate IDAT-3, and the results showed that it was highly acclaimed in terms of user-friendliness, fun, and usefulness.

© 2014, Fuji Technology Press. All rights received.To meet the higher requirements of human-machine interface technology, a robot with human-following capability, a classic but significant problem, is discussed in this paper. We first propose a human detection method that uses only a single laser range scanner to detect the waist of the target person. Second, owing to the limited speed of a robot and the potential risk of obstructions, a new human-following algorithm is proposed. The speed and acceleration of a robot are adaptive to the human-walking speed and the distance between the human and robot. Finally, the performance of the proposed control system is successfully verified through a set of experimental results obtained using a two-wheelmobile robot working in a real environment under different scenarios.

This paper shows a novel walking training system for foot-eye coordination. To design customizable trajectories for different users conveniently in walking training, a new system which can track and record the actual walking trajectories by a tutor and can use these trajectories for the walking training by a trainee is developed. We set the four items as its human-robot interaction design concept: feedback, synchronization, ingenuity and adaptability. A foot model is proposed to define the position and direction of a foot. The errors in the detection method used in the system are less than 40 mm in position and 15 deg in direction. On this basis, three parts are structured to achieve the system functions: Trajectory Designer, Trajectory Viewer and Mobile Walking Trainer. According to the experimental results,we have confirmed the systemworks as intended and designed such that the steps recorded in Trajectory Designer could be used successfully as the footmarks projected in Mobile Walking Trainer and foot-eye coordination training would be conducted smoothly.

Multi-user laser pointers system, in which more than single people could use the laser pointers concurrently, has promising applications, such as group discussion, appliances control for several handicapped persons, controlling the large displays by a few users and the like. Conventional methods employed in the above-mentioned applications are common mouse, gesture control utilizing the Kinect sensor or leap motion, including the single laser pointer. Common mouse and single laser pointer could not meet the requirements, which let a few users operate simultaneously. And gesture control is limited to a far distance. Compared to multi-user laser pointers, the majority research focuses on single laser pointer. However, multiple users would suffer the dilemma in which each one is not able to point at the target on the screen any time, at far distance, just because the laser pointer is controlled by only one person. The paper proposes a novel way to make it possible that 3 laser pointers are pointing at the screen without intervening between each other. Firstly, the foreground with all the dynamic spots is extracted. On the foreground, these spots will be searched all the time by grasping the contours of them. Hence, various information of the spots like the coordinate, pixel value, area and so on is obtained. Secondly, a square image containing the whole spot is referred to as the input of the designing back propagation neural network. The BPNN output indicates the category to which the laser pointer belongs. An experiment verifies that it works well under certain light conditions (12-727lux) if green laser pointers are used.

This paper introduces the real-time fingers naming method then illustrates how hand languages could be recognized by hand gesture and relation between fingertips. Moreover, this paper provides information on how to calculate and name each finger then using arm for dynamically adjust and improve the stability while the hand is moving. Supported by the study of relation between fingertips, palms, and arms, this proposed method can recognize hand gesture and translate these signs into numbers according to the standard sign language. The approach using in this paper relies on the image depth and the RGB image to identify hands, arms and fingertips. Then, the relation between each part is used to recognize a finger name regardless of the different direction of movement. Also, this report describes how to implement the proposed method into the ASUS Xtion as a sensor.

Laser spot clicking technique is the term of remote interaction technique between human and computer using a laser pointer as a pointing device. This paper is focused on the performance test of two laser spot clicking techniques. An off-the-shelf laser pointer has a toggle switch to generate a laser spot, the presence (ON) or absence (OFF) of this spot and its combination are the candidates of the interaction technique. We conducted empirical study that compared remote pointing technique performed using combination of ON and OFF of the laser spot, ON-OFF and ON-OFF-ON, and using a desktop mouse as a baseline comparison. We present quantitative performance test based on Fitts' test using a one-direction tapping test in ISO/TS 9241-411 procedure; and assessment of comfort using a questionnaire. We hope this result give contribution to the interaction technique using laser pointer as a pointing device especially in selecting the appropriate clicking technique for real application. Our results suggest ON-OFF technique has positive advantages over ON-OFF-ON technique such as the throughput and comfort.

Robot human-following is an important part of interaction between robot and people. Generally, speed of mobile robots is limited and far slower than human beings naturally walking speed. In order to catch the human rapidly, in this paper, we introduce a control method which uses adaptive acceleration of robots speed. The speed of robots mainly depends on the human speed and distance. Also the robots speed acceleration is adaptive to the distance between human and robots. The proposed control is successfully verified through experiments. © 2013 IEEE.

Image-projective Desktop Arm Trainer IDAT is designed to improve the upper limb therapy effect. This paper is focused on its design concept. The most important structural feature of IDAT is that it is non-invasive and unconstrained. Setting on the desk apart from the trainee sitting makes no body contact. It works based on the trainee's voluntary movement, although it has neither physical assist nor tactile feedback on conventional systems. IDAT is developed based on the design concept that the direct interaction is critical in terms of the motivation increase. Instead of a joystick, a handle, a robotic arm and a display screen on conventional systems, IDAT uses the screen projected on a desktop and produces the visual reaction at the time on the spot where the trainee operates. It inspires the vivid, lively and actual feeling to the trainee. © 2013 IEEE.

This paper describes the application of step-on interface (SOI) to therapy. SOI consists of a projector and a sensor such as range scanner, and its special feature is using a projected screen as bidirectional interface through which the information is presented from a robot to a user and the user instructions are delivered to the robot. The human-friendly amusing mobile robot HFAMRO equipped with SOI on a mobile platform is a playing-tag-robot which can be used for gait training. The image-projective desktop arm trainer IDAT is designed set on a desk in front of a trainee sitting. This kind of system is adjustable and customized to each individual by setting parameters, using multimedia channels, or uploading the program. Therefore it could provide the motivation and accomplishment to a trainee and maintain his/her enthusiasm and interest.

This paper presents one of the interaction models between humans and machines using a camera, the projector, and the spot-light from a laser pointer device. A camera was attached on the top of the projector, and the projector directed a direction screen display on the wall, while the user pointed a laser pointer to the desired location on the direction screen display. It is confirmed that this system can handle some distortion conditions of the direction screen display, such as an oblique rectangle, horizontal trapezoid distortion, and vertical trapezoid distortion as well as some surface illuminance - 127, 425, 630, and 1100 lux; and the system is designed to be used for static and moving objects. The coordinates that were obtained from the distorted screen can be used to give commands to a specific machine, robot, and application.

Aim to improve upper limb rehabilitation effect, we design and develop an Image-projective Desktop Arm Trainer (IDAT). Compared with conventional therapy, IDAT provides a more effective and interesting training method. IDAT's goal is to maintain and improve patients' upper limb function by training their eye-hand coordination. We select step-on interface (SOI) as the input system which makes trainees can operate IDAT with hand directly. Trainees can make a customized training setting. It can provide motivation and accomplishment to trainees and maintain their enthusiasm and interest. So IDAT provides a much different human robot interaction with pervious upper limb rehabilitation robots that equip a joystick or controller to operate remotely. We propose this idea in 2007 and have applied SOI on some mobile robots. Now we apply it on IDAT to make a new way to upper limb rehabilitation.

We propose and implement a solution for applying an infrared radiation (IR) image sensor to step-on interface (SOI). SOI is a kind of natural human-robot interface what consists of a projector and a laser range scanner (LRG) sensor. And it enables any interactive touch applications on desktop or floor. We attempt to introduce an IR image sensor such as ASUS Xtion to SOI instead of LRG sensor. In this paper, we will describe the procedure that how to use the Xtion and detect touched point. We distinguish user's hand from background (surface) depends on depth data from ASUS Xtion, and detecting touching action when finger almost close to the background. The proposed processes involve IR depth image acquisition, seeking for hand and its contours by thresholding, recognition of touched areas and computing of theirs center position. The research enables ASUS Xtion be applied on SOI in a simple way. Moreover, this system can realize touch interaction on any surface.

A laser pointer can be used as an input interface in human-machine interaction. Such utilization, however, can be problematic and one of the main issues is the lack of good reliability in the laser spotlight detection. Another problem is how to interpret the user's movement of the spotlight into commands for the application. This paper proposes a method for a laser spotlight detection. The aim is to improve the practicality and reliability of the previous approaches. We use the maximum pixel value as a multiplier in determining the threshold. Maximum pixel value is obtained from environment brightness at a specified time. For the second problem we propose a simple interpretation of incidents that allows the user to use the application, with three main events: laser-move, hover, and single-click. There is no need for users and program to wait a specified time span to be able to interact with each other, and the user can directly give commands to the application after the single-click event. These approaches result in better reliability, easier operation of the application by the user, and allow opportunity for development of a system for rehabilitative, recreation, and input interface devices in the future.

This paper presents the informative motion study to make a human-coexistence robot useful. First the usage and the design and marketing of a human-coexistence robot are considered. Next, the informative motion study that we are tackling to improve the human-coexistence robot's personal affinity is explained advocating the informative kinesics for human-machine system. As an example of application deployment, study on continuous movement (usual movement) and preliminary operation (prior operation) is shown.

This paper presents the study results on the handover movement informing a receiver of the weight load as an example of the informative motion for the humansynergetic robot. To design and generate the movement depending on the weight load, the human movement is measured and analyzed, and four items are selected as the parameter to vary - the distance between target point and transferred point (in front-back direction), the distance between highest point and transferred point (in vertical direction), the elbow rotation angle, and the waist joint angle. The fitted curve of the parameter variation depending on the weigh load is obtained from the tendency of the subjects' movement data. The movement data for an arbitrary weight load is generated processing the standard data at 0 kg of weight load so that each parameter follows the fitted curve. From the questionnaire survey, although it is difficult for a receiver to estimate the exact weight load, he may distinguish the heavy weight load from the light weight load so that the package will be received safely and certainly.

The friendly amusing mobile (FAM) function in which a robot and a user can interact through a motion is proposed applying the mobile robot step-on interface (SOI) in which the user can direct robot movement or operation by stepping or pointing on the button which shows the contents to desire on the operation screen projected on a floor. HFAMRO-2 mobile robot is developed to realize playing a light tag and three applications-animal tail stepping, bomb fuse stamping, and footprint stepping-are produced as a trial after the design policy.

To expand use of the mobile robot Step-On Interface
(SOI), originally targeting maintenance, training, and recovery of human physical and cognitive functions, we introduce a “Truly-Tender-Tailed” (T3, pronounced tee-cube) tag-playing robot as a “Friendly Amusing Mobile” (FAM) function. Displaying a previously prepared bitmap (BMP) image and speeding up display make it easy to design button placement and other screen parameters using a painting software package. The BMP-image scope matrix simplifies step detection and recognition and the motion trajectory design editor facilitates robot behavior design.

This video shows the basic functions of HFAMRO-2 equipped with the step-on interface (SOI). In the SOI the projected screen is used as a bilateral interface. It not only presents information from the equipment to the user but also delivers the instructions from the user to the equipment. HFAMRO is intended to represent the concept based on which robots interact with users. It assumes, for example, the ability to play `tag' - in this case, playing tag with light, similar to `shadow' tag. The HFAMRO-2 mobile robot, developed to study the SOI's application with mobility, has two sets of the SOI consisting of a projector and a range scanner on a mobile platform. The projector displays a direction screen on a travel surface and the two-dimensional range scanner detects and measures the user's stepping to specify the selected button.

This paper introduces a tag-playing robot as a "Friendly Amusing Mobile" (FAM) function to expand the mobile robot Step-On Interface (SOI) use and to promote the human-robot interaction through a motion, targeting maintenance, training, and recovery of human physical and cognitive functions in the elderly, physically challenged, injured, etc. Displaying a previously prepared bitmap (BMP) image becomes the display update rate faster and makes it easy to design button placement and other screen parameters using a painting software package. The scope matrix generated from the BMP image simplifies the step detection and recognition, and the motion trajectory design editor facilitates the robot behavior design.

This paper discusses discrimination and implementation of emotions on a zoomorphic robot aiming at designing emotional robot movements and improving robot friendliness. We consider four emotions; joy, anger, sadness, and fear. Two opposite viewpoints, performer and observer, are considered to acquire emotional movement data for analysis. The performer subjects produce emotional movements and movements that are easy to be recognized as expressions of the intended emotion are selected among the measured movements by the observer subjects. Discriminating the emotion embedded in a movement is tried using feature values based on the Laban movement analysis (LMA) and the principal component analysis (PCA). By the discrimination using PCA, the resultant rates of the correct discrimination are about 70% for all four emotions. The features of emotional movements are presumed from the coefficients of the discrimination function obtained in the emotion discrimination using PCA. Emotions are implemented by converting a setup for basic movements by employing the design principle based on the movement features. The result of the verification experiment suggests that four emotional movements are divided into two groups; the joy and anger group and the sadness and fear group. The emotional movements of joy or anger are dynamic, large-scale and frequent being relatively easy to interpret the intended emotion, while the emotional movements of sadness or fear are static, small-scale and little making difficult to understand the feature of movements.

In studying dynamic remodeling of environmental mapping around a mobile robot operated remotely while data measured by the robot range sensor is sent from the robot to the operator, we introduce the Line & Hollow method and the Cell & Hollow method for environmental mapping. Results for the three types of environmental situation clarifies features and effectiveness of our approach. In Line & Hollow method, an isosceles triangle is set based on the range data. The base line is pulled to express obstacle shape and the inside is hollowed out to express vacant space. In Cell & Hollow method, the cell value corresponding to the range data is incremented, and an obstacle is assumed to be exist if the cell value exceeds the ascending threshold. The cell value is decremented on the line between the cell that measured data indicates and the cell located at the sensor, and the obstacle is deleted if the value drops below the descending threshold. We confirmed that environmental mapping for either reflects a dynamic environmental change.

To improve HFAMRO-1 mobile robot maneuverability and safety, we added a step-on interface (SOI) to direct robotic or mechatronic tasks and operations (HFAMRO: “human-friendly amusing” mobile robot). To do so, a projector displays a direction screen on a floor or other surface, and an operator specifies a button showing the selected movement by stepping or pointing. We modified the direction screen so that among buttons displayed on two lines, stepped-on buttons directing movement are displayed only on the lower line. We also shortened retention time and had selected movement executed only when the foot was removed from the stepped-on button. The robot has 2 SOIs and multiple projection screens, and can be controlled from either direction for the same function. We synchronized direction and preliminary-announcement screens to inform passersby which way the robot would move. Using range scanner data, the robot distinguishes feet from other objects based on size and autonomous movement fusion control to avoid obstacles is implemented.

This paper shows the result of the characteristics measurement of a two-dimensional active range scanner, URG made by Hokuyo Automatic Co., Ltd. which was released in 2004 and is spreading widely as an external sensor for mobile robot. The following items were clarified from the characteristics measurement of URG-X002S in various conditions. (1) In the case that the object has a gloss surface or a black surface, the error rate (the frequency that the scanner judges to be impossible to measure and an error code is output) rises when the oblique angle of object becomes large and also the distance to object becomes long. (2) In the case that the object has a white surface or a rough surface, not only the error rate is zero but also the margin of error becomes dozens of millimeters and the varying is small, if the oblique angle is smaller than 60 deg and the distance is shorter than 1 m. (3) The lateral error is negligibly small if the distance to detect is shorter than 1 m. Moreover it shows the result of the examination to apply the range scanner in the Step-On Interface (SOI), in which the scanner is used for detection and measurement of the stepping of an operator. Based on the measured results, we designed the judgment method of the stepping, the installation position of the scanner, and the placement of buttons in the direction screen to apply the range scanner to the SOI for operation of a mobile robot.© Takafumi Matsumaru
Licensee Bentham Open.

This paper presents the result of the study on handover movement informing a receiver of the weight load as an example of the informative motion of a human-coexistence type robot (human-friendly robot). Movement of human body when performing handover task to a person was measured and analyzed first, and the features of variance of movement depending on the weight load were clarified. Then, the questionnaire survey was carried out to verify whether people could read out the difference in weight load from the variance of movement, in which the movement was reproduced using a simulator. It may be said that a receiver can judge whether a package is heavy or not, although it is difficult to estimate the exact weight load. Accordingly, if the variance of the noted points is included in the design of the handover movement of a humanoid robot, people as a receiver will be able to estimate the weight load to make easy to receive a package safely and certainly.

This paper presents the result of trials of the discrimination of emotion from movement and the addition of emotion in movement on a teddy bear robot, aiming at both expressing a robot's emotion by movement and improving a robot's personal affinity. We addressed four kinds of emotion joy, anger, sadness, and fear. In this research, two standpoints were considered a performer and an observer to establish the data of emotional movement used for analysis. The data of movement were collected as a performer's standpoint and they were sorted out in an observer's standpoint. In discriminating the emotion included in movement from the data of movement, both the method using Laban's feature quantity and the method using principal component analysis were tried. By the discrimination using principal component analysis, about 70% of rate of correct discrimination was obtained on all the four emotions. The feature of movement that each emotion could be interpreted was presumed from the coefficient of the discrimination function obtained in the discrimination using principal component analysis. Using the obtained feature the design principle of movement to add emotion into a basic movement was defined. From the verification experiment, it was suggested that the movement which people can interpret the intended emotion with relatively high probability would be produced about joy and anger. About fear and sadness, since the movement to express those emotions has small and little motions, it would be difficult to distinguish the feature and to produce clear emotional movement.

This paper proposes the step-on interface (SOI) to operate a mobile robot in which a projector projects and displays a direction screen on a floor and a user specifies a button showing the selected movement by stepping or pointing. The HFAMRO-1 mobile robot has been developed to demonstrate the SOI's potential (HFAMRO: "human-friendly amusing" mobile robot). The SOI of HFAMRO-1 consists of a projector and a range scanner on an omni-directional mobile platform. From operational comparison with other input interfaces, it is confirmed that we can direct the robot movement using our own foot. We had some students who does not specialize robotic systems try to operate HFAMRO-1 with their shoes, and all trial students could specify the button to operate the robot satisfactorily and everyone mastered the SOI immediately. © Matsumaru and Akai.

This paper presents the result of the experimental examination by "passing each other" and "positional prediction" in simulated interactive situation between people and mobile robot. We have developed four prototype robots based on four proposed methods for preliminarily announcing and indicating to people the speed and direction of upcoming movement of mobile robot moving on two-dimensional plane. We observed significant difference between when there was a preliminary-announcement and indication (PAI) function and when there was not even in each experiment. Therefore the effect of preliminary-announcement and indication of upcoming operation was declared. In addition the feature and effective usage of each type of preliminary-announcement and indication method were clarified. That is, the method of announcing state of operation just after the present is effective when a person has to judge to which direction he should get on immediately due to the feature that simple information can be quickly transmitted. The method of indicating operations from the present to some future time continuously is effective when a person wants to avoid contact or collision surely and correctly owing to the feature that complicated information can be accurately transmitted. We would like to verify the result in various conditions such as the case that traffic lines are obliquely crossed.

This paper discusses the mobile robot PMR-5 with the preliminary-announcement and display function which indicates the forthcoming operations to the people near the robot by using a projector. The projector is set on a mobile robot and a 2-dimensional frame is projected on a running surface. In the frame, not only the scheduled course but also the states of operation can be clearly announced as the information about movement. We examine the presentation of the states of operation such as stop or going back including the time information of the scheduled course on the developed robot. Scheduled course is expressed as the arrows considering the intelligibility at sight. Arrow expresses the direction of motion directly and the length of arrow can announce the speed of motion. Operation until 3-second-later is indicated and three arrows classified by color for each second are connected and displayed so these might show the changing of speed during 3-second period. The sign for spot revolution and the characters for stop and going back are also displayed. We exhibited the robot and about 200 visitors did the questionnaire evaluation. The average of 5-stage evaluation is 4.5 points and 3.9 points for the direction of motion and the speed of motion respectively. So we obtained the evaluation that it is intelligible in general.

We propose approaches and equipment for preliminarily
announcing and indicating to people the speed and direction of movement of mobile robots moving on a two-dimensional plane. We introduce the four approaches categorized into (1) announcing the state just after the present and (2) indicating operations from the present to some future time continuously. To realize the approaches, we use omni-directional display (PMR-2), flat-panel display (PMR-6), laser pointer (PMR-1), and projection equipment (PMR-5) for the announcement unit of protobots. The four protobots were exhibited at the 2005 International Robot Exhibition (iREX05). We had visitors answer questionnaires in a 5-stage evaluation. The projector robot PMR-5 received the highest evaluation score among the four. An examination of differences by gender and age suggested that some people prefer simple information, friendly expressions, and a minimum of information to be presented at one time.

This research aims to propose the method and equipment to preliminary-announce and indicate the surrounding people both the speed of motion and the direction of motion of the mobile robot that moves on a two-dimensional plane. This paper discusses the mobile robot PMR-6, in which the liquid crystal display (LCD) is set up on the mobile unit, and the state of operation at 1.5 s before the actual motion is indicated. The basis of the content to display is 'arrow' considering the intelligibility for people even at first sight. The speed of motion is expressed as the size (length and width) of the arrow and its color based on traffic signal. The direction of motion is described with the curved condition of the arrow. The characters of STOP are displayed in red in case of stop. The robot was exhibited to the 2005 International Robot Exhibition held in Tokyo. About 200 visitors answered to the questionnaires. The average of five-stage evaluation is 3.56 and 3.97 points on the speed and on the direction respectively, so the method and expression were evaluated comparatively intelligible. As for the gender, the females appreciated about the speed of motion than the males on the whole. Concerning the age, some of the younger age and the upper age admired highly about the direction of motion than the middle age.

This paper proposed the model to estimate the load on the lumber vertebra considering not only the value presumed from posture but also the increased amount by Valsalva maneuver of extrapolating from the vital capacity. The pressure force can be estimated to be reduced by about 30% by the effect of the abdominal pressure by using the proposed model as said so far. From the error of the presumed value of ground reaction force from an actual measurement, it is thought that the presumed accuracy of the lumbar vertebra load using the proposed model is smaller than 10%. Furthermore, two operations with extreme start-on posture were compared on transition of the compressive force and shear force on the lumbar vertebra. It was brought out that a start-on posture signficantly affects the maximum load on the lumber vertebra. The result suggests that optimal start-on posture may be between two postures.

This paper discusses the design and the basic characteristic of the mobile robot PMR-1 with the preliminaryannouncement and display function of the forthcoming operation (the direction of motion and the speed of motion) to the people around the robot by drawing a scheduled course on a running surface using light-ray. The laser pointer is used as a light source and the light from the laser pointer is reflected in a mirror. The light-ray is projected on a running surface and a scheduled course is drawn by rotating the reflector around the pan and the tilt axes. The preliminary-announcement and display unit of the developed mobile robot can indicate the operation until 3-second-later preliminarily, so the robot moves drawing the scheduled course from the present to 3-second-later. The experiment on coordination between the preliminary-announcement and the movement has been carried out, and we confirmed the correspondence of the announced course with the robot trajectory both in the case that the movement path is given beforehand and in the case that the robot is operated with manual input from a joystick in real-time. So we have validated the coordination algorithm between the preliminary-announcement and the real movement.

The aim of this research is to establish the quantitative evaluation method for lifting task and to clarify the safety posture and optimal motion. This paper firstly examines the operation from differenet three start-on postures with four items: the maximum compressive force and share force on L5/S1 joint, the energy efficiency index, and the degree of contribution of each joint. As a result it has been shown that the load not only to lumbar vertebra but to knee joint should be emphasized and the pose-C is an appropriate start-on posture in which the knee is flexed almost at right-angled and the upper body is raised. Moreover from the simulation using minimum jerk model we found a proper operation, but the actual operation has earlier timing to extend the lower body than presumed. So not only the criteria on the whole operation or the relative criteria but also the absolute criteria paying attention to a portion like knee joint moment seems necessary to study the optimal lifting motion.

This paper discusses the design method of the bodily shape and motion of a humanoid robot to raise not only the emotional but informative interpersonal-affinity of a humanoid robot. Concrete knowledge and a concrete opinion are classified into the movement prediction from its configuration and the movement prediction from continuous motion or preliminary motion, and they are discussed mentioning the application and usage. Specifically, the bodily shape and motion, which is easier to predict and understand the capability and performance and the following action and intention of a humanoid robot for the surrounding people who are looking at it, are considered.

The aim of this research is to establish the quantitative evaluation method for lifting task and to clarify the safety posture and optimal motion. This paper examines the motion from three kinds of start-on posture with the acceptance rate. Operation-A starts from the pose-A in which the knee is extended to the maximum. Operation-B starts from the pose-B in which the knee is flexed to the maximum and the upper body is raised as much as possible. Operation-C starts from the pose-C in which the knee is flexed at almost right-angled and the upper body is raised. The acceptance rate is the estimated population rate who can permit the joint moment during the lifting operation, based on the presumed moment and the coefficient of variation of the acceptable marginal moment on each joint. The compressive force on lumbar vertebrae computed from the L5/S1 joint moment at 85[%] of the acceptance rate (about Av.-1SD) has not been over the standard value of a previous research. So we have set 85[%] as the safety standard acceptance rate, then we settled judging-A (over 95[%], recommended), judging-B (85-95[%], should note), and judging-C (under 85[%], should modify). Although the ankle joint on operation-A and the knee joint on operation-B is judged as C rank, every joint on operation-C shows high acceptance rate. So the validity of operation-C has been clarified quantitatively.

This paper discusses the mobile robot PMR-5 with the preliminary- announcement and display function which indicates the forthcoming operations to the people near the robot by using a projector. The projector is set on a mobile robot and a 2-dimensional frame is projected on a running surface. In the frame, not only the scheduled course but also the states of operation can be clearly announced as the information about movement. We examine the presentation of the states of operation such as stop or going back including the time information of the scheduled course on the developed robot. Scheduled course is expressed as the arrows considering the intelligibility at sight. Arrow expresses the direction of motion directly and the length of arrow can announce the speed of motion. Operation until 3-second-later is indicated and three arrows classified by color for each second are connected and displayed so these might show the changing of speed during 3-second period. The sign for spot revolution and the characters for stop and going back are also displayed. We exhibited the robot and about 200 visitors did the questionnaire evaluation. The average of 5-stage evaluation is 3.9 points and 4.5 points for the direction of motion and the speed of motion respectively. So we obtained the evaluation that it is intelligible in general. ©2006 IEEE.

This paper discusses the design and the basic characteristic of the mobile robot PMR-1 with the preliminary-announcement and display function of the forthcoming operation (the direction of motion and the speed of motion) to the people around the robot by drawing a scheduled course on a running surface using light-ray. The laser pointer is used as a light source and the light from the laser pointer is reflected in a mirror. The light-ray is projected on a running surface and a scheduled course is drawn by rotating the reflector around the pan and the tilt axes. The preliminary-announcement and display unit of the developed mobile robot can indicate the operation until 3-second-later preliminarily, so the robot moves drawing the scheduled course from the present to 3-second-later. The experiment on coordination between the preliminary-announcement and the movement has been carried out, and we confirmed the correspondence of the announced course with the robot trajectory both in the case that the movement path is given beforehand and in the case that the robot is operated with manual input from a joystick in real-time. So we have validated the coordination algorithm between the preliminary-announcement and the real movement.

人工機械システムに対する恐怖感や違和感を排除するには,外見を人間型や動物型にするだけでなく,サイズと性能・機能の関係などにおいて,人間の常識,経験知・暗黙知に合致した形態・動作が必要だと考える.具体的には,移動ロボットやヒューマノイド・ロボットにおいて,その次の行動・意図が,それを見ている周囲の人間によって予測しやすい形態・動作を考えている.本稿は,まず人間機械系の情報動作学の応用展開における研究項目と手順を説明し,つぎにこれまでに入手した資料から得られている知見のいくつかを,(1)形態からの運動予測と(2)連続動作・予備動作からの運動予測に分類し,その応用・適用方法に言及しながら論じる.

This paper explains the PMR-2R ( prototype mobile robot - 2 revised), the mobile robot with the eyeball expression as the preliminary-announcement and display of the robot's following motion. Firstly, we indicate the importance of the preliminary-announcement and display function of the mobile robot's following motion for the informational affinity between human being and a robot, with explaining the conventional methods and the related works. We show the proposed four methods which are categorized into two types: one type which indicates a state just after the moment and the other type which displays from the present to some future time continuously. Then we introduce the PMR-2R, which has the omni-directional display, the magicball, on which the eyeball expresses the robot's following direction of motion and the speed of motion at the same time. From the evaluation experiment, we confirmed the efficiency of the eyeball expression to transfer the information. We also obtained the announcement at around one or two second before the actual motion may be appropriate. And finally we compare the four types of eyeball expression: the one-eyeball type, the two-eyeball type, the will-o'-the-wisp type, and the armor- helmet type. From the evaluation experiment, we have declared the importance to make the robot's front more intelligible especially to announce the robot's direction of motion.

This paper examines the combination control of the manual operation and the autonomous motion to improve the maneuverability and safety on teleoperation of mobile robot. The autonomous motion which works to support the manual operation processing information from simple range sensors on mobile robot is examined with using a computer simulation. Three types of autonomous motion, revolution (RV), following (FL), and slowdown (SD), are proposed and the way to be equipped on the system are examined. In revolution, robot turns autonomously when robot approaches some obstacle too much. In following, robot translates parallel keeping its orientation to go along the form of the obstacle. In slowdown, robot is restricted its translation speed according to the distance to the obstacle and the translation speed of the robot. Features of each autonomous motion is declared; transit time and mileage become shorter with the revolution or the following, and the contact between robot and obstacle is almost avoided with the slowdown. When the distance to some obstacle to apply some autonomous motion is adjusted depending on the translation speed of the mobile robot, too much addition of autonomous motion against operator's intention is reduced so the maneuverability can be improved. When the revolution or the following is incorporated with the slowdown, even though the transit time is prolonged, the number of near miss for the robot to some obstacles is reduced so the safety can be improved.

This paper examines the combination control of the manual operation and the autonomous motion on tele-operation of mobile robot. The autonomous motion that is suitable for the situation when the robot passes through a passage with bends is examined to improve the maneuverability with using a computer simulation. The situation that the manually operated robot contacts with the sidewall of a passage is investigated with the experiments. It is pointed out that the contact tends to occur around the entrance and exit of bends: a robot tends to contact with the inside near the entrance of the bend and the outside around the exit of the bend. The situation that the operator uses the autonomous motion of the mobile robot under the combination control is also investigated with the experiments. The operator tends to use the autonomous following (FL) near the entrance, and the autonomous revolution (RV) is effective to make the robot return to the center of the passage around the exit. From these situation analyses the selective-revolution/following (S-R/F) is developed in which the situation is estimated based on the direction of the manual operation of the operator and the direction of the obstacle from the robot then the autonomous revolution or the autonomous following is selected and applied according to the estimated situation. The new technique is equipped with the simulation system and it is confirmed that the autonomous revolution against the operator's intention is not applied based on the situation, so the maneuverability can be improved.

This paper examines the preliminary-announcement and display function of human-friendly robot's following action and intention, especially about the direction of motion and the speed of motion for mobile robot which moves on a 2-dimentional plane. We proposed 2 types of methods: the method indicating a state just after the moment (lamp and party-blowouts) and the method displaying from the present to some future time continouously (beam-of-light and projector). Simulation system has been developed to confirm the effectiveness of the preliminary-announcement and display. Effctiveness can be evaluated by the mobile robot chasing. The mobile robot moves about at a random speed in a random direction. Subject person moves the operation robot by using joystick with looking at the preliminary-announcement on the mobile robot. So the method to display (lamp/blowout/beam) and the timing to announce (0.5-3.0 [s] before the actual motion of the robot) are evaluated numerically with the position/direction gap. The data is processed not only as the average and the standard deviation but also with the two-way ANOVA and t-screening. It was examined on a translation and a rotation separately and then on a 2-dimentional plane.<br>We have found that the method displaying from the present to some future time continuously (beam) is easy to understand. But some amount of length of the displayed path in necessary, which means an appropriate timing depending on conditions. The optimal timing for indicating a state is almost 1.0-1.5 [s] before. If the time difference is too much long, the position/direction gap become large due to the poor memory and the operational mistake. If that is too short the operation will be late owing to the reaction delay. Moreover it seems that human being tends to understand some information with a transforming object than with a changing-color object, and continuous changing is easier to understand than distributed changing.

Human-friendly robot which supports and assists human being directly beside human being is expected as it has become an aging and decrease-in-the-birthrate society. In addition to the safety function to avoid dangers for human being, we think the informational affinity function is important such as to announce surrounding people the robot's following motion before beginning to move. A person can anticipate and guess other's motion considering human body's functional features and behavioral pattern as common sense. This paper discusses the preliminary-announcement and display function of mobile robot's following motion, the direction of motion and the speed of motion, by using a omni-directional display, magicball (R). We have been developing the mobile robot PMR-2 which tells its following action with eyeball expressions.

Human-friendly robot which supports and assists human being directly beside human being is expected as it has become an aging and decrease-in-the-birthrate society. In addition to the safety function to avoid dangers for human being, we think the informational affinity function is important such as to announce surrounding people the robot's following motion before beginning to move. A person can anticipate and guess other's motion considering human body's functional features and behavioral pattern as common sense. This paper discusses the preliminary-announcement and display function of mobile robot's following motion, the direction of motion and the speed of motion, by using a omni-directional display, magicball (R). This examines the four types of eyeball expression, one-eyeball type, two-eyeball type, will-o'-the wipe type, and armor-helmet type.

This paper examines the preliminary-announcement and display function of the robot's following action and intention, especially about the direction of motion and the speed of motion for mobile robot which moves on a 2-dimentional plane. Based on the result of the fundamental examination, the validity of the preliminary-announcement and display at around 1 second before the actual motion has been verifiedon a 2-dimentional plane. Moreover we have found that human being tends to understand the change of speed of the mobile robot more with some transforming object than with some changing-color object. Comparing two types of the proposed methods, the efficiency of the preliminary-announcement and display by indicating the continuous information as drawing with beam-light more than by displaying a state as lighting lamp or using blowouts have been declared. Moreover, we have found that the scheduled path needs some amount of length for a person to understand the mobile robot's following action.

Human-friendly robot which supports and assists human being directly beside human being is expected as it has become an aging and decrease-in-the-birthrate society. In addition to the safety function to avoid dangers for human being, we think the informational affinity function is important such as to announce surrounding people the robot's following motion before beginning to move. A person can anticipate and guess other's motion considering human body's functional features and behavioral pattern as common sense. This paper discusses the preliminary announcement and display function of mobile robot's following motion, the direction of motion and the speed of motion, by using a omni-directional display, magicball (R). This examines the four types of eyeball expression, one-eyeball type, two-eyeball type, will-o'-the wipe type, and armor-helmet type.

Human-friendly robot which supports and assists human being directly beside human being is expected as it has become an aging and decrease-in-the-birthrate society. In addition to the safety function to avoid dangers for human being, we think the informational affinity function is important such as to announce surrounding people the robot's following motion before beginning to move. A person can anticipate and guess other's motion considering human body's functional features and behavioral pattern as common sense. This paper discusses the preliminary announcement and display function of mobile robot's following motion, the direction of motion and the speed of motion, by using a onmi-directional display, magicball (R). We have been developing the mobile robot PMR-2 which tells its following action with eyeball expressions.

This paper discusses the preliminary-announcement and display function of the robot's following action and intention, especially about the direction of motion and the speed of motion for mobile robot which moves on a 2-dimentional plane. We started with a software simulation about the announcement method indicating a state just after the moment, with lighting lamps and with using blowouts (elastic arrow) on translation and rotation separately. As a result, the following three remarks have been obtained: Not only the direction of motion but also the speed of motion is effective to estimate the following translation. Not the rotation speed but the rotation angle (the target direction) is efficient to understand the following revolution. Around I-second before the actual motion is the optimal timing to recognize the mobile robot's following motion both on translation and rotation. Moreover we have verified the validity of the preliminary-announcement and display also for the general movement on a plane.

This paper examines the combination control in which remote operation is combined with autonomous behaviors with the aim to realize the remote operation of mobile robot which moves in human-coexisting environment. We considers the distance and direction to an obstacle and the speed of motion of the mobile robot for revolution, following, and slowdown, which we have proposed as the autonomous action element in combination control. Fuzzy reasoning and vector components are used. From the experiment by three subject persons, almost the same result has been obtained. When the distance and direction to an obstacle and the speed of motion of the mobile robot are considered, there doesn't seem to be a great difference in following, but mileage becomes shorter in revolution and transit time is reduced in slowdown.

This paper examined the combination control in which remote operation is combined with autonomous behaviours with the aim to realize the remote operation of mobile robot which moves in human-coexisting environment. In revolution, following, and slowdown which we had proposed as combination control the autonomous action element in consideration of the distance and direction to an obstacle and the speed of motion of the mobile robot at that time has been examined Fuzzy reasoning and vector component are used to achieve this consideration. Although there was no big difference in following, mileage becomes short in revolution, and transit time was shortened in slowdown. Furthermore in order to avoid a near miss with an obstacle completely, slowdown is incorporated with revolution or following. Verification experiment in software simulation has been carried out with the same subject persons. Although it takes a little longer transit time, other special bad influence was not appeared.

This paper describes the preliminary-announcement and display of action and intention of mobile robot which moves in human-coexisting environment. First, the affinity to realize a human-friendly robot is mentioned. Next, the usefulness of remote-operated human-friendly mobile robot is shown with explaining several applications. And it is declared that the informational affinity as the function to announce and display beforehand the following action and intention of robot towards human being is necessary with explaining conventional mobile machine and remote-operated mobile robot. Then the method using "blowout" is proposed to inform both speed of motion and direction of motion simultaneously. Copyright (C) 2001 IFAC.

This paper discusses the preliminary-announcement and display method of action and intention which is carried with a robot which moves in a human-coexisting environment, It is pointed out that the speed of motion and the direction of motion are important to announce the action of a mobile robot. The method by using blowout is proposed to tell surrounding people about these two kinds of information. Method to announce the speed of motion in translation is examined using a software simulator concerning the case that a robot is equipped with blowout, lamp, or no apparatus. Consequently, the effectiveness of the preliminary-announcement and display using blowout and the importance of information about the speed of motion are confirmed. Moreover method to announce the direction of motion in rotation is examined concerning the case to announce the target direction or the rotation speed. As a result, it is found that the target direction is efficient to announce in rotation of a mobile robot.

This paper proposes three action strategies for remote operation of mobile robot in human coexistence environment as an application of network robotics towards human coexistence robot. Navigation of mobile robots from remote site only in picture information is very difficult. To realize a human coexistence type robot is a challenging subject. So we proposes (1) the combination control method with remote operation and autonomous motion with priority on the remote operation, (2) the method of extending the task-based data exchange which we have been proposed, and (3) the method of sharing information between coexisting human and mobile robot by preliminary announcement display of action intention of mobile robot.

This paper proposes the task based data exchange for teleoperation systems through communication network as an efficient transmission method of data between an operation device and a remote robot. On the task-based data exchange, the more important information according to the contents and conditions of the task which the robot performs is given priority to transmit, for example, by altering the contents of the transmitted data. We have built the experimental system in which the master arm in Tsukuba and the slave arm in Kawasaki are connected through N-ISDN and the standard techniques are utilized, such as TCP/IP, socket, JPEG, etc. A series of experimental task has been effectively carried out by the task based data exchange, that is, the crank operation which consists of grasp and revolution. The communication network with capacity limitation was used effectively and the high maneuverability in real-time with bilateral servo control has been realized. Then the effectiveness of the task based data exchange has been confirmed.

This paper proposes the task-based data exchange for teleoperation systems through a communication network as an efficient method of transmitting data between an operation device and a remote robot. Oil the task-based data exchange, the more important information according to the contents and conditions of the task which the robot performs is given transmission priority, for example, by altering the contents of the transmitted data. We have built an experimental system iii which the master arm in Tsukuba and the slave arm in Kawasaki, separated by about 100 km, are connected through N-ISDN and the standard techniques are utilized, such as TCP/IP, socket, JPEG, etc. A series of experimental tasks has been effectively carried out by the task-based data exchange, that is, crank operation which consists of grasping and revolution. The communication network with capacity limitations was used effectively, and the high maneuverability in real-time with bilateral servo control has been realized. The effectiveness of the task-based data exchange have been confirmed.

A design concept of TOMMS, TOshiba Modular Manipulator system, has been already proposed to achieve a modular manipulator system which can be assembled into any desired configuration to provide adaptability to tasks, using as few kinds and types of modules as possible, without special handling such as modification of control software. To realize the concept, we developed a constitution and configuration recognition method of the assembled manipulator using electric resistance which is simple, practical and reliable. Moreover, to actualize the system which can offer the best suitable manipulator constitution and configuration for the desired task, we developed a workability judgment method considering the degeneracy of degreeds of freedom (d. o. f.) of the manipulator and the conditions of the desired. These methods were applied to the trial system TOMMS-1 and their efficiency and practicality were confirmed.

The TOshiba Modular Manipulator System, TOMMS, consists of joint modules, link modules, and a control unit with a joystick. As to the trial manufacturing, a manipulator with 3 d. o. f. is assembled using three joint modules and optional link modules into any desired configuration and shape, for instance, a horizontal type and a vertical type. The assembled manipulator is connected to the control unit, and the position of the end tip of the manipulator is controlled using the joystick without special handling. There is only one type of joint module and link module. There are three input ports and two output ports on the joint module. The distance between the fore side and the back side of the link module is adjustable. The Jacobian matrix is applied to the control software. Control experiments were carried out and the efficiency of the design concept of TOMMS for mechanical hardware and control software was confirmed.

This paper describes a control method for manipulators which work by pressing the end-effector to the workpiece under constant force (e.g. grinding and cleaning) and also positioning the end-tip to everywhere on the workpiece using the operation device. Based on ergonomics, "the operator coordinate system" is introduced which is determined from the glance line to the workpiece and the both eyes of the operator. Further, "the corresponding-to-operational-motion type control method" is proposed, on which method the direction of motion of the operation device and the reactive direction of motion of the end-effector are corresponded in the operator coordinate system. Especially for the workpiece with wave shape, "the corresponding-to-objective-shape type control method" is designed, on which method the winding line and the valley line of the wave are recognized during the work and the directions of motion of the operation device are corresponded to these lines. These methods have been applied to the remote control system including the joystick and the lightweight manipulator, so the efficiency of these methods have been confirmed.

A design concept of TOMMS, TOshiba Modular Manipulator System, has been already proposed to achieve a modular manipulator system which can be assembled into any desired constitution and configuration to provide adaptability to tasks, using as few kinds and types of modules as possible, without special handling such as modification of control software. To realize the concept, we developed a constitution and configuration recognition method of the assembled manipulator using electric resistance which is simple, practical and reliable. Moreover, we developed a workability judgment method considering the degeneracy of degrees of freedom (d.o.f.) of the manipulator and the conditions of the desired task These methods were applied to the trial system TOMMS-1 and their efficiency and practicality were confirmed.

This paper describes the development of Windshield Cleaning Robot System : WSC. This system is intended to be used for Boeing 747, commonly called jumbo jet, parked at airports prior to service. The objects to be cleaned are spots on windshields caused by collision with dusts, insects, and birds during takeoff and landing. The intention of this new system is that one operator would perform the whole work in 10 minutes. So the system consists of the manipulator (the arm and the cleaning device), the installation unit, the control unit, and the operation unit. A position and force control method is applied to this system. The target position of the arm tip is modified using the signals from the force sensor and the joystick. In accordance with this control method, the pressure force is kept constant and the tip is moved so as to follow the shape of the windshields. The various safety features provided include interference limit to limit the area of movement. System experiments were carried out and the effectivity applying lightweight manipulator with long arms to this work was confirmed.