Although exists several drivers' gaze direction classifiers to prevent traffic accidents caused by inattentive driving, making this classification while the driver's face is temporarily or permanently occluded remains exceptionally challenging. For example, drivers using masks, sunglasses, or scarves and daily light variations are non-ideal conditions that recurrently appear in an everyday driving scenario and are frequently overlooked by the existing classifiers. This paper presents a single camera framework gaze zone classifier that operates robustly even during non-uniform lighting, non-frontal face pose, and faces undergo temporal or permanent occlusions. The usage of a normalized dense aligned face pose vector, the classification result of a pre-processed right eye area pixels, and the classification result of a pre-processed left eye area pixels is the cornerstone of the feature vector used in our model. The key of this paper is double-folded: firstly, the usage of a normalized dense alignment for a robust face, landmark, and head-pose direction detection and secondly, the processing of the right and left eye images using computer vision and deep learning techniques for refining, modifying, and finally labeling eyes information. Experiments on a challenging dataset involving non-uniform lighting, non-frontal face pose, and faces with temporal or permanent occlusions show each feature's importance towards making a robust gaze zone classifier under unconstrained driving situations.

The lack of sleep (typically <6 hours a night) or driving for a long time are the reasons of drowsiness driving and caused serious traffic accidents. With pandemic of the COVID-19, drivers are wearing masks to prevent infection from it, which makes visual-based drowsiness detection methods difficult. This paper presents an EEG-based driver drowsiness estimation method using deep learning and attention mechanism. First of all, an 8-channels EEG collection hat is used to acquire the EEG signals in the simulation scenario of drowsiness driving and normal driving. Then the EEG signals are pre-processed by using the linear filter and wavelet threshold denoising. Secondly, the neural network based on attention mechanism and deep residual network (ResNet) is trained to classify the EEG signals. Finally, an early warning module is designed to sound an alarm if the driver is judged as drowsy. The system was tested under simulated driving environment and the drowsiness detection accuracy of the test set was 93.35%. Drowsiness warning simulation also verified the effectiveness of proposed early warning module.

The skin is an important organ which enables humans to interact with the unstructured environment around. It is perfectly soft and covers the entire body providing immediate feedback even when that part is not directly in the field of vision. With the human skin as an inspiration, in this paper, we develop a novel completely soft robot skin for tactile sensing. The skin utilizes a new type of material called as Permanent Magnet Elastomer (PME) to replace the traditionally used hard permanent magnet for hall effect based tactile sensors. PME is formed by mixing Neodymium particles in a polymer base and using strong magnetization (up to 6 T) for anisotropy and to achieve strong and complete magnetization. The 6-axis soft PME is a perfect replacement for powerful hard magnets. We also do a thorough analysis of this material by infusing it in different types of silicone and as a result the most suitable combinations are selected. Performance tests show that the sensor can detect minute forces like 0.1 N. Moreover, the hysteresis test is carried out and the hysteresis error for our skin is found to be only 1.402%. An overloading test is also performed by loading the skin up to 64 N to check the robustness. In conclusion, the skin can produce reliable Triaxial force measurements and we present two models of it for smaller and large force range measurements respectively.

Nowadays, the use of electric vehicles is increasing leading to a growing demand for more efficient use of lithium-ion batteries. The state-of-charge (SOC) has been estimated in previous studies to optimize energy management of batteries. For more efficient battery utilization, detecting degradation is important. However, it is difficult for conventional methods to distinguish the effect of the model parameters including different time constants. Identifying model parameters of multiple RC parallel branches, which represent the impedance of wider frequency ranges, is a necessary requirement to detect the degradation of parts. In this study, we present a method for estimating the model parameters of multiple RC parallel branches. We designed the Markov Chain Monte Carlo algorithm by setting a search range limit and moving window, which enable estimation of the model parameters of parallel branches of different time constants. Through validation of the algorithm based on simulation, the model parameters of a third-order circuit were estimated to be within the error range of 15.2 %. In addition, impedance was calculated from the estimated model parameters in the test using a real battery dataset. The error of impedance was less than 10 % from 0.01 to 100 Hz which was sufficiently low to monitor the change of the parameters owing to degradation. As the impedance in the high-frequency band above 0.1 Hz is more likely to change because of degradation, the proposed method can be used to monitor the model parameters that change as a result of degradation.

The underground facilities require to be inspected regularly for long term and safe-use. Several mobile inspection systems are widely used, but some crucial problems with the limited inspection distance still exist due to the unstable wireless communication and cable restrictions in complex facilities. To solve these problems, we propose a wheeled robot chain control system (RCCS) for underground facilities inspection based on visible light communication (VLC) and solar panel receivers. The RCCS consists of a leader and follower robot and can extend the inspection range with special "optical signal relay" technology. VLC can provide not only illumination for inspection but also less attenuation, higher information security, and stronger antielectromagnetic interference capability, compared with conventional radio communication in complex underground facilities. Using solar panels can increase the receiving area of optical signals, environmental adaptability, and system robustness, compared with conventional photodiode (PD) receivers. Moreover, to ensure longer and more stable communication, we develop an optical signal relay mechanism along-side an attenuation control module. We performed several experiments to evaluate the signal waveform, communication quality, effective communication region, and cooperative movement. The results indicated that the proposed system could extend the inspection range with higher communication quality and environmental adaptability than VLC-based system with PD receivers.

Professional drivers are required to safely transport passengers and/or properties of customers to their destinations, so they must keep being mentally and physically healthy. Health problems will largely affect driving performance and sometimes cause loss of consciousness, which results in injury, death, and heavy compensation. Conventional systems can detect the loss of consciousness or urgently stop the vehicle to prevent accidents, but detection of symptoms of diseases and providing support before the driver loses consciousness is more reasonable. It is challenging to earlier detect symptoms with high confidence. Toward solving these problems, we propose a new method with a multi-sensor based driver monitoring system to detect cues of symptoms quickly and a verbal interaction system to confirm the internal state of the driver based on the monitoring results to reduce false positives. There is almost no data that records abnormal conditions while driving and tests with unhealthy participants are dangerous and ethically unacceptable, so we developed a system with pseudo-symptom data and did outlier detection only with normal driving data. From data collection experiments, we defined the confidence level derived from cue signs. The results of evaluation experiments showed that the proposed system worked well in pseudo headache and drowsiness detection scenarios. We found that signs of drowsiness varied with individual drivers, so the multi-sensor based driver monitoring system was proved to be effective. Moreover, we found that there were individual differences in how the cue signs appeared, so we can propose an online re-training method to make the system adapt to individual drivers.

For the safety, underground facilities are required to be inspected regularly, especially with image analysis. Traditional wireless and wired transmission techniques have a weakness of limited transmission range in narrow underground environments. In this study, a new image transmission method based on visible light communication (VLC) has been thus proposed. Two types of detectors as an image signal receiver have been tested and discussed in the following experiments. The photodiodes (PDs) are widely used as a common image signal detector in VLC technology, but image signal detection using solar panels (SPs) has not been studied. PDs have a higher sensitivity and faster response time but a limited detection area and high cost. Besides, PDs require the lens to focus light. On the other hand, SPs have much larger optical signal receiving areas and stronger optical signal capture capabilities. They can realize lens-free detection and are inexpensive. These features of PD were firstly verified in experiments with several receiving areas and angles of detectors. The experimental result revealed that PD had better image detection and recovery capabilities than those of SP. Then, we tbund that a larger receiving area obtained by using double PDs/SPs improved the brightness of the restored image. In a supplementary experiment, the influence of different RGB optical components on VLC, especially the VLC-based image transmission, has been investigated by using two-dimensional Fourier transform frequency analysis. We found that the red optical component significantly increased the intensity and energy of the restored image as the image low-frequency signals were larger than the restored image using ordinary mixed white light, and moreover, the blue optical component decreased the low-frequency part of the image. (C) 2021 Optical Society of America under the terms of the OSA Open Access Publishing Agreement

Conventional soft robotic grippers have been developed based on soft pneumatic actuators or using smart soft materials, but they need external pressure sources, are easily deteriorated or used in millimetric-scale, which leads to increase of the independency, instability, and vulnerability. In this study, we propose an EPM-MRE (Electropermanent Magnet-Magnetorheological elastomer) actuation system, which strengthens the independency and stability of soft actuation by non-contact magnetic force. We established its fundamental principle and investigated parameter design through developing a suction cup as a robotic gripper. We prototyped an EPM-MRE actuated suction cup based on magnetic-charge and tensile modeling as well as optimized the structure of EPM by introducing axisymmetric EPM with frustum pole and suction cup by introducing bi-silicone structure to uniformly activate MRE membrane. The activation of EPM by different current pulses and suction force affected by contact shapes and air gaps were tested. Evaluations showed that the suction cup could be activated with 10 ms and generate a maximum suction force of 9.2 N at a steady state with 0 J energy consumption. In conclusion, the EPM-MRE soft actuation system can be used as a robotic gripper.

Distracted driving is one of the most common causes of traffic accidents around the world. Recognizing the driver's gaze direction during a maneuver could be an essential step for avoiding the matter mentioned above. Thus, we propose a gaze zone classification system that serves as a base of supporting systems for driver's situation awareness. However, the challenge is to estimate the driver's gaze inside not ideal scenarios, specifically in this work, scenarios where may occur self-occlusions or non-aligned head and eyes direction of the driver. Firstly, towards solving miss classifications during self-occlusions scenarios, we designed a novel protocol where a 3D full facial geometry reconstruction of the driver from a single 2D image is made using the state-of-the-art method PRNet. To solve the miss classification when the driver's head and eyes direction are not aligned, eyes and head information are extracted. After this, based on a mix of different data pre-processing and deep learning methods, we achieved a robust classifier in situations where self-occlusions or non-aligned head and eyes direction of the driver occur. Our results from the experiments explicitly measure and show that the proposed method can make an accurate classification for the two before-mentioned problems. Moreover, we demonstrate that our model generalizes new drivers while being a portable and extensible system, making it easy-adaptable for various automobiles.

Transmission systems that enable speed, torque, and direction conversion with high responsiveness and backdrivability are strongly required for higher performance robotic systems. Engagement states can be changed by clutches, but traditional clutches cannot change the direction and provide sufficient backdrivability. On the other hand, direction conversion can be realized by gear box such as bevel gears, but its backdrivability is poor. Thus, we newly develop a prototype transmission system with backdrivablity and responsiveness integrating clutch and direction converter using magnetorheological fluid (MRF). MRF is a functional fluid consisted of magnetic particles and carrier fluids which can change its viscosity rapidly and continuously according to the strength of magnetic field. MRF clutch consists of driving and driven shaft connected by vanes and coils for controlling MRF. MRF direction converter consists of bevel gears, brake, and MRF for reversing the output direction. In addition to traditional functions, such as speed and torque conversion, the proposed power transmission system provides five working modes: forward direction, reverse direction, and three kinds of free, according to states of the MRF clutch, MRF direction converter, and brake. Preliminary experiments revealed that the proposed transmission system could adequately realize implemented functions.

In semi-autonomous vehicles (SAE level 3) that requires drivers to takeover (TO) the control in critical situations, a system needs to judge if the driver have enough situational awareness (SA) for manual driving. We previously developed a SA estimation system that only used driver's glance data. For deeper understanding of driver's SA, the system needs to evaluate the relevancy between driver's glance and surrounding vehicle and obstacles. In this study, we thus developed a new SA estimation model considering driving-relevant objects and investigated the relationship between parameters. We performed TO experiments in a driving simulator to observe driver's behavior in different position of surrounding vehicles and TO performance such as the smoothness of steering control. We adopted support vector machine to classify obtained dataset into safe and dangerous TO, and the result showed 83% accuracy in leave-one-out cross validation. We found that unscheduled TO led to maneuver error and glance behavior differed from individuals.

The number of medical-condition-caused car accidents (MCCCAs) in transport industry (bus, truck, and taxi) recently increases. MCCCAs including cerebrovascular and cardiovascular disease lead to loss of consciousness, thus result in injury and loss of life, and heavy compensation payment. Toward this problem, conventional systems detect closing of eyes, and fallen down state as to prevent car collisions. However, the support is taken after driver losing consciousness. To prevent MCCCAs, it is important to find out abnormal signs before driver losing consciousness. It is challenging to detect abnormal signs not only early but also with high confidence level (CL). This paper proposes a novel method that multi-modally monitors driver to detect abnormal signs which can be cues for estimating a driving-disable state in future and performs voice interaction based on the result of monitoring to clarify the internal state of the driver. Considering no data of abnormal signs, this study developed the system using normal data and pseudo abnormal data, and method of outlier detection was used for abnormal signs detection. As results of experiment, we found the relationship between cue signs and CL, and the proposed system can detect 'sleepiness' state with accuracy of 80%. Voice interaction system did not increase driver's mental demand.

In this paper, we present a suction cup by applying magnetorheological elastomers (MREs) that can be used for the wall climbing robot and picking application. MRE is a smart material in which the ferromagnetic particles are dispersed in the elastomer. In conventional research, MREs are often used as a vibration absorber, since the physical or mechanical properties can be altered upon the application of a magnetic field. In this paper, we focus on the expansion of the use of MRE material. When the magnetic field is added, the MRE will be attracted to the magnetic generator due to the ferromagnetic particles inside. The basic mathematical model between magnetic force and elasticity is discussed by us, which supports the working principle of building MRE suction cup. The MRE suction cup contains a permanent magnetic field generator unit and a MRE membrane unit. A servo motor is used to control the ON/OFF of the magnetic generator, so that the MRE membrane can be activated and deactivated. In this prototype, we focus on picking objects with flat surfaces as the first attempt. The proposed suction cup can achieve appropriate suction force, low noise, energy-saving. In this paper, the operation method of MREs, the design of the suction cup, and the evaluation of the prototype are conducted.

The maintenance of water pipelines is quite essential since the leakage, deformation, and damage can bring fatal problems to the stable supply and safe use of water resource. In our former study, the transmission approach of information (image, sensor data, etc.) from water pipe based on visible light relay communication technology has been proposed since the visible light has less signal attenuation, stronger anti-electromagnetic interference capability, and smaller bit error ratio (BER) than traditional wireless solutions in the water pipe. However, each inspection sensor in the pipe is still powered by the battery, and thus, the operation time of the sensor is severely restricted by the battery consumption. Solar panel has been introduced in the visible light communication (VLC) technology due to its large signal receiving area and higher efficiency than traditional photodiode (PD) receivers. In this study, the performance of the solar panel receiver for VLC and energy harvesting (EH) has been analyzed. Since the LED owns low irradiation energy for energy harvesting, the received power of solar panel is low. To solve these problems, a solution by using sunlight for EH and LED light for VLC was proposed. We performed several experiments to evaluate the energy harvesting based on current and voltage, and VLC quality based on waveform and BER. The results indicated that the hybrid transmission method could increase the receiving current of solar panel with less influence on the VLC quality.

One of the most important problems in teleoperation of heavy machinery is that the work efficiency of teleoperation is lower than half that of a typical boarding operation. This difference is primarily caused by operators' difficulty in creating mental representations (i.e., cognitive maps) of work sites. Operators have two opportunities to acquire information, namely before work and during work, because the introduction of teleoperation requires about one week. Therefore, we have developed a view system to be used before work to provide environmental information concerning work sites on the basis of human spatial cognition. Cognitive maps can be built by acquiring knowledge from two perspectives-the survey perspective and the route perspective. We display an external view from any viewpoint to acquire knowledge from a survey perspective and a view from an operator's viewpoint, which can be modified by the operator's intention to acquire knowledge from the route perspective. Experimental results using a simulator suggested that a proposed view system could help operators acquire cognitive maps, which may lead to a decrease in task time, the number of stops, and the moving distance and an increase in speed during grasping.

Work efficiency of unmanned construction is 40% lower than that of on-board operation because of lack of visual information. Despite previous researches to acquire external views using drones and image processing, finding out optimum and allowable camera placements on actual construction machinery for chosen scenario and machine has been difficult. This work therefore documents the derivation of optimum and allowable camera placements with skilled subjects and using an actual construction machinery for digging and releasing tasks. The experimental results indicate an optimum pan angle of 90 degrees, allowable pan angles of 60 degrees-90 degrees and allowable tilt angles of > 45 degrees during digging, and allowable pan angles of 45 degrees-135 degrees, an optimum tilt angle of 90 degrees, and al-lowable tilt angles of 60 degrees-90 degrees during releasing. These results can be referred to the camera placements for unmanned construction sites to improve work efficiency. We will derive the camera placements for other scenarios.

We previously developed a four-arm, four-flipper disaster response robot called OCTOPUS to perform complex tasks at severe disaster sites. Its 22 degrees of freedom (DOF) enable advanced mobility and workability, but requires a minimum of two operators to control it. Considering the limitations of the current automated systems and the need to reduce human resources, it is practical to control OCTOPUS through a shared control. This entails a human-automation (HA) control system that independently allocates control authority between a human and automated system so as to simultaneously cover all DOF in what we call 'separative shared control'. To establish an HA control system, control authority allocation (CAA) must be carefully designed. We first performed experiments with a human-human (HH) control system using a VR simulator. We determined four tasks and five CAAs and then selected the optimum CAA by analyzing the results. We then developed an HA control system utilizing the selected CAA that replaces either human operator with a simple automated system. We compared the performance of the HH and HA control systems in the above four tasks using the VR simulator and found that the HA achieved less workload and higher subjective usability than the HH.

Owing to manpower shortages, robots are expected to be increasingly integrated into society in the future. Moreover, robots will be required to navigate through crowded environments. Thus, we proposed a new method of autonomous movement compatible with physical contact signaling used by humans. The method of contact was investigated before using an arm with six degrees of freedom (DoF), which increases the cost of the robot. In this paper, we propose a novel method of navigating through a human crowd by using a conventional driving system for autonomous mobile robots and an involute-shaped hand with an one-DoF arm. Finally, the effectiveness of the method was confirmed experimentally.

In this study, we designed a basic framework for diagnosing a video presentation system suitable for individual operator engaging robot teleoperation. The individual suitability of video presentation systems can be only evaluated through the result of work and there are no evaluation frameworks. We first created 16 video types based on the conventional video presentation system and six diagnosis tasks, and then assigned the operators to a video type according to the work performance. The results of diagnosis revealed that the video type that maximizes work performance differed in the individuals, and the individual-suitable video type obtained by diagnosis provided better work performance, even in an unknown environment.

In order to enable autonomous mobile robots to move in human-crowded environments, active inducement including using voice and light contact can be one of effective solutions. However, to ensure safety and effectiveness, the robots must predict how the crowds will behave according to the robot approach as well as among humans forming crowds. In this study, we thus develop a navigation system that predicts multiple human movement based on inducible social force model (i-SFM), which enables the robots to select proximal paths with active inducement including human contacts, and determines the path by considering the movement efficiency of both robot and humans. Experimental results indicated that the proposed method could predict the movement of the human crowds and improve movement efficiency, compared with conventional method.

Personal mobility (PM) taking parts in society is a challenge due to two major problems as follows. The one is difficulty in estimating passability precisely and proactively. The other is difficulty in controlling PM in a narrow space. Human operators are essentially not good at dealing with the problems, so we propose a proactive operation support system based on estimating passability index related to the height and width of passages. For height, the system detects steps taller than 5 cm, visualizes them on monitor, and provides emergency stop if collision is estimated. For width, the system extracts labelled clusters of walls and calculates distance for adjacent walls, visuals passages on monitor, and decrease the speed based on the width. The experimental results indicated that the proposed system enhanced the efficiency and safer operation, compared with no-support.

For providing services where environments exist multiple humans or obstacles, robots need to move with considering multiple humans simultaneously, react quickly to the behavioral changes of pedestrians, choose acceleration/deceleration appropriately, and predict human’s behavioral changes. We thus propose a multiple moving obstacles avoiding method, called Iterative Dynamic Waypoint Navigation (IDWN). The method consists of (i) interference estimation, to predict future interference with a pedestrian, (ii) cluster estimation, to avoid pedestrians as a cluster who cannot be avoided without entering their personal space, (iii) path generator, to calculate waypoints of default speed, acceleration and deceleration paths, (iv) iterative search, to repeat the above three functions for all pedestrians in the environment, and (v) cost calculation, to select the lowest cost trajectory. For reducing the calculation cost, some heuristics, omitting candidates which theoretically increase the cost, are introduced. We implemented the method into a simulator and conducted evaluation experiment by comparing with several conventional methods. As the result of the experiment, we verified that IDWN worked well in versatile dynamic environments with multiple moving obstacles with less computation time.

Highly automated vehicles require takeover (TO) to shift the control authority to drivers in an emergency, but the driver only have about 4–10 seconds for TO. Driver’s situation awareness should be thus monitored for safe TO. In this study, we develop a situation awareness estimation method based on cognitive behavior. We made TO situations with different conditions in the driving simulator, and corrected the driver gaze data and car position, and labels (safe or dangerous). We classified obtained dataset into safe and dangerous by using a support vector machine, and used the classifier as a reference model for cognitive behavior. We found that the reference model could classify the correctness of situation awareness with high accuracy.

Teleoperation of humanoid robots for disaster response tasks, controlled by a master slave system using a head-mounted display, requires a detailed frame of view for precise operation. We previously investigated an automatic zoom method, and images zoomed when the robot arm entered the work area could improve teleoperation performance. However, automatic zooming failed to consider the operator's intentions, including zoom levels and timing, causing high mental workload and motion sickness for operators. Therefore, this study proposes a zoom method with a low mental workload and reduced level of motion sickness. Motion sickness is less likely to be induced under active conditions, wherein the operator can manually control the image. Moreover, upper limbs should not be used to operate the zoom feature because such use can disturb the master slave operation. Furthermore, a zoom method should reflect the operator's intentions, including zoom levels and timing. Thus, we develop a zoom method based on head movement, which can be regarded as typical gaze movement. The results of experiments using a real robot system indicated that the proposed method could reduce the mental workload and work time without inducing motion sickness, compared with zoom operation using a foot pedal interface.

Work efficiency in unmanned construction is about 40% less than that in boarding operation owing to a lack of visual information. Thus, previous studies have developed methods to display rich visual information by adding multiple views. However, providing excess visual information can cause 'cognitive tunneling' in operators, which makes operators subconsciously attend to a limited view and ignore others. Therefore, we developed a cognitive untunneling multi-view system that enables operators to voluntarily select appropriate views depending on situations. The experimental results using a scale model indicate that the proposed view system could decrease work time, stoppage time and help operators switch views frequently and release their attention from a cab view compared with a conventional view system. These results suggest that the proposed view system can increase work efficiency and help operators avoid cognitive tunneling. In the future, we will develop a system to identify situations and change views automatically.

Due to functional limitations in certain situations, the driver receives a request to intervene from automated vehicles operating level 3. Unscheduled intervention of control authority would lead to insufficient situational awareness, then this will make dangerous situations. The purpose of this study is thus to propose tactical-level input (TLI) method with a multimodal driver-vehicle interface (DVI) for the human-centered intervention. The proposed DVI system includes touchscreen, hand-gesture, and haptic interfaces that enable interaction between driver and vehicle, and TLI along with such DVI system can enhance situational awareness. We performed unscheduled takeover experiments using a driving simulator to evaluate the proposed intervention system. The experimental results indicate that TLI can reduce reaction time and driver workload, and moreover, most drivers preferred the use of TLI than manual takeover.

The regular inspection of underground facilities such as pipelines is absolutely essential. Pipeline leakage caused by corrosion and deformation must be detected in time, otherwise, it may cause fatal disasters for human beings. In our previous research, a robot chain system (RCS) based on visible light relay communication (VLRC) for pipe inspection has been developed. This system can basically realize the light-based transmission of control command signals and illuminance-based coordinated movement, whereas the collection and transmission approach of the pipe leakage image have not been studied. Compared with former in-pipe wireless communication techniques, VLRC can not only overcome the instability and inefficiency of in-pipe data transmission but also extend the communication range with high transmission rates. The most important feature is that it can provide a stable illumination and high-quality communication for pipe inspection robot and finally improve the energy efficiency. Hence, the aim of this article is to analyze the performance of VLRC-based image transmission in the pipe and in the future provide a high-quality, long-range, and high-efficiency image transmission for complex infrastructure inspection with RCS. The transmission systems based on two signal transmission modes analog image signal relay transmission (AISRT) and digital image frame relay transmission (DIFRT) have been proposed. Multiple experiments including the waveform test, the test of transmission features with different bit error rate (BER), and in the different mediums were conducted between these two systems. The experiment revealed that DIFRT was superior to the AISRT in terms of the relatively high-quality image transmission and reconstruction quality. It could better overcome the attenuation brought by the absorption and scattering effects and finally increased the transmission range than former communication methods. The DIFRT system could also operate at 50 kbps with relatively low BER whether in the air or water. The technique in this research could potentially provide a new strategy for implementations in the stable, effective, high-speed, and long-range image transmission of the robots in some other special environments such as tunnel, mine, and underwater, etc.

Unmanned construction which includes teleoperation of construction machinery has been introduced after disasters in terms of operators’ safety. One of the most crucial problems in unmanned construction is that work efficiency is less than half compared with usual boarding operation owing mainly to lack of visual information. Environmental views from cameras installed at disaster sites are necessary for precise manipulation even if operators can watch a wide and three-dimensional cab view. Thus, we have investigated optimum and suitable pan and tilt angle of environmental views for precise manipulation including grasping and releasing by using a 1/20 scale model. Experimental analysis using an actual construction machinery is indispensable to apply results acquired by using a scale model for unmanned construction. Therefore, in the present work, we investigate optimum and suitable range of environmental views for precise operation by using an actual construction machinery. We focus on digging and releasing because construction machineries are usually used for digging gravel or sand, and release it to designated area including bed sections of dump trucks. The task of experiments using ZAXIS 35U was determined based on the model task of unmanned construction, including digging modeled to hook a target object and releasing it in the designated area. The results indicate that the optimum pan angle for digging is 60°, and suitable range of pan angle for digging is from 45° to 90°. Furthermore, the results suggest that the optimum tilt angle is 60°, and tilt angle between 45° and 90° affects a little on work efficiency.

Human-aware navigation is an essential requirement for autonomous robots in human-coexisting environments. The goal of conventional navigation is to find a path for a robot to pass through safely and efficiently without colliding with human. Note that if such a path cannot be found, the robot stops until a path is clear. Thus, such collision-avoidance based passive navigation does not work in a congested or narrow space. To avoid this freezing problem, the robot should induce humans to make a space for passing by an adequate inducement method, such as body movement, speech, and touch, depending on the situation. A robot that deliberately clears a path with such actions may make humans uncomfortable, so the robot should also utilize inducements to avoid causing negative feelings. In this study, we propose a fundamental framework of interactive navigation with situation-adaptive multimodal inducement. For a preliminary study, we target a passing scenario in a narrow corridor where two humans are standing and adopt a model-based approach focusing on common parameters. The suitable inducement basically varies depending on the largest space through which a robot can pass, distance between the robot and a human, and human behavior such as conversing. We thus develop a situation-adaptive inducement selector on the basis of the relationship between human-robot proximity and allowable inducement strength, considering robot efficiency and human psychology. The proposed interactive navigation system was tested across some contextual scenarios and compared with a fundamental path planner. The experimental results indicated that the proposed system solved freezing problems, provided a safe and efficient trajectory, and improved humans' psychological reaction although the evidence was limited to robot planner and hardware design we used as well as certain scenes, contexts, and participants.

Magnetorheological Fluid (MRF) is a functional fluid in which apparent viscosity changes quickly, continuously and reversibly according to the strength of the magnetic field applied. On the basis of the peculiar features of MRFs various studies have been conducted mainly on functional fluids in the power fluid field. MRF consists of magnetic substances of the size 1-10 mu m which are dispersed in lubrication oil having low viscosity and specific gravity. There is no MRF which can be used in robots for precise control because it cannot guarantee the reliability. We here aim to acquire advanced robot mechanism by fusing material-mechanical technology and crossing hypothesis-experiment cycle. As a result, we found that anti-sedimentation of MRF can be regarded as the most important features for implementation in robotic mechanisms. In this paper, the anti-sedimentation MRF is manufactured and tested in a MRF damper. We first obtained a common method to manufacture the anti-sedimentation MR fluid, by designing the medium, the magnetic substance, and the additive. In the sedimentation test, the proposed MR fluid had the minimum separation compared to the commercial product. Secondly, in experiment using the MR damper, the anti-sedimentation MRF had superior stability compared to the commercial product.

Highly-automated vehicles operating in level 3 issue a takeover request (TOR) to transfer the control authority from the autonomated driving (AD) system to a human driver when they encounter system limitations. In such 'unscheduled' situations, the driver is required to immediately re-engage in the driving task both physically and cognitively, and perform suitable action, e.g. lane change. Thus, evaluating driver engagement by the AD system would lead to safe takeover. Physical engagement is easily estimated but there are few studies on evaluating cognitive engagement. In this study, we thus developed a driver situational awareness estimation system based on glance information. We first defined seven standard glance areas and driver glance classification model using a convolutional neural network. We then obtained a large amount of glance data when both safe and dangerous takeover situations (lane change) by using a driving simulator, and we derived the standard glance model including the glance area and time, in order to estimate whether driver gained enough cognitive re-engagement in real-time. To evaluate the effectiveness of the proposed model, we created a situational awareness assist system to visually indicate regions with insufficient glance. As a result, we found that the assist system drastically improved driving performance and reduced the number of accidents during takeover.

The gas pipeline requires regular inspection since the leakage brings damage to the stable gas supply. Compared to current detection methods such as destructive inspection, using pipeline robots has advantages including low cost and high efficiency. However, they have a limited inspection range in the complex pipe owing to restrictions by the cable friction or wireless signal attenuation. In our former study, to extend the inspection range, we proposed a robot chain system based on wireless relay communication (WRC). However, some drawbacks still remain such as imprecision of evaluation based on received signal strength indication (RSSI), large data error ratio, and loss of signals. In this article, we thus propose a new approach based on visible light relay communication (VLRC) and illuminance assessment. This method enables robots to communicate by the light signal relay', which has advantages in good communication quality, less attenuation, and high precision in the pipe. To ensure the stability of VLRC, the illuminance-based evaluation method is adopted due to higher stability than the wireless-based approach. As a preliminary evaluation, several tests about signal waveform, communication quality, and coordinated movement were conducted. The results indicate that the proposed system can extend the inspection range with less data error ratio and more stable communication.

Detailed views are necessary for precise manipulation in teleoperation of humanoid robots controlled by the master slave system with head mounted display. Thus, we have proposed an autonomous zoom method based on the positions of the hands, which could increase work efficiency. However, it caused motion sickness and high cognitive load owing to little reflection of operators’ intention. In this study, we thus propose a low cognitive-load zooming interface with little motion sickness. Motion sickness can be reduced by active conditions with manual control rather than passive conditions. Moreover, the interface should let operators reflect their intention to realize a low cognitive load zoom. Furthermore, the interface should be without upper limbs because operators control robots by their upper limbs. Thus, we developed the zoom interface with head movements which are typical movements during gazing. The experimental results suggest that the proposed interface can reduce sickness and cognitive load.

To make full use of the advantages of automatic driving while retaining the enjoyment and freedom of the manual driving, we have proposed a tactical-level input (TLI) that allows the driver to input dynamic driving tasks such as parking and turning. In the future state input using touch interface, eye movement between the interface and front is troublesome and it takes time to operate the screen. In the command input using haptic interface, it cannot reserve input to a distant place and the amount of information is small. To solve these problems, we develop a visual-haptic interface by integrating them, which has a planar-movable haptic device and an input/output state indicator using transparent screen. The results of experiments using a driving simulator indicate that the proposed interface solve the above disadvantages while remaining the advantages.

Autonomous mobile robots are expected to work in a human-robot coexistence environment, and they require to have a proximal navigation framework considering approach and contact with humans. Such a framework requires a function to select a path based on estimating “the risk of approach” and comparing it with “the benefit of efficient movement by approach”. To move even in the proximal state, we develop a proximal navigation planning considering the distance between the human and robot (time to interference). The control framework estimates risks and benefits, and adjusts the risks as necessary, while performing proximal movement. The results of the experiment showed that the proposed navigation system could estimate the risk and benefit of approach, adjust it through inducement, and select a path that is natural and safe for the robot and human.

In this study, we designed a framework to evaluate quantitatively the operation status of human-operated work machines represented by work efficiency. The framework requires to reveal the sensor data affecting the operating conditions, the system consists of (a) formulating between explanatory variables (sensor data) and the purpose variable (operational status) and (b) extracting explanatory variables contributing to the objective variable. As a result of the experiments, the proposed system could grasp and evaluate the operating situation in a pluralistic and quantitative manner.

Work efficiency of teleoperation of heavy machinery is less than a half of boarding operation because of difficulties in creating mental representation of work sites. Thus, we proposed a pre-offering view system for teleoperators to remember environmental information, which helps operators plan paths and working strategies, but some operators forgot their planning. Problems of previous researches including the navigation field are not considering changes of work states including moving and reaching and cognitive load. Therefore, in this paper, we develop a planning reminder system with low cognitive load which can adopt changes of work states. We display different AR depending on work states. Furthermore, we display AR only for current and next work states while operators are in low cognitive load to avoid increasing cognitive load to watch AR. The experimental results using a scale model indicates that the proposed system can decrease work time and cognitive load.

In this study, we propose a method of pipeline robot’s navigation based on fuzzy logic control (FLC) for passing through elbows or T-junctions. A CCD camera is used for locating the region of interest (ROI) in elbow or T-junction. Moreover, ROIs formed by reflection of LED light and the edge of pipe’s dark hole are considered as input variables in the FLC system. By analyzing outputs of the system, we can control the robot’s speed and yaw angle in real time. Compared with conventional studies on pipeline robot’s navigation method, the proposed method can be more precise and faster by using FLC algorithm and analyzing ROI with fewer sensors. Finally, we performed a simulation validation, and the results showed that the robot could adapt to known pipe environments and realize navigation in straight part, elbow, and junction of pipe.

When highly-automated vehicles operating in level 3 encounter system limitations, they issue a takeover request (TOR) to transfer the control authority from the automated driving (AD) system to a human driver. In such ‘unscheduled’ situations, the driver needs immediately re-engage in the driving task both physically and cognitively. In this study, we developed a driver situational awareness estimation system based on glance information. We collected a lot of glance data when both safe and dangerous takeover situations by using a driving simulator, and we derived the standard glance model including the glance area and time. To evaluate the effectiveness of the model, we created a glance guidance system to visually indicate regions with insufficient glance. We found that the guidance system improved driving performance and reduced the number of accidents.

In this study, we develop an environment adaptability enhancement system with active information search in real space and high efficiency motion search in virtual space for multi-degree of freedom disaster response robot. For such robots to work safely, estimating the work achievability before executing it is important. However, in disaster sites, failed motions such as falls increase the risk of secondary disasters, so it is difficult to learn motions by trial and error. Thus, the robot obtains the environment information in real space and reproduces them in virtual space, and then performs motion search efficiently in virtual space. Due to the limitation of the reproduction accuracy, for finer adjustment, the robot performs active touches to acquire ground properties such as friction and hardness in real space. The experimental results showed that the proposed system could judge work availability by trying robot motions and provide the robot with efficient motions.

One of the most crucial problems associated with unmanned construction is low work efficiency due mainly to lack of visual information. External side views are necessary for efficient manipulation, but the optimum and allowable pan and tilt angles of side views have not been investigated. We thus conducted experiments using a scale model. However, experiments using actual construction machinery are necessary to confirm that these experimental results using a scale model can be applied to actual machines. Thus, the purpose of the present study is to investigate the optimum and allowable ranges of pan and tilt angles by using actual construction machinery. We focus on digging and releasing because these manipulation tasks require external views considering that precise operation is necessary. We conduct experiments based on a model task, including digging, which involves hook a target object, and releasing it in a designated area. The results suggest that the optimum pan angle for digging is 60 degrees, and the allowable range is 45 degrees-90 degrees. Moreover, the results indicate that the optimum tilt angle is 60 degrees, and the allowable range is 45 degrees-90 degrees.

Herein, unmanned construction technology that allows remote control of construction machinery has been introduced for post-disaster recovery efforts. One of the most significant problems in unmanned construction is that work efficiency is less than half of that in manned operations owing to the lack of visual information. Thus, deriving optimum and allowable external sideway views can lead to higher efficiencies. In this study, we investigate optimum and allowable pan and tilt angles of external views for grasping and placing tasks as these tasks are required in disaster sites. Humans primarily recognize objects from canonical views with the least amount of occlusions, and with an allowable rotation of 30 degrees. Thus, we hypothesize that external views with pan and tilt angles of 90 degrees could be canonical views for grasping and placing; the allowable range of both pan and tilt angles was +/- 30 degrees. We conducted experiments using a 1/20 scale model to derive an optimum and allowable range of pan and tilt angles. The experimental results suggest that the optimum pan angle is 90 degrees, and the allowable range for grasping is +/- 30 degrees; the range for placing was from 30 degrees to 135 degrees. In addition, the results indicate that the optimum tilt angle is 60 degrees, and the allowable range is +/- 30 degrees.

Pipeline inspection robots are recently expected to be deployed to pipelines with longer distance and more complex configuration. These robots require high performance in wireless communication among robots and outside. The radio frequency or infrared radiation frequency can be one of the promising wireless communication method, but they would have drawbacks in pipeline environments, such as electromagnetic interference and low energy efficiency. In this study, we focus on visible light communication (VLC), which has a high transmission rate, strong anti-interference capability, and large bandwidth, compared with the above methods. VLC also has an interesting feature that can simultaneously realize data communication and illuminating dark pipeline environment for pipe robot. As a preliminary study of VLC application, we developed a control system based on VLC for a wheeled robot, which includes a VLC transmitter and receiver. The transmitter and receiver can encode and decode the light signals. Finally, we tested the signal waveform of transmitter and receiver, and evaluate the performance of this system in real pipeline robot. The experimental results revealed that the proposed VLC system could achieve a reliable signal transmission and provide well control and illumination for the robot in the pipe.

How to improve task performance and how to control a robot in extreme environments when just a few sensors can be used to obtain environmental information are two of the problems for disaster response robots (DRRs). Compared with conventional DRRs, multi-arm multi-flipper crawler type robot (MAMFR) have high mobility and task-execution capabilities. Because, crawler robots and quadruped robots have complementary advantages in locomotion, therefore we have the vision to combine both of these advantages in MAMFR. Usually, MAMFR (like four-arm four-flipper robot OCTOPUS) was designed for working in extreme environments such as that with heavy smoke and fog. Therefore it is a quite necessary requirement that DRR should have the ability to work in the situation even if vision and laser sensors are not available. To maximize terrains adaption ability, self-balancing capability, and obstacle getting over capability in unstructured disaster site, as well as reduce the difficulty of robot control, we proposed a semi-autonomous control system to realize this compound locomotion method for MAMFRs. In this control strategy, robot can explore the terrain and obtain basic information about the surrounding by its structure and internal sensors, such as encoder and inertial measurement unit. Except that control system also can recognize the relative positional relationship between robot and surrounding environment through its arms and crawlers state when robot moving. Because the control rules is simple but effective, and each part can adjust its own state automatically according to robot state and explored terrain, MRMFRs have better terrain adaptability and stability. Experimental results with a virtual reality simulator indicated that the designed control system significantly improved stability and mobility of robot in tasks, it also indicated that robot can adapt complex terrain when controlled by designed control system.

Gas pipeline requires to be inspected regularly for leakages caused by natural disaster. Robots are widely used for pipeline inspection since they are more convenient than manual inspection. Several problems, however, exist due to the restriction by complex pipe networks. The most significant one is limited inspection range caused by restriction of cable length or wireless signal attenuation. In this paper, we proposed a concept of wireless relay communication to assist robot to extend the inspection range, and we newly developed a tracked robot chain system. In this system, each robot serves as a relay communication node. Leakage information of pipes are transmitted via these relay nodes. To ensure the stability of relay communication between adjacent robots, we adopted RSSI (received signal strength indication)-based evaluation method for cooperative and coordinated movement of robot chain system. Moreover, wireless application layer communication protocol (WALCP) was used to increase the stable performance of wireless relay communication. Each robot can self-navigate based on distance measurement module, which enables robots to pass through an elbow junction. Multiple experiments to evaluate relay transmission efficiency, RSSI-based cooperative movement, and comprehensive performance were conducted. Results revealed that our proposed system could realize relatively accurate relay transmission and RSSI-based coordinated movement.

Some electric-powered wheelchairs are recently redefined as personal mobility devices. Their users are not only elderly or handicapped people, but also passengers with large baggage or pedestrians going from station to destination, i.e., last-mile transport. Consequently, people with different operation skills and expectations on personal mobility would become new users of this kind of devices. Safe and comfort travel in human co-existing environment such as sidewalks and airports is a social expectation for personal mobility. In order to realize this, understanding the operation skill of each user by a practical and simple method is essential. This paper thus introduced a skill level classification method by machine learning using only joystick data as input. In order to determine the number of skill level clusters, basic 26 features of joystick operation data are used for unsupervised clustering (single-linkage). We then made evaluation indexes by using speed, speed control, and direction control. For a five-level classification by using gradient boosting as supervised learning, we achieved a 67% accuracy (tolerance: 0) and a 98% accuracy (tolerance: 1). Further analysis of the feature importance of gradient boosting revealed key features to a good operation. Results also show that skill level differed among people with different driving experiences.

Efficient navigation in a congested environment is immensely difficult for the current state-of-the-art mobile robots owing to the challenges related to both sensing and actuation. Consequently, conventional approaches in mobile robotics research prioritize safety by slowing down or stopping robot's movement under challenging circumstances. This paper considers an alternative approach in which the robot utilizes forearm contact to create space for itself and to act as a safety buffer, during very close or congested navigation interactions. First, two categories of contact methods are defined based on different navigation scenarios. These contact methods are analyzed through different contact positions and force directions in a set of comparative experiments. The results of the comparative experiments are then incorporated into a contact-based framework that outputs the necessity, and the appropriate form of forearm contact. Finally, a set of evaluation experiments are performed to test the usability and effectiveness of the constructed framework. The results indicate that the proposed framework effectively outputs appropriate contact according to the situation, and aids the robot while navigating through space-constrained scenarios.

Smooth and efficient robot navigation among humans is a crucial requirement for successful integration of robots in human society. Towards this end, an indispensable characteristic of robot action is legibility while communicating its intention. However, unlike humans, present robots cannot convey its intention through human-like non-verbal communication. This paper explores the use of projection indicators for communicating directional intent of a robot across three different 'crossing scenarios' as a means of overcoming the shortcomings of the robot's non-verbal communication abilities. The results of the study show statistically significant improvement in perceived feelings of the measured attributes when using the auxiliary communication method. The studied method also improves cooperation from the participants. Nevertheless, the improvement in perceived feeling does not necessarily replicate in terms of smoothness across all the scenarios.

Automated vehicles operating in level 3 may request the human driver to intervene in certain situations due to system limitations. Unscheduled transferring of control to manual driving will create safety issues as consequences of inadequate situational awareness and sudden increase of driver workload. In this study, we propose and evaluate tactical-level input (TLI) method with a multimodal human-machine interface (HMI) for driver intervention in short-term system limitations. The HMI system consists of touchscreen, gesture, and haptic interfaces enabling bilateral driver-vehicle interaction. TLI along with the HMI capable of multimodal feedback can provide situation-adaptive spatial information which enhance the driver situational awareness in a short time. To evaluate the proposed system we conducted driving experiments involving unscheduled takeover situations in urban environment using a driving simulator. We analyzed driver reaction times, physiological responses including heart rate, skin conductance and subjective workload as well as qualitative feedback comparing with manual takeover. The results show that TLI can reduce driver workload, reaction times, and improve driver behavior. Moreover, 90% of drivers preferred to use TLI method over manual takeover.

Pipeline robot are widely used since pipelines require to be inspected regularly for leakages caused by natural disaster, etc. Most robots which rely heavily on manual operation are incapable of self-navigation in pipe. Moreover incorrect operations would degrade the efficiency, and sometimes damage the robots especially when they pass through elbows or junctions. Some robots can realize navigation based on multi-sensor such as position sensitive detector and laser sensor, but navigation performance for such robots will be greatly influenced by the performance of these sensors, and space to install large number of sensors is limited. In this study, we propose an approach of pipeline robot's navigation based on fuzzy logic control (FLC) algorithm for passing through elbows or T-junctions. A CCD camera installed on the robot is used for locating region of interest (ROI) in elbow or junction. Moreover, ROIs formed by reflection of robot's LED light and edge of pipe's dark hole are considered as input variables in the FLC system. By analyzing system outputs, we can control the robot's speed and yaw angle in real time. Compared with conventional studies on pipeline robot's navigation method, the proposed method can be more precise and faster by using FLC algorithm and analyzing ROI with fewer sensors. Finally, we conducted a simulation validation, and the results showed that the robot was capable of adapting to known pipe environments and realizing navigation in straight part, elbow, and junction of pipe.

In this paper, a mechanism which can adjust inertia significantly is presented. Such mechanism contains a flywheel, a group of clutches, a direction converter, and a magnetorheological hydraulic rotary actuator. According to the physical definition, inertia is the resistance of object changing in its state of motion, which always connects to the mass of the object. Inertia is also known as an inherent property of an object. Conventionally, among impedance factors of the mechanical system, inertia seems to be unchangeable, not like damping factor or elasticity. However, changing inertia might bring different behavior to mechanical networks. In this study, instead of changing the mass of the object, by controlling the flywheel energy collected from the motor pump to release to the object, the resistance of actuator motion can be adjusted. We call this the pseudo-inertia adjustable mechanism (pIAM). The clutches are used for controlling the engaging and disengaging of the flywheel, energy releasing direction and magnitude. The mechanism design, mathematical model, and experimental results are included in this paper. According to the results, the machine is able to release the maximum torque about 7 Nm and the maximum speed about 800 degrees/s with 1000 rpm of the flywheel speed, which reveals the feasibility of this mechanism.

Autonomous driving systems (ADSs) could reduce the degrees of human control, and which would result in lack of flexibility of controlling the vehicle. In this study, we proposed a method to allow the driver to control its lateral and longitudinal motions with a time lag, in order to increase the flexibility. We first derived a set of vehicle movements, such as lane changing, overtaking, merging, turning, speed changing, and parking, and related them to a set of control functions that the driver can input. We then developed a joystick-type interfaces with kinesthetic (force) and tactile (vibration) feedback for vehicle control, called haptic force feedback interface (HIF). This interface enables bidirectional interaction between the driver and the ADS, which allows the system to deny inputs from the driver due to collision danger as well as to inform environmental and vehicle states. We performed comparison experiments with a driving simulator among HIF and a previously developed hand-gesture interface (GIF), in addition to manual driving with steering wheel and pedal and autonomous driving with touchscreen interface to designate the destination. The results of experiments showed that the proposed HIF significantly reduced the average input time and input error compared with GIF, and drivers preferred the HIF due to its ability to provide immediate, active, and passive feedback, compared with manual and autonomous driving.

Human sensing enables intelligent vehicles to provide driver-adaptive support by classifying perceived workload into multiple levels. Objective of this study is to classify driver workload associated with traffic complexity into five levels. We conducted driving experiments in systematically varied traffic complexity levels in a simulator. We recorded driver physiological signals including electrocardiography, electrodermal activity, and electroencephalography. In addition, we integrated driver performance and subjective workload measures. Deep learning based models outperform statistical machine learning methods when dealing with dynamic time-series data with variable sequence lengths. We show that our long short-term memory based recurrent neural network model can classify driver perceived-workload into five classes with an accuracy of 74.5%. Since perceived workload differ between individual drivers for the same traffic situation, our results further highlight the significance of including driver characteristics such as driving style and workload sensitivity to achieve higher classification accuracy.

Currently, robots are used for pipeline inspection since it is more convenient than manual inspection. In this paper, we proposed a concept of wireless relay communication to assist robot to extend the inspection range. Specifically, we developed a tracked robot-chain system. Leakage information of pipes are transmitted through multiple robots. To ensure the stability of relay communication between adjacent robots, we adopt RSSI-based evaluation method. We conducted some experiments to evaluate relay transmission efficiency, RSSI-based cooperative movement, and comprehensive performance. The results revealed that our proposed robot chain system could realize relatively accurate data relay transmission and basically RSSI-based coordinated movement.

Disaster response robot OCTOPUS with four arms and four flippers has been developed in order to adapt to complex disaster environments. In this study, we propose a new locomotion method by cooperating the arms and flippers for multi-limb robots. The proposed locomotion would be effective to traverse unstructured terrain more stably and efficiently by combining the advantages of crawling by crawlers and walking by arms. Electrically-driven OCTOPUS was used to verify the proposed locomotion method. Results of experiments on climbing a step indicated that the proposed method enabled the robot to climb the step, and confirmed that it would be effective to increase terrain adaptability of robot in unstructured environment.

In robot teleoperation using a master-slave system, we have proposed a scale-gain adjustment system in order to increase the time efficiency and endurance capacity. Here, the presentation of images to an operator is also important for accurately recognizing environmental information. As a preliminary study of evaluating image scaling, we investigated the effectiveness of image scaling methods. We developed several image scaling systems by combining scaling factor, scaling method (manual/automatic), and changing method (continuous/discontinuous), considering proprioception, and visual momentum. Experimental results revealed that a system with manual and continuous change was better for the number of successes than others.

In extreme disaster sites, external sensors such as laser range finders do not work properly, and such sensors may be broken in accident. Risk-tolerance robots should continue to perform tasks, even if the external sensors cannot work. In this preliminary study, for proximate environment understanding without using external sensors, we developed a groping method, in which the robot actively touches the environment using the end-point of the arm, and then reconstructs a three-dimensional local map around the robot based on the recorded contact information. The results of the experiments conducted by using four-arm four-flipper robot OCTOPUS indicated that the proposed groping actions could recognize step and pit, and calculate the position and size of object, and confirm that the robot successfully removed the object.

The purpose of this research is to make robot possible to use contacts in a practical environment. Therefore, considering the psychology and movement of the person by contact, we propose a movement control system with light contact for autonomous mobile robots, which can make the most appropriate contact to the situation even in a dynamic environment. To realize this contact system, experiments are carried out in which the robot actually touches human with various methods under various circumstances. On the basis of the experimental results, we embodied the movement control method with light contact, and evaluated. The results indicated that the proposed system could select contacts according to the situation.

This study proposes a scale-gain adjustment method for same structure type master-slave systems (SSMS), considering complexity, continuity, and time-limitation in disaster response work. SSMS systems have advantages in which master's movement itself can connect directly to slave's movement. However, human operators sometimes make the performance of the slave system degrade due to their human factors such as the limitation of control accuracy and physical fatigue. Therefore, we develop a method to changing a scale and gain of the relationship between master and slave movements. Experimental results indicated that our proposed system improved work performance and decreased fatigue.

In this study, we develop a torus interface to enable a driver to provide multimodal bidirectional interaction with driving automation system, as a driver-vehicle interface with higher implementability for tactical-level inputs. The driver can input tactical driving tasks (e.g., turn right) by using six-axis moving input (e.g., push and tilt) with kinesthetic feedback. The interface system offers bidirectional and multimodal interaction, i.e., kinesthetic, visual, and haptic, in order to improve a driving experience. We conducted experiments using a simulator to evaluate the multimodal interaction method compared with unimodal methods. The results showed that the multimodal method enhanced the responsiveness and accuracy of the input to system and made the driver to intuitively and quickly understand the proposal from the system.

Four-arm four-flipper disaster response robot called OCTOPUS, which has 26 degrees of freedom (DOF), has been developed to engage in complex disaster response work. OCTOPUS adopts a multi-operator single-robot (MOSR) control system, and two operators manually control the robot. The pattern of control-authority allocation (CAA) for two operators largely affects performance of MOSR systems, so this study conducts usability tests in various CAAs, and derives a reasonable CAA pattern. So far, there are no uniform standard allocation rules for flippers and crawlers in multi-limb robots like OCTOPUS, therefore we specify five CAA patterns for investigation. Three fundamental tasks that whole body of the robot must be cooperatively controlled were conducted to test the usability in each CAA pattern. From the results of analysis, we found that pattern 1, which one operator controls front two arms, flippers, and crawlers, and another operator controls remaining back parts, had best scores for all tasks, as well as pattern 2 had the best distribution of workload between two operators.

Traffic complexity is one of the factors affecting driver workload. In order to study the relationship between traffic complexity levels and workload, a designed experiment is required, especially to vary traffic flow parameters systematically in a simulated environment. This paper describes the experimental design of a simulator study for developing a computational model to estimate the behavior of driver workload based on traffic complexity. Driving simulators allow creating and testing different traffic scenarios and manipulating independent variables to improve the quality of data, as compared to real world experiments. Physiological responses such as heart rate, skin conductance, and pupil size have been found to be related to workload. By adapting a data-driven method, we integrated electrocardiography sensors, electro-dermal activity sensors, and eye-tracker to acquire driver physiological signals and gaze information. Preliminary results show a positive correlation between traffic complexity levels and corresponding physiological responses, performance, and subjective measures.

The gas pipeline networks cover approximately 200,000, km in urban region in Japan. The risk caused by severe leakage, cracking, or corrosion will lead to a big accidence for human. Therefore, the gas pipeline robots for regular inspection and maintenance are very essential for the safe operation and supply of gas. In this paper, as a fundamental study of pipeline inspection robots, a new small wireless tracked robot for gas pipeline inspection and maintenance has been developed. Due to robot’s special kinematics, it is capable of moving in straight pipe, turn at elbow and also climb the inclined pipe flexibly and efficiently. Compared with previous robots, it owns two working modes: self-navigation (automated control) and manual control mode for pipeline inspection. Moreover, a preliminary experiment has been implemented with a typical M-pipeline in 10, cm diameter/4, m length. The experiment results revealed that self-navigation could improve the efficiency of inspection.

Disaster response crawler robot OCTOPUS has four arms and four flippers for better adaptability to disaster environments. To further improve the robot mobility and terrain adaptability in unstructured terrain, we propose a new locomotion control method called compound motion pattern (CMP) for multi-limb robots like OCTOPUS. This hybrid locomotion by cooperating the arms and flippers would be effective to adapt to the unstructured terrain due to combining the advantages of crawling and walking. As a preliminary study on CMP, we proposed a fundamental and conceptual CMP while clarifying problems in constructing CMP, and developed a semi-autonomous control system for realizing the CMP. Electrically-driven OCTOPUS was used to verify the reliability and correctness of CMP. Results of experiments on climbing a step indicate that the proposed control system could obtain relatively accurate terrain information and the CMP enabled the robot to climb the step. We thus confirmed that the proposed CMP would be effective to increase terrain adaptability of robot in unstructured environment, and it would be a useful locomotion method for advanced disaster response robots.

Due to the huge microwave attenuation in small steel gas pipelines, the wireless in-pipe robots are unable to complete the inspection and maintenance in long distance. In this research, an approach of a reliable communication and localization method based on Received Signal Strength Indication (RSSI) theory and ultrasonic measurement for gas pipeline robot chain is proposed. Without any GPS modules in the pipe, the robots are able to measure and calculate a certain distance from the adjacent one by detecting and estimating the received wireless signal strength. Such approach enables the all members among the robot chain to cooperate and coordinate, and finally complete the pipeline inspection and maintenance. In addition, this paper describes a "leader-follower" control approach in pipe to support the RSSI-based communication. The leader robot is responsible for the task of whole pipe inspection with wireless remote control by operator, and the follower robots act as the wireless "signal relay station" for communication. Finally, such pipe robot chain cooperate and coordinate together, so that they are able to complete the inspection successfully. We evaluate the performance of the proposed method by using virtual reality simulation. The simulation result indicated that the proposed method could realize the reliable wireless communication and localization for the pipeline robots.

This paper presents a new manipulation mode for multi-arm and multi-flipper robots to improve the manipulation ability of them. Multi-arm and multi-flipper crawler robots designed for disaster response or other uses are naturally divided into two function parts from the design stage. There are the crawlers and the flippers for robot locomotion tasks, and the arms for manipulation tasks. In this design concept, the flippers and arms are hardly to systematically and actively cooperate with each other, and these control systems cannot fully play the structural features of this kind of robot. To solve this problem, we proposed a compound manipulation mode (CMM) for multi -arm multi-flipper robot. In this control mode, the flippers and arms are controlled as a whole in manipulation tasks, the flippers will support the arms to own bigger manipulation space, to have better manipulation posture and to optimize the position of robot center of gravity (COG) to make sure robot in good stable state. Four arm and four flipper robot OCTOPUS was used as the test platform, and the verification experiments were carried out. The results indicated that CMM control mode can achieve all the proposals, therefore the concept of CMM control mode is successful and useful.

In teleoperation of heavy machines, work efficiency will be 50% lower than manned operation because operators cannot obtain effective information about work sites due to the limitation of current monitoring systems. Operators would have opportunities to obtain such information before work (about a week required to introduce teleoperation systems) and during work. As a fundamental study to support operator's spatial cognition, we developed views to provide spatial information of work sites before work. Humans have cognitive maps which are created based on knowledge acquired from survey and route perspectives. To make operators acquire the above two knowledge, we provide a bird's-eye view that can be changed by operators to acquire a knowledge from survey perspective, and a view from operator's viewpoint that can be changed by operator's intention to acquire a knowledge from route perspective. To evaluate two pre-offering views, we preformed experiments using a virtual reality simulator. The results indicated that a view to acquire a knowledge from survey perspective could help operators plan totally and one to acquire a knowledge from route perspective could help operators plan locally, and could increase work efficiency.

This paper proposes a basic near-environmental recognition framework based on groping for risk-tolerance disaster response robot (DRR). In extreme disaster sites, including high radiation and heavy smog, external sensors such as cameras and laser range finders do not work properly, and such sensors may be broken in accidents in the tasks. It is hoped that DRRs can continue to perform tasks, even if the external sensors cannot work, and at least, they can safely evacuate from the site. In this preliminary study, for recognizing near environments without using external sensors, we proposed a groping method. In this method, a robot actively touches the environment using arms or other movable parts, records the contact information, and then reconstructs a three-dimensional local map around the robot by the detected information, e.g., robot arm's position and reactive force. The proposed groping system can recognize the existence of three situations, such as an object, step, and pit, and those geometry, by exploring the designated space using arms. The groping strategy was designed considering both robot specification, time limitation, and required resolution. Experiments were performed using four-arm and four-crawler robot OCTOPUS. The results indicate that the proposed framework could recognize step, pit, and object, and calculate the position and size of the object, and confirm that the robot successfully removed the object on the basis of groped data.

In a previous study, we have developed a four-arm four-crawler disaster response robot called 'OCTOPUS'. Advanced disaster response robots are expected to be capable of both mobility, such as entering narrow spaces over unstructured ground, and workability, such as preforming complex debris-removal work. We have confirmed experimentally that the four arms could make the robot perform complex tasks while ensuring stabilization when climbing steps while the four flippers could make it traverse rough terrain while avoiding toppling over when conducting manipulation task. OCTOPUS, renamed as H-OCTOPUS, is oil-hydraulically driven to perform outdoor demolition of heavy debris, and is teleoperated by two operators. In this study, we develop a prototype electrically-driven OCTOPUS, called E-OCTOPUS, to manipulate various light-objects mainly indoors such as valve operations in nuclear power plants. For reducing the size and weight while maximizing task performance, we introduced a mutual complementary strategy between its arms and flippers. To validate the capability of E-OCTOPUS, we performed preliminary experiments involving climbing high steps and manipulating and cutting wires by cooperating the four arms and four flippers. The results indicated that E-OCTOPUS could complete the tasks by coordinating its four arms and four flippers.

Intelligent vehicles operating in different levels of automation require the driver to fully or partially conduct the dynamic driving task (DDT) and to conduct fallback performance of the DDT, during a trip. Such vehicles create the need for novel human-machine interfaces (HMIs) designed to conduct high-level vehicle control tasks. Multimodal interfaces (MMIs) have advantages such as improved recognition, faster interaction, and situation-Adaptability, over unimodal interfaces. In this study, we developed and evaluated a MMI system with three input modalities
touchscreen, hand-gesture, and haptic to input tactical-level control commands (e.g. lane-changing, overtaking, and parking). We conducted driving experiments in a driving simulator to evaluate the effectiveness of the MMI system. The results show that multimodal HMI significantly reduced the driver workload, improved the efficiency of interaction, and minimized input errors compared with unimodal interfaces. Moreover, we discovered relationships between input types and modalities: location-based inputs-Touchscreen interface, time-critical inputs-haptic interface. The results proved the functional advantages and effectiveness of multimodal interface system over its unimodal components for conducting tactical-level driving tasks.

This paper proposes a practical scheme for measuring the mass of an object grasped by the end-effector of a large-scale hydraulic manipulator. Such a measurement system requires high accuracy and robustness considering the nonlinearity and uncertainty in hydraulic pressure-based force measurement during rigorous outdoor work. It is thus difficult to precisely model system behaviors and completely remove error force components (white-box modeling) under such conditions, so our scheme adopts a less-error data selection approach to relatively improving the accuracy and reliability of the measurand (gray-box modeling). It first removes dominant error forces, i.e., gravity and dynamic friction forces, then defines the on-load state by evaluating measurement conditions to omit data in indeterminate conditions, then extracts data during the object-grasp state identified by a grasp motion model and removes high-frequency components by a simple low-pass filter, and finally integrates data from multiple sensors using the posture-based priority and averages all selected data. Evaluation experiments were conducted using an instrumented hydraulic arm. Results indicate that our scheme can precisely measures the mass of the grasped object under various detection conditions with fewer errors.

A new magnetorheological piston head design inspired by toroidal electromagnets was proposed in previouswork as an alternative to conventional annular dampers. A prototype with a circular valve array integrated inside the piston head was built and tested to measure its passive performance. The mechanical, electromagnetic, and hydraulic models used in the new design were explained, and the relevant parameters of the actuator were analyzed to construct a mathematical model to estimate its performance. These works showed the feasibility of the concept and its potential as an alternative to current damper technology, but lacked a baseline for comparison. This paper reviews and widens this groundwork. It adds a magnetic finite element method study to address the previously found leakage, and a new set of experiments, including a force controller, to compare its performance against a conventional annular piston head. The new findings show how the current force limitations can be overcome by striking a balance between the coil space and the size of electromagnet cores to achieve the performance of current dampers. They also highlight its potential in force control applications to provide a wider range of customization options, such as number and size of holes, electromagnets, and coils, and a better use of the active area of the gap.

We previously developed a four-arm four-flipper disaster response robot called OCTOPUS, with both advanced mobility and workability, and we have experimentally confirmed the usefulness of OCTOPUS. This robot has 22 degrees of freedom, so it is currently controlled by two operators. Considering the demand to reduce human resources, we need to introduce a human-automation (HA) shared control. As a preliminary study to end this, we investigate a control-authority allocation (CAA) suitable for various tasks and environment through experiments on a human-human (HH) control system by using a virtual reality simulator. We specified four tasks and five CAA patterns with different allocations among flippers and crawler, and selected a CAA by analyzing the results in terms of completion time, workload, and usability.

In previous studies on the magnetorheological piston, the system performance was evaluated only as a traditional damping system. Here, the piston is connected to a pump to create a hydraulic compliant actuator, and is capable to control the piston speed and force independently by using the pump and electromagnet voltages, respectively. Different combinations of these parameters can be used to achieve diverse system properties; e.g. low response time or energy efficiency. Several experiments are conducted to evaluate its performance, including force, friction, speed, and step response. The results indicated that the potential of the devices to be used as an active system for compliant hydraulic robotic applications.

In teleoperation of heavy machines, work efficiency is lower than in ordinary manned operations. One of the reasons is that it is difficult for operators to imagine the situation of work sites. Thus, we proposed views to acquire spatial knowledge before work based on the mechanism of spatial cognition. Humans have cognitive maps which can be divided into survey and route perspective. In this study, we focus on the route perspective because operators usually watch a cab view. It is knowledge acquired from subjective viewpoint and can be easily acquired from active movement. Therefore, we decided views which are from operator's viewpoint and can be changed by operator's intention as views to acquire route perspective. We conducted experiments using a virtual reality simulator to evaluate the proposed views. The results indicated that the proposed views could decrease the number of stops and increase speed in grasp.

In this study, we investigate relationship between robot motion and human psychology in collaborative movement situations. To realize collaborative movement, the robot needs to consider psychological aspect. This makes robot to estimate human intention and psychological states, and convey robot's intention to human. For evaluating human's psychology directly, we use 9 kinds with 3 strength emotion items. We apply semantic differential method and use 14 adjective pairs of impressions. The subjects are asked to answer questionnaires in 20 scenarios where the robot passes through the corridor including two humans. The result of experiments using an actual mobile robot with one arm indicated that motion and psychology related such as that robot's touching motion for letting human make way evoke surprise and humanlike.

In this study, we propose a backdrivable rotatory actuator using magnetoreological fluids and several fundamental control modes for actual applications. The base design of this actuator is a vane motor considering the easiness of modification, and the passages needed for a variable viscosity mechanism are located in the vane because it can offer high backdrivability. The control modes are designed based on a finite state machine using PID control independently of the speed and force of the actuator. Several experiments are conducted to evaluate basic actuator's performance and controller's performance. The results indicated that the actuator could move adequately as the implemented control mode.

In order to maneuver a human-operated work machine efficiently and safely, the operator is required to have not only operational skill to control the machine as intended, but also planning skill to determine operational strategy such as safe operation and efficient procedure. However, there are few studies on quantifying planning skill. In this paper, planning skill for machine operators is tried to quantify by focusing on operational procedure and attention. We create two experiments to derive feature value for operational procedure and attention. The results showed that shorter travel distance of the end-point of the manipulator for procedure and larger number of cooperated joints for attention leads to improving work efficiency.

Teleoperation of disaster response robots with four limbs are required to manage sequential control-transition between hand movement and body movement. Furthermore, it is required that robots have to be operated intuitively since such disaster response robots have multiple degree of freedom. In order to deal with these requirements, we developed a master-slave interface system using magnetic sensors and data gloves for hand movement so that robots can be operated intuitively, and using foot pedals for body movement so that systems can manage sequential transition between hand and body movement. To evaluate the proposed system, we conducted a transport task using a VR simulator. The results indicated that proposed interface system could manage sequential transition between hand and body movement and this system enabled an intuitive operation.

In order to realize a safe and secure society, it is considered necessary to constantly monitor social infrastructure and traffic infrastructure. To realize that, it is necessary to collect desired information autonomously and autonomously, sharing information with each other through a network. In this research, as a basic study, we made a prototype of IoT sensor device that self-generates by monitoring phenomena to be monitored. The design requirements of the proposed IoT sensor device are (1) detecting the phenomenon, and (2) supplementing the electric power of the circuit with the self-power generation inside the device. Although the device was operated, slow device startup, slow information transmission and difficulty in power supply were confirmed, thus improvement of device is necessary.

This study develops a touchscreen-based human-machine interface prototype that allows the driver to use simple touch gestures to input vehicle control commands specifically in levels 2 and 3. The driver can instantly command a set of lateral inputs by location-based TLI, e.g., lane changing, by designating a desired location on the map, e.g., lane, by double tapping and swiping. Situational awareness is enhanced, for e.g., when approaching an intersection, by using visual and auditory prompts. We conducted experiments using a simulator to evaluate TLI method compared with the operational- (OLI, level 0) and strategic-level input (SLI, level 4) methods. The results show that TLI method offers both the flexibility of OLI as well as comfort of SLI, and drivers prefer to use all three methods depending on the driving environment.

The gas pipelines need to be inspected in the case of existence of pipeline's inner problems. Usually, inspection for leakage of gas pipeline has been carried out by wired probes or wired robots equipped with a CCD camera. However, the above inspection approaches are restricted by the cable in the pipe. This study thus focuses on an inspection approach based on microwave communication. However, obtaining the transmission properties (i.e., transmission loss, RSSI (received signal strength indication), maximum transmission distance, etc) in gas pipeline is crucial to determine which frequency is feasible to the further development. In this paper, a communication system for microwave communication in 2.4 and 5.8 GHz band is developed. Then, the transmission loss and RSSI of these microwaves in 2.5 x 100 cm (straight), 5 x 100cm (straight) and 5 x 200cm M-type curved steel gas pipe (with L-junctions) are then measured. The experimental results revealed that the microwave in 2.4 GHz band owns much stronger communication capability in complicated gas pipeline in consideration of few transmission loss, and three approximate linear functions could be modeled on the basis of the measured data.

The aim of this study is to provide a machine operator with enhanced visibility and more adaptive visual information suited to the work situation, particularly advanced unmanned construction. Toward that end, we propose a method for autonomously controlling multiple environmental cameras. Situations in which the yaw, pitch, and zoom of cameras should be controlled are analyzed. Additionally, we define imaging objects, including the machine, manipulators, and end points; and imaging modes, including tracking, zoom, posture, and trajectory modes. To control each camera simply and effectively, four practical camera roles with different combinations of the imaging objects and modes were defined: overview machine, enlarge end point, posture-manipulator, and trajectory-manipulator. A real-time role assignment system is described for assigning the four camera roles to four out of six cameras suitable for the work situation (e.g., reaching, grasping, transport, and releasing) on the basis of the assignment-priority rules. To test this system, debris-removal tasks were performed in a virtual reality simulation to compare performance among fixed camera, manual control camera, and autonomous control camera systems. The results showed that the autonomous system was the best of the three at decreasing the number of grasping misses and erroneous contacts and simultaneously increasing the subjective usability and time efficiency.

In this paper, we present a virtual reality (VR) simulator for four-arm disaster response robot OCTOPUS, which has high capable of both mobility and workability. OCTOPUS has 22 degrees of freedom (DOF), so it is quite difficult to operate OCTOPUS. We thus developed a VR simulator for training operation, revealing operations and systems to be improved, and developing operator support systems. VR simulator can conduct experimental environments at lower cost and higher efficiency. Our VR simulator consisted of VR environment and human-machine interface such as operation-input and video- and sound-output, based on robot operation system (ROS) and Gazebo. Four tasks such as rough terrain translation, step climbing, obstacle stepping over, and object transport were conducted to evaluate OCTOPUS itself and our VR simulator. The results indicate that operators could complete all the tasks and reveal key operations for stable task execution.

A new design of a magnetorheological (MR) piston prototype intended for passive force control in robotic applications is introduced. It is based in a novel toroidal array of valves, contained within the piston head, which are used to control the output force of the actuator in order to achieve a high degree of reliability, size efficiency, and safety, by exploiting the material properties of MR fluids and permalloy metals. This paper describes the main points in the development of the MR piston prototype, the mathematical modeling of the magnetic circuit, and the results of the experiments conducted using a universal testing machine to evaluate the passive performance of the prototype. Results show the feasibility and performance of the new toroidal magnetic circuit of the MR piston prototype.

This paper proposes a framework for analyzing comprehensive work tendency and operational skills using a frequency plot map of the end-point of a manipulator as a fundamental study of work analysis using long-term data for human-operated work machines. The proposed framework creates a two-dimensional end-point plot map with its origin fixed at the yaw joint of the manipulator to deal with arbitrary usage conditions. It then extracts arbitrary characteristics by using hierarchical feature extraction filters defined by feature values such as movement, and load. Finally, it quantifies the filtered map by gridding and normalization to visually grasp its frequency distribution.

In this study, a fundamental method to tune a basic input/output gain (BIOG) according to usage conditions is proposed for improving work performance in human-machine systems. For improving operability and workability, an I/O gain tuning is effective, but frequent and drastic changes degrade the operability by confusing the machine dynamics. The proposed tuning system adjusts a BIOG at long intervals on the basis of comprehensive features in operator and work content obtained from histogram of control lever input. The target value is set to the normal distribution, meaning that all ranges of the control lever are evenly used in a spring-type lever, as leveling histogram independent of an operator and work content provides the consistent operational feeling, which leads to comfortable operability. To meet practicality and effectiveness, a BIOG curve is set to a polygonal line involving a break point and a saturation point that are tuned by equivalent transform of area differences between the obtained histogram and normal distribution curves. Results of experiments conducted by using a hydraulic arm system showed that the proposed BIOG tuning system improves time efficiency by reducing the area difference while increasing the subjective usability compared with a conventional fixed BIOG system.

Human's ability to perceive intent plays a crucial role in achieving smooth and efficient navigation. At the present state, even with the state-of-the-art anthropomorphic robots, displaying human-like non-verbal communication (kinesics) is a challenging task. This poses a significant difficulty in performing legible navigation behavior for robots. In this paper, we look into light (turn indicator) and screen (arrow indicator) indicators as a means of overcoming the shortcomings of the robot's non-verbal communication abilities. Our results show a statistically significant improvement in perceived comfort, predictability, and performance with the use of light indicators.

We developed a four-arm four-crawler advanced disaster response robot called OCTOPUS. Disaster response robots are expected to be capable of both mobility, e.g., entering narrow spaces over very rough unstable ground, and workability, e.g., conducting complex debris-demolition work. However, conventional disaster response robots are specialized in either mobility or workability. Moreover, strategies to independently enhance the capability of crawlers for mobility and arms for workability will increase the robot size and weight. To balance environmental applicability with the mobility and workability, OCTOPUS is equipped with a mutual complementary strategy between its arms and crawlers. The four arms conduct complex tasks while ensuring stabilization when climbing steps. The four crawlers translate rough terrain while avoiding toppling over when conducting demolition work. OCTOPUS is hydraulic driven and teleoperated by two operators. To evaluate the performance of OCTOPUS, we conducted preliminary experiments involving climbing high steps and removing attached objects by using the four arms. The results showed that OCTOPUS completed the two tasks by adequately coordinating its four arms and four crawlers and improvement in operability needs.

Autonomous vehicles will significantly change the existing driver-vehicle relationship, since only a destination input from the human driver will suffice. However, reduced degree of human-control could result in lack of driving pleasure and excitement. Thus, we proposed a method to increase the flexibility in controlling of an autonomous vehicle by allowing the driver to control its lateral and longitudinal motions with a time lag. We first derived a set of vehicle movements to improve driver experience and related them to a set of control functions that the driver can input. We then created two types of driver-vehicle interfaces (DVIs) for vehicle control; a haptic interface with kinesthetic and tactile feedback, and a hand-gesture interface with augmented reality feedback. The joystick-type haptic interface provides feedback on driver input by dynamically varying its degrees of freedom through controlling the current supplied to axis motors, and through vibration motors. The gesture interface is based on Leap Motion controller and provides visual feedback to driver. We conducted driving experiments in a VR simulator using twenty drivers to evaluate the effectiveness of these DVIs. The results showed that haptic interface significantly reduced the average input time and input error, and drivers preferred the haptic interface due to its ability to provide immediate, active, and passive feedback.

This work focuses on the analysis of the performance parameters of a new magneto-rheological device. Previously, a mathematical model of the actuator head was introduced; however, the complex relations between its parameters made its optimisation challenging, causing the device not to reach its full potential. In this work, the previous model is updated to fit a revised version of the toroidal design, as well as an annular design for comparison. The relations between different parameters are studied to find their trade-offs, and understand how they affect the performance of the actuators. Finally, an optimisation based on an iterative search for valid permutations of parameters, is used to find optimal combinations. Two prototypes are built and tested to validate the results of the optimisation. The study revealed the critical parameters of each design, which mostly depend on the relations between the magnetic flux density, and the active area exposed to the MRF; and successfully optimised the performance of the devices. However, work needs to be done to create a more complete tool with concrete guidelines for specific applications.

This paper presents a virtual reality (VR) simulator for four-arm disaster response robot OCTOPUS, which has high capable of both mobility and workability. OCTOPUS has 26 degrees of freedom (DOF) and is currently teleoperated by two operators, so it is quite difficult to operate OCTOPUS. Thus, we developed a VR simulator for training operation, developing operator support system and control strategy. Compared with actual robot and environment, VR simulator can reproduce them at low cost and high efficiency. The VR simulator consists of VR environment and human-machine interface such as operation-input and video-and sound-output, based on robot operation system (ROS) and Gazebo. To enhance work performance, we implement indicators and data collection functions. Four tasks such as rough terrain passing, high-step climbing, obstacle stepping over, and object transport were conducted to evaluate OCTOPUS itself and our VR simulator. The results indicate that operators could complete all the tasks but the success rate differed in tasks. Smooth and stable operations increased the work performance, but sudden change and oscillation of operation degraded it. Cooperating multi-joint adequately is quite important to execute task more efficiently.

Legible robot behavior is a key element for smooth and efficient navigation among humans. However, at present, even the state-of-the-art robots cannot communicate its internal state of directional intent by displaying human-like non-verbal cues. In this paper, we explore various modes for communicating directional intent of a robot across three different scenarios as a means of overcoming the shortcomings of the robot's non-verbal communication abilities. Specifically, we look into turn indicators, display indicators, and their combinations with sound and investigate their effectiveness across different passing scenarios. Our study shows us that using auxiliary communicating methods significantly improves the perceived feelings of our dependent measures. Further, communicating intention also helps in improving cooperation. However, the effectiveness greatly varies with the modes and the passing scenarios. In the case of 90 degrees crossing scenario, even though participants have positive perceived feelings, this does not necessarily translate into smooth and efficient navigation.

The advantages of mechanical compliance have driven the development of devices using new smart materials. A new kind of magnetorheological piston based on a toroidal array of magnetorheological valves, has been previously tested to prove its feasibility. However, being an initial prototype its potential was still limited by its complex design, and low output force. This study presents the revisions done to the design with several improvements targeting key performance parameters. An improved annular piston design is also introduced as comparison with conventional devices. The toroidal and annular piston head prototypes are built and tested, and their force performance compared with the previous iteration. The experimental results show an overall performance improvement of the toroidal assembly. However, the force model used in the study still fails to accurately predict the magnetic flux at the gaps of the piston head. This deviation is later verify and corrected using a FEM analysis. The force performance of the new toroidal assembly is on par with the commonplace annular design. It also displays a more linear behaviour, at the expense of lower energy efficiency. Finally, it also shows potential for a greater degree of customisation to meet different system requirements.

Compliant actuation is an indispensable element for safe physical human robot interaction. However, there is a lack of devices, which can integrate the high power density of hydraulic actuators with intrinsically safe mechanisms. This drove the development a new type of hydraulic magnetorheological piston. This device includes a novel toroidal array of magnetorheological valves in its head. In previous studies, the system performance was evaluated only as a traditional damping system. Now, the piston is connected to a pump to create a hydraulic compliant actuator. This novel compliant piston is capable to control the piston speed and force independently by using the pump and electromagnet voltages respectively. In this way, different combinations of these parameters can be used to achieve diverse system properties; e.g. low response time or energy efficiency. Several experiments are conducted to evaluate its performance, including force, friction, speed, and step response. The results display the potential of the devices to be used as an active system for compliant hydraulic robotic applications. They also hint to the possibilities to improve by using a more sophisticated control system for its speed and force.

Disaster response robot with four arms and flippers OCTOPUS has high mobility and task-execution capabilities. Owing to the higher number of degrees of freedom, OCTOPUS is controlled by two operators, however this kind of robots is inherently difficult to be operated. To design easy-to-use human machine interfaces and intelligent control systems, we need to analyze and quantify a reasonable operation strategy in multi-operator control systems. Thus, three different types of essential disaster response tasks were conducted by using OCTOPUS and we analyzed results of operations and work performance, by focusing on each operator and each pair. As the results, we derived basic operation strategies as follow; operators with higher number of simultaneously-operated joints (Ns) can control OCTOPUS more smoothly, and pairs with higher rate of cooperated operations (Rc) can finish tasks more efficiently. We also found that Ns and Rc can be used to quantify operational skills. Revealed strategies and parameters could be useful to design new human-machine interface and intelligent control system.

Autonomous vehicles would make the future roads safer by keeping the human driver out of the loop. However, reduced degree of human-control could result in loss of the feeling of driving for some drivers. Therefore, in this study we proposed a method of interaction between the driver and autonomous vehicle by allowing the driver to control the vehicle's lateral and longitudinal motions. We adopted hand gestures as input modality because it can reduce driver's visual and cognitive demands. We first derived seven fundamental vehicle maneuvers to improve driver experience, and related them to seven independent hand gestures. We then created a hand gesture interface to control an autonomous vehicle, using Leap Motion as the gesture recognition platform. We conducted driving experiments involving twenty drivers in a virtual reality driving simulator to investigate the effectiveness of this interface for vehicle control. We evaluated the driving experience and drivers' opinions regarding the gestural interface. The results proved that semi-autonomous controlling using the hand gesture interface significantly reduced drivers' perceived work-load.

In unmanned construction, work efficiency is lower than that in manned construction due to lack of visual information. Thus, we previously developed an autonomous camera control system to provide various visual information suited to work states through multiple displays. However, that system increased the cognitive load on operators, and required them to have much experience to choose appropriate views for various situations. Next, we should investigate the degree of effectiveness for each view in a certain state. Thus, in this study, we analyzed gaze patterns to clarify which are the displays that operators often watch in work states, i.e., moving, grasping, transport, and releasing. We then derived which gaze patterns have higher work performance, including time efficiency and safeness. We clustered gaze patterns using Ward's method, which is a criterion applied in hierarchical clustering. To evaluate the objective of this study, we conducted experiments involving debris transport tasks, using a virtual reality simulator. The results indicated that gaze patterns differed in operators and we found that better time efficiency related to specific gaze patterns for each work state.

Intelligent passenger vehicles with autonomous capabilities will be commonplace on our roads in the near future. These vehicles will reshape the existing relationship between the driver and vehicle. Therefore, to create a new type of rewarding relationship, it is important to analyze when drivers prefer autonomous vehicles to manually-driven (conventional) vehicles. This paper documents a driving simulator-based study conducted to identify the preferences and individual driving experiences of novice and experienced drivers of autonomous and conventional vehicles under different traffic and road conditions. We first developed a simplified driving simulator that could connect to different driver-vehicle interfaces (DVI). We then created virtual environments consisting of scenarios and events that drivers encounter in real-world driving, and we implemented fully autonomous driving. We then conducted experiments to clarify how the autonomous driving experience differed for the two groups. The results showed that experienced drivers opt for conventional driving overall, mainly due to the flexibility and driving pleasure it offers, while novices tend to prefer autonomous driving due to its inherent ease and safety. A further analysis indicated that drivers preferred to use both autonomous and conventional driving methods interchangeably, depending on the road and traffic conditions.

In this paper, we analyzed spatial perception ability based on human eyesight for teleoperation. Recognizing the space relationship between different objects by human eyesight from a two dimensional monitor quickly and correctly is a hard task. On the basis of the analysis about visual perception error, we make some assumptions about the alignment of multi monitors. The reasons which cause visual perception error were then investigated using our virtual reality simulator,and experiments were implemented to verify our assumptions. Results show when objects were fixed, continuously moving the viewpoint around the fixed objects, the position of viewpoint obviously affected spatial perception accuracy of human subjects. We also found that subjects tend to estimate object position deeper than its real position.

In recent years there has been interest in the development of magnetorheological actuators for robotic applications with adjustable compliance. In this paper we explore the conceptualization, design and test of a new kind magnetorheological piston head design. Inside the head of this new damper a circular assembly of magnetorheological valves is used to control the damping force of the piston. The results show the feasibility of the new design, and its potential for compliant robotic applications.

In this study, we focused on creating different scenarios and traffic conditions in a driving simulator to evaluate the differences in driving experience in conventional and autonomous vehicles. We conducted experiments using two groups of drivers and evaluated their subjective workload and preference for each driving method for different traffic conditions and scenarios.

In unmanned construction, operators watch multi views to know situations at a disaster site. However, such a vision system requires much cognitive load of the operators to determine monitors and its parts which they should watch according to situation. In order to decrease cognitive load, we developed a system to provide appropriate monitors and its parts by inducing visual attention using augmented reality.

A basic input/output gain (BIOG) tuning system for a human-operated machine was previously proposed. It adjusts a BIOG at long intervals to conform the histogram of control input, representing global features in operator and work content, to a normal distribution curve, meaning that all ranges of control lever are evenly used. Experimental results showed that the system improved time efficiency and the subjective usability. As the next step, we analyze features of work and operator on the basis of input-output information in the BIOG tuning system. We first analyze relationship among I/O information, workability, and operability, and then analyze time-series variation of I/O information. The results indicate that some axes largely affected both workability and operability. We also found from the results that the form of BIOG map in each axis reveals the feature of operator and work content such as the movement direction of end-point where precise operations require, and also reveals improvements on the BIOG tuning system.

Advanced intelligent vehicles capable of autonomous driving will be commercially available in near future. To build a beneficial relationship between driver and vehicle, it is important to analyze how the drivers would react to autonomous driving compared to conventional human-driven driving. In this study, we develop a driving simulator for analyzing individual driving experience in several road conditions for autonomous and human-driven vehicles. The driving simulator incorporates fundamental functions to connect arbitrary interfaces, create virtual environments consisting of scenarios and events that drivers encounter in real-world driving, and reproduce fully autonomous driving. The results of preliminary experiments showed that experienced drivers opt for human-driven driving mainly due to the flexibility and driving fun it offers, while novices tend to prefer autonomous driving due to its inherent easiness and safety.

An autonomous environmental camera control system we previously developed can provide operators with various and variable visual information suited to the work situation for unmanned construction. However, cognitive load of operators may increase, so we propose a selective visual attention and situational awareness support system using augmented reality (AR) to enhance the ability of visual attention. The components include (i) monitor frame, (ii) obscuration, (iii) distance arrow, and (iv) end-point vertical arrow, with variable color and flashing effect to facilitate understanding the situation. Moreover, the system adaptively draws visual attention by changing types and parameters of AR components according to situations. Debris removal tasks were conducted using a VR simulator to compare without and with AR support. Results showed that the AR support reduced the frequent changes of point of gaze and provided the sense of depth and attention to potential collisions.

This paper proposes a sensor model in three dimensional (3D) space, where the height of sensor above ground is variable and the position of sensor on the ground is limited. Traditional sensor models treat the sensor placement area as a film and there is no placement limits as well. As a consequence, the number of sensors is very large and the performance is not such ideal as simulation. In this paper, we develop a novel model for sensor placement, which makes sensors possible to be set at different height above ground, so that the coverage rate can be analyzed in 3D space and the simulation result can be more dependable. We also propose a corresponding 3D coverage rate evaluation method to be used for our proposed 3D model. In order to test our proposed model and evaluation method, the system is designed and numerical and graphical simulators are developed. Experiments are conducted to quantify trade-off relationship between coverage rate and cost, i.e., the number of sensors, which can make sensor placement strategy more practical. From the experimental results, we confirmed that the proposed 3D model and evaluation method is effective in a 3D space.

Unmanned construction machines are used after disasters. Compared with manned construction, time efficiency is lower because of incomplete visual information, communication delay, and lack of tactile experience. We have developed an autonomous camera control system to supply appropriate visual information. In order to let operator easy to avoid unexpected collision, we first introduce the frame for each viewport according to its contents. And then, we exchange the danger view with an overlook view. The purpose of this study is to attract operators' attention to the danger area when there is a probable collision if operator won't stop or change their action. Besides, we also want to reveal the relationship between gaze habit and performance. The experimental results conducted using our virtual reality simulator confirms that operator tends to watch danger view when it appears. In addition, work strategy may influence the danger avoidance and work time efficiency in opposite way.

Intelligent vehicles capable of autonomous driving will be commercially available in near future. To formulate a beneficial relationship between driver and vehicle, it is important to analyze how the drivers would react to autonomous vehicles compared to human-driven (conventional) vehicles. In this study, we focused on analyzing individual driving experience in several road conditions for autonomous and conventional vehicles among experienced and novice drivers. We first developed a simplified driving simulator that can connect arbitrary interfaces, create virtual environments consisting of scenarios and events that drivers encounter in real-world driving, and implement fully autonomous driving. We then conducted experiments to clarify differences of driving experiences for autonomous driving between the two groups. The experimental results showed that experienced drivers opt for conventional driving mainly due to the flexibility and driving fun it offers, while novices tend to prefer autonomous driving due to its inherent easiness and safety. An in-depth analysis indicated that drivers preferred to use both the driving methods interchangeably depending on the road and traffic conditions.

Passive compliance is useful for robotic arms to ensure their safety. Often springs are used, but they are problematic because they reduce the achievable accelerations and can lead to underdamped oscillations. Torque limiters enhance the safety, but usually the torque limit cannot be adjusted to a desired torque. Electronically adjustable torque limiters, also known as series clutch actuators, have several benefits, especially for robotic arms, but they also have severe limitations. This paper suggests incorporating series clutch actuators into a gravity compensated arm. Consequently, gravity should not limit the isotropically achievable force anymore and in the case of power outage the arm keeps its position. The benefits and limitations of a series clutch actuator in a gravity compensated arm are discussed, and a prototype of such an arm is presented. Commercially available magnetic friction clutches are used. Preliminary experiments demonstrate that the safety can be increased.

This paper describes a combined tactile sensor and haptic interface that can change its stiffness using magnetorheological fluids (MR fluid). The tactile sensor consists of 6 distributed capacitive sensors that can sense the location and the amount of applied force. Above the sensors is a chamber filled with MR fluid. By changing the magnetic field, the hardness of the MR fluid, and thereby of the haptic interface, can be changed. Fast changes of the magnetization direction lead to a sensation of vibration. The resulting device can be used for novel haptic input devices or for robotic grippers. A prototype device has been constructed, and the effects of the varying magnetic field and the resulting varying stiffness of the MR fluid on the distributed force sensing with the capacitive sensors has been evaluated. We discovered that the measured forces vary very little with changes in the strength of the magnetic field.

In densely populated scenarios with cramped spaces, it is very difficult to achieve safe and efficient navigation without cooperation from humans. One way in which we can seek cooperation from humans is by using contact to influence them to give way. However, such a method may incur certain psychological implications and therefore requires an acceptability check to ensure whether such action is acceptable or not. For this purpose, we investigate the participant subjective response towards robot-initiated touch during the course of navigation. We conducted a 2 (robotic experience vs. none) x 2 (warning vs. none) between-subject experiment with 44 people in which a mobile robotic platform exerted contact on an unaware and obstructing participant to make way towards its goal. Our results show that prior experience with robots produces slightly better response even though the results are not statistically significant. However, a verbal warning prior to contact yielded much more favorable response. In general, the participants did not find contact to be uncomfortable and were not opposed to robot-initiated contact if deemed necessary.

A new design of a magnetorheological piston prototype intended for passive or active force control in robotic applications for human robot interaction is introduced. It is based in a novel toroidal array of valves, contained within the piston head, which are used to control the output force of the actuator in order to achieve a high degree of reliability, size efficiency, and safety, by exploiting the material properties of magnetorheological fluids and permalloy metals. This paper describes the main points in the development of the magnetorheological piston prototype, the mathematical modelling of the magnetic circuit, and the results of the experiments conducted using a universal testing machine to evaluate the passive performance of the prototype. Results show the feasibility and performance of the new toroidal magnetic circuit of the magnetorheological hydraulic piston prototype. Improvements in order to be able to test the active performance of the design together with a pump setup are proposed.

Unmanned construction machines are used after disasters. Compared with manned construction, time efficiency is lower because of incomplete visual information, communication delay, and lack of tactile experience. We have developed an autonomous camera control system to supply appropriate visual information. In order to inform operators the potential data in the views, we introduced several augmented reality (AR) elements to induce visual attention to suitable images in a scenario. The purpose of this study is to develop an attention inducement system using AR technique and evaluate it under different conditions. We first improve our autonomous camera control system and then implement several kinds of AR items suitable for each work situation. The experimental results conducted using our virtual reality simulator confirm that AR items can supply a better positional relationship between machine and objects in the environment which makes operator handle the conditions as experienced. The results from eye-tracker data also indicate that AR items can induce visual attention to images suitable for situations.

In densely populated scenarios with cramped spaces, it is very difficult to achieve safe and efficient navigation without cooperation from humans. One way in which we can seek cooperation from humans is by using contact to influence them to give way. However, such a method may incur certain psychological implications and therefore requires an acceptability check to ensure whether such action is acceptable or not. For this purpose, we investigate the participant subjective response towards robot-initiated touch during the course of navigation. We conducted a 2 (robotic experience vs. none) x 2 (warning vs. none) between-subject experiment with 44 people in which a mobile robotic platform exerted contact on an unaware and obstructing participant to make way towards its goal. Our results show that prior experience with robots produces slightly better response even though the results are not statistically significant. However, a verbal warning prior to contact yielded much more favorable response. In general, the participants did not find contact to be uncomfortable and were not opposed to robot-initiated contact if deemed necessary.

A practical operator support system for disaster response work using construction machinery is applied to object-break operations using a dual-arm machine. An object-break operation, which separates an object into two different parts by applying massive pulling force, requires skillful force and trajectory controls. Operators typically have trouble visually perceiving the movement of the manipulator because the target object is strongly restrained. As a result, after the object has been separated into two pieces, the operator may continue performing lever operations because of unavoidable perception reaction time and the manipulator may collide with the surrounding environment. A control input cancellation system based on object-break identification is proposed to reduce the time difference between the object break and operation stoppage. A support system for disaster response situations must be safe and efficient and accommodate operators with various skill levels; thus, common physical phenomena related to the object breakage should be exploited. The system cancels control inputs to stop the movement of the manipulator when an object-break flag is detected, which is defined as a situation where the force applied to each joint of a manipulator decreases rapidly when a lever operation and load on the inside of the grapple are detected. Demolition experiments were conducted using a hydraulic dual-arm system. The results indicate that the displacement of the end-point after the object breaks is greatly decreased and the completion time of the task is reduced as well.

In this paper, an object grasp motion, which is a requisite condition to make a demolition machine grasp an object, is pragmatically modeled, considering accurate and robust identification. Grasping an object is a highly difficult task that requires safe and precise operations, particularly in disaster response work. Identifying a grasp or non-grasp state is essential for providing operational support. These types of outdoor machines lack visual and tactile sensors, so pragmatically available lever operation and cylinder pressure sensors are adopted as parameters for modeling. The grasp motion is simply defined by using sequential transitions of the on-off state of the operation signal and cylinder pressure data for the grapple and the manipulator. The results of experiments conducted to transport objects using an instrumented hydraulic arm indicated that the modeled grasp motion model effectively identifies a grasp or non-grasp state with high accuracy, independently of operators and work environments.

This paper proposes a quantification method for a comprehensive work flow in construction work for describing work states in more detail on the basis of analyzing state transitions of primitive static states (PSS), which consist of 16 symbolic work states defined by using on-off state of the lever operations and joint loads for the manipulator and end-effector. On the basis of the state transition rules derived from a transition-condition analysis, practical state transitions (PST), which are common and frequent transitions in arbitrary construction work, are defined. PST can be classified into essential state transition (EST) or nonessential state transitions (NST). EST extracts common phases of work progress and estimates positional relations between a manipulator and an object. NST reveals wasted movements that degrade the efficiency and quality of work. To evaluate comprehensive work flows modeled by combining EST and NST, work-analysis experiments using our instrumented setup were conducted. Results indicate that all the PSS definitely changes on the basis of PST under various work conditions, and work analysis using EST and NST easily reveals work characteristics and untrained tasks related to wasted movements.

A virtual reality (VR) simulator is developed to aid in advancing teleoperated construction machines for disaster response work. VR simulators, which can measure arbitrary data, allow the operator to reproduce desired situations repeatedly, and change the machine and environmental configurations more easily than is possible in real environments, can create teleoperation technologies and quantitatively evaluate them, and can improve operational skills in complex disaster response works. As basic components of a VR simulator, a VR environment, operation-input, and videooutput components are developed. The VR environment is built using a basic graphics library and dynamics engine for simplification. The operation-input component consists of control levers for a demolition machine that has a grapple and environmental cameras with yaw, pitch, and zoom functions. The videooutput component consists of a two-dimensional monitor that can display an in-vehicle camera view, multiple environmental camera views, and the machine status. Experiments conducted show that operators can adequately transport debris in the VR environment while watching views on the monitor from the in-vehicle and environmental cameras. The experiments also reveal the characteristics that reduce the machine's time efficiency.

A method to autonomously control multiple environmental cameras for providing more adaptive visual information suited to the work situation for advanced unmanned construction is proposed. Situations in which the yaw, pitch, and zoom of cameras should be controlled were analyzed and imaging objects and imaging modes were defined. To control each camera simply and effectively, four practical camera roles combined with the imaging objects and modes were defined. A role assignment system was then developed to assign the four camera roles to four out of six cameras suitable for the work situation, on the basis of the assignment priority rules. Debris removal tasks were performed by using a VR simulator to compare fixed, manual control, and autonomous systems. Results showed that the autonomous system was the best of the three at decreasing the number of grasping misses and increasing the subjective usability while improving the time efficiency.

A method to autonomously control multiple environmental cameras, which are currently fixed, for providing more adaptive visual information suited to the work situation for advanced unmanned construction is proposed. Situations in which the yaw, pitch, and zoom of cameras should be controlled were analyzed and imaging objects including the machine, manipulator, and end-point and imaging modes including tracking, zoom, posture, and trajectory modes were defined. To control each camera simply and effectively, four practical camera roles combined with the imaging objects and modes were defined as the overview-machine, enlarge-end-point, posture-manipulator, and trajectory-manipulator. A role assignment system was then developed to assign the four camera roles to four out of six cameras suitable for the work situation, e.g., reaching, grasping, transport, and releasing, on the basis of the assignment priority rules, in the real time. Debris removal tasks were performed by using a VR simulator to compare fixed camera, manual control, and autonomous systems. Results showed that the autonomous system was the best of the three at decreasing the number of grasping misses and error contacts and increasing the subjective usability while improving the time efficiency.

Unmanned construction machines are used after disasters. Compared with manned construction, time efficiency is lower because of incomplete visual information, communication delay, and lack of tactile experience. Visual information is the most fundamental items for planning and judgment, however, in current vision systems, even the posture and zoom of cameras are not adjusted. To improve operator's visibility, these parameters must be adjusted in accordance with the work situation. The purpose of this study is thus to analyze effective camera images from some comparison experiments, as a fundamental study of advanced visual support. We first developed a virtual reality simulator to enable experimental conditions to be modified easier. To effectively derive required images, experiments with two different camera positions and systems (fixed cameras and manually controllable cameras) were then conducted. The results indicate that enlarged views to show the manipulator is needed in object grasping and tracking images to show the movement direction of the manipulator is needed in largely end-point movement. The result also confirms that the operational accuracy increases and blind spot rate decreases by using the manual system, com-pared with fixed system.

A method to tune a basic input/output gain (BIOG) according to usage conditions for human-machine systems is proposed for improving work performance. Adapting a BIOG is effective in terms of improving operability and work efficiency, but frequent changes using a gain scheduling strategy degrade the operability by confusing the machine dynamics. The proposed tuning system adjusts a BIOG at long intervals on the basis of comprehensive features in operator and work content obtained from the histogram of control lever input. The target value is set to the normal distribution, meaning that all ranges of the control lever are evenly used in a spring-type lever, as leveling the histogram independent of an operator and work content provides the consistent operational feeling, which leads to comfortable operability. To ensure practicality and effectiveness, a BIOG curve is set to a polygonal line involving a break point and a saturation point that are tuned by equivalent transform of area differences between the obtained histogram and normal distribution curves. Two types of experimental task were performed using a hydraulic arm system. Results showed that the proposed BIOG tuning system improves time efficiency by reducing the area difference while increasing the subjective usability compared with a conventional fixed BIOG system.

A practical framework for measuring the mass of an object grasped by the end-effector of a large-scale hydraulic manipulator, such as construction manipulators, is proposed. Such a measurement system requires high accuracy and robustness considering the nonlinearity and uncertainty in hydraulic pressure-based force measurement. It is thus difficult to precisely model the system behaviors and completely remove error force components, so our framework adopts a less-error data selection approach to improving the reliability of the measurand. It first detects the on-load state to extract reliable data for mass measurement, including evaluating measurement conditions to omit sensors in indeterminate conditions and redefining three-valued outputs such as on, off, or not determinate, to improve robustness, then extracts data during the object-grasp state identified by the grasp motion model and removes high-frequency component by a simple low-pass filter, to improve accuracy, and finally integrates date from plural sensors using the posture-based priority and averages all selected data, to improve reliability. Evaluation experiments were conducted using an instrumented hydraulic arm. Results indicate that our framework can precisely measures the mass of the grasped object in various detection conditions with less errors.

In this paper, a method to accurately estimate whether a grapple grasps an object or not for construction machinery is proposed on the basis of extending the requisite condition for grasp, considering robustness and practicality. Accurate grasp estimation requires confirmation that the hand and arm loads continue to be observed during arbitrary manipulator movements. Enhancement conditions are thus defined by using force and movement at the end-point, including (i) vertical downward force, (ii) longer direction movement, and (iii) horizontal direction reciprocating movement. In transport experiments conducted by using an instrumented hydraulic arm, results indicated that the ratio of error estimation decreases as formed conditions increase, and confirmed that the proposed method estimates grasp/non-grasp accurately and robustly.

This paper proposes a framework for visualizing comprehensive work tendency using a frequency plot map of the end-point of a manipulator as a fundamental study of work analysis using long-term data for human-operated work machines. A visualization system requires high generality and commonality to enable to extract various characteristics to be extracted on an arbitrary time scale independently of the work machine specifications, work contents and environment, and machine operator. The proposed framework meets this requirement by first creating a two-dimensional end-point plot map with its origin fixed at the yaw joint of the manipulator to deal with arbitrary usage conditions. It then extracts arbitrary characteristics by using hierarchical feature extraction filters, including binary, quantity, and advanced filters, defined by three kinds of essential data: operation, movement, and load. Finally, it quantifies the filtered map by gridding and normalization to visually grasp its frequency distribution. Two experiments in which the work environments and completion time differed were conducted using an instrumented hydraulic arm. Results indicated that the comprehensive work tendency revealed by using the proposed framework corresponds to the actual work results independently of the various conditions. © 2013 IEEE.

The purpose of this paper is to model and identify an object grasp motion, which is a requisite condition to make a demolition machine grasp an object. Grasping an object is a highly difficult task that requires safe and precise operations, so identifying a grasp or non-grasp is necessary for providing operational support, particularly in disaster response work. These types of outdoor machines lack visual and tactile sensors, so practically available lever operation and cylinder pressure sensors are adopted. The grasp motion is modeled by using sequential transitions of the on-off state of the control signal and cylinder pressure data for the grapple and the manipulator. The results of experiments conducted to transport objects using an instrumented hydraulic arm indicated that the modeled grasp motion effectively identifies a grasp or non-grasp with high accuracy, independently of operators and environments.

This paper proposes a practical framework to estimate whether or not a grapple installed in demolition machines is in a grasp state. Object grasp is a highly difficult task that requires safe and precise operations, so identifying a grasp or non-grasp state is important for assisting an operator. These types of outdoor machines lack visual and tactile sensors, so the proposed framework adopts practically available lever operation and cylinder pressure sensors. The grasp is formed by a grasp motion, which is operations to make the grapple pinch an object, and the grasp state, where the grapple holds the object in any manipulator movements. Thus, the framework determinately confirms the grasp motion through the requisite conditions defined by using sequential changes of binarized operation and pressure data for the grapple and the manipulator, and stochastically confirms the grasp state through the enhancement conditions defined by using force and movement vectors including vertical downward force, movement in the longer direction, and horizontal reciprocating movement. The results of experiments conducted to transport objects using an instrumented hydraulic arm indicated that the proposed framework is effective for identifying grasp/non-grasp with high accuracy, independently of various operators and environments.

This paper describes a tele-operation simulator using a virtual reality (VR) environment for advanced unmanned construction. VR simulators, which can measure arbitrary data, reproduce the same situations, and change the machine and environmental configurations more easily, compared with actual environments, are effective to create tele-operation technologies and quantitatively evaluate them as well as to improve operational skills in complex disaster response works. In this basic study, a VR simulator including a VR environment, operation-input, and video-output components was developed. (i) The VR environment is built using a basic graphic library and dynamics engine. (ii) The operation-input component consists of control levers for a demolition machine with a grapple and environmental cameras with yaw, pitch, and zoom functions. (iii) The video-output component consists of a two-dimensional monitor for displaying an in-vehicle camera view, environmental camera views, and machine status. The results of experiments conducted to transport debris using the VR simulator indicated that the operators adequately completed the debris transport in the VR environment while watching the monitor views from the in-vehicle and environmental cameras.

A pragmatic framework for detecting (identifying the on-off state of) the external force applied to a construction manipulator (front load) by using a hydraulic sensor is proposed. Such a load detecting system requires high accuracy and robustness considering the uncertainty in pressure-based force measurement. The proposed framework first identifies the dominant error force component, including self-weight and driving force, using theoretical and experimental estimation and binarizes the analog cylinder external force. It then evaluates detection conditions to address indeterminate conditions such as stroke-end, singular posture, and impulsive or oscillatory force and redefines three-valued outputs such as on, off, or not determinate (ND). It finally outputs the front load decision by combining all the cylinder decisions to improve robustness through priority analysis. Experiments were conducted using an instrumented hydraulic arm. Results indicate that the proposed framework detects on, off, and ND outputs of the front load more accurately, robustly, and stably in various detection conditions.

The purpose of this paper is to develop a fundamental external-force-detection framework for construction manipulators. Such an industrial application demands the practicality that satisfies detection requirements such as the accuracy and robustness while ensuring (i) a low cost, (ii) wide applicability, and (iii) a simple detection algorithm. For satisfying (i) and (ii), our framework first adopts a hydraulic sensor as a force sensor. However, hydraulic-pressure readings essentially include error force components. These components depend strongly on the joint kinetic state and differ in the identification difficulty owing to a nonlinear and uncertain hydromechanical system. For satisfying (ii) and (iii), our framework thus focuses on the dominant error-force components classified by the control input states, such as self-weight, cylinder driving, and oscillating forces, and identifies and removes them by using a theoretical model, an experimental estimation, and a waveform analysis without complex modeling, respectively. Experiments were conducted using an instrumented hydraulic arm system. The results of a no-load task indicate that our framework greatly lowers the threshold to determine the on-off state of external force application, independent of the joint kinetic states. The results of an on-load task confirm that our framework robustly identifies the off states in which an external force is not applied to the hydraulic cylinder.

In this paper, state transitions of primitive static states (PSS), which consist of 16 symbolic work states defined by using on-off state of the lever operations and joint loads for the manipulator and end-effector, are analyzed for describing work states in more detail. From a transition-condition analysis, practical state transitions (PST), which are frequent transitions in arbitrary construction work, are defined. PST can be classified into essential (EST) or nonessential state transitions (NST). EST extracts common phases of work progress and estimates positional relations between a manipulator and an object. NST reveals wasted movements that degrade the efficiency and quality of work. Work-analysis experiments using our instrumented setup were conducted. Results indicate that all the PSS definitely changes on the basis of PST under various work conditions, and EST and NST easily reveals work characteristics and untrained tasks related to wasted movements.

This paper proposes a quantification method for a comprehensive work flow in construction work for describing work states in more detail on the basis of analyzing state transitions of primitive static states (PSS), which consist of 16 symbolic work states defined by using on-off state of the lever operations and joint loads for the manipulator and end-effector. On the basis of the state transition rules derived from a transition-condition analysis, practical state transitions (PST), which are common and frequent transitions in arbitrary construction work, are defined. PST can be classified into essential (EST) or nonessential state transitions (NST). EST extracts common phases of work progress and estimates positional relations between a manipulator and an object. NST reveals wasted movements that degrade the efficiency and quality of work. To evaluate comprehensive work flows modeled by combing EST and NST, work-analysis experiments using our instrumented setup were conducted. Results indicate that all the PSS definitely changes on the basis of PST under various work conditions, and work analysis using EST and NST easily reveals work characteristics and untrained tasks related to wasted movements.

This paper describes an operational support system to simplify removing objects from a building structure in demolition work using dual-arm construction machinery. An operator cannot recognize the manipulator movement visually when a target object is strongly restrained in an unmovable environment, so the operator continues large lever-operations for a while after the object has been removed. This could lead to a collision against the surrounding environment. An operational support in construction machinery filed requires the compatibility among efficiency, quality, and safety and the commonality among various operator skill levels. The proposed support system thus identifies the moment when an object has been removed and cancels any operational inputs to ensure safety work when generating intensive force applied to a manipulator, focusing only on the common physical phenomena. An object-removal flag is defined as a situation where force applied to the end-point of a manipulator decrease rapidly in which lever operation and hand load are detected. Demolition experiments are conducted using a hydraulic dual arm system. Result indicates that the displacement of end-effector after removal and the number of error contacts are decreased while decreasing time consumption.

In this paper, a quantitative analysis method for a comprehensive work flow in construction work for identifying work states in more detail is proposed. The proposed method is based on analyzing state transitions of simplified primitive static states (s-PSS), which consist of four symbolic work states defined by using the on-off state of lever operations and manipulator loads. First, practical state transitions (PST), which are common and frequent transitions in arbitrary construction work, are defined on the basis of the transition rules, according to which an operation flag changes arbitrarily and a load flag changes only during a lever operation. Second, PST is notionally classified into essential (EST) and nonessential (NST) state transitions whose definitions change depending on the task phase, including reaching, contacting, loadworking, and releasing. Third, closed loops formed by EST represent work content and those formed by NST represent wasted movements. In work-analysis experiments using our instrumented setup, results indicated that all s-PSS definitely changes on the basis of PST under various experimental conditions and that work analysis using EST and NST easily reveals untrained tasks related to wasted movements.

This paper reports a practical framework for measuring the external force vector applied to a construction manipulator using hydraulic sensors. Such a measuring system inevitably includes measuring errors owing to difficult-to-reduce modeling errors, so our framework introduces relative measuring accuracy improvement. It is organized into (i) quantifying internal error range (IER) using the summation of the maximum measuring error of static and dynamic friction forces, (ii) calculating error force vectors using IER to select cylinders with less errors, and (iii) outputting the external force vector using the selected cylinders. Experiments were conducted using an instrumented hydraulic arm. Results indicate that our framework enables to improve the relative measuring accuracy, independently of various postural and kinematic conditions.

This paper proposes a practical framework for detecting (identifying the on-off state of) the external force applied to a construction manipulator (front load) by using a hydraulic sensor. Such a detection system requires high accuracy and robustness considering the uncertainty in pressure-based force measurement. Our framework is thus organized into (i) identifying the dominant error force component (self-weight and driving force) using theoretical and experimental estimation and binarizing the analog external cylinder force, (ii) evaluating detection conditions to address indeterminate conditions such as stroke-end, singular posture, and impulsive or oscillatory force and redefining three-valued outputs such as on, off, or not determinate (ND), and (iii) outputting the front load decision by combining all the cylinder decisions to improve robustness through priority analysis. Experiments were conducted using an instrumented hydraulic arm. Results indicate that our framework adequately detects on, off, and ND outputs of the front load in various detection conditions without misidentification.

This paper proposes a practical framework for measuring the external force applied to a construction manipulator (front load vector) by using a hydraulic sensor. Such a force measurement system requires high accuracy and robustness considering the uncertainty in the construction machinery field, but it inevitably includes measurement errors owing to difficult-to-reduce modeling errors. Our framework thus adopts a relative accuracy improvement strategy without correcting the models for the practicality. It comprises (i) quantifying the internal error range (IER) by using the sum of the maximal measurement errors of static and dynamic friction forces, which change in postural and kinematic conditions, (ii) calculating the error force vector by using IER to select cylinders (sensors) that have less error, and (iii) outputting the front load vector using the cylinders whose error force vector is minimum. Experiments were conducted using an instrumented hydraulic arm. The results indicate that our framework can enhance the relative accuracy of external force measurement independently of various postural and kinematic conditions.

This paper proposes an analysis method for a comprehensive work flow in construction work for identifying work states in more detail on the basis of analyzing state transitions of simplified primitive static states (s-PSS), which consist of four symbolic work states defined by using on-off state of the lever operations and manipulator loads. First, practical state transitions (PST), which are common and frequent transitions in arbitrary construction work, can be defined on the basis of the transition rules, according to which an operation flag changes arbitrary and load flag only changes during a lever operation. Second, PST can be classified into essential (EST) or nonessential state transitions (NST), and NST changes its definition depending on the task phase. Third, EST and NST represent work contents and wasted movements, respectively, so work-analysis experiments using our instrumented setup were conducted. Results indicate that all the s-PSS definitely changes on the basis of PST under various experimental conditions and that work analysis using EST and NST easily reveals untrained tasks related to wasted movements. © 2011 IEEE.

The state identification framework we propose supports complex construction machinery operations using a dual arm. Such support requires compatibility with different types of support and commonality among various operator skill levels. Our framework is organized into (i) real-time task phase identification defined using joint load applied based on environment constraints and (ii) time-series attentional condition identification defined as an internal work-state condition classified by the operational support necessity level and dependent on the vectorial or time-series data selected by the identified task phase. Experiments are conducted using the instrumented hydraulic dual arm system for transport and removal tasks, including complex dual-arm operations. Results show that the number of erroneous contacts, internal force applied, and mental workload decreased without any increase in time, confirming that operational support based on our framework greatly improves individual operator work performance.

This paper proposes a framework to detect (identify on-off state of) external force applied to hydraulic cylinder for construction manipulator, utilizing hydraulic sensor. Hydraulic sensor inherently detects not only external force but also self-weight, driving, and inertial forces. These error components generate depending on joint-kinetic state and they differ in identification-difficulty due to hydromechanical uncertainty and nonlinearity. Theoretical model-based, experimental estimation-based, and waveform analysis-based identification methods were respectively applied in static, uniform motion, and accelerated motion states. Experiments were conducted by utilizing an instrumented hydraulic arm system. Results indicated that the proposed framework greatly lowered a threshold to detect external force, independently of various kinetic conditions.

This paper proposes a state identification framework to support the complicated dual-arm operations in construction work. The operational support in construction machinery filed requires the compatibility with different types of support and the commonality among various operator skill levels. The proposed framework is therefore organized into two functions: real-time task phase identification and time-series attentional condition identification. The task phase is defined by utilizing the joint load applied according to the environment constraint condition. The attentional condition is defined as one of the internal work-state condition classified by the necessity level of operational support, and is dependent on the vectorial or time-series date selected by the identified task phase. Experiments are conducted using the hydraulic dual arm system to perform transporting and removing tasks. Results show that the number of error contacts, internal force applied, and mental workload is decreased without time-consumption increase. The result confirmed that the proposed framework greatly contribute to improving each operator's work performance. ©2010 IEEE.

This paper reports a newly developed hydraulic dual robotic arm system to create and evaluate intelligent systems, which support complicated machine operations, for advanced construction machinery. This kind of machine system (test-bed) requires functions to quantify its dynamic characteristics and operational difficulty (influential factors). In particular, construction manipulator nonlinearly changes its dynamics depending on various internal and external factors. To quantify such influential factors, our proposed test-bed is equipped with standard hydromechanical system which includes two manipulators and grapples, and practical sensing system which detects control input, oil pressure, oil temperature, and cylinder stroke data. Using the developed test-bed, fundamental experiments were conducted to clarify hydromechanical system characteristics. Experimental results indicate that the test-bed quantifies its hysteresis, pressure loss, and time delay, and show that variable dynamics complicates intuitive and precise machine operations and external force measurement. This analysis confirms the developed machine system is useful to quantify influential factors for creating intelligent system. © 2010 IEEE.

We understand that primitive static states (PSS), which we have defined, are useful for work or skill analysis, but PSS are not useful for operational support because PSS determine with using only on-off information. In this study, we propose a work state identification for operational support, which includes an identification of the most basic construction work property by modifying PSS and an extraction of operational support providing flag by applying modified-PSS to time-series data. Experiments in transporting and removing tasks using our hydraulic dual arm system indicated that operator supports based on the proposed identification method improves work performance, including decreasing the time taken to complete a task, and the number of error operation, and reducing the moving distance of end-effector after breaking an object.

This paper reports a newly developed hydraulic dual robotic arm system for research and development on advanced intelligent construction machinery that has two joysticks to control the two manipulators with twelve degrees of freedom, control valves with load-sensing system, and sensors for detecting lever operation, cylinder oil pressure, and cylinder stroke information. To clarify important points at addressing an intelligent system, we evaluated basic characteristics of hydraulic machine system. Experimental results showed that hydraulic machine system has measurable time delay from lever input to cylinder actuation, which makes a cooperative operation with two arms difficult. We also found that magnitude of cylinder load greatly changes by lever operation and joint angle, which requires a sophisticated external load detecting method.

Double-front construction machinery (DFCM), which has been developed for adaptation to complicated work, demands higher operational skills to control double fronts with more multiple joints. To handle DFCM skillfully for performing more complicated works, intelligent functions that can autonomously distinguish between diverse states and also provide informational and operational supports to their operators are inevitably required. In particular, a state identification method strongly requires high reliability and robustness due to the complexity of the construction work environment and the variety of the operator's skill level. However, most of the current construction machinery has unique functions that only reproduce the movements that are originated by the operator. In this study, which focuses on DFCM, we addressed the need for a new conceptual design of an operator support system (OSS) and evaluated it using our newly developed simulator. Our experimental results showed that the OSS improves the work performance, including decreasing the operational time for completing a task, reducing the mental workload on the operators, and the number of error operations. © 2009 IEEE.

Advanced operated-work machines, which have been designed for complicated tasks and which have complicated operating systems, requires intelligent systems that can provide the quantitative work analysis needed to determine effective work procedures and that can provide operational and cognitive support for operators. Construction work environments are extremely complicated, however, and this makes state identification, which is a key technology for an intelligent system, difficult. We therefore defined primitive static states (PSS) that are determined using on-off information for the lever inputs and manipulator loads for each part of the grapple and front and that are completely independent of the various environmental conditions and variation in operator skill level that can cause an incorrect work state identification. To confirm the usefulness of PSS, we performed experiments with a demolition task by using our virtual reality simulator. We confirmed that PSS could robustly and accurately identify the work states and that untrained skills could be easily inferred from the results of PSS-based work analysis. We also confirmed in skill-training experiments that advice information based on PSS-based skill analysis greatly improved operator's work performance. We thus confirmed that PSS can adequately identify work states and are useful for work analysis and skill improvement.

Intelligent functions that can autonomously identify the current work states and also provide informational or operational support to their operators are inevitably required for double-front construction machinery (DFCM), which has been developed for complicated tasks. In this study, which focuses on DFCM, we address the need for a new conceptual design of an operator support system. In particular, a state identification method strongly requires high reliability and robustness to address the complexity of the construction work environment and the variety of the operator's skill level. We, therefore, define primitive static states (PSS) that are determined using on-off information for the]ever inputs and manipulator loads for each part of the grapple and front. We develop an intelligent system that provides a reduction of operational gain to make precise work easier and an indication of an enlarged image of the end-effector from a different viewpoint to assist depth perception based on PSS identification, and evaluate it using our newly developed simulator. Our experimental results show that the operator support system improves the work performance, including decreasing the operational time for completing a task, reducing the mental workload on the operators and the number of error operations. (C) Koninklijke Brill NV, Leiden and The Robotics Society of Japan, 2009

This paper reports a newly developed simulator for operation skill training in Double-Front Construction Machinery (DFCM) that allows novices to virtually experience tough operations repeatedly using DFCM under various conditions, including dangerous congestion. First, we selected several situations targeted where the DFCM needs to be used to provide a high level of operation skills: sorted dismantling for recycling and reusing resources, rescue and recovery work in disaster areas, and building construction. In addition, we developed an operation skill-training simulator that enables novice operators to repeatedly train with the high level of operation skills needed to easily and safely handle the DFCM in even more complicated works. This simulator system has two joysticks (set in front of a monitor) to dependently control the two fronts of the animated DFCM on the monitor. Several modes involving basic construction tasks are provided and the effects of improvement in operability achieved by the training simulator can be verified. Evaluation experiments indicated that repeated training using the simulator successfully decreased the operation time to complete a task and enhanced positioning accuracy in cooperative transportation with the two fronts. The results confirm the effectiveness of the developed simulator. Futhermore, it was confirmed that informational or operational support based on knowledge provided from experiment results enabled work performance greatly improved.

In complicated work machines represented by Double-Front Construction Machinery, an intelligent system to which can provide operational and cognitive supports for an operator is essential. In construction work, a) work objects change consistently, b) environment is complicated, and c) operation methods are different by operators. We understand that there are so many unknown parameters that state identification technique to cover all is very difficult. Therefore, we think that the state unit in total independence of varieties of environmental condition and operator's skill level condition is important. We designed the state unit - PSS (Primitive Static States) with ON / OFF information of lever input and manipulator load divided into "end-effecter" and "except end-effecter". PSS classified working states in 256 (double front) ways every sampling period. As a result of having experimented with between different operators and between different environments with PSS, we confirmed that PSS have properties "in total independence of varieties of environmental condition and operator's skill".

This paper reports a newly developed simulator for operation skill training in Double-Front Construction Machinery (DFCM) that allows novices to virtually experience tough operations repeatedly using DFCM under various conditions, including dangerous congestion. First, we selected several situations targeted where the DFCM needs to be used to provide a high level of operation skills: sorted dismantling for recycling and reusing resources, rescue and recovery work in disaster areas, and building construction. In addition, we developed an operation skill-training simulator that enables novice operators to repeatedly train with the high level of operation skills needed to easily and safely handle the DFCM in even more complicated works. This simulator system has two joysticks (set in front of a monitor) to dependency control the two fronts of the animated DFCM on the monitor. Several modes involving basic construction tasks are provided and the effects of improvement in operability achieved by the training simulator can be verified. Evaluation experiments indicated that repeated training using the simulator successfully decreased the operation time to complete a task and enhanced positioning accuracy in cooperative transportation with the two fronts. The results confirm the effectiveness of the developed simulator. © 2008 IEEE.

We developed a new simulator of operation skill training for Double-Front construction Machinery. As for a past construction machinery simulator, the one developed for a lot of educational for dangerous acknowledgment, and to train the operation skill is few. It was simple digging work by the backhoe, and work of the crane. In this research, Double-Front construction machinery which we make a target increases the number of fronts should be manipulated and demands an advanced operation skill. Moreover, the discretion dismantlement and the disaster relief and restoration are developed as assumption work as for Double-Front construction machinery. It is understood that the meaning of the developed simulator is high from these two points. As a result of the proof experiment, we obtained data with a different in the feature transition of expert and beginner's end-effecter orbits. Because the difference was confirmed, it becomes useful as a training machine for the skill improvement by being analyze this in the future and feeding back to the operator appropriately.

The multi-degree-of-freedom disaster response robot ‘OCTOPUS’ requires an ‘environment-adaptive motion generation function’ to fully utilize its advanced ‘body.’ The disaster site is unknown and complex, so there is a limit to the amount of control rules that can be described in advance by humans. To address this issue, we have developed a basic framework for an adaptive reinforcement system that learns behaviors in a virtual space generated based on environmental information acquired in a real space. In this study, we develop a motion learning system based on task classification and reuse of learned control rules. As a result of experiments on rough terrain, we confirmed that the proposed system could accomplish a moving task on any unknown terrain in a shorter time than the conventional system by the task classification and reusing control laws.

The purpose of this study is to develop a basic control system that enhances mechanical and geometric adaptability for magnetorheological fluid (MRF) robot arms with high backdrivability and high output power, which we have developed in a previous study. A prototype control system was developed to control the force of the endpoint by controlling pressure in the actuator. The results of gravity compensation and surface copying motion experiments were conducted. The experimental results showed that proposed controller could effectively control the MRF robot arm.

For personal mobility(PM) operators, it is difficult to move through a crowd and it is important to estimate collision risks and to control driving velocity properly. In this study, we propose an automatic velocity control system for PM based on “Passability Index (PI),” which indicates difficulty of driving while pedestrians are passing. The system uses an RGB-D camera and LRFs to estimate a pedestrian’s position and velocity. To calculate PI, the system calculates width of passage during passing considering pedestrian’s movement. Then, PM adopts a proper passing velocity depending on PI. Experimental results show that the proposed system helped operators to drive safer compared to manual (non-supported) driving.

Personal mobility (PM) taking parts in society is a challenge due to two major problems as follows. The one is difficulty in estimating passability precisely and proactively. The other is difficulty in controlling PM in a narrow space. Human operators are essentially not good at dealing with the problems, so we propose a proactive operation support system based on estimating passability index related to the height and width of passages. For height, the system detects steps taller than 5 cm, visualizes them on monitor, and provides emergency stop if collision is estimated. For width, the system extracts labelled clusters of walls and calculates distance for adjacent walls, visuals passages on monitor, and decrease the speed based on the width. The experimental results indicated that the proposed system enhanced the efficiency and safer operation, compared with no-support.

In this study, we designed a basic framework for diagnosing a video presentation system suitable for individual operator engaging robot teleoperation. The individual suitability of video presentation systems can be only evaluated through the result of work and there are no evaluation frameworks. We first created 16 video types based on the conventional video presentation system and six diagnosis tasks, and then assigned the operators to a video type according to the work performance. The results of diagnosis revealed that the video type that maximizes work performance differed in the individuals, and the individual-suitable video type obtained by diagnosis provided better work performance, even in an unknown environment.

Autonomous mobile robots are expected to work in a human-robot coexistence environment, and they require to have a proximal navigation framework considering approach and contact with humans. Such a framework requires a function to select a path based on estimating “the risk of approach” and comparing it with “the benefit of efficient movement by approach”. To move even in the proximal state, we develop a proximal navigation planning considering the distance between the human and robot (time to interference). The control framework estimates risks and benefits, and adjusts the risks as necessary, while performing proximal movement. The results of the experiment showed that the proposed navigation system could estimate the risk and benefit of approach, adjust it through inducement, and select a path that is natural and safe for the robot and human.

Detailed views are necessary for precise manipulation in teleoperation of humanoid robots controlled by the master slave system with head mounted display. Thus, we have proposed an autonomous zoom method based on the positions of the hands, which could increase work efficiency. However, it caused motion sickness and high cognitive load owing to little reflection of operators’ intention. In this study, we thus propose a low cognitive-load zooming interface with little motion sickness. Motion sickness can be reduced by active conditions with manual control rather than passive conditions. Moreover, the interface should let operators reflect their intention to realize a low cognitive load zoom. Furthermore, the interface should be without upper limbs because operators control robots by their upper limbs. Thus, we developed the zoom interface with head movements which are typical movements during gazing. The experimental results suggest that the proposed interface can reduce sickness and cognitive load.

In this study, we develop an environment adaptability enhancement system with active information search in real space and high efficiency motion search in virtual space for multi-degree of freedom disaster response robot. For such robots to work safely, estimating the work achievability before executing it is important. However, in disaster sites, failed motions such as falls increase the risk of secondary disasters, so it is difficult to learn motions by trial and error. Thus, the robot obtains the environment information in real space and reproduces them in virtual space, and then performs motion search efficiently in virtual space. Due to the limitation of the reproduction accuracy, for finer adjustment, the robot performs active touches to acquire ground properties such as friction and hardness in real space. The experimental results showed that the proposed system could judge work availability by trying robot motions and provide the robot with efficient motions.

In this study, we designed a framework to evaluate quantitatively the operation status of human-operated work machines represented by work efficiency. The framework requires to reveal the sensor data affecting the operating conditions, the system consists of (a) formulating between explanatory variables (sensor data) and the purpose variable (operational status) and (b) extracting explanatory variables contributing to the objective variable. As a result of the experiments, the proposed system could grasp and evaluate the operating situation in a pluralistic and quantitative manner.

When highly-automated vehicles operating in level 3 encounter system limitations, they issue a takeover request (TOR) to transfer the control authority from the automated driving (AD) system to a human driver. In such ‘unscheduled’ situations, the driver needs immediately re-engage in the driving task both physically and cognitively. In this study, we developed a driver situational awareness estimation system based on glance information. We collected a lot of glance data when both safe and dangerous takeover situations by using a driving simulator, and we derived the standard glance model including the glance area and time. To evaluate the effectiveness of the model, we created a glance guidance system to visually indicate regions with insufficient glance. We found that the guidance system improved driving performance and reduced the number of accidents.

To make full use of the advantages of automatic driving while retaining the enjoyment and freedom of the manual driving, we have proposed a tactical-level input (TLI) that allows the driver to input dynamic driving tasks such as parking and turning. In the future state input using touch interface, eye movement between the interface and front is troublesome and it takes time to operate the screen. In the command input using haptic interface, it cannot reserve input to a distant place and the amount of information is small. To solve these problems, we develop a visual-haptic interface by integrating them, which has a planar-movable haptic device and an input/output state indicator using transparent screen. The results of experiments using a driving simulator indicate that the proposed interface solve the above disadvantages while remaining the advantages.

In extreme disaster sites, external sensors such as laser range finders do not work properly, and such sensors may be broken in accident. Risk-tolerance robots should continue to perform tasks, even if the external sensors cannot work. In this preliminary study, for proximate environment understanding without using external sensors, we developed a groping method, in which the robot actively touches the environment using the end-point of the arm, and then reconstructs a three-dimensional local map around the robot based on the recorded contact information. The results of the experiments conducted by using four-arm four-flipper robot OCTOPUS indicated that the proposed groping actions could recognize step and pit, and calculate the position and size of object, and confirm that the robot successfully removed the object.

This study develops a touchscreen-based human-machine interface prototype that allows the driver to use simple touch gestures to input vehicle control commands specifically in levels 2 and 3. The driver can instantly command a set of lateral inputs by location-based TLI, e.g., lane changing, by designating a desired location on the map, e.g., lane, by double tapping and swiping. Situational awareness is enhanced, for e.g., when approaching an intersection, by using visual and auditory prompts. We conducted experiments using a simulator to evaluate TLI method compared with the operational- (OLI, level 0) and strategic-level input (SLI, level 4) methods. The results show that TLI method offers both the flexibility of OLI as well as comfort of SLI, and drivers prefer to use all three methods depending on the driving environment.

In order to maneuver a human-operated work machine efficiently and safely, the operator is required to have not only operational skill to control the machine as intended, but also planning skill to determine operational strategy such as safe operation and efficient procedure. However, there are few studies on quantifying planning skill. In this paper, planning skill for machine operators is tried to quantify by focusing on operational procedure and attention. We create two experiments to derive feature value for operational procedure and attention. The results showed that shorter travel distance of the end-point of the manipulator for procedure and larger number of cooperated joints for attention leads to improving work efficiency.

An autonomous environmental camera control system we previously developed can provide operators with various and variable visual information suited to the work situation for unmanned construction. However, cognitive load of operators may increase, so we propose a selective visual attention and situational awareness support system using augmented reality (AR) to enhance the ability of visual attention. The components include (i) monitor frame, (ii) obscuration, (iii) distance arrow, and (iv) end-point vertical arrow, with variable color and flashing effect to facilitate understanding the situation. Moreover, the system adaptively draws visual attention by changing types and parameters of AR components according to situations. Debris removal tasks were conducted using a VR simulator to compare without and with AR support. Results showed that the AR support reduced the frequent changes of point of gaze and provided the sense of depth and attention to potential collisions.

A method to autonomously control multiple environmental cameras for providing more adaptive visual information suited to the work situation for advanced unmanned construction is proposed. Situations in which the yaw, pitch, and zoom of cameras should be controlled were analyzed and imaging objects and imaging modes were defined. To control each camera simply and effectively, four practical camera roles combined with the imaging objects and modes were defined. A role assignment system was then developed to assign the four camera roles to four out of six cameras suitable for the work situation, on the basis of the assignment priority rules. Debris removal tasks were performed by using a VR simulator to compare fixed, manual control, and autonomous systems. Results showed that the autonomous system was the best of the three at decreasing the number of grasping misses and increasing the subjective usability while improving the time efficiency.

In this paper, a method to accurately estimate whether a grapple grasps an object or not for construction machinery is proposed on the basis of extending the requisite condition for grasp, considering robustness and practicality. Accurate grasp estimation requires confirmation that the hand and arm loads continue to be observed during arbitrary manipulator movements. Enhancement conditions are thus defined by using force and movement at the end-point, including (i) vertical downward force, (ii) longer direction movement, and (iii) horizontal direction reciprocating movement. In transport experiments conducted by using an instrumented hydraulic arm, results indicated that the ratio of error estimation decreases as formed conditions increase, and confirmed that the proposed method estimates grasp/non-grasp accurately and robustly.

In this paper, state transitions of primitive static states (PSS), which consist of 16 symbolic work states defined by using on-off state of the lever operations and joint loads for the manipulator and end-effector, are analyzed for describing work states in more detail. From a transition-condition analysis, practical state transitions (PST), which are frequent transitions in arbitrary construction work, are defined. PST can be classified into essential (EST) or nonessential state transitions (NST). EST extracts common phases of work progress and estimates positional relations between a manipulator and an object. NST reveals wasted movements that degrade the efficiency and quality of work. Work-analysis experiments using our instrumented setup were conducted. Results indicate that all the PSS definitely changes on the basis of PST under various work conditions, and EST and NST easily reveals work characteristics and untrained tasks related to wasted movements.

This paper reports a practical framework for measuring the external force vector applied to a construction manipulator using hydraulic sensors. Such a measuring system inevitably includes measuring errors owing to difficult-to-reduce modeling errors, so our framework introduces relative measuring accuracy improvement. It is organized into (i) quantifying internal error range (IER) using the summation of the maximum measuring error of static and dynamic friction forces, (ii) calculating error force vectors using IER to select cylinders with less errors, and (iii) outputting the external force vector using the selected cylinders. Experiments were conducted using an instrumented hydraulic arm. Results indicate that our framework enables to improve the relative measuring accuracy, independently of various postural and kinematic conditions.

We understand that primitive static states (PSS), which we have defined, are useful for work or skill analysis, but PSS are not useful for operational support because PSS determine with using only on-off information. In this study, we propose a work state identification for operational support, which includes an identification of the most basic construction work property by modifying PSS and an extraction of operational support providing flag by applying modified-PSS to time-series data. Experiments in transporting and removing tasks using our hydraulic dual arm system indicated that operator supports based on the proposed identification method improves work performance, including decreasing the time taken to complete a task, and the number of error operation, and reducing the moving distance of end-effector after breaking an object.

This paper reports a newly developed hydraulic dual robotic arm system for research and development on advanced intelligent construction machinery that has two joysticks to control the two manipulators with twelve degrees of freedom, control valves with load-sensing system, and sensors for detecting lever operation, cylinder oil pressure, and cylinder stroke information. To clarify important points at addressing an intelligent system, we evaluated basic characteristics of hydraulic machine system. Experimental results showed that hydraulic machine system has measurable time delay from lever input to cylinder actuation, which makes a cooperative operation with two arms difficult. We also found that magnitude of cylinder load greatly changes by lever operation and joint angle, which requires a sophisticated external load detecting method.

In complicated work machines represented by Double-Front Construction Machinery, an intelligent system to which can provide operational and cognitive supports for an operator is essential. In construction work, a) work objects change consistently, b) environment is complicated, and c) operation methods are different by operators. We understand that there are so many unknown parameters that state identification technique to cover all is very difficult. Therefore, we think that the state unit in total independence of varieties of environmental condition and operator's skill level condition is important. We designed the state unit - PSS (Primitive Static States) with ON / OFF information of lever input and manipulator load divided into "end-effecter" and "except end-effecter". PSS classified working states in 256 (double front) ways every sampling period. As a result of having experimented with between different operators and between different environments with PSS, we confirmed that PSS have properties "in total independence of varieties of environmental condition and operator's skill".