To smoothly and efficiently enable autonomous mobile robots to move in human crowd environments, path planning based on human movement-prediction, reactive movement toward human movement, and active motion-inducing physical interaction to avoid getting stuck are required. In this study, we developed a reactive, proactive, and inducible proximal crowd navigation (PCN) method that is based on a newly developed inducible social force model (i-SFM). The PCN first reactively generates multiple paths including avoidance, proximal, and inducible physical-touch paths based on the waypoint-i-SFM fused-path planning method, proactively predicts human movement based on i-SFM, selects an optimal path based on the movement cost including robot-movement efficiency and the crowd-invasion index (impulsive and touch forces applied to the humans), and finally reactively moves in the crowd based on the forces applied on it. The proposed PCN method works not only when the robot enters crowds but also when it is already in a crowd and avoids humans who enter the crowd, so we also applied our PCN method to both situations. Simulation and experimental results indicated that our PCN method can adequately predict the movement of human crowds and achieve higher movement efficiency without stuck, compared with a conventional non-proximal navigation methodusing the SFM.

This article explores a scheme of tuning a basic input-output (I/O) gain (BIOG) depending on usage conditions for improving the work performance of human-operated machines. I/O gain tuning is effective for improving operability and workability, but continual and/or huge changes makes operators confuse the machine dynamics, and then degrade the operability. Thus, the proposed tuning system tunes the BIOG at long terms according to comprehensive characteristics of the work content and operator, collected from a control lever input histogram. The target value is specified by a Gaussian distribution, meaning that all ranges of the spring-type control lever are equally used. This approach was chosen because leveling the control lever histogram independently of the operator and work content provides a consistent operational experience, leading to comfortable operations. In this article, BIOG curves are specified as a polygonal line involving a break and saturation point. These points are adjusted by reciprocal conversion of differential areas between the Gaussian distribution and obtained histogram curves. Experimental results obtained with a robot arm indicated that the developed BIOG tuning system could improve time efficiency by decreasing the differential area and increase the subjective usability as compared to a conventional fixed BIOG system. Moreover, the relationship between the histogram and BIOG curve could reveal features of both the work content and the operator's skill level.

Disaster response robots are expected to perform complicated tasks such as traveling over unstable terrain, climbing slippery steps, and removing heavy debris. To complete such tasks safely, the robots must obtain not only visual-perceptual information (VPI) such as surface shape but also the haptic-perceptual information (HPI) such as surface friction of objects in the environments. VPI can be obtained from laser sensors and cameras. In contrast, HPI can be basically obtained from only the results of physical interaction with the environments, e.g., reaction force and deformation. However, current robots do not have a function to estimate the HPI. In this study, we propose a framework to estimate such physically interactive parameters (PIPs), including hardness, friction, and weight, which are vital parameters for safe robot-environment interaction. For effective estimation, we define the ground (GGM) and object groping modes (OGM). The endpoint of the robot arm, which has a force sensor, actively touches, pushes, rubs, and lifts objects in the environment with a hybrid position/force control, and three kinds of PIPs are estimated from the measured reaction force and displacement of the arm endpoint. The robot finally judges the accident risk based on estimated PIPs, e.g., safe, attentional, or dangerous. We prepared environments that had the same surface shape but different hardness, friction, and weight. The experimental results indicated that the proposed framework could estimate PIPs adequately and was useful to judge the risk and safely plan tasks.

Although exists several drivers' gaze direction classifiers to prevent traffic accidents caused by inattentive driving, making this classification while the driver's face is temporarily or permanently occluded remains exceptionally challenging. For example, drivers using masks, sunglasses, or scarves and daily light variations are non-ideal conditions that recurrently appear in an everyday driving scenario and are frequently overlooked by the existing classifiers. This paper presents a single camera framework gaze zone classifier that operates robustly even during non-uniform lighting, non-frontal face pose, and faces undergo temporal or permanent occlusions. The usage of a normalized dense aligned face pose vector, the classification result of a pre-processed right eye area pixels, and the classification result of a pre-processed left eye area pixels is the cornerstone of the feature vector used in our model. The key of this paper is double-folded: firstly, the usage of a normalized dense alignment for a robust face, landmark, and head-pose direction detection and secondly, the processing of the right and left eye images using computer vision and deep learning techniques for refining, modifying, and finally labeling eyes information. Experiments on a challenging dataset involving non-uniform lighting, non-frontal face pose, and faces with temporal or permanent occlusions show each feature's importance towards making a robust gaze zone classifier under unconstrained driving situations.

The lack of sleep (typically <6 hours a night) or driving for a long time are the reasons of drowsiness driving and caused serious traffic accidents. With pandemic of the COVID-19, drivers are wearing masks to prevent infection from it, which makes visual-based drowsiness detection methods difficult. This paper presents an EEG-based driver drowsiness estimation method using deep learning and attention mechanism. First of all, an 8-channels EEG collection hat is used to acquire the EEG signals in the simulation scenario of drowsiness driving and normal driving. Then the EEG signals are pre-processed by using the linear filter and wavelet threshold denoising. Secondly, the neural network based on attention mechanism and deep residual network (ResNet) is trained to classify the EEG signals. Finally, an early warning module is designed to sound an alarm if the driver is judged as drowsy. The system was tested under simulated driving environment and the drowsiness detection accuracy of the test set was 93.35%. Drowsiness warning simulation also verified the effectiveness of proposed early warning module.

The skin is an important organ which enables humans to interact with the unstructured environment around. It is perfectly soft and covers the entire body providing immediate feedback even when that part is not directly in the field of vision. With the human skin as an inspiration, in this paper, we develop a novel completely soft robot skin for tactile sensing. The skin utilizes a new type of material called as Permanent Magnet Elastomer (PME) to replace the traditionally used hard permanent magnet for hall effect based tactile sensors. PME is formed by mixing Neodymium particles in a polymer base and using strong magnetization (up to 6 T) for anisotropy and to achieve strong and complete magnetization. The 6-axis soft PME is a perfect replacement for powerful hard magnets. We also do a thorough analysis of this material by infusing it in different types of silicone and as a result the most suitable combinations are selected. Performance tests show that the sensor can detect minute forces like 0.1 N. Moreover, the hysteresis test is carried out and the hysteresis error for our skin is found to be only 1.402%. An overloading test is also performed by loading the skin up to 64 N to check the robustness. In conclusion, the skin can produce reliable Triaxial force measurements and we present two models of it for smaller and large force range measurements respectively.

Nowadays, the use of electric vehicles is increasing leading to a growing demand for more efficient use of lithium-ion batteries. The state-of-charge (SOC) has been estimated in previous studies to optimize energy management of batteries. For more efficient battery utilization, detecting degradation is important. However, it is difficult for conventional methods to distinguish the effect of the model parameters including different time constants. Identifying model parameters of multiple RC parallel branches, which represent the impedance of wider frequency ranges, is a necessary requirement to detect the degradation of parts. In this study, we present a method for estimating the model parameters of multiple RC parallel branches. We designed the Markov Chain Monte Carlo algorithm by setting a search range limit and moving window, which enable estimation of the model parameters of parallel branches of different time constants. Through validation of the algorithm based on simulation, the model parameters of a third-order circuit were estimated to be within the error range of 15.2 %. In addition, impedance was calculated from the estimated model parameters in the test using a real battery dataset. The error of impedance was less than 10 % from 0.01 to 100 Hz which was sufficiently low to monitor the change of the parameters owing to degradation. As the impedance in the high-frequency band above 0.1 Hz is more likely to change because of degradation, the proposed method can be used to monitor the model parameters that change as a result of degradation.

The underground facilities require to be inspected regularly for long term and safe-use. Several mobile inspection systems are widely used, but some crucial problems with the limited inspection distance still exist due to the unstable wireless communication and cable restrictions in complex facilities. To solve these problems, we propose a wheeled robot chain control system (RCCS) for underground facilities inspection based on visible light communication (VLC) and solar panel receivers. The RCCS consists of a leader and follower robot and can extend the inspection range with special "optical signal relay" technology. VLC can provide not only illumination for inspection but also less attenuation, higher information security, and stronger antielectromagnetic interference capability, compared with conventional radio communication in complex underground facilities. Using solar panels can increase the receiving area of optical signals, environmental adaptability, and system robustness, compared with conventional photodiode (PD) receivers. Moreover, to ensure longer and more stable communication, we develop an optical signal relay mechanism along-side an attenuation control module. We performed several experiments to evaluate the signal waveform, communication quality, effective communication region, and cooperative movement. The results indicated that the proposed system could extend the inspection range with higher communication quality and environmental adaptability than VLC-based system with PD receivers.

Professional drivers are required to safely transport passengers and/or properties of customers to their destinations, so they must keep being mentally and physically healthy. Health problems will largely affect driving performance and sometimes cause loss of consciousness, which results in injury, death, and heavy compensation. Conventional systems can detect the loss of consciousness or urgently stop the vehicle to prevent accidents, but detection of symptoms of diseases and providing support before the driver loses consciousness is more reasonable. It is challenging to earlier detect symptoms with high confidence. Toward solving these problems, we propose a new method with a multi-sensor based driver monitoring system to detect cues of symptoms quickly and a verbal interaction system to confirm the internal state of the driver based on the monitoring results to reduce false positives. There is almost no data that records abnormal conditions while driving and tests with unhealthy participants are dangerous and ethically unacceptable, so we developed a system with pseudo-symptom data and did outlier detection only with normal driving data. From data collection experiments, we defined the confidence level derived from cue signs. The results of evaluation experiments showed that the proposed system worked well in pseudo headache and drowsiness detection scenarios. We found that signs of drowsiness varied with individual drivers, so the multi-sensor based driver monitoring system was proved to be effective. Moreover, we found that there were individual differences in how the cue signs appeared, so we can propose an online re-training method to make the system adapt to individual drivers.

For the safety, underground facilities are required to be inspected regularly, especially with image analysis. Traditional wireless and wired transmission techniques have a weakness of limited transmission range in narrow underground environments. In this study, a new image transmission method based on visible light communication (VLC) has been thus proposed. Two types of detectors as an image signal receiver have been tested and discussed in the following experiments. The photodiodes (PDs) are widely used as a common image signal detector in VLC technology, but image signal detection using solar panels (SPs) has not been studied. PDs have a higher sensitivity and faster response time but a limited detection area and high cost. Besides, PDs require the lens to focus light. On the other hand, SPs have much larger optical signal receiving areas and stronger optical signal capture capabilities. They can realize lens-free detection and are inexpensive. These features of PD were firstly verified in experiments with several receiving areas and angles of detectors. The experimental result revealed that PD had better image detection and recovery capabilities than those of SP. Then, we tbund that a larger receiving area obtained by using double PDs/SPs improved the brightness of the restored image. In a supplementary experiment, the influence of different RGB optical components on VLC, especially the VLC-based image transmission, has been investigated by using two-dimensional Fourier transform frequency analysis. We found that the red optical component significantly increased the intensity and energy of the restored image as the image low-frequency signals were larger than the restored image using ordinary mixed white light, and moreover, the blue optical component decreased the low-frequency part of the image. (C) 2021 Optical Society of America under the terms of the OSA Open Access Publishing Agreement

Conventional soft robotic grippers have been developed based on soft pneumatic actuators or using smart soft materials, but they need external pressure sources, are easily deteriorated or used in millimetric-scale, which leads to increase of the independency, instability, and vulnerability. In this study, we propose an EPM-MRE (Electropermanent Magnet-Magnetorheological elastomer) actuation system, which strengthens the independency and stability of soft actuation by non-contact magnetic force. We established its fundamental principle and investigated parameter design through developing a suction cup as a robotic gripper. We prototyped an EPM-MRE actuated suction cup based on magnetic-charge and tensile modeling as well as optimized the structure of EPM by introducing axisymmetric EPM with frustum pole and suction cup by introducing bi-silicone structure to uniformly activate MRE membrane. The activation of EPM by different current pulses and suction force affected by contact shapes and air gaps were tested. Evaluations showed that the suction cup could be activated with 10 ms and generate a maximum suction force of 9.2 N at a steady state with 0 J energy consumption. In conclusion, the EPM-MRE soft actuation system can be used as a robotic gripper.

Teleoperation of humanoid robots for disaster response tasks, controlled by a master slave system using a head-mounted display, requires a detailed frame of view for precise operation. We previously investigated an automatic zoom method, and images zoomed when the robot arm entered the work area could improve teleoperation performance. However, automatic zooming failed to consider the operator's intentions, including zoom levels and timing, causing high mental workload and motion sickness for operators. Therefore, this study proposes a zoom method with a low mental workload and reduced level of motion sickness. Motion sickness is less likely to be induced under active conditions, wherein the operator can manually control the image. Moreover, upper limbs should not be used to operate the zoom feature because such use can disturb the master slave operation. Furthermore, a zoom method should reflect the operator's intentions, including zoom levels and timing. Thus, we develop a zoom method based on head movement, which can be regarded as typical gaze movement. The results of experiments using a real robot system indicated that the proposed method could reduce the mental workload and work time without inducing motion sickness, compared with zoom operation using a foot pedal interface.

How to improve task performance and how to control a robot in extreme environments when just a few sensors can be used to obtain environmental information are two of the problems for disaster response robots (DRRs). Compared with conventional DRRs, multi-arm multi-flipper crawler type robot (MAMFR) have high mobility and task-execution capabilities. Because, crawler robots and quadruped robots have complementary advantages in locomotion, therefore we have the vision to combine both of these advantages in MAMFR. Usually, MAMFR (like four-arm four-flipper robot OCTOPUS) was designed for working in extreme environments such as that with heavy smoke and fog. Therefore it is a quite necessary requirement that DRR should have the ability to work in the situation even if vision and laser sensors are not available. To maximize terrains adaption ability, self-balancing capability, and obstacle getting over capability in unstructured disaster site, as well as reduce the difficulty of robot control, we proposed a semi-autonomous control system to realize this compound locomotion method for MAMFRs. In this control strategy, robot can explore the terrain and obtain basic information about the surrounding by its structure and internal sensors, such as encoder and inertial measurement unit. Except that control system also can recognize the relative positional relationship between robot and surrounding environment through its arms and crawlers state when robot moving. Because the control rules is simple but effective, and each part can adjust its own state automatically according to robot state and explored terrain, MRMFRs have better terrain adaptability and stability. Experimental results with a virtual reality simulator indicated that the designed control system significantly improved stability and mobility of robot in tasks, it also indicated that robot can adapt complex terrain when controlled by designed control system.

Four-arm four-flipper disaster response robot called OCTOPUS, which has 26 degrees of freedom (DOF), has been developed to engage in complex disaster response work. OCTOPUS adopts a multi-operator single-robot (MOSR) control system, and two operators manually control the robot. The pattern of control-authority allocation (CAA) for two operators largely affects performance of MOSR systems, so this study conducts usability tests in various CAAs, and derives a reasonable CAA pattern. So far, there are no uniform standard allocation rules for flippers and crawlers in multi-limb robots like OCTOPUS, therefore we specify five CAA patterns for investigation. Three fundamental tasks that whole body of the robot must be cooperatively controlled were conducted to test the usability in each CAA pattern. From the results of analysis, we found that pattern 1, which one operator controls front two arms, flippers, and crawlers, and another operator controls remaining back parts, had best scores for all tasks, as well as pattern 2 had the best distribution of workload between two operators.

Traffic complexity is one of the factors affecting driver workload. In order to study the relationship between traffic complexity levels and workload, a designed experiment is required, especially to vary traffic flow parameters systematically in a simulated environment. This paper describes the experimental design of a simulator study for developing a computational model to estimate the behavior of driver workload based on traffic complexity. Driving simulators allow creating and testing different traffic scenarios and manipulating independent variables to improve the quality of data, as compared to real world experiments. Physiological responses such as heart rate, skin conductance, and pupil size have been found to be related to workload. By adapting a data-driven method, we integrated electrocardiography sensors, electro-dermal activity sensors, and eye-tracker to acquire driver physiological signals and gaze information. Preliminary results show a positive correlation between traffic complexity levels and corresponding physiological responses, performance, and subjective measures.

The gas pipeline networks cover approximately 200,000, km in urban region in Japan. The risk caused by severe leakage, cracking, or corrosion will lead to a big accidence for human. Therefore, the gas pipeline robots for regular inspection and maintenance are very essential for the safe operation and supply of gas. In this paper, as a fundamental study of pipeline inspection robots, a new small wireless tracked robot for gas pipeline inspection and maintenance has been developed. Due to robot’s special kinematics, it is capable of moving in straight pipe, turn at elbow and also climb the inclined pipe flexibly and efficiently. Compared with previous robots, it owns two working modes: self-navigation (automated control) and manual control mode for pipeline inspection. Moreover, a preliminary experiment has been implemented with a typical M-pipeline in 10, cm diameter/4, m length. The experiment results revealed that self-navigation could improve the efficiency of inspection.

Disaster response crawler robot OCTOPUS has four arms and four flippers for better adaptability to disaster environments. To further improve the robot mobility and terrain adaptability in unstructured terrain, we propose a new locomotion control method called compound motion pattern (CMP) for multi-limb robots like OCTOPUS. This hybrid locomotion by cooperating the arms and flippers would be effective to adapt to the unstructured terrain due to combining the advantages of crawling and walking. As a preliminary study on CMP, we proposed a fundamental and conceptual CMP while clarifying problems in constructing CMP, and developed a semi-autonomous control system for realizing the CMP. Electrically-driven OCTOPUS was used to verify the reliability and correctness of CMP. Results of experiments on climbing a step indicate that the proposed control system could obtain relatively accurate terrain information and the CMP enabled the robot to climb the step. We thus confirmed that the proposed CMP would be effective to increase terrain adaptability of robot in unstructured environment, and it would be a useful locomotion method for advanced disaster response robots.

In teleoperation of heavy machines, work efficiency will be 50% lower than manned operation because operators cannot obtain effective information about work sites due to the limitation of current monitoring systems. Operators would have opportunities to obtain such information before work (about a week required to introduce teleoperation systems) and during work. As a fundamental study to support operator's spatial cognition, we developed views to provide spatial information of work sites before work. Humans have cognitive maps which are created based on knowledge acquired from survey and route perspectives. To make operators acquire the above two knowledge, we provide a bird's-eye view that can be changed by operators to acquire a knowledge from survey perspective, and a view from operator's viewpoint that can be changed by operator's intention to acquire a knowledge from route perspective. To evaluate two pre-offering views, we preformed experiments using a virtual reality simulator. The results indicated that a view to acquire a knowledge from survey perspective could help operators plan totally and one to acquire a knowledge from route perspective could help operators plan locally, and could increase work efficiency.

This paper proposes a basic near-environmental recognition framework based on groping for risk-tolerance disaster response robot (DRR). In extreme disaster sites, including high radiation and heavy smog, external sensors such as cameras and laser range finders do not work properly, and such sensors may be broken in accidents in the tasks. It is hoped that DRRs can continue to perform tasks, even if the external sensors cannot work, and at least, they can safely evacuate from the site. In this preliminary study, for recognizing near environments without using external sensors, we proposed a groping method. In this method, a robot actively touches the environment using arms or other movable parts, records the contact information, and then reconstructs a three-dimensional local map around the robot by the detected information, e.g., robot arm's position and reactive force. The proposed groping system can recognize the existence of three situations, such as an object, step, and pit, and those geometry, by exploring the designated space using arms. The groping strategy was designed considering both robot specification, time limitation, and required resolution. Experiments were performed using four-arm and four-crawler robot OCTOPUS. The results indicate that the proposed framework could recognize step, pit, and object, and calculate the position and size of the object, and confirm that the robot successfully removed the object on the basis of groped data.

Intelligent vehicles operating in different levels of automation require the driver to fully or partially conduct the dynamic driving task (DDT) and to conduct fallback performance of the DDT, during a trip. Such vehicles create the need for novel human-machine interfaces (HMIs) designed to conduct high-level vehicle control tasks. Multimodal interfaces (MMIs) have advantages such as improved recognition, faster interaction, and situation-Adaptability, over unimodal interfaces. In this study, we developed and evaluated a MMI system with three input modalities
touchscreen, hand-gesture, and haptic to input tactical-level control commands (e.g. lane-changing, overtaking, and parking). We conducted driving experiments in a driving simulator to evaluate the effectiveness of the MMI system. The results show that multimodal HMI significantly reduced the driver workload, improved the efficiency of interaction, and minimized input errors compared with unimodal interfaces. Moreover, we discovered relationships between input types and modalities: location-based inputs-Touchscreen interface, time-critical inputs-haptic interface. The results proved the functional advantages and effectiveness of multimodal interface system over its unimodal components for conducting tactical-level driving tasks.

This paper proposes a practical scheme for measuring the mass of an object grasped by the end-effector of a large-scale hydraulic manipulator. Such a measurement system requires high accuracy and robustness considering the nonlinearity and uncertainty in hydraulic pressure-based force measurement during rigorous outdoor work. It is thus difficult to precisely model system behaviors and completely remove error force components (white-box modeling) under such conditions, so our scheme adopts a less-error data selection approach to relatively improving the accuracy and reliability of the measurand (gray-box modeling). It first removes dominant error forces, i.e., gravity and dynamic friction forces, then defines the on-load state by evaluating measurement conditions to omit data in indeterminate conditions, then extracts data during the object-grasp state identified by a grasp motion model and removes high-frequency components by a simple low-pass filter, and finally integrates data from multiple sensors using the posture-based priority and averages all selected data. Evaluation experiments were conducted using an instrumented hydraulic arm. Results indicate that our scheme can precisely measures the mass of the grasped object under various detection conditions with fewer errors.

A new magnetorheological piston head design inspired by toroidal electromagnets was proposed in previouswork as an alternative to conventional annular dampers. A prototype with a circular valve array integrated inside the piston head was built and tested to measure its passive performance. The mechanical, electromagnetic, and hydraulic models used in the new design were explained, and the relevant parameters of the actuator were analyzed to construct a mathematical model to estimate its performance. These works showed the feasibility of the concept and its potential as an alternative to current damper technology, but lacked a baseline for comparison. This paper reviews and widens this groundwork. It adds a magnetic finite element method study to address the previously found leakage, and a new set of experiments, including a force controller, to compare its performance against a conventional annular piston head. The new findings show how the current force limitations can be overcome by striking a balance between the coil space and the size of electromagnet cores to achieve the performance of current dampers. They also highlight its potential in force control applications to provide a wider range of customization options, such as number and size of holes, electromagnets, and coils, and a better use of the active area of the gap.

The gas pipelines need to be inspected in the case of existence of pipeline's inner problems. Usually, inspection for leakage of gas pipeline has been carried out by wired probes or wired robots equipped with a CCD camera. However, the above inspection approaches are restricted by the cable in the pipe. This study thus focuses on an inspection approach based on microwave communication. However, obtaining the transmission properties (i.e., transmission loss, RSSI (received signal strength indication), maximum transmission distance, etc) in gas pipeline is crucial to determine which frequency is feasible to the further development. In this paper, a communication system for microwave communication in 2.4 and 5.8 GHz band is developed. Then, the transmission loss and RSSI of these microwaves in 2.5 x 100 cm (straight), 5 x 100cm (straight) and 5 x 200cm M-type curved steel gas pipe (with L-junctions) are then measured. The experimental results revealed that the microwave in 2.4 GHz band owns much stronger communication capability in complicated gas pipeline in consideration of few transmission loss, and three approximate linear functions could be modeled on the basis of the measured data.

The aim of this study is to provide a machine operator with enhanced visibility and more adaptive visual information suited to the work situation, particularly advanced unmanned construction. Toward that end, we propose a method for autonomously controlling multiple environmental cameras. Situations in which the yaw, pitch, and zoom of cameras should be controlled are analyzed. Additionally, we define imaging objects, including the machine, manipulators, and end points; and imaging modes, including tracking, zoom, posture, and trajectory modes. To control each camera simply and effectively, four practical camera roles with different combinations of the imaging objects and modes were defined: overview machine, enlarge end point, posture-manipulator, and trajectory-manipulator. A real-time role assignment system is described for assigning the four camera roles to four out of six cameras suitable for the work situation (e.g., reaching, grasping, transport, and releasing) on the basis of the assignment-priority rules. To test this system, debris-removal tasks were performed in a virtual reality simulation to compare performance among fixed camera, manual control camera, and autonomous control camera systems. The results showed that the autonomous system was the best of the three at decreasing the number of grasping misses and erroneous contacts and simultaneously increasing the subjective usability and time efficiency.

In this study, a fundamental method to tune a basic input/output gain (BIOG) according to usage conditions is proposed for improving work performance in human-machine systems. For improving operability and workability, an I/O gain tuning is effective, but frequent and drastic changes degrade the operability by confusing the machine dynamics. The proposed tuning system adjusts a BIOG at long intervals on the basis of comprehensive features in operator and work content obtained from histogram of control lever input. The target value is set to the normal distribution, meaning that all ranges of the control lever are evenly used in a spring-type lever, as leveling histogram independent of an operator and work content provides the consistent operational feeling, which leads to comfortable operability. To meet practicality and effectiveness, a BIOG curve is set to a polygonal line involving a break point and a saturation point that are tuned by equivalent transform of area differences between the obtained histogram and normal distribution curves. Results of experiments conducted by using a hydraulic arm system showed that the proposed BIOG tuning system improves time efficiency by reducing the area difference while increasing the subjective usability compared with a conventional fixed BIOG system.

Human's ability to perceive intent plays a crucial role in achieving smooth and efficient navigation. At the present state, even with the state-of-the-art anthropomorphic robots, displaying human-like non-verbal communication (kinesics) is a challenging task. This poses a significant difficulty in performing legible navigation behavior for robots. In this paper, we look into light (turn indicator) and screen (arrow indicator) indicators as a means of overcoming the shortcomings of the robot's non-verbal communication abilities. Our results show a statistically significant improvement in perceived comfort, predictability, and performance with the use of light indicators.

We developed a four-arm four-crawler advanced disaster response robot called OCTOPUS. Disaster response robots are expected to be capable of both mobility, e.g., entering narrow spaces over very rough unstable ground, and workability, e.g., conducting complex debris-demolition work. However, conventional disaster response robots are specialized in either mobility or workability. Moreover, strategies to independently enhance the capability of crawlers for mobility and arms for workability will increase the robot size and weight. To balance environmental applicability with the mobility and workability, OCTOPUS is equipped with a mutual complementary strategy between its arms and crawlers. The four arms conduct complex tasks while ensuring stabilization when climbing steps. The four crawlers translate rough terrain while avoiding toppling over when conducting demolition work. OCTOPUS is hydraulic driven and teleoperated by two operators. To evaluate the performance of OCTOPUS, we conducted preliminary experiments involving climbing high steps and removing attached objects by using the four arms. The results showed that OCTOPUS completed the two tasks by adequately coordinating its four arms and four crawlers and improvement in operability needs.

Autonomous vehicles will significantly change the existing driver-vehicle relationship, since only a destination input from the human driver will suffice. However, reduced degree of human-control could result in lack of driving pleasure and excitement. Thus, we proposed a method to increase the flexibility in controlling of an autonomous vehicle by allowing the driver to control its lateral and longitudinal motions with a time lag. We first derived a set of vehicle movements to improve driver experience and related them to a set of control functions that the driver can input. We then created two types of driver-vehicle interfaces (DVIs) for vehicle control; a haptic interface with kinesthetic and tactile feedback, and a hand-gesture interface with augmented reality feedback. The joystick-type haptic interface provides feedback on driver input by dynamically varying its degrees of freedom through controlling the current supplied to axis motors, and through vibration motors. The gesture interface is based on Leap Motion controller and provides visual feedback to driver. We conducted driving experiments in a VR simulator using twenty drivers to evaluate the effectiveness of these DVIs. The results showed that haptic interface significantly reduced the average input time and input error, and drivers preferred the haptic interface due to its ability to provide immediate, active, and passive feedback.

This work focuses on the analysis of the performance parameters of a new magneto-rheological device. Previously, a mathematical model of the actuator head was introduced; however, the complex relations between its parameters made its optimisation challenging, causing the device not to reach its full potential. In this work, the previous model is updated to fit a revised version of the toroidal design, as well as an annular design for comparison. The relations between different parameters are studied to find their trade-offs, and understand how they affect the performance of the actuators. Finally, an optimisation based on an iterative search for valid permutations of parameters, is used to find optimal combinations. Two prototypes are built and tested to validate the results of the optimisation. The study revealed the critical parameters of each design, which mostly depend on the relations between the magnetic flux density, and the active area exposed to the MRF; and successfully optimised the performance of the devices. However, work needs to be done to create a more complete tool with concrete guidelines for specific applications.

This paper presents a virtual reality (VR) simulator for four-arm disaster response robot OCTOPUS, which has high capable of both mobility and workability. OCTOPUS has 26 degrees of freedom (DOF) and is currently teleoperated by two operators, so it is quite difficult to operate OCTOPUS. Thus, we developed a VR simulator for training operation, developing operator support system and control strategy. Compared with actual robot and environment, VR simulator can reproduce them at low cost and high efficiency. The VR simulator consists of VR environment and human-machine interface such as operation-input and video-and sound-output, based on robot operation system (ROS) and Gazebo. To enhance work performance, we implement indicators and data collection functions. Four tasks such as rough terrain passing, high-step climbing, obstacle stepping over, and object transport were conducted to evaluate OCTOPUS itself and our VR simulator. The results indicate that operators could complete all the tasks but the success rate differed in tasks. Smooth and stable operations increased the work performance, but sudden change and oscillation of operation degraded it. Cooperating multi-joint adequately is quite important to execute task more efficiently.

Legible robot behavior is a key element for smooth and efficient navigation among humans. However, at present, even the state-of-the-art robots cannot communicate its internal state of directional intent by displaying human-like non-verbal cues. In this paper, we explore various modes for communicating directional intent of a robot across three different scenarios as a means of overcoming the shortcomings of the robot's non-verbal communication abilities. Specifically, we look into turn indicators, display indicators, and their combinations with sound and investigate their effectiveness across different passing scenarios. Our study shows us that using auxiliary communicating methods significantly improves the perceived feelings of our dependent measures. Further, communicating intention also helps in improving cooperation. However, the effectiveness greatly varies with the modes and the passing scenarios. In the case of 90 degrees crossing scenario, even though participants have positive perceived feelings, this does not necessarily translate into smooth and efficient navigation.

The advantages of mechanical compliance have driven the development of devices using new smart materials. A new kind of magnetorheological piston based on a toroidal array of magnetorheological valves, has been previously tested to prove its feasibility. However, being an initial prototype its potential was still limited by its complex design, and low output force. This study presents the revisions done to the design with several improvements targeting key performance parameters. An improved annular piston design is also introduced as comparison with conventional devices. The toroidal and annular piston head prototypes are built and tested, and their force performance compared with the previous iteration. The experimental results show an overall performance improvement of the toroidal assembly. However, the force model used in the study still fails to accurately predict the magnetic flux at the gaps of the piston head. This deviation is later verify and corrected using a FEM analysis. The force performance of the new toroidal assembly is on par with the commonplace annular design. It also displays a more linear behaviour, at the expense of lower energy efficiency. Finally, it also shows potential for a greater degree of customisation to meet different system requirements.

Compliant actuation is an indispensable element for safe physical human robot interaction. However, there is a lack of devices, which can integrate the high power density of hydraulic actuators with intrinsically safe mechanisms. This drove the development a new type of hydraulic magnetorheological piston. This device includes a novel toroidal array of magnetorheological valves in its head. In previous studies, the system performance was evaluated only as a traditional damping system. Now, the piston is connected to a pump to create a hydraulic compliant actuator. This novel compliant piston is capable to control the piston speed and force independently by using the pump and electromagnet voltages respectively. In this way, different combinations of these parameters can be used to achieve diverse system properties; e.g. low response time or energy efficiency. Several experiments are conducted to evaluate its performance, including force, friction, speed, and step response. The results display the potential of the devices to be used as an active system for compliant hydraulic robotic applications. They also hint to the possibilities to improve by using a more sophisticated control system for its speed and force.

Disaster response robot with four arms and flippers OCTOPUS has high mobility and task-execution capabilities. Owing to the higher number of degrees of freedom, OCTOPUS is controlled by two operators, however this kind of robots is inherently difficult to be operated. To design easy-to-use human machine interfaces and intelligent control systems, we need to analyze and quantify a reasonable operation strategy in multi-operator control systems. Thus, three different types of essential disaster response tasks were conducted by using OCTOPUS and we analyzed results of operations and work performance, by focusing on each operator and each pair. As the results, we derived basic operation strategies as follow; operators with higher number of simultaneously-operated joints (Ns) can control OCTOPUS more smoothly, and pairs with higher rate of cooperated operations (Rc) can finish tasks more efficiently. We also found that Ns and Rc can be used to quantify operational skills. Revealed strategies and parameters could be useful to design new human-machine interface and intelligent control system.

Autonomous vehicles would make the future roads safer by keeping the human driver out of the loop. However, reduced degree of human-control could result in loss of the feeling of driving for some drivers. Therefore, in this study we proposed a method of interaction between the driver and autonomous vehicle by allowing the driver to control the vehicle's lateral and longitudinal motions. We adopted hand gestures as input modality because it can reduce driver's visual and cognitive demands. We first derived seven fundamental vehicle maneuvers to improve driver experience, and related them to seven independent hand gestures. We then created a hand gesture interface to control an autonomous vehicle, using Leap Motion as the gesture recognition platform. We conducted driving experiments involving twenty drivers in a virtual reality driving simulator to investigate the effectiveness of this interface for vehicle control. We evaluated the driving experience and drivers' opinions regarding the gestural interface. The results proved that semi-autonomous controlling using the hand gesture interface significantly reduced drivers' perceived work-load.

In unmanned construction, work efficiency is lower than that in manned construction due to lack of visual information. Thus, we previously developed an autonomous camera control system to provide various visual information suited to work states through multiple displays. However, that system increased the cognitive load on operators, and required them to have much experience to choose appropriate views for various situations. Next, we should investigate the degree of effectiveness for each view in a certain state. Thus, in this study, we analyzed gaze patterns to clarify which are the displays that operators often watch in work states, i.e., moving, grasping, transport, and releasing. We then derived which gaze patterns have higher work performance, including time efficiency and safeness. We clustered gaze patterns using Ward's method, which is a criterion applied in hierarchical clustering. To evaluate the objective of this study, we conducted experiments involving debris transport tasks, using a virtual reality simulator. The results indicated that gaze patterns differed in operators and we found that better time efficiency related to specific gaze patterns for each work state.

In this paper, we analyzed spatial perception ability based on human eyesight for teleoperation. Recognizing the space relationship between different objects by human eyesight from a two dimensional monitor quickly and correctly is a hard task. On the basis of the analysis about visual perception error, we make some assumptions about the alignment of multi monitors. The reasons which cause visual perception error were then investigated using our virtual reality simulator,and experiments were implemented to verify our assumptions. Results show when objects were fixed, continuously moving the viewpoint around the fixed objects, the position of viewpoint obviously affected spatial perception accuracy of human subjects. We also found that subjects tend to estimate object position deeper than its real position.

In recent years there has been interest in the development of magnetorheological actuators for robotic applications with adjustable compliance. In this paper we explore the conceptualization, design and test of a new kind magnetorheological piston head design. Inside the head of this new damper a circular assembly of magnetorheological valves is used to control the damping force of the piston. The results show the feasibility of the new design, and its potential for compliant robotic applications.

In this study, we focused on creating different scenarios and traffic conditions in a driving simulator to evaluate the differences in driving experience in conventional and autonomous vehicles. We conducted experiments using two groups of drivers and evaluated their subjective workload and preference for each driving method for different traffic conditions and scenarios.

In unmanned construction, operators watch multi views to know situations at a disaster site. However, such a vision system requires much cognitive load of the operators to determine monitors and its parts which they should watch according to situation. In order to decrease cognitive load, we developed a system to provide appropriate monitors and its parts by inducing visual attention using augmented reality.

This paper proposes a sensor model in three dimensional (3D) space, where the height of sensor above ground is variable and the position of sensor on the ground is limited. Traditional sensor models treat the sensor placement area as a film and there is no placement limits as well. As a consequence, the number of sensors is very large and the performance is not such ideal as simulation. In this paper, we develop a novel model for sensor placement, which makes sensors possible to be set at different height above ground, so that the coverage rate can be analyzed in 3D space and the simulation result can be more dependable. We also propose a corresponding 3D coverage rate evaluation method to be used for our proposed 3D model. In order to test our proposed model and evaluation method, the system is designed and numerical and graphical simulators are developed. Experiments are conducted to quantify trade-off relationship between coverage rate and cost, i.e., the number of sensors, which can make sensor placement strategy more practical. From the experimental results, we confirmed that the proposed 3D model and evaluation method is effective in a 3D space.

Unmanned construction machines are used after disasters. Compared with manned construction, time efficiency is lower because of incomplete visual information, communication delay, and lack of tactile experience. We have developed an autonomous camera control system to supply appropriate visual information. In order to let operator easy to avoid unexpected collision, we first introduce the frame for each viewport according to its contents. And then, we exchange the danger view with an overlook view. The purpose of this study is to attract operators' attention to the danger area when there is a probable collision if operator won't stop or change their action. Besides, we also want to reveal the relationship between gaze habit and performance. The experimental results conducted using our virtual reality simulator confirms that operator tends to watch danger view when it appears. In addition, work strategy may influence the danger avoidance and work time efficiency in opposite way.

Intelligent vehicles capable of autonomous driving will be commercially available in near future. To formulate a beneficial relationship between driver and vehicle, it is important to analyze how the drivers would react to autonomous vehicles compared to human-driven (conventional) vehicles. In this study, we focused on analyzing individual driving experience in several road conditions for autonomous and conventional vehicles among experienced and novice drivers. We first developed a simplified driving simulator that can connect arbitrary interfaces, create virtual environments consisting of scenarios and events that drivers encounter in real-world driving, and implement fully autonomous driving. We then conducted experiments to clarify differences of driving experiences for autonomous driving between the two groups. The experimental results showed that experienced drivers opt for conventional driving mainly due to the flexibility and driving fun it offers, while novices tend to prefer autonomous driving due to its inherent easiness and safety. An in-depth analysis indicated that drivers preferred to use both the driving methods interchangeably depending on the road and traffic conditions.

Passive compliance is useful for robotic arms to ensure their safety. Often springs are used, but they are problematic because they reduce the achievable accelerations and can lead to underdamped oscillations. Torque limiters enhance the safety, but usually the torque limit cannot be adjusted to a desired torque. Electronically adjustable torque limiters, also known as series clutch actuators, have several benefits, especially for robotic arms, but they also have severe limitations. This paper suggests incorporating series clutch actuators into a gravity compensated arm. Consequently, gravity should not limit the isotropically achievable force anymore and in the case of power outage the arm keeps its position. The benefits and limitations of a series clutch actuator in a gravity compensated arm are discussed, and a prototype of such an arm is presented. Commercially available magnetic friction clutches are used. Preliminary experiments demonstrate that the safety can be increased.

This paper describes a combined tactile sensor and haptic interface that can change its stiffness using magnetorheological fluids (MR fluid). The tactile sensor consists of 6 distributed capacitive sensors that can sense the location and the amount of applied force. Above the sensors is a chamber filled with MR fluid. By changing the magnetic field, the hardness of the MR fluid, and thereby of the haptic interface, can be changed. Fast changes of the magnetization direction lead to a sensation of vibration. The resulting device can be used for novel haptic input devices or for robotic grippers. A prototype device has been constructed, and the effects of the varying magnetic field and the resulting varying stiffness of the MR fluid on the distributed force sensing with the capacitive sensors has been evaluated. We discovered that the measured forces vary very little with changes in the strength of the magnetic field.

In densely populated scenarios with cramped spaces, it is very difficult to achieve safe and efficient navigation without cooperation from humans. One way in which we can seek cooperation from humans is by using contact to influence them to give way. However, such a method may incur certain psychological implications and therefore requires an acceptability check to ensure whether such action is acceptable or not. For this purpose, we investigate the participant subjective response towards robot-initiated touch during the course of navigation. We conducted a 2 (robotic experience vs. none) x 2 (warning vs. none) between-subject experiment with 44 people in which a mobile robotic platform exerted contact on an unaware and obstructing participant to make way towards its goal. Our results show that prior experience with robots produces slightly better response even though the results are not statistically significant. However, a verbal warning prior to contact yielded much more favorable response. In general, the participants did not find contact to be uncomfortable and were not opposed to robot-initiated contact if deemed necessary.

A new design of a magnetorheological piston prototype intended for passive or active force control in robotic applications for human robot interaction is introduced. It is based in a novel toroidal array of valves, contained within the piston head, which are used to control the output force of the actuator in order to achieve a high degree of reliability, size efficiency, and safety, by exploiting the material properties of magnetorheological fluids and permalloy metals. This paper describes the main points in the development of the magnetorheological piston prototype, the mathematical modelling of the magnetic circuit, and the results of the experiments conducted using a universal testing machine to evaluate the passive performance of the prototype. Results show the feasibility and performance of the new toroidal magnetic circuit of the magnetorheological hydraulic piston prototype. Improvements in order to be able to test the active performance of the design together with a pump setup are proposed.

Unmanned construction machines are used after disasters. Compared with manned construction, time efficiency is lower because of incomplete visual information, communication delay, and lack of tactile experience. We have developed an autonomous camera control system to supply appropriate visual information. In order to inform operators the potential data in the views, we introduced several augmented reality (AR) elements to induce visual attention to suitable images in a scenario. The purpose of this study is to develop an attention inducement system using AR technique and evaluate it under different conditions. We first improve our autonomous camera control system and then implement several kinds of AR items suitable for each work situation. The experimental results conducted using our virtual reality simulator confirm that AR items can supply a better positional relationship between machine and objects in the environment which makes operator handle the conditions as experienced. The results from eye-tracker data also indicate that AR items can induce visual attention to images suitable for situations.

In densely populated scenarios with cramped spaces, it is very difficult to achieve safe and efficient navigation without cooperation from humans. One way in which we can seek cooperation from humans is by using contact to influence them to give way. However, such a method may incur certain psychological implications and therefore requires an acceptability check to ensure whether such action is acceptable or not. For this purpose, we investigate the participant subjective response towards robot-initiated touch during the course of navigation. We conducted a 2 (robotic experience vs. none) x 2 (warning vs. none) between-subject experiment with 44 people in which a mobile robotic platform exerted contact on an unaware and obstructing participant to make way towards its goal. Our results show that prior experience with robots produces slightly better response even though the results are not statistically significant. However, a verbal warning prior to contact yielded much more favorable response. In general, the participants did not find contact to be uncomfortable and were not opposed to robot-initiated contact if deemed necessary.

A practical operator support system for disaster response work using construction machinery is applied to object-break operations using a dual-arm machine. An object-break operation, which separates an object into two different parts by applying massive pulling force, requires skillful force and trajectory controls. Operators typically have trouble visually perceiving the movement of the manipulator because the target object is strongly restrained. As a result, after the object has been separated into two pieces, the operator may continue performing lever operations because of unavoidable perception reaction time and the manipulator may collide with the surrounding environment. A control input cancellation system based on object-break identification is proposed to reduce the time difference between the object break and operation stoppage. A support system for disaster response situations must be safe and efficient and accommodate operators with various skill levels; thus, common physical phenomena related to the object breakage should be exploited. The system cancels control inputs to stop the movement of the manipulator when an object-break flag is detected, which is defined as a situation where the force applied to each joint of a manipulator decreases rapidly when a lever operation and load on the inside of the grapple are detected. Demolition experiments were conducted using a hydraulic dual-arm system. The results indicate that the displacement of the end-point after the object breaks is greatly decreased and the completion time of the task is reduced as well.

In this paper, an object grasp motion, which is a requisite condition to make a demolition machine grasp an object, is pragmatically modeled, considering accurate and robust identification. Grasping an object is a highly difficult task that requires safe and precise operations, particularly in disaster response work. Identifying a grasp or non-grasp state is essential for providing operational support. These types of outdoor machines lack visual and tactile sensors, so pragmatically available lever operation and cylinder pressure sensors are adopted as parameters for modeling. The grasp motion is simply defined by using sequential transitions of the on-off state of the operation signal and cylinder pressure data for the grapple and the manipulator. The results of experiments conducted to transport objects using an instrumented hydraulic arm indicated that the modeled grasp motion model effectively identifies a grasp or non-grasp state with high accuracy, independently of operators and work environments.

This paper proposes a quantification method for a comprehensive work flow in construction work for describing work states in more detail on the basis of analyzing state transitions of primitive static states (PSS), which consist of 16 symbolic work states defined by using on-off state of the lever operations and joint loads for the manipulator and end-effector. On the basis of the state transition rules derived from a transition-condition analysis, practical state transitions (PST), which are common and frequent transitions in arbitrary construction work, are defined. PST can be classified into essential state transition (EST) or nonessential state transitions (NST). EST extracts common phases of work progress and estimates positional relations between a manipulator and an object. NST reveals wasted movements that degrade the efficiency and quality of work. To evaluate comprehensive work flows modeled by combining EST and NST, work-analysis experiments using our instrumented setup were conducted. Results indicate that all the PSS definitely changes on the basis of PST under various work conditions, and work analysis using EST and NST easily reveals work characteristics and untrained tasks related to wasted movements.

A method to autonomously control multiple environmental cameras, which are currently fixed, for providing more adaptive visual information suited to the work situation for advanced unmanned construction is proposed. Situations in which the yaw, pitch, and zoom of cameras should be controlled were analyzed and imaging objects including the machine, manipulator, and end-point and imaging modes including tracking, zoom, posture, and trajectory modes were defined. To control each camera simply and effectively, four practical camera roles combined with the imaging objects and modes were defined as the overview-machine, enlarge-end-point, posture-manipulator, and trajectory-manipulator. A role assignment system was then developed to assign the four camera roles to four out of six cameras suitable for the work situation, e.g., reaching, grasping, transport, and releasing, on the basis of the assignment priority rules, in the real time. Debris removal tasks were performed by using a VR simulator to compare fixed camera, manual control, and autonomous systems. Results showed that the autonomous system was the best of the three at decreasing the number of grasping misses and error contacts and increasing the subjective usability while improving the time efficiency.

Unmanned construction machines are used after disasters. Compared with manned construction, time efficiency is lower because of incomplete visual information, communication delay, and lack of tactile experience. Visual information is the most fundamental items for planning and judgment, however, in current vision systems, even the posture and zoom of cameras are not adjusted. To improve operator's visibility, these parameters must be adjusted in accordance with the work situation. The purpose of this study is thus to analyze effective camera images from some comparison experiments, as a fundamental study of advanced visual support. We first developed a virtual reality simulator to enable experimental conditions to be modified easier. To effectively derive required images, experiments with two different camera positions and systems (fixed cameras and manually controllable cameras) were then conducted. The results indicate that enlarged views to show the manipulator is needed in object grasping and tracking images to show the movement direction of the manipulator is needed in largely end-point movement. The result also confirms that the operational accuracy increases and blind spot rate decreases by using the manual system, com-pared with fixed system.

A method to tune a basic input/output gain (BIOG) according to usage conditions for human-machine systems is proposed for improving work performance. Adapting a BIOG is effective in terms of improving operability and work efficiency, but frequent changes using a gain scheduling strategy degrade the operability by confusing the machine dynamics. The proposed tuning system adjusts a BIOG at long intervals on the basis of comprehensive features in operator and work content obtained from the histogram of control lever input. The target value is set to the normal distribution, meaning that all ranges of the control lever are evenly used in a spring-type lever, as leveling the histogram independent of an operator and work content provides the consistent operational feeling, which leads to comfortable operability. To ensure practicality and effectiveness, a BIOG curve is set to a polygonal line involving a break point and a saturation point that are tuned by equivalent transform of area differences between the obtained histogram and normal distribution curves. Two types of experimental task were performed using a hydraulic arm system. Results showed that the proposed BIOG tuning system improves time efficiency by reducing the area difference while increasing the subjective usability compared with a conventional fixed BIOG system.

A practical framework for measuring the mass of an object grasped by the end-effector of a large-scale hydraulic manipulator, such as construction manipulators, is proposed. Such a measurement system requires high accuracy and robustness considering the nonlinearity and uncertainty in hydraulic pressure-based force measurement. It is thus difficult to precisely model the system behaviors and completely remove error force components, so our framework adopts a less-error data selection approach to improving the reliability of the measurand. It first detects the on-load state to extract reliable data for mass measurement, including evaluating measurement conditions to omit sensors in indeterminate conditions and redefining three-valued outputs such as on, off, or not determinate, to improve robustness, then extracts data during the object-grasp state identified by the grasp motion model and removes high-frequency component by a simple low-pass filter, to improve accuracy, and finally integrates date from plural sensors using the posture-based priority and averages all selected data, to improve reliability. Evaluation experiments were conducted using an instrumented hydraulic arm. Results indicate that our framework can precisely measures the mass of the grasped object in various detection conditions with less errors.

This paper proposes a framework for visualizing comprehensive work tendency using a frequency plot map of the end-point of a manipulator as a fundamental study of work analysis using long-term data for human-operated work machines. A visualization system requires high generality and commonality to enable to extract various characteristics to be extracted on an arbitrary time scale independently of the work machine specifications, work contents and environment, and machine operator. The proposed framework meets this requirement by first creating a two-dimensional end-point plot map with its origin fixed at the yaw joint of the manipulator to deal with arbitrary usage conditions. It then extracts arbitrary characteristics by using hierarchical feature extraction filters, including binary, quantity, and advanced filters, defined by three kinds of essential data: operation, movement, and load. Finally, it quantifies the filtered map by gridding and normalization to visually grasp its frequency distribution. Two experiments in which the work environments and completion time differed were conducted using an instrumented hydraulic arm. Results indicated that the comprehensive work tendency revealed by using the proposed framework corresponds to the actual work results independently of the various conditions. © 2013 IEEE.

The purpose of this paper is to model and identify an object grasp motion, which is a requisite condition to make a demolition machine grasp an object. Grasping an object is a highly difficult task that requires safe and precise operations, so identifying a grasp or non-grasp is necessary for providing operational support, particularly in disaster response work. These types of outdoor machines lack visual and tactile sensors, so practically available lever operation and cylinder pressure sensors are adopted. The grasp motion is modeled by using sequential transitions of the on-off state of the control signal and cylinder pressure data for the grapple and the manipulator. The results of experiments conducted to transport objects using an instrumented hydraulic arm indicated that the modeled grasp motion effectively identifies a grasp or non-grasp with high accuracy, independently of operators and environments.

This paper proposes a practical framework to estimate whether or not a grapple installed in demolition machines is in a grasp state. Object grasp is a highly difficult task that requires safe and precise operations, so identifying a grasp or non-grasp state is important for assisting an operator. These types of outdoor machines lack visual and tactile sensors, so the proposed framework adopts practically available lever operation and cylinder pressure sensors. The grasp is formed by a grasp motion, which is operations to make the grapple pinch an object, and the grasp state, where the grapple holds the object in any manipulator movements. Thus, the framework determinately confirms the grasp motion through the requisite conditions defined by using sequential changes of binarized operation and pressure data for the grapple and the manipulator, and stochastically confirms the grasp state through the enhancement conditions defined by using force and movement vectors including vertical downward force, movement in the longer direction, and horizontal reciprocating movement. The results of experiments conducted to transport objects using an instrumented hydraulic arm indicated that the proposed framework is effective for identifying grasp/non-grasp with high accuracy, independently of various operators and environments.

This paper describes a tele-operation simulator using a virtual reality (VR) environment for advanced unmanned construction. VR simulators, which can measure arbitrary data, reproduce the same situations, and change the machine and environmental configurations more easily, compared with actual environments, are effective to create tele-operation technologies and quantitatively evaluate them as well as to improve operational skills in complex disaster response works. In this basic study, a VR simulator including a VR environment, operation-input, and video-output components was developed. (i) The VR environment is built using a basic graphic library and dynamics engine. (ii) The operation-input component consists of control levers for a demolition machine with a grapple and environmental cameras with yaw, pitch, and zoom functions. (iii) The video-output component consists of a two-dimensional monitor for displaying an in-vehicle camera view, environmental camera views, and machine status. The results of experiments conducted to transport debris using the VR simulator indicated that the operators adequately completed the debris transport in the VR environment while watching the monitor views from the in-vehicle and environmental cameras.

A pragmatic framework for detecting (identifying the on-off state of) the external force applied to a construction manipulator (front load) by using a hydraulic sensor is proposed. Such a load detecting system requires high accuracy and robustness considering the uncertainty in pressure-based force measurement. The proposed framework first identifies the dominant error force component, including self-weight and driving force, using theoretical and experimental estimation and binarizes the analog cylinder external force. It then evaluates detection conditions to address indeterminate conditions such as stroke-end, singular posture, and impulsive or oscillatory force and redefines three-valued outputs such as on, off, or not determinate (ND). It finally outputs the front load decision by combining all the cylinder decisions to improve robustness through priority analysis. Experiments were conducted using an instrumented hydraulic arm. Results indicate that the proposed framework detects on, off, and ND outputs of the front load more accurately, robustly, and stably in various detection conditions.

This paper proposes a quantification method for a comprehensive work flow in construction work for describing work states in more detail on the basis of analyzing state transitions of primitive static states (PSS), which consist of 16 symbolic work states defined by using on-off state of the lever operations and joint loads for the manipulator and end-effector. On the basis of the state transition rules derived from a transition-condition analysis, practical state transitions (PST), which are common and frequent transitions in arbitrary construction work, are defined. PST can be classified into essential (EST) or nonessential state transitions (NST). EST extracts common phases of work progress and estimates positional relations between a manipulator and an object. NST reveals wasted movements that degrade the efficiency and quality of work. To evaluate comprehensive work flows modeled by combing EST and NST, work-analysis experiments using our instrumented setup were conducted. Results indicate that all the PSS definitely changes on the basis of PST under various work conditions, and work analysis using EST and NST easily reveals work characteristics and untrained tasks related to wasted movements.

This paper describes an operational support system to simplify removing objects from a building structure in demolition work using dual-arm construction machinery. An operator cannot recognize the manipulator movement visually when a target object is strongly restrained in an unmovable environment, so the operator continues large lever-operations for a while after the object has been removed. This could lead to a collision against the surrounding environment. An operational support in construction machinery filed requires the compatibility among efficiency, quality, and safety and the commonality among various operator skill levels. The proposed support system thus identifies the moment when an object has been removed and cancels any operational inputs to ensure safety work when generating intensive force applied to a manipulator, focusing only on the common physical phenomena. An object-removal flag is defined as a situation where force applied to the end-point of a manipulator decrease rapidly in which lever operation and hand load are detected. Demolition experiments are conducted using a hydraulic dual arm system. Result indicates that the displacement of end-effector after removal and the number of error contacts are decreased while decreasing time consumption.

In this paper, a quantitative analysis method for a comprehensive work flow in construction work for identifying work states in more detail is proposed. The proposed method is based on analyzing state transitions of simplified primitive static states (s-PSS), which consist of four symbolic work states defined by using the on-off state of lever operations and manipulator loads. First, practical state transitions (PST), which are common and frequent transitions in arbitrary construction work, are defined on the basis of the transition rules, according to which an operation flag changes arbitrarily and a load flag changes only during a lever operation. Second, PST is notionally classified into essential (EST) and nonessential (NST) state transitions whose definitions change depending on the task phase, including reaching, contacting, loadworking, and releasing. Third, closed loops formed by EST represent work content and those formed by NST represent wasted movements. In work-analysis experiments using our instrumented setup, results indicated that all s-PSS definitely changes on the basis of PST under various experimental conditions and that work analysis using EST and NST easily reveals untrained tasks related to wasted movements.

This paper proposes a practical framework for detecting (identifying the on-off state of) the external force applied to a construction manipulator (front load) by using a hydraulic sensor. Such a detection system requires high accuracy and robustness considering the uncertainty in pressure-based force measurement. Our framework is thus organized into (i) identifying the dominant error force component (self-weight and driving force) using theoretical and experimental estimation and binarizing the analog external cylinder force, (ii) evaluating detection conditions to address indeterminate conditions such as stroke-end, singular posture, and impulsive or oscillatory force and redefining three-valued outputs such as on, off, or not determinate (ND), and (iii) outputting the front load decision by combining all the cylinder decisions to improve robustness through priority analysis. Experiments were conducted using an instrumented hydraulic arm. Results indicate that our framework adequately detects on, off, and ND outputs of the front load in various detection conditions without misidentification.

This paper proposes a practical framework for measuring the external force applied to a construction manipulator (front load vector) by using a hydraulic sensor. Such a force measurement system requires high accuracy and robustness considering the uncertainty in the construction machinery field, but it inevitably includes measurement errors owing to difficult-to-reduce modeling errors. Our framework thus adopts a relative accuracy improvement strategy without correcting the models for the practicality. It comprises (i) quantifying the internal error range (IER) by using the sum of the maximal measurement errors of static and dynamic friction forces, which change in postural and kinematic conditions, (ii) calculating the error force vector by using IER to select cylinders (sensors) that have less error, and (iii) outputting the front load vector using the cylinders whose error force vector is minimum. Experiments were conducted using an instrumented hydraulic arm. The results indicate that our framework can enhance the relative accuracy of external force measurement independently of various postural and kinematic conditions.

This paper proposes an analysis method for a comprehensive work flow in construction work for identifying work states in more detail on the basis of analyzing state transitions of simplified primitive static states (s-PSS), which consist of four symbolic work states defined by using on-off state of the lever operations and manipulator loads. First, practical state transitions (PST), which are common and frequent transitions in arbitrary construction work, can be defined on the basis of the transition rules, according to which an operation flag changes arbitrary and load flag only changes during a lever operation. Second, PST can be classified into essential (EST) or nonessential state transitions (NST), and NST changes its definition depending on the task phase. Third, EST and NST represent work contents and wasted movements, respectively, so work-analysis experiments using our instrumented setup were conducted. Results indicate that all the s-PSS definitely changes on the basis of PST under various experimental conditions and that work analysis using EST and NST easily reveals untrained tasks related to wasted movements. © 2011 IEEE.

This paper proposes a state identification framework to support the complicated dual-arm operations in construction work. The operational support in construction machinery filed requires the compatibility with different types of support and the commonality among various operator skill levels. The proposed framework is therefore organized into two functions: real-time task phase identification and time-series attentional condition identification. The task phase is defined by utilizing the joint load applied according to the environment constraint condition. The attentional condition is defined as one of the internal work-state condition classified by the necessity level of operational support, and is dependent on the vectorial or time-series date selected by the identified task phase. Experiments are conducted using the hydraulic dual arm system to perform transporting and removing tasks. Results show that the number of error contacts, internal force applied, and mental workload is decreased without time-consumption increase. The result confirmed that the proposed framework greatly contribute to improving each operator's work performance. ©2010 IEEE.

This paper reports a newly developed hydraulic dual robotic arm system to create and evaluate intelligent systems, which support complicated machine operations, for advanced construction machinery. This kind of machine system (test-bed) requires functions to quantify its dynamic characteristics and operational difficulty (influential factors). In particular, construction manipulator nonlinearly changes its dynamics depending on various internal and external factors. To quantify such influential factors, our proposed test-bed is equipped with standard hydromechanical system which includes two manipulators and grapples, and practical sensing system which detects control input, oil pressure, oil temperature, and cylinder stroke data. Using the developed test-bed, fundamental experiments were conducted to clarify hydromechanical system characteristics. Experimental results indicate that the test-bed quantifies its hysteresis, pressure loss, and time delay, and show that variable dynamics complicates intuitive and precise machine operations and external force measurement. This analysis confirms the developed machine system is useful to quantify influential factors for creating intelligent system. © 2010 IEEE.

Double-front construction machinery (DFCM), which has been developed for adaptation to complicated work, demands higher operational skills to control double fronts with more multiple joints. To handle DFCM skillfully for performing more complicated works, intelligent functions that can autonomously distinguish between diverse states and also provide informational and operational supports to their operators are inevitably required. In particular, a state identification method strongly requires high reliability and robustness due to the complexity of the construction work environment and the variety of the operator's skill level. However, most of the current construction machinery has unique functions that only reproduce the movements that are originated by the operator. In this study, which focuses on DFCM, we addressed the need for a new conceptual design of an operator support system (OSS) and evaluated it using our newly developed simulator. Our experimental results showed that the OSS improves the work performance, including decreasing the operational time for completing a task, reducing the mental workload on the operators, and the number of error operations. © 2009 IEEE.

Advanced operated-work machines, which have been designed for complicated tasks and which have complicated operating systems, requires intelligent systems that can provide the quantitative work analysis needed to determine effective work procedures and that can provide operational and cognitive support for operators. Construction work environments are extremely complicated, however, and this makes state identification, which is a key technology for an intelligent system, difficult. We therefore defined primitive static states (PSS) that are determined using on-off information for the lever inputs and manipulator loads for each part of the grapple and front and that are completely independent of the various environmental conditions and variation in operator skill level that can cause an incorrect work state identification. To confirm the usefulness of PSS, we performed experiments with a demolition task by using our virtual reality simulator. We confirmed that PSS could robustly and accurately identify the work states and that untrained skills could be easily inferred from the results of PSS-based work analysis. We also confirmed in skill-training experiments that advice information based on PSS-based skill analysis greatly improved operator's work performance. We thus confirmed that PSS can adequately identify work states and are useful for work analysis and skill improvement.

Intelligent functions that can autonomously identify the current work states and also provide informational or operational support to their operators are inevitably required for double-front construction machinery (DFCM), which has been developed for complicated tasks. In this study, which focuses on DFCM, we address the need for a new conceptual design of an operator support system. In particular, a state identification method strongly requires high reliability and robustness to address the complexity of the construction work environment and the variety of the operator's skill level. We, therefore, define primitive static states (PSS) that are determined using on-off information for the]ever inputs and manipulator loads for each part of the grapple and front. We develop an intelligent system that provides a reduction of operational gain to make precise work easier and an indication of an enlarged image of the end-effector from a different viewpoint to assist depth perception based on PSS identification, and evaluate it using our newly developed simulator. Our experimental results show that the operator support system improves the work performance, including decreasing the operational time for completing a task, reducing the mental workload on the operators and the number of error operations. (C) Koninklijke Brill NV, Leiden and The Robotics Society of Japan, 2009

This paper reports a newly developed simulator for operation skill training in Double-Front Construction Machinery (DFCM) that allows novices to virtually experience tough operations repeatedly using DFCM under various conditions, including dangerous congestion. First, we selected several situations targeted where the DFCM needs to be used to provide a high level of operation skills: sorted dismantling for recycling and reusing resources, rescue and recovery work in disaster areas, and building construction. In addition, we developed an operation skill-training simulator that enables novice operators to repeatedly train with the high level of operation skills needed to easily and safely handle the DFCM in even more complicated works. This simulator system has two joysticks (set in front of a monitor) to dependently control the two fronts of the animated DFCM on the monitor. Several modes involving basic construction tasks are provided and the effects of improvement in operability achieved by the training simulator can be verified. Evaluation experiments indicated that repeated training using the simulator successfully decreased the operation time to complete a task and enhanced positioning accuracy in cooperative transportation with the two fronts. The results confirm the effectiveness of the developed simulator. Futhermore, it was confirmed that informational or operational support based on knowledge provided from experiment results enabled work performance greatly improved.

This paper reports a newly developed simulator for operation skill training in Double-Front Construction Machinery (DFCM) that allows novices to virtually experience tough operations repeatedly using DFCM under various conditions, including dangerous congestion. First, we selected several situations targeted where the DFCM needs to be used to provide a high level of operation skills: sorted dismantling for recycling and reusing resources, rescue and recovery work in disaster areas, and building construction. In addition, we developed an operation skill-training simulator that enables novice operators to repeatedly train with the high level of operation skills needed to easily and safely handle the DFCM in even more complicated works. This simulator system has two joysticks (set in front of a monitor) to dependency control the two fronts of the animated DFCM on the monitor. Several modes involving basic construction tasks are provided and the effects of improvement in operability achieved by the training simulator can be verified. Evaluation experiments indicated that repeated training using the simulator successfully decreased the operation time to complete a task and enhanced positioning accuracy in cooperative transportation with the two fronts. The results confirm the effectiveness of the developed simulator. © 2008 IEEE.