Distracted driving is one of the most common causes of traffic accidents around the world. Recognizing the driver's gaze direction during a maneuver could be an essential step for avoiding the matter mentioned above. Thus, we propose a gaze zone classification system that serves as a base of supporting systems for driver's situation awareness. However, the challenge is to estimate the driver's gaze inside not ideal scenarios, specifically in this work, scenarios where may occur self-occlusions or non-aligned head and eyes direction of the driver. Firstly, towards solving miss classifications during self-occlusions scenarios, we designed a novel protocol where a 3D full facial geometry reconstruction of the driver from a single 2D image is made using the state-of-the-art method PRNet. To solve the miss classification when the driver's head and eyes direction are not aligned, eyes and head information are extracted. After this, based on a mix of different data pre-processing and deep learning methods, we achieved a robust classifier in situations where self-occlusions or non-aligned head and eyes direction of the driver occur. Our results from the experiments explicitly measure and show that the proposed method can make an accurate classification for the two before-mentioned problems. Moreover, we demonstrate that our model generalizes new drivers while being a portable and extensible system, making it easy-adaptable for various automobiles.

The number of medical-condition-caused car accidents (MCCCAs) in transport industry (bus, truck, and taxi) recently increases. MCCCAs including cerebrovascular and cardiovascular disease lead to loss of consciousness, thus result in injury and loss of life, and heavy compensation payment. Toward this problem, conventional systems detect closing of eyes, and fallen down state as to prevent car collisions. However, the support is taken after driver losing consciousness. To prevent MCCCAs, it is important to find out abnormal signs before driver losing consciousness. It is challenging to detect abnormal signs not only early but also with high confidence level (CL). This paper proposes a novel method that multi-modally monitors driver to detect abnormal signs which can be cues for estimating a driving-disable state in future and performs voice interaction based on the result of monitoring to clarify the internal state of the driver. Considering no data of abnormal signs, this study developed the system using normal data and pseudo abnormal data, and method of outlier detection was used for abnormal signs detection. As results of experiment, we found the relationship between cue signs and CL, and the proposed system can detect 'sleepiness' state with accuracy of 80%. Voice interaction system did not increase driver's mental demand.

Due to functional limitations in certain situations, the driver receives a request to intervene from automated vehicles operating level 3. Unscheduled intervention of control authority would lead to insufficient situational awareness, then this will make dangerous situations. The purpose of this study is thus to propose tactical-level input (TLI) method with a multimodal driver-vehicle interface (DVI) for the human-centered intervention. The proposed DVI system includes touchscreen, hand-gesture, and haptic interfaces that enable interaction between driver and vehicle, and TLI along with such DVI system can enhance situational awareness. We performed unscheduled takeover experiments using a driving simulator to evaluate the proposed intervention system. The experimental results indicate that TLI can reduce reaction time and driver workload, and moreover, most drivers preferred the use of TLI than manual takeover.

Highly-automated vehicles operating in level 3 issue a takeover request (TOR) to transfer the control authority from the autonomated driving (AD) system to a human driver when they encounter system limitations. In such 'unscheduled' situations, the driver is required to immediately re-engage in the driving task both physically and cognitively, and perform suitable action, e.g. lane change. Thus, evaluating driver engagement by the AD system would lead to safe takeover. Physical engagement is easily estimated but there are few studies on evaluating cognitive engagement. In this study, we thus developed a driver situational awareness estimation system based on glance information. We first defined seven standard glance areas and driver glance classification model using a convolutional neural network. We then obtained a large amount of glance data when both safe and dangerous takeover situations (lane change) by using a driving simulator, and we derived the standard glance model including the glance area and time, in order to estimate whether driver gained enough cognitive re-engagement in real-time. To evaluate the effectiveness of the proposed model, we created a situational awareness assist system to visually indicate regions with insufficient glance. As a result, we found that the assist system drastically improved driving performance and reduced the number of accidents during takeover.

Automated vehicles operating in level 3 may request the human driver to intervene in certain situations due to system limitations. Unscheduled transferring of control to manual driving will create safety issues as consequences of inadequate situational awareness and sudden increase of driver workload. In this study, we propose and evaluate tactical-level input (TLI) method with a multimodal human-machine interface (HMI) for driver intervention in short-term system limitations. The HMI system consists of touchscreen, gesture, and haptic interfaces enabling bilateral driver-vehicle interaction. TLI along with the HMI capable of multimodal feedback can provide situation-adaptive spatial information which enhance the driver situational awareness in a short time. To evaluate the proposed system we conducted driving experiments involving unscheduled takeover situations in urban environment using a driving simulator. We analyzed driver reaction times, physiological responses including heart rate, skin conductance and subjective workload as well as qualitative feedback comparing with manual takeover. The results show that TLI can reduce driver workload, reaction times, and improve driver behavior. Moreover, 90% of drivers preferred to use TLI method over manual takeover.