Scientific research project

Abstract

Drowsy driving is one of the common causes of road accidents resulting in injuries, even death, and significant economic losses to drivers, road users, families, and society. There have been many studies carried out in an attempt to detect drowsiness for alert systems. However, a majority of the studies focused on determining eyelid and mouth movements, which have revealed many limitations for drowsiness detection. Besides, physiological measures-based studies may not be feasible in practice because the measuring devices are often not available on vehicles and often uncomfortable for drivers. In this research, we therefore propose two efficient methods with three scenarios for doze alert systems. The former applies facial landmarks to detect blinks and yawns based on appropriate thresholds for each driver. The latter uses deep learning techniques with two adaptive deep neural networks based on MobileNet-V2 and ResNet-50V2. The second method analyzes the videos and detects the driver's activities in every frame to learn all features automatically. We leverage the advantage of the transfer learning technique to train the proposed networks on our training dataset. This solves the problem of limited training datasets, provides fast training time, and keeps the advantage of the deep neural networks. Experiments were conducted to test the effectiveness of our methods compared with other methods. Empirical results demonstrate that the proposed method using deep learning techniques can achieve a high accuracy of 97%. This study provides meaningful solutions in practice to prevent unfortunate automobile accidents caused by drowsiness.

Introduction

The American National Highway Traffic Safety Administration (https://www.nhtsa.gov (accessed on 4 August 2021)) has an estimated 100,000 accidents reported each year mainly due to drowsy driving. This results in more than 1550 deaths, 71,000 injuries, and 12.5 billion dollars of property damage. According to the National Safety Council (https://www.nsc.org (accessed on 4 August 2021)), 13% of drivers admitted to falling asleep behind the wheel at least once a month and 4% of them resulted in accidents. Morgenthaler et al. announced that drowsiness is one of the main causes of traffic accidents in their study [1]. It is estimated that about 10–15% of car accidents are related to lack of sleep. The sleep questionnaire obtained from professional drivers [2] showed that more than 10.8% of drivers are drowsy while driving at least once a month, 7% had caused a traffic accident, and 18% had near-miss accidents due to drowsiness. These alarming statistics point to the need for capable systems for monitoring drowsy drivers to prevent unfortunate traffic accidents that may occur.

Proposed Method

Method 1 - Drowsiness Detection and Prediction Based on Facial Landmarks: The proposed method is inspired by the methods introduced in preceding studies [5,6,7]. However, the threshold of eye-opening is fixed for all drivers, leading to inaccuracies for drivers with large or small eyes. To overcome this problem, we improve the determination of the adaptive eye-opening threshold (𝐸𝐴𝑅𝑇ℎ𝑟𝑒𝑠ℎ) for each driver. The model of the proposed method is shown in Figure 1. The values of the EAR and 𝐸𝐴𝑅𝑇ℎ𝑟𝑒𝑠ℎ are first computed for each driver. Then, we determine the drowsiness level of drivers by a comparison of 𝐸𝐴𝑅 and 𝐸𝐴𝑅𝑇ℎ𝑟𝑒𝑠ℎ. This work is carried out repeatedly during driving. This method does not rely on the yawning frequency, since it would be inaccurate in cases where drivers wear a mask or talk while driving. The details of this method are described as follows.

Figure 1. Model of the proposed method 1 for detecting drowsiness using facial landmarks.

Method 2 - Drowsiness Detection and Prediction Using Deep Learning: In this method, the drowsiness detection using deep neural networks (DNNs) includes two phases of the training and testing, as illustrated in Figure 2. In the first phase, we train the proposed network models by training a dataset after pre-processing and feature extraction. In the second phase, we evaluate the network models with a test set for drowsiness detection.

Figure 2. Model of the proposed method 2 for detecting drowsiness using deep learning.

Results

Table 1 provides experiments for drowsiness detection from videos for the three proposed scenarios. In cases a and e, it does not work (no detection) for scenario 1 due to the inability to determine the eye-opening threshold in input frames; scenarios 2 and 3 give correct results of detection but the determination time of scenario 2 is slower; the driver completely sleeps in scenario 2, while the driver has their head tilted in scenario 3. On the other hand, in scenario 2, face noise occurs when the driver’s face is placed too close to the camera. In case b, all three scenarios can detect the driver, but scenario 2 gives incorrect results. In case c, all three scenarios give the correct results. In case d, scenario 1 does not give a warning when the driver is drowsy, scenario 2 gives correct results, while scenario 3 only shows signs of eye strain. It can be seen that scenario 3 has more signs of drowsiness, which manifest as feelings of tiredness and eye strain without having a clear face.

Table 1. Illustration of drowsiness detection and prediction on videos for three scenarios.

Table 2 shows the experimental results for the proposed scenarios. These scenarios give the correct results in all cases. However, in case a, scenarios 2 and 3 give results when the driver completely closes their eyes and shows signs of tilting their head. In case b, scenarios 2 and 3 still give correct identification results in the case where the driver wears a mask; scenario 2 is bases the identification on the head being tilted and eyes being closed, while scenario 3 requires only a slight tilt of the head and lightly closed eyes; scenario 1 only relies on the eye-opening value, which can lead to confusion when the driver only closes their eyes but does not completely fall asleep; thus, in scenario 1, we need to adjust the blink counter depending on each driver to alert when the driver completely falls asleep. In cases c and d, the driver is in non-drowsy status and the results of all three scenarios are correct.

Table 2. Some experimental results of drowsiness detection for three scenarios

Conclusions

In this thesis, I propose two methods with three scenarios for driver’s drowsiness detection systems. The proposed method with scenario 1 uses facial landmarks to detect drowsiness. This method analyzes the videos and detects drivers’ faces in every frame using image processing techniques. Experiments were conducted to test the efficacy of the proposed approaches. The results show that the proposed methods can achieve a high accuracy of 97% using deep learning techniques. Method 2 with scenario 3 provides more accurate results than scenario 2 and method 1 because it detects most of the drowsiness in various experimental contexts. From the above results, the proposed method using deep learning techniques can be useful for monitoring the fatigue of drivers to give early warning of falling asleep, avoiding unfortunate traffic accidents.

References