Scientific research project

Abstract

In recent years, with the rapid development of transportation in our country, the number of traffic accidents has become increasingly alarming. One of the main causes of traffic accidents is a lack of concentration or driving while fatigued and drowsy, especially among long-distance drivers. To address this issue, there is a need for a system that can detect early signs of drowsiness and fatigue, helping to reduce and limit the number of accidents caused by drowsy driving. Various methods have been proposed by authors both domestically and internationally. In this study, we successfully explored a deep learning-based approach by developing and training a model to detect driver drowsiness using image data of drowsy and non-drowsy states, leveraging two neural network models: MobileNetV2 and ResNet-50V2. Experimental results indicate that the proposed method achieves a high accuracy rate of 96% to 97%.

Introduction

Drowsy is a common phenomenon when doing excessive tasks such as driving cars, long-distance motorbikes, studying stressfully during exam season, working at the computer a lot. According to statistics from the National Traffic Safety Board, sleep-related traffic accidents account for up to 30% of all traffic accidents in a year. In 2019 alone, many serious accidents occurred due to drivers falling asleep. In addition, according to statistics from the “Driver Care Day 2019” co-organized by the National Traffic Safety Board, Michelin, and the Automobile Hospital, the cause of accidents is falling asleep while driving, accounting for about 21%, with 6,400 deaths annually and losses of up to 109 billion dollars (excluding damages to property) for accidents related to drowsiness while driving, and 30% of drivers have admitted to dozing behind the wheel.

There is a term called “microsleep” or “sleepiness,” which is defined as a short and unintended period of sleep that can occur at any time due to fatigue or some prolonged effort of consciousness. Microsleep can only last for a few seconds (it can be between 1 and 30 seconds), and during this time the brain can fall into an uncontrollable rapid sleep, which can be extremely dangerous, especially while driving or in situations that require focused attention. There is a sign that the driver is not awake: yawning, blinking constantly and having difficulty opening his eyes, inability to concentrate, inability to keep his head straight, disturbed mind, feeling tired, and blurred vision

In this context, detecting signs of drowsiness while driving is an urgent and critical issue due to the growing incidence of traffic accidents partially caused by driver drowsiness. To address this issue, I use several deep neural network (DNN) models to evaluate detection through cameras and image datasets. Specifically, I employ convolutional neural networks MobileNetV2 and ResNet-50V2 to train the model.

Proposed Method

Datasets description: To ensure safety during experimentation, we created a test dataset consisting of sleepy and non-sleepy faces, sourced from datasets from Bing Search API, Kaggle, RMFD, and iStock by selecting images and videos recorded from cameras related to the driver's drowsy state. The main advantages of this work are its efficiency, low cost, safety, and ease of data collection. In these images and videos, the driving environment is relatively similar to real-world road experiments. This study conducted experiments on a dataset comprising 1,431 drowsy images and 2,360 non-drowsy images. Subsequently, the drowsy dataset images were processed (face detection and cropping) to produce an optimized dataset for training.

Table 1: Datasets description

The drowsiness detection system for drivers proposed in this study follows the following procedure: The input video file is split into individual frames, and then the subject is detected in each frame. After processing the source video, the result determines whether the person is drowsy. To accomplish this task, we use a Deep Neural Network (DNN). This approach divides the problem into three main components: preprocessing, feature extraction, and classification. The classification results are based on the training process, which indicates the category of the image data and its accuracy.

Figure 1: System Diagram for Drowsiness Detection Based on Image Dataset

- Step 1. data preprocessing: In this step, We extract images from input videos at a rate of 25 frames per second. Initially, we detect faces in these images using the SSD (Single Shot MultiBox Detector) network with ResNet-10. The facial images are then normalized to a size of 224 × 224 to create the training dataset.

- Step 2. feature extraction and classification: To detect and classify the target objects, feature extraction is performed on the dataset from Step 1. I use feature extraction methods with two network models: ResNet-50V2 and MobileNetV2, by modifying certain layers of these networks to adapt them for drowsiness detection. We construct an adaptive network by adding layers such as Flatten, ReLU (Dense), Dropout, and Dense (Sigmoid) and by replacing the Average Pooling 2D layer of the MobileNetV2 network. This aims to address overfitting issues and help the model converge faster. Additionally, we propose another adaptive network by adding layers such as Flatten, two ReLU (Dense), Dropout, and Dense (Sigmoid) to ResNet-50V2.

Display light sensor parameters on an LCD screen:

Table 2: Scenarios of the proposed model

Results

Figures 2 and 3 illustrate the average accuracy and the average testing time of the two scenarios in detecting drowsiness based on experimental results.

Figure 2: Testing accuracy of scenarios; Figure 3: Testing time of scenarios

Figure 4 presents images illustrating experimental results from the camera across the two proposed scenarios. Contextually, the results for each case in the two scenarios are accurate. Observing row a), the subject in a non-drowsy state results in accurate outcomes for both scenarios. In row b), both Scenario 1 and Scenario 2 yield results when the subject has their eyes completely closed and shows signs of tilting or nodding. Row c) examines the case of wearing a mask but still produces accurate results; Scenario 1 relies on signs of head tilting and fully closed eyes, while Scenario 2 only requires a slight head tilt and signs of partially closed eyes. In row d), where the subject wears sunglasses, neither scenario can distinguish between awake or asleep, although Scenario 2 detects when the subject shows signs of leaning to one side.

a) Scenario 1

b) Scenario 2

Figure 4: Illustration of Experimental Results from the Camera Across Two Proposed Scenarios

Based on the comparison table and the results above, the method using deep learning techniques can be beneficial for monitoring driver fatigue to provide early warnings about drowsiness, thereby preventing unfortunate traffic accidents. The experimental results demonstrate that this method is feasible and adaptable for the development of drowsiness warning systems, especially mobile applications.

Table 3: Comparison of Accuracy of Drowsiness Detection Methods

Conclusions

The thesis offers an overview of techniques designed to tackle driver distraction caused by fatigue and drowsiness, aiming to enhance traffic safety for drivers, communities, and society as a whole. We developed a driver drowsiness detection system with notable outcomes, including the collection of image datasets featuring both drowsy and non-drowsy faces. This involved sourcing non-drowsy face images from Bing Search API, Kaggle, RMFD, and other platforms, followed by processing drowsy face images through detection and cropping to create a refined training dataset. Additionally, we improved two deep neural network models, MobileNetV2 and ResNet-50V2, achieving impressive accuracy rates of 96% to 97% on video and image datasets.

References