An Efficient Approach for Detecting Driver Drowsiness Based on Deep Learning

Abstract

Drowsy driving is one of the common causes of road accidents resulting in injuries, even death, and significant economic losses to drivers, road users, families, and society. There have been many studies carried out in an attempt to detect drowsiness for alert systems. However, a majority of the studies focused on determining eyelid and mouth movements, which have revealed many limitations for drowsiness detection. Besides, physiological measures-based studies may not be feasible in practice because the measuring devices are often not available on vehicles and often uncomfortable for drivers. In this research, we therefore propose two efficient methods with three scenarios for doze alert systems. The former applies facial landmarks to detect blinks and yawns based on appropriate thresholds for each driver. The latter uses deep learning techniques with two adaptive deep neural networks based on MobileNet-V2 and ResNet-50V2. The second method analyzes the videos and detects driver’s activities in every frame to learn all features automatically. We leverage the advantage of the transfer learning technique to train the proposed networks on our training dataset. This solves the problem of limited training datasets, provides fast training time, and keeps the advantage of the deep neural networks. Experiments were conducted to test the effectiveness of our methods compared with other methods. Empirical results demonstrate that the proposed method using deep learning techniques can achieve a high accuracy of 97%. This study provides meaningful solutions in practice to prevent unfortunate automobile accidents caused by drowsiness.

Introduction

The American National Highway Traffic Safety Administration estimates that 100,000 accidents occur annually due to drowsy driving, resulting in over 1,550 deaths, 71,000 injuries, and $12.5 billion in property damage. Studies show that 10-15% of car accidents are linked to lack of sleep, with over 10% of drivers experiencing drowsiness while driving at least once a month.Many systems have been developed to detect drowsy driving. Rateb et al. proposed a real-time driver drowsiness detection system using deep neural networks, with over 80% accuracy, though limited by only detecting eye landmarks. Akalya et al. developed a method using Raspberry Pi and a Haar cascade classifier to detect eye blinks. Mohana and Sheela presented a method detecting eye closure and yawning but faced limitations with fixed thresholds. Other studies used convolutional neural networks (CNNs) and machine learning techniques, reaching accuracies as high as 97.1%.Despite these advances, most systems focus on eye and mouth detection, overlooking other physiological factors. The paper proposes two new approaches: (1) using facial landmarks to adaptively determine blink and yawn thresholds; and (2) using deep learning with transfer learning on MobileNet-V2 and ResNet-50V2. These methods achieve 97% accuracy and improve the feasibility of detecting drowsiness in real-world systems.

Proposed Method

1.Step 1: Pre-Processing In this step, we extract video frames from input videos. The rate of selecting images from videos is 25 frames per second. These images are flipped to accurately identify the facial landmarks

Phase 2: TestingThe training dataset is passed through deep neural networks (DNNs). We develop two adaptive networks for drowsiness detection inspired by ResNet-50V2 and MobileNet-v2, with improvements to some layers to adapt to drowsiness detection. We use transfer learning with pre-trained weights from Bing Search API, Kaggle, and RMFD, and re-train on our dataset to fine-tune the parameters, resulting in faster learning and shorter training time. Training stops when the Loss_value is not reduced.

Step 2: Feature Extraction and Training
The training dataset is passed through deep neural networks (DNNs). We develop two adaptive networks for drowsiness detection inspired by ResNet-50V2 and MobileNet-v2, with improvements to some layers to adapt to drowsiness detection. We use transfer learning with pre-trained weights from Bing Search API, Kaggle, and RMFD, and re-train on our dataset to fine-tune the parameters, resulting in faster learning and shorter training time. Training stops when the Loss_value is not reduced.

Results