Home › AI/ML Projects › Driver Drowsiness Detection Using Python & dlib

In this complete guide, you will build a real-time driver drowsiness detection system using Python, OpenCV, and dlib’s 68-point facial landmark model. The system detects closed eyes (EAR), yawning (MAR), and head tilting — triggering an audio alarm and Twilio SMS alert when fatigue is detected.

Project Overview & How It Works

Drowsy driving causes over 1.35 million road deaths per year globally (WHO). This project builds an AI-based fatigue monitor that runs on a laptop or Raspberry Pi with a webcam and sounds an alarm before the driver falls asleep at the wheel.

The system detects three signs of driver fatigue:

1. Eye closure (EAR): Detects when eyes stay closed for more than 1.5 seconds.
2. Yawning (MAR): Detects wide mouth opening indicating fatigue.
3. Head tilt (Head Pose): Detects when the driver’s head droops forward.

Understanding the Eye Aspect Ratio (EAR) Algorithm

The Eye Aspect Ratio is the key algorithm of this project. It was first introduced by Tereza Soukupova and Jan Cech in their 2016 paper “Real-Time Eye Blink Detection using Facial Landmarks”.

The EAR is computed from 6 landmark points around each eye. p1 and p6 are the left and right corners; p2/p5 and p3/p4 are upper and lower eyelid points at the outer and inner thirds respectively.

📌 Important: EAR stays approximately constant when the eye is open, and drops rapidly to near zero when closed. A single blink lasts ~200 ms; drowsiness is flagged after 20+ consecutive frames of low EAR (~1.5 seconds at 13 fps).

Prerequisites & Library Installation

⚠️ dlib installation note: dlib requires CMake and a C++ compiler. On Windows, install Visual C++ Build Tools first. On Linux/Mac it installs directly via pip.

Install on PC / Laptop

pip install opencv-python dlib imutils scipy
pip install pygame pyttsx3 twilio playsound
pip install numpy Pillow
# Output: Successfully installed opencv-python-4.9.0 dlib-19.24.2...

Install on Raspberry Pi (dlib compiles from source — allow 20 min)

sudo apt-get install cmake libopenblas-dev liblapack-dev
pip install dlib --verbose   # Takes ~20 min on RPi 4

Parts Required

Component	Specification	Purpose	Cost (INR)
Laptop / PC with Webcam	Any modern webcam, 720p+	Face capture	Already available
OR Raspberry Pi 4	4 GB RAM + Pi Camera v2	Embedded version	₹5,500–₹7,500
Speaker / Buzzer	3.5 mm audio jack speaker	Alarm sound	₹200–₹600
Twilio Account	Free trial: 1 number + credits	SMS alert	Free trial
shape_predictor_68 model	dlib pre-trained model (95.4 MB)	Facial landmarks	Free download
alarm.wav	Any short alert sound file	Audio alert	Free

How dlib 68-Point Facial Landmarks Work

dlib’s shape_predictor_68_face_landmarks.dat predicts 68 precise landmark coordinates on a human face. Each index range maps to a specific feature:

Landmark Index Range	Facial Feature
0 – 16	Jawline (17 points)
17 – 21	Left eyebrow (5 points)
22 – 26	Right eyebrow (5 points)
27 – 35	Nose bridge + nostrils
36 – 41	🔴 Left eye (6 points — used for EAR)
42 – 47	🔴 Right eye (6 points — used for EAR)
48 – 67	🔴 Mouth / lips (used for yawn detection)

Step 1 — Download the dlib Landmark Model

• 📥 Direct download: shape_predictor_68_face_landmarks.dat.bz2 (59 MB compressed)
• After downloading, extract it to your project folder.
• Final file: shape_predictor_68_face_landmarks.dat (95.4 MB)

$ bzip2 -dk shape_predictor_68_face_landmarks.dat.bz2
# Extracted: shape_predictor_68_face_landmarks.dat (95.4 MB)
$ ls -lh *.dat
# -rw-r--r-- 1 user staff 95M shape_predictor_68_face_landmarks.dat

Step 2 — Core Drowsiness Detection Code (EAR Algorithm)

This script implements the core EAR-based eye monitoring system. It detects when a driver’s eyes have been closed for more than 20 consecutive frames (~1.5 seconds at 13 fps) and triggers an alarm.

Core EAR Script — drowsiness_core.py

# ============================================================
# Driver Drowsiness Detection — Core EAR Algorithm
# Website: iotprojectsandtrainings.in
# Based on Eye Aspect Ratio (EAR) by Soukupova & Cech (2016)
# ============================================================

import cv2, dlib, numpy as np
from imutils import face_utils
from scipy.spatial import distance as dist
import pygame, time

EYE_AR_THRESH = 0.25    # EAR below this = eyes closed
EYE_AR_CONSEC = 20      # Consecutive frames before alarm (~1.5s at 13fps)
ALARM_PATH    = 'alarm.wav'

pygame.mixer.init()
pygame.mixer.music.load(ALARM_PATH)

detector  = dlib.get_frontal_face_detector()
predictor = dlib.shape_predictor('shape_predictor_68_face_landmarks.dat')

(lStart, lEnd) = face_utils.FACIAL_LANDMARKS_IDXS['left_eye']
(rStart, rEnd) = face_utils.FACIAL_LANDMARKS_IDXS['right_eye']

def eye_aspect_ratio(eye):
    """Compute Eye Aspect Ratio from 6 landmark points.
    EAR = (||p2-p6|| + ||p3-p5||) / (2 * ||p1-p4||)
    Returns float: ~0.3 open, ~0.0 closed."""
    A = dist.euclidean(eye[1], eye[5])
    B = dist.euclidean(eye[2], eye[4])
    C = dist.euclidean(eye[0], eye[3])
    return (A + B) / (2.0 * C)

cap = cv2.VideoCapture(0)
cap.set(cv2.CAP_PROP_FRAME_WIDTH, 640)
cap.set(cv2.CAP_PROP_FRAME_HEIGHT, 480)

frame_counter = 0
alarm_on      = False

print("✅ Drowsiness Detector started. Press 'q' to quit.")

while True:
    ret, frame = cap.read()
    if not ret:
        break

    gray  = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
    faces = detector(gray, 1)

    for face in faces:
        shape    = face_utils.shape_to_np(predictor(gray, face))
        leftEye  = shape[lStart:lEnd]
        rightEye = shape[rStart:rEnd]

        ear = (eye_aspect_ratio(leftEye) + eye_aspect_ratio(rightEye)) / 2.0

        color = (0, 255, 0) if ear >= EYE_AR_THRESH else (0, 0, 255)
        cv2.drawContours(frame, [cv2.convexHull(leftEye)],  -1, color, 1)
        cv2.drawContours(frame, [cv2.convexHull(rightEye)], -1, color, 1)

        if ear < EYE_AR_THRESH:
            frame_counter += 1
            if frame_counter >= EYE_AR_CONSEC:
                if not alarm_on:
                    alarm_on = True
                    pygame.mixer.music.play(-1)
                cv2.putText(frame, "DROWSINESS ALERT!", (10, 30),
                            cv2.FONT_HERSHEY_SIMPLEX, 1.1, (0, 0, 255), 3)
        else:
            frame_counter = 0
            if alarm_on:
                alarm_on = False
                pygame.mixer.music.stop()

        cv2.putText(frame, f"EAR: {ear:.2f}", (500, 30),
                    cv2.FONT_HERSHEY_SIMPLEX, 0.65, (255, 255, 255), 2)

    cv2.imshow("Driver Drowsiness Detection | iotprojectsandtrainings.in", frame)
    if cv2.waitKey(1) & 0xFF == ord('q'):
        break

cap.release()
cv2.destroyAllWindows()
pygame.mixer.quit()

Step 3 — Add Yawn Detection (MAR Algorithm)

Yawning is a strong indicator of driver fatigue. We detect it by computing the Mouth Aspect Ratio (MAR) — the same concept as EAR but applied to the inner lip landmarks (points 60–67).

# Add to your main drowsiness script
(mStart, mEnd) = face_utils.FACIAL_LANDMARKS_IDXS['inner_mouth']  # 60-67

MOUTH_AR_THRESH = 0.7    # MAR above this = yawning
YAWN_CONSEC    = 15      # Consecutive yawning frames

def mouth_aspect_ratio(mouth):
    """Compute Mouth Aspect Ratio (MAR).
    High MAR = wide open mouth = yawning."""
    A = dist.euclidean(mouth[1], mouth[7])
    B = dist.euclidean(mouth[2], mouth[6])
    C = dist.euclidean(mouth[3], mouth[5])
    D = dist.euclidean(mouth[0], mouth[4])
    return (A + B + C) / (3.0 * D)

Step 5 — SMS Alert & Voice Warning System

Twilio SMS + Text-to-Speech — sms_alert.py

from twilio.rest import Client
import datetime, threading, time, pyttsx3

ACCOUNT_SID  = 'YOUR_TWILIO_ACCOUNT_SID'
AUTH_TOKEN   = 'YOUR_TWILIO_AUTH_TOKEN'
FROM_PHONE   = '+1XXXXXXXXXX'
TO_PHONE     = '+91XXXXXXXXXX'

last_sms_time = 0
SMS_COOLDOWN  = 60   # Only 1 SMS per 60 seconds

def send_drowsiness_alert(ear_value):
    """Send SMS via Twilio in background thread."""
    global last_sms_time
    now = time.time()
    if now - last_sms_time < SMS_COOLDOWN:
        return
    def _send():
        try:
            c = Client(ACCOUNT_SID, AUTH_TOKEN)
            ts = datetime.datetime.now().strftime("%H:%M:%S on %d-%m-%Y")
            c.messages.create(
                body=(f"⚠️ DROWSINESS ALERT!\n"
                      f"Driver detected drowsy at {ts}.\n"
                      f"EAR value: {ear_value:.2f} (threshold: 0.25)"),
                from_=FROM_PHONE, to=TO_PHONE)
        except Exception as e:
            print(f"SMS failed: {e}")
    last_sms_time = now
    threading.Thread(target=_send, daemon=True).start()

# Text-to-Speech voice warning
tts_engine = pyttsx3.init()
tts_engine.setProperty('rate', 160)

def voice_alert(message="Wake up! You are falling asleep!"):
    def _speak():
        tts_engine.say(message)
        tts_engine.runAndWait()
    threading.Thread(target=_speak, daemon=True).start()

Step 6 — Full Combined System Code

This is the complete production-ready script combining EAR eye monitoring, MAR yawn detection, head pose estimation, audio alarm, and Twilio SMS alerts.

Complete Script — drowsiness_detector_full.py

# ============================================================
# COMPLETE Driver Drowsiness Detection System
# Combines: EAR + MAR + Head Pose + Alarm + Twilio SMS
# iotprojectsandtrainings.in | Final Year AI Project 2026
# ============================================================

import cv2, dlib, numpy as np, time
from imutils import face_utils
from scipy.spatial import distance as dist
import pygame, threading, datetime
from twilio.rest import Client
import pyttsx3

# ============ THRESHOLDS (TUNE THESE) ============
EAR_THRESH      = 0.25
EAR_CONSEC_FRM  = 20
MAR_THRESH      = 0.75
MAR_CONSEC_FRM  = 15
PITCH_THRESH    = 15

# ============ TWILIO CONFIG ============
TWILIO_SID   = 'YOUR_SID'
TWILIO_TOKEN = 'YOUR_TOKEN'
FROM_PHONE   = '+1XXXXXXXXXX'
TO_PHONE     = '+91XXXXXXXXXX'

pygame.mixer.init()
pygame.mixer.music.load('alarm.wav')
tts = pyttsx3.init()

detector  = dlib.get_frontal_face_detector()
predictor = dlib.shape_predictor('shape_predictor_68_face_landmarks.dat')

(lS, lE) = face_utils.FACIAL_LANDMARKS_IDXS['left_eye']
(rS, rE) = face_utils.FACIAL_LANDMARKS_IDXS['right_eye']
(mS, mE) = face_utils.FACIAL_LANDMARKS_IDXS['inner_mouth']

def ear(e):
    A = dist.euclidean(e[1], e[5]); B = dist.euclidean(e[2], e[4])
    C = dist.euclidean(e[0], e[3]); return (A + B) / (2.0 * C)

def mar(m):
    A = dist.euclidean(m[1], m[7]); B = dist.euclidean(m[2], m[6])
    C = dist.euclidean(m[3], m[5]); D = dist.euclidean(m[0], m[4])
    return (A + B + C) / (3.0 * D)

last_sms = 0
def send_sms_bg(ear_val):
    global last_sms
    if time.time() - last_sms < 60: return
    last_sms = time.time()
    def _send():
        try:
            c = Client(TWILIO_SID, TWILIO_TOKEN)
            c.messages.create(
                body=f"⚠️ Drowsy Driver Alert!\nEAR={ear_val:.2f}\n{datetime.datetime.now()}",
                from_=FROM_PHONE, to=TO_PHONE)
        except: pass
    threading.Thread(target=_send, daemon=True).start()

eye_counter = yawn_counter = 0
eye_alarm_on = yawn_alarm = False

cap = cv2.VideoCapture(0)
cap.set(cv2.CAP_PROP_FRAME_WIDTH, 640)
cap.set(cv2.CAP_PROP_FRAME_HEIGHT, 480)

print("🚗 Full Drowsiness Detection System Started")

while True:
    ret, frame = cap.read()
    if not ret: break

    gray  = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
    faces = detector(gray, 0)
    status = "👀 Monitoring..."
    status_color = (0, 200, 0)

    for face in faces:
        shape = face_utils.shape_to_np(predictor(gray, face))
        leftEye   = shape[lS:lE]; rightEye = shape[rS:rE]
        mouth_pts = shape[mS:mE]

        EAR_val = (ear(leftEye) + ear(rightEye)) / 2.0
        MAR_val = mar(mouth_pts)

        cv2.drawContours(frame, [cv2.convexHull(leftEye)],   -1, (0,255,0), 1)
        cv2.drawContours(frame, [cv2.convexHull(rightEye)],  -1, (0,255,0), 1)
        cv2.drawContours(frame, [cv2.convexHull(mouth_pts)], -1, (0,255,255), 1)

        if EAR_val < EAR_THRESH:
            eye_counter += 1
            if eye_counter >= EAR_CONSEC_FRM:
                if not eye_alarm_on:
                    eye_alarm_on = True
                    pygame.mixer.music.play(-1)
                    send_sms_bg(EAR_val)
                status = "🚨 DROWSY! EYES CLOSED"
                status_color = (0, 0, 255)
        else:
            eye_counter = 0
            if eye_alarm_on:
                eye_alarm_on = False; pygame.mixer.music.stop()

        if MAR_val > MAR_THRESH:
            yawn_counter += 1
        else:
            yawn_counter = 0; yawn_alarm = False

        cv2.putText(frame, f"EAR:{EAR_val:.2f} MAR:{MAR_val:.2f}",
                    (400,30), cv2.FONT_HERSHEY_SIMPLEX, 0.6, (255,255,255), 1)

    cv2.putText(frame, status, (10, 30), cv2.FONT_HERSHEY_SIMPLEX, 0.9, status_color, 2)
    cv2.imshow("Drowsiness Detector | iotprojectsandtrainings.in", frame)
    if cv2.waitKey(1) & 0xFF == ord('q'): break

cap.release(); cv2.destroyAllWindows()

Performance & Accuracy

🎯 Accuracy tip: The dlib CNN face detector (dlib.cnn_face_detection_model_v1()) is more accurate in low-light conditions but 5–10× slower than the HOG detector. Use HOG for real-time (30fps) and CNN for higher accuracy on a GPU or RPi 4.

Frequently Asked Questions (FAQ)

What is Eye Aspect Ratio (EAR) and why is it used for drowsiness detection?

Eye Aspect Ratio is derived from 6 facial landmark points around each eye. When eyes are open, EAR stays approximately constant (~0.3). When eyes close, EAR drops rapidly to near zero. Drowsiness is flagged when EAR stays below 0.25 for more than ~20 consecutive frames (~1.5 seconds).

Do I need a custom dataset to train this project?

No. This project uses dlib's pre-trained 68-point facial landmark model which is already trained on thousands of face images. There is no need to collect data or train a model. Simply download shape_predictor_68_face_landmarks.dat and use it directly.

Does this work at night or in low-light conditions?

Standard webcams perform poorly in low light. For vehicle use, add an IR camera module and switch dlib to its CNN-based detector: dlib.cnn_face_detection_model_v1('mmod_human_face_detector.dat'). This significantly improves night-time detection.

Can I deploy this on a Raspberry Pi inside a car?

Yes. Replace cv2.VideoCapture(0) with Picamera2 for the Pi Camera Module. Power the Raspberry Pi from the car's 12V socket via a USB-C car charger. The system runs at ~13 fps on a Raspberry Pi 4, which is sufficient for real-time drowsiness detection.

What happens if the driver is wearing glasses or sunglasses?

Regular prescription glasses work fine with dlib's landmark detector. However, dark sunglasses block the eye landmarks, making EAR unreliable. In this case, the system switches to head pose estimation (pitch angle) as the primary drowsiness indicator. Combining EAR + MAR + Head Pose improves robustness significantly.

What to Read Next