This paper describes a real-time prototype computer vision system for monitoring driver vigilance. It uses a remotely located CCD camera equipped with an active IR illuminator to acquire video images of the driver. Various visual cues typically characterizing the level of alertness of a person are extracted in real time and systematically combined to infer the fatigue level of the driver. The visual cues used include eyelid movement, gaze movement, head movement, and facial expression. A probabilistic model is developed to model human fatigue and to predict fatigue based on the visual cues obtained. The simultaneous use of multiple visual cues and their systematic combination yields a much more robust and accurate fatigue characterization than using a single visual cue. The system was validated under real life fatigue conditions with human subjects of different ethnic backgrounds, different genders, ages, with/without glasses, and under different illumination conditions, and it was found reasonably robust, reliable and accurate in fatigue characterization.

Abstract—An eye detection algorithm based on Convolutional Neural Networks (CNN) architecture was developed. The algorithm was designed to detect eyes in video images from a remote gaze estimation system that is part of a gaze-controlled human-computer interface. The CNN for eye detection has two stages of convolutional and sub-sampling layers followed by a fully connected feed forward neural network with a total of 1227 trainable parameters. Experiments with 3 subjects showed that for the full range of expected head movements, the CNN achieved a detection rate of 100%, for images with fully opened eyes, and a false alarm rate of 2.65 X 10-4 %. The CNN failed to detect eyes that were either partially or completely covered by the eyelids. The CNN for eye detection did not require pre-processing or normalization and was shown to be robust to changes in scale, rotation and illumination of the eyes.