Bridging the gap between human intent and robotic control using eye tracking.

The Engineering Deep Dive

Part of my thesis research addressed the problem in eye tracking, where the difficulty of distinguishing between a user simply looking at an object and a user intending to interact with it. Using a Gazepoint GP3 Eye Tracker, I captured raw gaze vectors and processed them in real-time using a custom Python/Matlab pipeline. I developed a shape vector analysis model that looked at the velocity, acceleration, and fixation stability of the eye. By identifying specific intent patterns, such as, a specific dwell time combined with saccade time the system could trigger commands on a NAO humanoid robot or computer. This allowed users with limited mobility to control a NAO robot using only their eyes, effectively turning gaze into a input device.

The Technical Post Mortem

My research focused on the problem of communicating intent from a non standard input. I built a pipeline that calculated shape vectors of a gaze to distinguish intent from observation. I developed a custom API to bridge this model with the NAO humanoid robot, allowing mobility limited users to communicate via gaze gestures.

Engineering Constraints

Solving for the 'Impossible' means navigating rigid physical and computational limits:

  • Distinguishing involuntary eye movements (saccades) from active user intent.
  • Real time serialization of intent to control NAO robot or computer.
  • Normalizing data across varying ambient light and pupil conditions.