Magic Leap 2 uses cameras and infrared LEDs to estimate the location and orientation of your eyes in order to improve the quality of rendering (by estimating the center of perspective for the system’s virtual scene cameras) and to enable interaction scenarios using, for example, the gaze vectors and/or blink state of the eyes.
Fit and Calibration
In order to track your eyes, the eye cameras need to have a clear view of the eyes. The “Custom Fit” app on the device can guide you to the proper fit which will ensure that the eye cameras have a good view of the eyes, and that the user has the best view of the displays.
Once the eye cameras have a good view of the user’s eyes, the Magic Leap 2 immediately begins estimating the location of the eyes in space; measuring inter-pupillary distance (the distance between your eyes), recognizing blinks, and classifying the types of eye motions being observed. However, to get a good estimate of where you are looking the eye tracker needs to be calibrated because the location of the fovea varies from person to person. This process is carried out in the second half of the Custom Fit application.
Supported Range of Operation
Eye tracking provides the most accurate estimates of where a user is looking within approximately 15 degrees of the eye’s “neutral gaze", or the eye position when the eye muscles are relaxed - like if you looked into your own eyes in a mirror. This is true of most eye trackers, and is related to the way that the angle of the line of sight is modeled mathematically. Most people naturally move their heads to roughly center an object in their sight rather than direct their gaze at extreme angles, but it is possible (although uncomfortable) to do this. Outside of 15 degrees, the eye tracking accuracy will degrade steadily.
Additionally, although eye tracking doesn’t have any preferred gaze directions, looking downward presents more difficulty than other directions because the upper eyelid tends to drop when looking down and this can prevent the eye cameras from seeing the reflections of the infrared LEDs that are used to estimate the position of the eyes.
A quick note about fixation distance (the distance in meters from the mid-point between the users eyes and the intersection of both eyes gaze directions): while eye tracking can estimate the distance to the user’s fixation point, this is purely based on the convergence of the eyes. The consequence of that is that fixating at3 meters or 10 meters will both present as essentially parallel gaze vectors. The distance is only reliable in the less than 3 meter range.
Eye Tracking and Power Management
In addition to supporting rendering and interactions, Eye tracking is also used to improve battery life by setting the device to standby mode when the device is not being worn. In our own experience working with this feature internally we determined that often users would like to be able to close their eyes when listening to spatial audio from the device, or to temporarily place the headset atop their head and then wear it again. To support these types of use cases the power management aspect of eye tracking is more “lenient” than the other aspects of the system (which require a good view of the pupil for instance). The power management aspect of eye tracking is best thought of as a “face detector” – it keeps the device on, even if your eyes are closed or the device is on your forehead. However, if the eye cameras end up in your hair the device might not be able to distinguish up-close hair from being in a bag and go into standby. On the other hand, if you place the device on a bare arm or leg, then eye tracking cannot distinguish “arm skin” from “forehead skin” and this will also prevent the device from going to standby.
Eye tracking can work with contact lenses. Sometimes well-worn soft contact lenses can become rough and/or fit poorly which can lead to spurious reflections from the infrared LEDs and cause degraded performance. If you are experiencing poor eye tracking reliability and your contacts are both old and disposable, then it may be worth trying a fresh pair. If you don't wear contact lenses, you might be tempted to try to wear the Magic Leap headset over their glasses. This will cause eye tracking to fail due to extra reflections from the surface of the glasses lenses (not to mention it might be a little uncomfortable).
Magic Leap offers prescription inserts that have been designed to fit the headset and not interfere with eye tracking’s operation. See Prescription Inserts for more information.
Eye tracking data is only processed locally on the device. For complete details please see our transparency policy: https://www.magicleap.com/eye-tracking