Microsoft is developing an Eye-Gaze typing technology that allows users to write with their eyes

Microsoft might be releasing an Eye-Gaze typing technology that would help users type and interact with applications solely using their eyes, according to the patent, which was published recently and can be read here.

The technology employs a so-called ‘dwell-free’ typing, meaning users can type just by looking at keys on a screen. But it can also be used to interact with other applications by gazing at them: for instance, if a user looks at a button in a web browser, the system can click the button for them.

As eye movements can be jittery, and the recorded gaze might be non-uniformed, the noisy gaze will be intercepted by the dwell-free response system, which, as the paper explains, is a system that does not rely on dwell time for identifying a desired letter or word of a user.

This means users won’t have to gaze at the buttons for extended periods, as the technology would use the dwell-free system to make sense of the user’s gaze accurately and come up with solutions.

Employed in an operating system, such as Windows, this technology could let users control various settings such as audio volume, or brightness by moving their eyes. It would not be limited to just certain applications, as it can be extended to become a new way to interact with devices.

The most interesting aspect of this technology is that it also employs an AI-based model that gathers user information to make smart predictions and suggestions. If the user is looking at a certain area when navigating an application or operating system, the technology will associate these gazings with certain actions.

Microsoft is also making the Eye-Gaze technology predictive by using a so-called prediction processor, which guesses the word or action based on where they’re looking. The technology is also using smart algorithms to come up with quick responses to the user’s actions.

The Redmond-based tech giant has been experimenting with the eye-gaze technology on its HoloLens headsets, although in that case, the experience exists in a mixed-reality setting. Apple’s Vision Pro device also employs a similar technology, but users still require hand movements happening in a mixed reality setting, as well.

This technology, however, would be employed on all devices, including, but not limited to mobile phones, tablets, and personal computers, and it can be run on any application that has text-input capabilities, such as text messaging apps, email apps, or search engine apps.

Could we expect this kind of technology in Microsoft Edge or the future AI-based Windows? Microsoft has already experimented with gaze-controlled settings in Windows 11, but this is the next level, and quite frankly, it’s both exciting and terrifying.