Google’s AI Can Detect When A Stranger Is Looking At Your Phone

Google is working on a new security feature that could protect against strangers stealing glances at your phone over your shoulder. Now, a couple of Google researchers have figured out a way to detect when someone is looking at your phone.

The software is called an “e-screen protector,” and is currently just in the research stage. It’s quite simple: it uses your front-facing camera in combination with some face- and gaze-detection algorithms to identify anyone looking at your display.

According to Ryu and Schroff, the program can recognize a second face in just two milliseconds, and works across a number of angles, poses, and lighting conditions. And while more details will be announced at the presentation, a demo of the software in action can already been seen in the unlisted, but public video below.

To achieve such fast recognition, it seems the team’s program is using TensorFlowLite, Google’s latest venture into AI and machine learning which uses the processor in your phone to perform complex visual analysis rather than needing to ping beefier servers in the cloud.

The system takes just two milliseconds to detect a stranger’s gaze, and 47 milliseconds to recognize a face. It also takes 115 milliseconds to detect a face in each frame. With that speed, even a passing glance is likely to be spotted!
“Because of the quick, robust, and accurate gaze detection mobile model, we can now easily identify the face identity and gaze simultaneously in real time,” they explain in a brief description.
“Hence, the application, an electronic screen protector, can enable its enrolled users to continue reading private and confidential contents on your mobile device, while protecting their privacy from onlookers in a crowded space such as the subway or an elevator.”

In the demo video, above, you can see it reacting almost instantaneously to a suspicious looky-loos, switching the user’s screen to catch the culprit red-faced, and tagging them with some Snapchat-style rainbow vomit for good effect. The video shows that the algorithm can detect a stranger’s gaze within milliseconds.

In an early test version of the functionality shown off to Quartz, the app recognizes that someone is looking at the screen and immediately switches from the Hangouts app to a camera viewfinder which highlights the person looking at your phone. Once they turn away, it switches back to the previously open app.

Researchers at Google, developed a method to use the front-facing camera on a smartphone (2016 Pixel) to detect not just when another person’s face is in view, but when their eyes are actually looking at the phone’s display.

Although we can’t say whether or not Google will ever bring this feature to Android, it’s a perfect example of the sort of small software tweaks AI can offer mobile devices. How long would it take for this functionality to make its way into the greater Android ecosystem—when?

The “E-Screen Protector” project will be presented more in-depth at the Neural Information Processing Systems conference in Long Beach, California next week.

To Top