At Google I/O, AI was a theme throughout the event, but another game-changing assistive technology was announced that lets your face control a computer instead of using a mouse. Google’s Project GameFace literally turns your face into a game controller. If even the best mouse or trackball is prving difficult or impossible, this free app can significantly improve your computing experience.
Alan Truly tries Project GameFace while watching Lance Carr.
Google’s blog post describes how Project GameFace was inspired by the plight of video game streamer Lance Carr, who has muscular dystrophy. A fire led to the loss of Carr’s expensive head-tracking mouse, which had allowed him to keep gaming.
Related Videos
Google worked with Carr to develop Project GameFace, a solution that uses a standard webcam and works on any Windows PC. You can download the app from the GitHub project or fork, and customize it for your own particular needs if you know Python.
I installed and tried it. My built-in webcam worked for pointer control, but was too blown out by a nearby window to detect my eyebrows and mouth well. I hooked up an Insta360 Link, which includes face-tracking technology to pan and tilt as I move around. The two technologies blended seamlessly and everything worked great after some fine-tuning.
The cursor speed was much too fast, so I dropped from the default setting of 40 to 15 vertically and 20 horizontally. With these adjustments, I could move my mouse pointer to any point on my screen simply by turning and tilting my head. This isn’t eye-tracking. It seems to follow my nose.
Google’s Project GameFace has adjustments for cursor speed.
The app recognizes six face gestures: open mouth, mouth left and right, raise left and right eyebrow, and lower right eyebrow. It’s unclear why Lower left eyebrow isn’t an option. These gesture settings, combined with head movement for directional control, make it relatively easy to bind keys and mouse clicks, allowing you to play a game or control a computer with only your face.
There’s plenty of room for improvement here. Ideally, groups of gesture bindings and cursor speeds should be saved for each app so you don’t have to find a single configuration that works for everything. It would also be more convenient to require a confirmation click, with a countdown to revert, after adjusting cursor speed. It took me a few minutes to tame my wild mouse pointer when I first switched on face tracking.
Overall, Project GameFace is a wonderful adaptation that should make life much easier for many people who could otherwise struggle to navigate a computer or play a game. More tech companies should be taking advantage of gesture control enabled by a webcam.