Facebook turned to artificial intelligence so that its 3D Photos feature will no longer require the use of Portrait mode on dual-camera smartphones.
Facebook’s 3D Photos, first revealed at the 2018 F8 developer conference and rolled out a few months later, utilizes the capabilities of dual-camera setups to create images with depth and movement when the smartphone is tilted. While any mobile device is capable of viewing the 3D Photos, the effect could only be created by phones with a Portrait mode.
In a blog post, Facebook said that single-camera smartphones will now be able to make 3D Photos through A.I. that produces 3D images from standard 2D pictures. The system is capable of inferring the 3D structure of any image, whether it was just taken with the smartphone’s camera or if it was taken years ago. This also enables 3D selfies, made with images taken with single front-facing cameras.
Facebook trained a neural network that can estimate the distance from the camera for each pixel, which eliminates the need for the second camera. The A.I. needed to overcome technical challenges that include correctly inferring the position of a wide variety of subject matter, as well as optimizing the system so that mobile devices will be able to complete the process in just a fraction of a second.
In order to overcome the challenges, Facebook said that it trained the neural network on millions of public 3D images and their corresponding depth maps while utilizing various mobile optimization techniques that have previously been developed by Facebook A.I.
The next step in the application of the technology is on videos, though that presents the challenge of consistent frame depths for a seamless transition into 3D effects. However, once Facebook figures this out, it will enable a whole new kind of content that creators can share on the social network.
Facebook’s advancements in 3D Photos are similar to the efforts of LucidPix, an A.I. app teased during CES 2020 that converts 2D pictures into 3D images. During the showcase in January, LucidPix was in beta with more than 250,000 testers and was looking to launch in the second quarter of this year for iOS and Android.