Facebook hinted at a brand-new feature called 3D images in May, and it is exactly what it sounds like. However, not much was stated about it other than a little video and the name. The computational photography team at the company, however, has just disclosed the research that underpins the feature’s operation, and after using it myself, I can attest that the outcomes are actually rather convincing.
Where is it?
In case you missed the teaser, 3D photographs will appear in your news feed alongside regular photos, with the exception that when you scroll by them, touch or click them, or tilt your phone, they react by changing perspective as though they were a window into a miniature diorama. It will function for common photographs of people and dogs as well as panoramas and landscapes.
I’m about as cynical as they come and it seems a little cheesy, but the effect immediately won me over. It feels more like a small magic window seeing into a time and place than a 3D model, even though that is exactly what it is due to the excellent depth illusion. This is how it seems.
What is The Real Purpose?
It’s interesting to note that the objective behind 3D images wasn’t to improve snapshots, but rather to make VR content development more accessible. All of it is synthetic, Kopf noted. Furthermore, no casual Facebook user has the skills or desire to create 3D models and populate a virtual environment.
The exception to this rule is panoramic and 360-degree imagery, which is typically broad enough to be usefully studied through VR. However, the experience is not much different from simply gazing at the picture that is floating around and printed on butcher paper. Not quite transformational. There is no sense of depth, that’s what.
How Does it Works?
The two cameras on the phone take a pair of pictures, and the gadget immediately works on its own to create a “depth map” out of them—an image that encodes the computed distance of everything in the frame. All of the major smartphone manufacturers, including Apple, Samsung, Huawei, and Google, have their own built-in systems for accomplishing this, albeit so far it has mostly been employed to produce artificial backdrop blur.
The issue with that is that the depth map that was produced lacks any sort of absolute scale; for instance, light yellow does not indicate a depth of 10 feet, whereas dark red indicates a depth of 100 feet. An photograph of a person taken a few feet to the left would show yellow representing one foot and red representing ten. If you capture more than one photo, let alone dozens or even hundreds, there won’t be any constant indicator of how far away an object actually is, making it difficult to stitch them together properly.