For decades, we’ve lived with an inconvenient technological truth: Cameras and other sensors cannot occupy the same space as our screens. It’s why, increasingly, smartphones rely on the dreaded “notch” as a way of maximizing screen-to-body ratios while preserving the front-facing camera and other sensors.
Some phone makers, from Oppo to OnePlus, get around this problem by using motorized pop-up cameras, while others have resorted to punching holes in displays to provide the camera with its own peephole. It’s also why even the latest high-end laptops still have pronounced bezels around their displays. The webcam needs a home and it seems no one is willing to live with a notch or hole-punch on a computer.
But it turns out that cameras and screens aren’t quite as incompatible as they seem. Thanks to improvements in manufacturing techniques, these two adversaries are about to end their long-standing territorial dispute. This isn’t a far-flung prediction; it’s happening right now.
By the end of 2021, we will see the first smartphones with invisible, under-display cameras (UDC). Laptops, tablets, and TVs will follow.
How will this be accomplished, and how will it change the way we use these devices?
Here’s a glimpse into our hidden camera future.
Why hide the camera?
Joseph Mucira / Pixabay
Complaining about a phone notch, hole-punch or a large screen bezel is the very definition of a first-world problem. And judging from Apple’s stellar sales numbers, none of these side effects of forward-facing cameras are dealbreakers for buyers.
But aesthetics aside, there are three major benefits to hiding cameras behind displays.
First, it lets you make phones that have true edge-to-edge screens. Videos and photos look better, and app developers can make use of every square millimeter for their designs — all while keeping the phone’s body as small as possible.
Second, from a design and manufacturing point of view, if cameras and sensors can be placed anywhere, with fewer restrictions on their size and visibility, it redraws the map for phone design. Bigger batteries, thinner phones, more sensors, and much better cameras are all potential upsides.
But the third, and arguably biggest, benefit is the ability to line up the camera with our gaze.
Cameras placed in bezels or notches create the now all-too-familiar, awkward downward gaze that happens during video calls. “Most of the time, you’re not actually looking at each other when you’re talking over video chat,” Michael Helander, CEO at Toronto-based OTI Lumionics told Digital Trends. “The current placement of videoconferencing cameras in all of these devices is really suboptimal.”
Helander has probably thought about this problem more than most. His company creates specialty materials that enable what was once impossible — making displays transparent enough that you can place a camera behind them.
Once a camera is sitting behind the display, it will finally make our video interactions look and feel like real, in-person interactions — a game changer that couldn’t come at a better time in our COVID-restricted world.
How do you make a display transparent?
In 2020, Xiaomi introduced the first transparent OLED TV for consumers. Xiaomi
Screen technology is dominated by two kinds of displays. The most common are liquid crystal displays (LCD), which include LED TVs and QLED TVs. The second, organic light-emitting diode (OLED), dominates smartphones and tablets, and is growing in use in laptops and even desktop monitors
LCDs are actually transparent when not in use — that’s why you see a gray background on a calculator screen wherever the black digit segments aren’t active. But taking advantage of this transparency to take a photo poses big technical hurdles, especially once you factor in the need for a backlight.
The active portion of an OLED display, on the other hand, is paper-thin. Its various layers are measured in nanometers, making it the perfect candidate for transparency. Its nanometer-thin top layer of metal is already translucent for visible light, but infrared light is totally blocked.
So the question becomes, how can you increase OLED’s transparency without damaging the display with cutouts?
One solution favored by Xiaomi and Oppo in their UDC prototypes is to rely on an OLED pixel’s inherent transparency. When an OLED pixel isn’t being used to emit light, it lets light in. So you can place a camera behind an OLED display and it will be able to gather enough light to capture images. But there’s a catch: You still need to place the camera at the top or bottom of the screen, because when the camera is active, the OLED pixels above it must be shut off, which creates a temporary black area on the screen. That approach is a solution to the notch and hole-punch problem, but it does nothing to solve the downward gaze issue.
Another way to achieve transparency is by creating small physical holes that fit between a display’s pixels, but that’s incredibly difficult in its own right.
The first commercially available phone with an under-display camera — the ZTE Axon 20 5G — uses this technique, but it also suffers from a less-than-ideal compromise. Modern smartphones have incredibly densely packed pixels. The iPhone 12 Pro has a 460ppi (pixels per inch) display, which means that there are more than 200,000 pixels in one square inch. Sony’s Xperia XZ Premium had a whopping 807ppi screen (more than 650,000 pixels per square inch).
Punching holes in between those pixels, even with a laser, is so tricky that ZTE had to remove some pixels from the area above the camera to buy some extra room. The result is a noticeably lower-resolution square on the screen.
A close-up shot of the ZTE Axon 20 5G screen, showing the camera area. Io Technology/YouTube
What’s also noticeable (because of the size of the holes) is the camera itself, which becomes visible at some angles.
A lower-resolution section of the screen might not bother you when it’s near the top, in an area that’s used mostly for inconsequential information. But few people would accept such an obvious reduction of resolution in the center of their phone’s display, which is what we would need to counteract the downward-gaze problem.
See and be seen
OTI Lumionics
But there is a third option. What if, instead of relying on transparent pixels, or punching holes in the display after assembly, you could create millions of tiny holes in each layer of an OLED display during manufacturing?
Through a process known as patterning, existing manufacturing techniques have brought us tantalizingly close to this scenario.
“We know how to do that in the TFT [thin-film transistor] layer,” Helander said. “We know how to do that in the bottom electrode. We know how to do that in the layer that makes up all of the different pixels.” But the top metal layer, also known as the cathode, isn’t created like these other layers, and that poses a unique engineering challenge.
Self-assembled holes
Electron microscope shot showing OTI Lumionics’ self-assembled holes in an OLED cathode layer. OTI Lumionics
The top metal cathode isn’t a sheet of metal in the conventional sense. Instead of bonding a separate metal sheet to the top of the display, metal molecules are vaporized and allowed to condense over the entire surface, a process known as vapor deposition.
Trouble is, once that metal layer has been deposited, there’s no way to pattern it. That brings us to OTI Lumionics’ secret sauce.
“The technology that we’ve developed is a way of patterning millions of tiny holes in that layer during the manufacturing process through what’s called self-assembly,” Helander said. “When you lay all these materials together, they’ll naturally form all of these little openings in the display, millions of them.”
Helander claims the self-assembly process works on any screen size, and lets manufacturers decide how many openings are needed — from just one to 1 billion.
Once these openings exist, visible and infrared light can pass through unimpeded.
Trust, but verify
As exciting as it is to think that we’ll soon be able to have much more natural video calls, placing a camera under a display puts an even bigger onus on manufacturers to provide trustworthy privacy measures.
We’ll need some kind of reliable indicator of when the camera is active and an equally reliable way of disabling it. Because it’s under the screen, there’s no way to physically block the lens without blocking content on the screen as well.
Apple recently updated iOS to show a small green dot near the notch when its forward-facing camera is in use, and an orange dot to show when the mic is active. That’s a good way to inform us of what’s going on, but we need something more.
Smart speakers like the Google Nest mini ship with physical switches that can be used to disable the microphones. Assuming that there’s no way to remotely overcome the switch’s position, it provides a very good level of trust. A similar mechanism on TVs, monitors, and laptops should come standard once cameras become invisible.
When will under-display cameras start showing up?
OTI Lumionics already has agreements in place with several Chinese smartphone manufacturers, but due to confidentiality restrictions, these companies can’t be named just yet. “Many of them have prototype phones that have been built and everything looks great,” Helander notes, “but none of them want to disclose anything publicly until they’re ready for their actual official product announcements.” He’s confident that we’ll see these new under-display camera models sometime in 2021, although they may remain a Chinese market exclusive until 2022.
How much will they cost?
I was fully prepared for Helander to tell me that only the most premium smartphones, commanding prices of $1,000 or more, would be first to market with UDCs. But the first models are expected to be midtier handsets. He attributes this to the ferocious competition among the Chinese brands in the $400 to $600 smartphone market, which has led to a willingness to try new features faster, even if they fail to catch on.