[youtube https://www.youtube.com/watch?v=3RYNThid23g]
Watching old documentary footage lets us get a glimpse of how life was in an earlier era. The effect isn’t totally convincing, though. Whether it’s incorrect projection speeds, decrepit film stock, or just the distinctly non-4K quality of old-fashioned cameras, watching this footage today still feels like, well, you’re watching old footage.
Cutting-edge artificial intelligence (A.I.) could help change that, however. This week, YouTube user Denis Shiryaev uploaded a 4K, 60fps version of the 1896 Lumière Brothers film “Arrival of a Train at La Ciotat.” Using A.I. upscaling technology (and an added soundtrack), the 1-minute film has been cleaned up to the point that, black-and-white color scheme aside, it could have been shot yesterday. Or tomorrow.
To create the updated film, Shiryaev used two publicly available enhancement programs, DAIN and Topaz Labs’ Gigapixel A.I. The first of these tools inserts new frames into a video clip to smooth the viewing experience. The second identifies details and structures within images and then “completes” them.
“We usually think of upscaling as scaling-up an image, creating more pixels by averaging out the existing ones, but this makes the result look extremely blurry,” Harry Robins, a film production student who works with A.I. upscaling tech, told Digital Trends. “With machine learning algorithms, the program has been given millions of images, all low resolution. Then it’s given a higher-resolution version of the same image [and] has to try and learn how to turn the first one into the second. After each iteration, it [learns to improve] upon itself.”
In the instance of the Lumière Brothers film, for instance, the ground has a texture that the A.I. picks up on. It then pastes on a higher-resolution texture from what it has learned. The same is true for other textures in the short film. While it’s not the perfect replacement for rescanning an original film negative in all cases, it’s nonetheless a super-impressive demo.
“There are stunning advances in deep learning, deep fake, 4K upscaling, and a host or incredibly exciting tools at the disposal of artists and restorers,” said Stuart Humphryes, a film colorizer and editor currently collaborating on a project involving A.I. upscaling. “Used in the right context, by a skilled hand, it can completely revolutionize the way people perceive archive film. It can bring immediacy and, with that, a closer emotional connection to the past.”