This Neural Network Enhances Phone Photos to ‘DSLR-Quality’

Want to turn your smartphone snapshots into DSLR-quality photos? A group of scientists in Switzerland is trying to help make that possible. They’ve created a neural network that aims to automatically enhance low-quality phone snapshots into “DSLR-quality photos.”

The research group at ETH Zurich detailed their new artificial intelligence photo enhancer in a paper titled “WESPE: Weakly Supervised Photo Enhancer for Digital Cameras.”

“Low-end and compact mobile cameras demonstrate limited photo quality mainly due to space, hardware and budget constraints,” the scientists write. “We propose a deep learning solution that translates photos taken by cameras with limited capabilities into DSLR-quality photos automatically.”

The scientists first trained a deep learning system on what makes a “DSLR-quality photo” by shooting photos of the exact same scenes using both smartphones (the iPhone 3Gs, BlackBerry Passport, and Sony Xperia Z) and a DSLR (a Canon 70D). Next, they also trained the system using a large set of DSLR-quality photos unrelated to the smartphone photos.

The results created using the large set of photos was superior in key ways to the set shot on-location with the DSLR, which “shows that training benefits from a data diverse dataset (different sources) of high-quality images with little noise levels, rather than a set of images from a single high-quality camera,” the scientists say.

So instead of using a giant dataset of original and enhanced pairs of photos, the system only requires a single set of photos from a new smartphone to operate since it’s trained using a set of high-quality photos that can be completely unrelated to the source camera photos.

Here are some before-and-after examples of how this neural network enhances smartphone photos:

Before. Shot on the iPhone
After
Before
After
Before. Shot on the Nexus 5X
After
Before
After
Before. Shot on the iPhone 6.
After
Before. Shot on the Xiaomi Redmi 3X.
After
Before. Shot on the HTC One M9.
After
Before. Shot on the HTC One M9.
After

As you can see from these sample photos, the neural network seems to have a bad habit of blowing out highlights (check out the clouds). These emerging technologies are continually refined over time, though, so perhaps it’s a weakness that will be addressed in the future.

While using AI to automatically enhance lower-quality photos is a problem that scientists across the world are tackling, these scientists at ETH Zurich say that their system’s advantage is being able to train itself for any new camera simply with a set of photos from that camera instead of using pairs of photos like traditional enhancement methods.

Want to try this neural network out yourself? The scientists have set up a webpage that lets you upload your own smartphone photos to see what the AI enhancement produces.

(via ETH Zurich via Engadget)

Discussion