Ever wonder why your photos never turn out as amazing those posted by your favourite Instagrammer? There’s probably a lot of post-processing happening in Photoshop you don’t see. But instead of poking at sliders for an hour, computer scientists want to make it incredibly easy for even amateur photographers to achieve results comparable to a professional’s.
Photos: Fujun Luan
In a paper recently posted to the arXiv pre-print server titled Deep Style Photo Transfer, Sylvain Paris and Eli Shechtman from Adobe, working with Fujun Luan and Kavita Bala from Cornell University, detail a new deep-learning approach to post-production and colour correction that can automatically apply the visual aesthetics of one photograph (lighting, colours, tone) to a completely different shot, with results that still look photorealistic.
Image-processing algorithms like this are not new, but the results often tend to have a painting-like aesthetic to them. Fine details get lost, straight lines get warped and distorted, and the colour changes are applied to broad regions of an image — which is far from ideal since it requires further processing afterwards to fix mistakes.
The goal here was a cleaner one-step transformation, so the research team turned to neural networks and deep learning. They’re phrases that are thrown around a lot today when it comes to artificial intelligence, but are essential approaches to automating a complex process like this. Specifically teaching software to spot and process every possible object on Earth is impossible, but by having it make corrections to thousands of sample images, with feedback on when it has done a good job and when it hasn’t, over time the algorithm will adapt and learn.
Eventually, without being taught what a building is or looks like, the algorithm will automatically know that colours appearing in the sky regions of a photo shouldn’t be applied to man-made structures. The new algorithm is also designed to only make tweaks to an image’s colours and tones, so it preserves details and doesn’t produce any warping side effects.
Photos: Fujun Luan
Are the results always perfect? No. As this photo of a snowy landscape being transformed into a lush tropical valley reveals, there are some things the algorithm can’t account for. The details of a field full of plants can’t really be recreated by just colour-correcting a snow-covered plain to appear green. But the researchers have found that the algorithm works about 80 per cent of the time without the need for any manual tweaking.
Eventually, this software could make its way into Photoshop or other image processing applications, and while talented photographers probably won’t like the idea of someone being able to instantly steal their style, the practical uses of a tool like this seem to outweigh those concerns. If a client asks for a night time version of a photo that doesn’t exist, a photographer could quickly make an altered version without losing an hour making manual tweaks in Photoshop.
And at the very least, it should help up everyone’s Instagram game.
[Deep Photo Style Transfer via DPReview via PetaPixel]