When the idea behind ClipDrop first appeared a few months ago, it seemed like magic: You could point the app at any object and then clip it out its background, creating a standalone image of anything from a plant to a coffee mug. It’s basically like copy and paste, but for anything. The app is now available in beta and I tried it… with mixed results.
Created by Parisian developers Cyril Diagne and Jonathan Blanchet, who co-founded a machine learning company called Init ML. ClipDrop is their first commercial product.
“We have more than 100k people registered on our beta waiting list,” said Diagne. He thinks the user experience — snap, snip, upload — is what makes this app stand apart.
“We spent considerable effort in making the user experience as fast, smooth, and intuitive as possible,” he said. “It has meant spending an unreasonable amount of time on various aspects of the application to remove as many steps and layers as possible.”
The project began as a research project that the two posted to Github. It took off, and hundreds of users and developers reached out wanting to try the app, according to Diagne.
“I posted a video of the prototype and it immediately got great traction. We’ve received thousands of messages from people across industries telling us how useful it would be in their work. So Jonathan and I spent the past few months making it available for anyone to use and download,” he said.
Aaand here it is..!!! ????
After months of hard work with @jblanchefr, @ClipDropApp beta (AR Copy Paste) is now publicly available on #Android, #iOS, #macOS, and #Windows
???? https://t.co/52eLMEfXNR ????
Here’s a thread of what you can already do with it ↓ 1/n#ML #AR #AI pic.twitter.com/0fQJQ8KRBv
— Cyril Diagne (@cyrildiagne) October 22, 2020
Should you try it? Absolutely. It’s really fun to “grab” things out of your environment and beam them to your desktop. And it works surprisingly well.
To be abundantly clear, this is a beta version of the app and definitely not indicative of its final performance. In the original videos, ClipDrop was supposed to let you yank real-world objects out of a scene and drop them into Photoshop. It does exactly what it supposed to do, but it’s also a bit finicky.
Take a look at this lovely scene. The printer is on a dark table and fairly well lit from the outside. ClipDrop was able to pull it out of the scene almost completely, leaving only a chunk of the table at the bottom as an unwanted artefact. Unfortunately, the shadows confused it a bit, making the Canon printer a little lumpen but definitely good enough to, say, drag into an eBay listing.
The same happened when I grabbed E.T. and his friend the Admiral. Parts of the scene didn’t quite escape the app’s algorithms, resulting in a rooster tail for Ackbar and a dark wedding ring for E.T.
As you can see, it’s not perfect. E.T. was actually behind Admiral Ackbar in the second photo and Ackbar’s 3D printed stand in grey plastic confused things further. That said, E.T. looks great, and there’s very little you have to do to stick him into an exciting forest scene.
Ultimately, what ClipDrop is doing is really difficult. Assessing visual planes is very hard, and doing it in a messy environment, be it on a desk or in nature, is almost impossible. The use case — taking a perfect picture of a jacket and then dropping it directly into a catalogue, for example — is not far off from what the app does right now.
You can also grab images from your desktop — a face in a crowd, say — and easily pull it out for editing. This is another fascinating feature that can save lots of Photoshop time.
Now for the bad news: You can try it five times before you have to pay. The OCR system, which grabs text out of the environment, is free, but to grab objects costs $US9.99 ($14) a month or $US39.99 ($56) a year. It’s a small price to pay for an app that can save you hours of time fiddling with select masks and eraser tools in Photoshop, but is definitely too much for the casual user.
The app does a few things really well and is well worth exploring if you’re doing any kind of image processing or even note-taking. Being able to grab text out of the real world isn’t a new trick nor is removing the background from an object, but historically both processes took quite a while and were quite frustrating.
ClipDrop isn’t perfect, but it’s fun. Just don’t go nuts and clip too many things at once. You only get five free clips, and once they’re gone, you’ve got to pay. Clip wisely.