Structure Sensor, the world’s first iPad 3D scanner, is the sixth highest-earning project ever funded on Kickstarter and one of the most highly anticipated gadgets of the year. Does it deliver on the promises that made thousands of backers feel like they were seeing the technology of 2020? It definitely feels like a glimpse of the future.
For the past week, I’ve had the chance to play with the Structure Sensor in my apartment and around the neighbourhood. I’ve described the experience below — but keep in mind, I’m neither a developer nor a gamer. Still, it was easy to see why this little device has so much potential.
What Is It?
Structure Sensor is a tiny, portable 3D scanner that snaps into your iPad, made by the startup Occipital.
At $US399 — or $US499 if you want the super cool photoreal scanning software Skanect — this clip-on bracket gives you access to two infrared LEDs and camera that clips on to fourth generation iPads, iPad Airs, and iPad Mini with Retina. It gives your mobile device a sixth sense: Space. In other words, Structure Sensor lets you “see” depth, which means you can do everything from creating a 3D model of your house, to playing augmented reality games right in your own living room, to 3D-scanning your dog.
Why does it matter?
Structure Sensor was Kickstarted to massive success last fall, blowing through its $US100,000 goal to raise $US1,290,439. Why was the internet so excited about this little gadget?
Well, most 3D scanners are huge, heavy contraptions that cost thousands of dollars. Occipital has figured out a way to make it tiny, inexpensive, and compatible with a device you’re likely to already own. “Instead of waiting for someone else to create that sensor, we turned ourselves into a hardware company to create the Structure Sensor,” its creators told me this spring.
But Structure Sensor isn’t just about 3D scanning or augmented reality. As the company has explained, it’s a platform. It’s not supposed to be a self-sustaining tool or toy. Rather, it’s an amazing piece of hardware and an SDK that makes this hardware into clay for you to shape to your own uses. The apps and programs will largely depend on the community of developers who have invested in it. But the fact that thousands of people ponied up the cash to get in on the ground floor bodes well.
Design
Structure Sensor’s nervous system is a tiny infrared sensor made by PrimeSense — the Israeli startup that makes the sensors for Kinect and was acquired by Apple last year — that reads near-infrared light. That’s the kind of light that changes based on the distance between the objects. Then, using a complex algorithm, it turns that data into a three dimensional model.
It’s a complex piece of hardware, and fitting it into an anodized aluminium frame that’s no wider than two fingers fell to New Deal Design, the same studio that created Lytro and FitBit. It shows: The sensor is incredibly sleek. It arrives in either a gunmetal grey or a deep blue and slips onto your iPad without leaving any marks whatsoever. There’s not much more to say about the industrial design, which is actually a good thing; it recedes into the background of the user experience itself.
Using It
Hooking the device up to your iPad is about as simple as putting on a new cover. You take it out of the box, screw in four tiny screws, and slide it over the edge. It locks in place with a clever tightening mechanism, and there’s a short cable that links the sensor to your iPad’s lightning dock. If the sensor needs a charge (since it contains its own battery), it plugs into any old outlet and charges quickly thanks to an enclosed cord.
Once you’re all set up — which should take ten minutes tops — you’re ready to start scanning. Occipital has developed four demos to go along with its little device, each showcasing a different aspect of what it can do.
First, there’s Object Scanner, which lets you capture spatial data about a space, object, or person, and export it to either a 3D modelling program or a 3D printer. It can take a while to get the hang of; you need to get into the groove of how fast to scan and how much to move. It’s also tricky to adjust the wireframe box that controls what the sensor is scanning. You do so by pinching to scale it up and down, but when it’s the size of a room, it can be hard to tell how much of your space it’s encompassing.
I tried scanning all kinds of spaces, and it’s important to note that the models it generates are relatively low-resolution. Rooms are rendering in very broad strokes, so you won’t be getting perfectly detailed 3D models. That said, it’s extremely accurate when it comes to measuring distance, and only takes a few seconds to scan, which is a major plus.
Viewer, meanwhile, is a simple demo that shows you all of the depth and surface normals the sensor is collecting at any given time.
Then there’s Ball Physics, a delightful little demonstration of how the sensor can model the physics of the real world onto a scene viewed on your iPad’s screen. You hold it up to your living room, say, and then you can drop balls into it to see how they bounce off of your furniture and floor on the screen, based on the IR data being estimated simultaneously:
Fetch is another augmented reality demo, but instead of physics, it’s a little introduction to the gaming capabilities of the sensor. You hold it up to the world, and suddenly, you’re playing a game of — what else? — fetch that takes place right in your living room, with this little guy doing all the heavy lifting:
But the real crowdpleaser is the independently-developed app ItSeez3D (tagline: Mobile Computer Vision) made by an eponymous Russian startup that’s been working on computer vision since 2005. This is arguably the most important app, since ItSeez is an independent company, and one of the first to participate in the Structure Sensor SDK. It’s an early glimpse of what’s possible when developers get their hands on the platform.
The program overlays a photo map onto the spatial data, creating surprisingly lifelike models (and uncanny valley-style portraits). It guides you through the scanning process with helpful tips:
And then it renders incredibly realistic models, not only of humans, but of objects, too. It made quick work of this (fake!) bluebird and other simple objects:
And when I decided to give it a huge challenge by scanning a giant pot of basil, which has a fairly complex geometry, ItSeez did a pretty serviceable job — despite my more than likely flawed technique:
Amazed that ItSeez could make any sense out of my plant, I decided to give it an even bigger challenge: The detailed geometry of a road bike. It didn’t go quite as planned — chalk it up to my inexperience as a scanner or my lack of bright light. But it was still fun as hell to see what the sensor would even make of it:
Keep in mind that those complex geometries aren’t the intended target of this app. You can make a much more complete rendering of a human face — and creepily 3D print it! — or a solid, photoreal replica of any object you choose. For example, ItSeez has uploaded dozens of human models to SketchFab, like this one:
Like
First of all, using Structure Sensor really does feel like using technology from ten years from now, today. It’s simply a different sensory experience than the one we’re used to. It’s also addictive: The more I scanned my apartment, the more detail I wanted to capture. Sure, I’d gotten the outline of my kitchen. But what’s a kitchen without the specifics? Soon I’d spent 30 minutes pacing around my kitchen, iPad in hand, like a crazed tourist trying to capture every detail from every conceivable angle.
Structure Sensor is dead easy to set up and get going, which is saying a lot when you’re talking about a technology that only a tiny fraction of the general population has any contact with, much less understands its potential. Though I can’t say I became a master scanner during the few days I spent with the sensor, I was up and scanning within 15 minutes of opening the box. And though getting the hang of it was frustrating, every success hooked me in even more.
No Like
This is not a perfect 3D scanner. In fact, it’s not perfect at any one specific thing that it does. But then, it doesn’t claim to be. Rather, Structure Sensor is solid at many, many things, a foundation for developers and companies who want to build their own apps and software using it. That said, if you’re looking for a cheap 3D scanner that can take super-detailed models, this isn’t it. The captured data is still very rough, at 640 x 480, so this is no match for more expensive scanners out there.
It’s also still fairly glitchy, and it takes a certain amount of patience and practice to get it to scan a full model. For example, my first dozen attempts to scan my living room resulted in bizarre models like this:
Should You Buy It?
The technology at the heart of Structure Sensor is rapidly proliferating throughout other product categories. Google’s Project Tango doesn’t have exactly the same electronic guts, but it does largely the same things that Structure Sensor can do. Apple, meanwhile, now owns the company that invented the sensor that’s inside Structure Sensor. Though we don’t quite know what its plans are for the technology, it’s likely that these sensors will be making their way in your life, in one device or another.
In that light, Structure Sensor is the independent, hackable version of a technology that’s about to be everywhere. So if you’re a developer, yes — you should buy it. It’s just as incredible a platform as it claims, and it has the potential to become a gateway for a huge slew of new apps, from augmented reality games to architectural rendering programs.
But for those who simple want a cool toy? Wait a few months. Not only to see how things with Google and Apple shake out, but to see what happens as thousands of developers and startups begin putting Structure Sensor’s SDK to good use. For now, it’s more of tool than a toy — and at $US499 with the software, it’s an expensive one.
Which brings us back to why Structure Sensor matters. Beyond a device, it represents a sea change in how consumer technology is made and marketed. Unlike the average iPad peripheral, the sensor isn’t a finished product. It’s the guts that could power a million other products, from apps to games. This is a participatory process: The consumers and independent companies who buy Structure Sensor will be actively contributing to the process of turning it into a piece of hardware that has thousands of different applications.
Occipital wants — and needs — your help to make Structure Sensor a success. And the more people and companies who accept the challenge, the better it will get. It’s early days, but this little sensor could become something truly great.