Redecorating a room is a big commitment, but one of the more intriguing uses for augmented reality is letting users preview new furniture pieces in a space. Thanks to a new feature Apple introduced to developers last month, you won’t have to move a single item before dropping a new virtual couch or coffee table into your real world space.
The feature is a new Swift API called RoomPlan that leverages Apple’s ARKit as well an iPhone or an iPad with a camera and LiDAR (short for Light Detection and Ranging). The scanner can easily generate detailed and accurate 3D floor plans, including features like windows, doors, and even various types of furniture that are automatically recognised and accurately measured. Then, it can edit the furniture out of your shot while keeping the room intact. When it was revealed At WWDC 2022 last month, Apple introduced the new API with a video that demonstrates what the RoomPlan API can do and how it can potentially be used in mobile apps.
Using augmented reality to try out a new piece of furniture in a room before you buy it isn’t an entirely new idea. IKEA’s been doing it for years through its mobile apps, and last month the Swedish furniture maker revealed a new design tool that can generate a static image of a room with all of the existing furniture digitally erased, allowing virtual furniture to be added and moved around. It was an interesting upgrade, but one that lacked the interactivity of being able to move around the emptied virtual space and see the added furniture from different angles.
Using the new RoomPlan API and a few other AR tools, digitally erasing existing furniture in an interactive 3D room is apparently something that’s quite feasible, as Russ Maschmeyer, who currently develops AR and VR experiences for Shopify, recently demonstrated on Twitter.
Lots of interest in this “reset button” for your room we built @Shopify that uses @apple’s new #RoomPlan API. Here’s how we built it… 👇 🧵1/12 pic.twitter.com/YdNP1vebVW
— Russ Maschmeyer (@StrangeNative) July 13, 2022
The video shows off a new augmented reality tool that Maschmeyer describes as a reset button for a room. After a LiDAR equipped Apple device is used to scan and capture imagery of the space, everything from furniture, to rugs on the floor, to even vents and AC units on the walls, can be instantly erased, leaving behind the interior designer’s equivalent of a blank canvas.
#RoomPlan outputs an untextured USDZ model composed of unit cubes; one for each door, window, wall, and room-defining object—but, surprise! Models don’t contain floor or ceiling elements. We align the model with the real world using ARWorldMap. [7/12] pic.twitter.com/MiagVaWb74
— Russ Maschmeyer (@StrangeNative) July 13, 2022
The RoomPlan API is only designed to generate a plain 3D model of a room, including crude shapes that represent furniture and other objects inside it. To preserve the original appearance of the captured room with everything in it magically erased, Maschmeyer and his team used the scanning device’s camera to capture textures of walls and floors that were then applied to the surfaces of the 3D model, which is overlaid on top of the existing room in real-time using augmented reality. This technique occasionally required some textures that were obscured by furniture to be digitally patched and extended, but that is functionality that can one day be incorporated into an app and handled automatically, as the new IKEA Kreativ tool does, but with full 3D interactivity for the user.
The days of bringing paint swatches home from the hardware store and holding them up to the wall while trying to imagine what the whole room will look like in that shade could be long gone. But more importantly, augmented reality’s biggest promise could be furniture shopping without ever having to navigate IKEA’s maze-like showrooms ever again.