Prior to the release of CompanyCam’s highly-anticipated AR measuring feature, I had the opportunity to sit down with two of the men behind it — Luke Hansen, CEO of CompanyCam, and Dr. Dan Maienschein, CompanyCam’s resident Doctor of Mathematics and the brain behind our newest feature.
Q: What is ‘augmented reality’?
Luke: Augmented reality (AR) is when you put things, digital objects, in the real world. You can perceive them in the real world through your phone or special glasses. A lot of people confuse augmented reality with virtual reality. Virtual reality puts you in a fully digital world — you can move around and see 100% digital creations.
With augmented reality, in the way we use it, you’re looking through your phone and seeing things your phone has put in the real world. The technology has finally caught to where most people can now use AR with the smartphones in their pocket.
Dr. Dan: As long as it’s an iPhone.
Luke: Or 1 of 2 Android phones that very few Android users possess.
Q: We talked about “what is AR?” but how does it actually work?
Dr Dan: In practical terms, you just enter the camera screen in CompanyCam. It looks like a normal camera screen, except there are a bunch of yellow dots. Those dots are essentially capturing unique features of the surface your camera is seeing. That’s how it can keep track of the position of the phone.
Q: How many dots will it find?
Dr. Dan: It depends on the surface. If you’re pointing the camera at carpet there will be far more dots than if you were pointing the camera at a blank wall — that would have zero points, because there are no unique features. The trick is, you have to point it at something that has texture.
Luke: Those points are how your camera and graphics card knows where an object is placed, specifically when you physically move around the room, and how it knows to keep the objects or measurements in place while you move. Essentially, the tool is taking the fancy math it’s doing with tracking all these points, combining it with the fancy math it’s doing with orientation . So what it’s doing is just moving measurements or objects around your screen and resizing them to create the illusion that it’s just sitting in the world.
Dr. Dan: One effect that you’ll see as you’re using this will be on the stuff you drop into the screen. Those things will sort of drift a little bit, which can be kind of annoying, but Apple’s done a pretty good job of correcting those errors. A little tip here is it helps to sort of wave your phone a bit or go back to where you started. It may take a little practice to get used to it.
Luke: You’ll quickly learn what it’s good at and what the limitations are. The key limitation is that you can’t track anything with no difference in it. So with a white wall, it won’t track anything, so as a tip you can place a business card or a sticker to create that texture. Anything you can tape on the wall should be enough for it to grab that surface.
Q: What exactly is CompanyCam doing with AR?
Luke: We want CompanyCam to be the best tool that contractors use to capture data in the field. Currently, they use CompanyCam to capture photos, notes on those photos, voice notes and they’ll use it in the future to record video. Now, with the augmented reality functionality that’s available we can do things like capture measurements, capture angles of a surface, and place 3‑D objects in the world. Plants, trees, air conditioner units, and a window on the wall of your house can be put in the world as if it were there.
We think this little suite of tools is going to be very useful for contractors in the field. They can do quick measurements that are relatively accurate and they can show off what their product may look like to the customer by placing objects in the world with augmented reality. We’re launching with a kind of limited set of functions that we know there’s a need for and then we’ll be looking for feedback on what else people would like to do with this.
Dr. Dan: I think an important thing is that Apple has just released this AR kit. So, there are going to a whole bunch of apps coming out that can do more or less the same thing as what we are offering. But what we’re doing with AR is completely integrated with CompanyCam — we have the whole cloud infrastructure as leverage. Our users can share those measurements, take photos of the measurement and share those really easily. I think that’s what CompanyCam is providing that a lot of these other apps won’t have.
Q: Why could AR measuring be more useful than using a regular tape measurer?
Luke: The fact that you always have your phone makes this functionality handier for estimating purposes. Our AR measuring seems to be accurate enough for kind of a rough estimate, but it will not be more accurate than a tape measurer, probably ever. There’s inherent limitations of the technology within computer vision.
It’s simple, quick, and extremely compelling to show a customer. You can just shoot your tape across the room and write that down in a notebook, which is handy, but when you can scan the room — measure all the dimensions — and take pictures it’s so much more beneficial. Those pictures will flow into CompanyCam where they will be saved, connected to that project, and allow you to refer back to those measurements in context. There’s an opportunity to really make it part of the workflow you’re already doing in CompanyCam, but we do need to stress that this is for estimating purposes only. However, for the quick estimates this is going to be really, really good for contractors. We’re trying to make something the best way it can be made for contractors.
Dr. Dan: There are some other things you can do aside from just measuring. For example, say you want to cut a hole in a wall. You know where you want to cut it from one side of the wall, you can drop something into your AR camera and just walk to the other side of the wall, you can still see that point you marked.
Luke: That’s really interesting.
Dr. Dan: I think over time we’ll come up with more interesting tools besides just measuring length and placing objects.
Luke: I’d also like to add that we’re kind of at that place where we’re using this to build existing tools. People know they need a tape measurer and we can build that type of functionality in here. We haven’t spent our time thinking on what the new possibilities could be. The example Dr. Dan just gave is one I’ve never considered, and seems obvious in hindsight, but there will be new possibilities for what we do with AR just through use and feedback from our users.
Q: In what industries do you believe this measuring tool will be most beneficial?
Luke: Off the top of my head, I think anywhere you’re doing estimating around the area of something, especially something that’s kind of an odd shape. Anything that requires a difficult measurement, this will handle automatically. For people like painters, carpet installers, roofers trying to measure pitch, measuring a room and getting quick dimensions should be great for estimates. You’ll know how many gallons of paint you need to buy, you can figure out about how many square inches of carpet you’re going to need, and in landscaping, I’m picturing a flower bed that curves around a house can easily be tracked.
Those are very obvious applications for this tool, but I think since we’re just at the beginning it’s hard to know where we can go with this further down the line. We’re going to have to learn from our customers.
Q: What is it about AR measuring and CompanyCam that make them a perfect fit together?
Luke: Dr. Dan kind of alluded to this. Inevitably, there will be plenty of apps that allow you to do AR things. Apple has made it possible and you still have to be pretty stinkin’ smart to put it together, but there’s a lot of smart people out there. There will be a ton of apps out there for this. But, for contractors — regardless of whether they can do AR measuring on 10 different apps — we are going to allow them to capture that information, the photos and the measurements, and have it available across their company. Our mission is to make field data capturing simple, and this is just another powerful tool.
Dr. Dan: Android right now has their version of AR, which we’re not utilizing just yet, but even if you have a company with a bunch of Android phones they can still get the benefits of the AR measuring because they can view those measurements later in CompanyCam.
Q: Do I have to have a certain type of phone or update in order to use this tool?
Dr. Dan: Yes, basically it works with iOS 11, so all iPhones that are 6s or newer, the iPad Pro (9.7, 10.5 or 12.9) and the iPad (2017).
Q: When will this feature be launched on the CompanyCam app?
Luke: AR is ready right now! So, you can get the app and start AR-ing.
Dr. Dan: Yeah, I guess I better get back to work.