M+E Connections

Facebook Doubles Down on AR, VR at F8 Developer Conference

Facebook executives doubled down on their aggressive plans and expectations for augmented reality (AR) and virtual reality (VR) at the F8 Facebook Developer Conference in San Jose April 18-19.

In addition to announcing the closed beta launch of its Facebook Spaces VR app for its Oculus Rift and Oculus Touch devices April 18, Facebook CEO Mark Zuckerberg used the opening keynote at the annual conference to reiterate how optimistic he is about AR and VR.

During an April 19 presentation at F8, Michael Abrash, chief scientist at the Oculus division, laid out an optimistic forecast for not only AR and VR, but also a “new wave” of computing that’s “just starting,” which he referred to as “virtual computing.”

“I think its impact may ultimately be even greater than that of personal computing,” Abrash said of virtual computing, which combines AR and VR. He added: “Twenty or 30 years from now, I predict that instead of carrying stylish smartphones everywhere, we’ll wear stylish glasses. Those glasses will offer VR, AR and everything in between, and we’ll wear them all day and use them in almost every aspect of our lives. The distinction between AR and VR will vanish. The real and virtual worlds will just mix and match throughout the day, according to our needs.”

Abrash said that, “in the nearer term, due to technological constraints, AR and VR will follow separate paths toward that mixed reality future.” He predicted that VR will offer “increasingly convincing experiences and it will gain the ability to capture the real world and bring it into your virtual world.” VR “will be the most immersive way to interact with the virtual world and it will revolutionize how we work and play,” he predicted.

But, despite VR’s bright future, Abrash said: “There is one key area that will never be VR’s strong suit: always on, go everywhere, mixed reality because no matter how good VR gets, few people would be comfortable socializing in person with someone whose eyes they can’t see.”

Abrash added that “in order to make virtual computing as much a part of everyday life as the smartphone is today, we’re going to need see-through augmented reality: transparent glasses that can show virtual images overlaid on the real world.” See-through AR glasses will be “the path to what I’ll call full AR: augmentation that enhances your vision and hearing seamlessly, that makes you smarter and more capable, that is light, comfortable, stylish, power-efficient, and socially acceptable enough to accompany you everywhere you go,” he said.

But Abrash conceded that such glasses don’t yet exist, and they could take five or even 10 years (or longer) to create.
The day earlier, Zuckerberg said he was “excited” about AR because “it’s going to make it so that we can create all kinds of things that, until today, have only been possible in the digital world, and we’re going to be able to interact with them and explore them together.”

At last year’s F8, Facebook laid out a 10-year roadmap “to give everyone in the world the power to share anything they want with anyone” and AR was one of the “key long-term” technologies it talked about, Zuckerberg noted.

Zuckerberg said during his April 18 keynote: “We all know where we want this to get eventually. We want glasses or eventually contact lenses that look and feel normal, but that let us overlay all kinds of information and digital objects on top of the real world.”

Such glasses would allow people to sit at the conference and start playing virtual chess games with their friends, he noted, adding: “You want to watch TV? We can put a digital TV on that wall and instead of it being a piece of hardware, it’s a $1 app instead of a $500 piece of equipment. So, think about how many of the things that we have in our lives actually don’t need to be physical. They can be digital and think about how much better and more affordable and accessible they’re going to be when they are.”
Facebook, meanwhile, is “making the camera the first augmented reality platform,” Zuckerberg said, but he added that was “just the first step.”

Goals include the ability to offer simultaneous localization and mapping (SLAM), to capture and interact with scenes in 3D with high precision, and object recognition offering real-time visual understanding, he said.
Zuckerberg also predicted there will be a “whole new genre of augmented reality games coming.” Noting that more than 1 billion people play interactive games now, he said it will be possible to play AR mobile games using a table in a doctor’s waiting room as a game board.

But he said: “This part of the platform is going to come a bit later this year.”

The Facebook CEO conceded that it’s “going to take a while for this to develop” using AR with smartphones. He said: “There’s a lot in here that we’re going to roll out over time. Your experience isn’t going to change dramatically overnight. It’s going to take a while to roll some of these things out and then even longer for developers to actually start building all of these experiences. But, over time, I do think that this is going to be a really important technology that changes how we use our phones and, eventually, all of technology.”

Zuckerberg’s AR/VR strategy is “not overly aggressive,” Wedbush Securities analyst Michael Pachter told the Media & Entertainment Services Alliance (MESA) by email April 19. “I agree that both VR and AR are real concepts, both will be a big deal, and the only constraint is that each requires equipment,” he said.

Pachter added that consumers are “not yet comfortable with VR headsets or AR glasses, and the equipment required to power both concepts is expensive.” But he said: “Ultimately, both will be powered by your phone (or your Apple TV or Fire TV), and Zuckerberg is right, the headsets will ultimately be small enough to fit on a contact lens.  That’s a long way away, but conceptually and directionally correct. 
My guess is that this is LATE next decade (so more than 10 years away), but it is very likely to happen eventually.”

In another presentation at F8 April 18, focused on Facebook Spaces, Rachel Franklin, head of Social VR at Facebook, said VR “has the promise to be the most powerful social platform.” She noted that the company integrated Spaces with Messenger video calls, and added: “Eventually we’re going to add new ways for developers to build more amazing things on the foundation that you’re seeing today. We’re already exploring this with a few early partners.” Facebook has “only just scratched the surface of social VR technology and we see a future where it will transform the way people around the world will stay connected with their communities and those people that they love,” she said.

Like Facebook’s AR and VR offerings, Facebook Messenger also uses artificial intelligence (AI), David Marcus, VP of messaging products, pointed out in another April 18 presentation. He told the conference: “We’ve made a lot of progress in the last 12 months, and we still have a lot of work to do.”

Marcus added: “Automation is going to be a critical part of our joint success, and we want to bring automation to the people who need it the most, which are small business owners because they don’t have time to figure out technology and respond to messages 24/7. They have a business to run.” Therefore, Facebook was launching Smart Replies, powered by AI, and “we’re starting very small with restaurants in the U.S. by enabling them to automate responses to messages” that restaurant owners get on Messenger, he said. The Smart Replies engine looks at questions being asked and responds on the user’s behalf, he explained.

Facebook also launched M Suggestions two weeks ago, which he said recommends things that include making plans, sharing one’s location, or sharing funny stickers. He added: “As M gets smarter, it will start making recommendations for chat extensions built by you, and that is going to take a little bit of time to get right.” For now, Facebook is fine-tuning it, he said. The company wants to see how people will interact with M Suggestions social and multiplayer recommendations, so it will launch an experience around food ordering with Delivery.com, he said.

Facebook, meanwhile, launched Games on Messenger last year and, since then, 1.5 billion games have been played, Marcus said. Now, the company is taking that “to the next level” by introducing “rich game play that will enable real-time gaming and turn-by-turn” games, and also by opening a game tab to find all the games a user is playing and find all challenges by friends, he said.