WebVR Connecting People, Pandas and You

Diego and Uve speaking at JS Monthly London in August, 2017
Great talks, fired to your inbox 👌
No junk, no spam, just great talks. Unsubscribe any time.

About this talk

We'll be covering the present and future of WebVR, and why it's important for cross device experiences. Then we'll look at a demo of an app that Diego and Uve built, and how they did it.


- I'm Diego. - I'm Uve. - We're here on behalf of Samsung Internet, and we're going to be speaking to you about WebVR. The title of the presentation is Connecting pandas, people ... Connecting people, pandas and you, sorry people come first. So yeah, let's get started. - Cool, let's see what we're gonna look at. So we're gonna do an introduction of ourselves, Samsung Internet, introduction of what WebVR is, we're gonna show you a demo we made, then we're gonna break down and see how we made it, do a summary of what we just went through, and then some resources, and the we'll do questions, and then after that we can all go to the pub. - Okay, so a little bit about ourselves. We are working at the Samsung Internet web advocacy team. Generally myself, I'm doing WebVR demos and code examples. And you? - Similar things, web demos, speaking events, and making small games, speaking at large events, and sometimes web stuff. - So we've mentioned a couple of times Samsung Internet. Can I see some hands if you are aware of what Samsung Internet is? Okay, keeps on getting better by the day. - Should we get them some stickers? - Yeah. - Keep your hands up. - Definitely. To answer that question, Samsung Internet is a browser that runs on all Android phones. It's important because Samsung is the second largest contributor to the Chromium project, after Google. Nice, nice. It's also an Evergreen browser so this means that we're pretty much a browser that is constantly being updated to keep up to date with the latest web standards including WebVR, Web Payments, CSS grade and all the new goodness that are coming our way. The reason for us to have a browser is we have a specific user experience for our user. So depending on the device that you have we might way to tailor, for example, facial detection and logging in with iris scanner, doing WebVR. - Focusing on privacy. - Focusing on privacy, yes as well, very important. If you wanna learn more about the browser you can head out to this URL. - Cool, so our team, as we said, helps build communities, speaks at events, makes blogs, demos, we're really friendly, come by and talk to us later we'll give you even more stickers, we have a lot with us. - Yes, exactly. - If you want to see our team, our link was on the slides, but. - So it's Samsunginter.net/team. - And the slides will be shared at the end of the talk. So if you miss anything don't feel bad, you can catch up. - So, some of the questions that we do get at events, even though they might sound a bit rude, is, "And why should I care that Samsung has a web browser?" Turns out that it is the third largest mobile browser in the UK, also in Europe and North America, and we're constantly contributing to different web APIs. For example, the service worker specification, we have a colleague in Korea that is working in the W3C group so we're constantly on the forefront of defining what's coming to the web. Also, pretty much if you're developing certain experiences, you want these experiences to work in the broadest group of devices that you have. So Samsung Internet does make sense to be testing and developing for. - And as of last week we are on all Android phones running Lollipop and above, so try a beat if you want. - Yes, version 6.2, you can go to the Google Play Store and download it, I think there's not a reason anymore to be using any other browser. - Not really. - Anyway, let's go back to the main topic of the presentation today. What is WebVR? Has anyone here been using WebVR? Has anyone heard about the WebVR API? - We'll give you more stickers. - Good we do actually have WebVR ones. You wanna go with the definition of WebVR? - I'll let you handle it. - Okay, so basically, WebVR, we have, we're gonna be able to take content and create VR experiences that will be delivered through a browser. Notice that there are two other components. First we have a device and it has a little star in it. This star is because this device can be a mobile phone, it can be a PC, or it can be a VR headset. And notice that we have, in the headset, two stars because the headset is not required. So you might be thinking, wait, it doesn't really make much sense, you know, we are talking about VR and you're saying that the headset is not required? Well, turns out that you can develop experiences that will run in this kind of like magic window, when you start peeking through the phone as if it were a window, and this is very engaging with your users and consumers. So these are some of the ways that we can develop WebVR experiences. Part of the benefits that we have in WebVR. - So, WebVR it's easy to share because you just click on a link, it opens up your browser and wow, look! You're in a VR experience! It's cross-platform, it runs on phones, computers, and all sorts of headsets, and it's accessible because you don't need a headset to view the content, and if someone wants to share a cool thing with you, you're gonna be able to see that when you're out and about and not necessarily have one of these bulky things with you. - True. So when you mention cross-device, we're pretty much saying that we can run in something that has only a phone, we can run in something as simple as a Google Cardboard, or we can run in something that is as complex as a high-end headset like the Oculus Rift or the HTC Vibe. Now, the specification itself does give you access not only to position and orientation of every VR device that's connected to the machine, but it also allows you to react accordingly, depending on which platform you are. Which, specifically talking about platforms, it's currently supported and being worked on at Samsung Internet, at Mozilla Firefox, Nightly, Microsoft Edge and Google Chrome and, well, the Chrome project. What about the others? - Well, you can use it on other browsers as well by using polyfill, so it won't be as good of an experience, but it will still work. - Exactly. So, part of the benefits is that the delivery platform that you're getting for your VR experience is just the widest that we have in existence today, which is the web. - Mmhmm. - And it's kind of interesting, because, if you start thinking about what the future of VR might look like, we can start saying, or, it's not that hard to think that VR content will be delivered the same way that video is being delivered nowadays. And if you take a look at Amazon Video and YouTube and Netflix, it's kind of like, it makes sense that the web can be one of these delivery platforms. But there's, of course, more. Because, since we're talking about a technology that's running inside the browser, you have access to every single web API that's already embedded and part of the browser itself. So, let's say that, if you wanted to have a physical web beacon in a physical location, and your user is walking in front of this building or this location or this bus stop, this museum, or this theatre, then he could get a notification popup that is prompting him to go into the browser, and once the browser launches, then he's getting a VR experience inside the browser. And if he has a headset at his reach, then he can just put the phone in the headset, or use a computer and go into the higher-end headsets, and basically we're saying that, from a physical object, to the browser, to the home screen, we're talking about progressive web apps. Which is also something that you can do, and hopefully we're going to be talking about next month. To your headset in less than 30 seconds. - Wow. - So, wow, boom. Mindblowing. I mean, we're talking, in 30 seconds you're getting from a physical location to a headset. Now, we're gonna show a small demo of an application that we've been building, so we need two volunteers. If you want to see what they are seeing, you can go to this URL, hungry-pandas.glitch.me, and what you're gonna be is joining us as a spectator on the game. - You can just click and drag, and you can swing about. - So we're joining, notice there that, okay, so this is the game, this is what they are doing at the moment. While Joe is only in a magic window mode with the phone, we have someone with the headset, and I'm just a user that's on a computer. So you can see how all of these is exactly the same code. I'm pretty sure some of you are in, as well. - More people are joining in the phone way. - Oh, okay, there might be a small bug ... But don't worry, pandas are sociable, so. - They're all having fun, it's cool. - So, you kind of get the idea. We can have an experience that's running on the web, therefore it's adapting to different devices - - So, more people have joined and, it's just one on top of each other. - [Volunteer 1] That looks very wrong from this angle. - Okay, so yeah. - Yeah, they actually do, yeah. - So, I guess we can continue more pandemonium. - Cool. - So basically, the demo that they were seeing, looks something like this once it goes in VR mode, there are plenty of frameworks that already take care of the boiler, start of getting the stereoscopic rendering and the barrel effect, etc, etc. So how did we code an example like this? Was it difficult? - We used magic. - We just, exactly, we used magic. So basically, there's this way of doing magic that is just a simple - - Combining a bunch of stuff, alchemy, it's called! - Yes, a list of simple ingredients like VR, multi-user, Physics, 3D models, a bit of some fish - - Dead fish, yeah. - And then, of course, magic. So, we're gonna start one by one, so you can get an idea of how easy it is to create an experience like this. Do you want to start with A-Frame? - Cool, so A-Frame is a WebVR framework - Cool, it's built on top off three.js, which is built, itself, on top of WebGL, so it's abstracting a lot of layers away, and makes it quite accessible to work with using basic HTML and Markdown, and it's compatible with all sorts of WebVR and headsets. - Yeah, and you can create, with A-Frame either flat experiences, or VR experiences. VR experiences that, at the same time, can be six degrees of freedom or three degrees of freedom, so if you want to do a high-end experience where you're walking across the room, you are able to do that with WebVR. - Yeah, and instead of building, like, custom VR experiences for every single device using different SDKs, A-Frame lets you just build it once and deploy on the web through multiple devices. - Exactly. Or, pretty much any other framework that just is build on top of WebVR. So we can talk about babylon.js as well, we can talk about different options that already exist. - Mmhmm, Polygram ... - Exactly. It's a bit more complicated if you want to talk about support. Then, again, the good news is that WebVR is supported and compatible with each and every single platform that you're seeing here to the left. So if you're thinking about doing an experience for Gear VR, Google Cardboard, Daydream, Oculus, VIVE or HoloLens, you're pretty much covered. Now you might see that there are not many checks in that list. Nonetheless, if you have the Polyfill then you can pretty much target every single platform that's there, operating system and platform, including iOS. - Cool, so A-Frame looks like this, which is a bunch of code. Also, once you build it up you can press control-O-and-I which will open a visual inspector which behaves a bit like Unity on Maya, where you can click and drag things, which makes it a bit easier to see what you're doing when you're making WebVR experiences. We're not going to go into the gritty details of how to learn A-Frame right here and now, but we do recommend A-Frame School which is this great interactive tutorial you can go through, which guides you through how the components work, with examples. - The second part is the Networked A-Frame. So basically, we had several users that were sharing one same virtual scene. The way that we did this is with a component that's already pre-built that allows you to get certain entities in the scene and share them across everyone that's connected. So it's open source and it's on GitHub. This is more or less another example that is done, you can see every single avatar is a different device, and it's mapping, automatically, orientation and position. So you can take, you can define a template, and it will be imported into the scene and it will respond to the movement that you have. How does Networked A-Frame work? We have a scene, and this scene, we're gonna give it a, kind of like a characteristic, a system that's the networked scene. And then, pretty much every single object in the VR scene that we defined as networked is going to be shared across all the devices that are connected. We are going to be using templates to represent the networked entity, so you can see in the game that the character was able to see the panda that he was, that's because we were enabling the template to be seen. And this template is going to be syncing automatically, orientation and position. So if you start moving, let's say, on a high-end VR experience, then this is going to be shown to the other people that are in the same scene as well. It can broadcast messages to connected peers and it's pretty much just based on WebRTC and WebSockets, so you can decide if you want to use one or the other. The other option, sorry, the other ingredient is Physics. - So we used A-Frame Physics System, which, next slide, just adds Physics into A-Frame so that you throw blocks around, and they all interact with each other, and that works using either dynamic or static bodies. Dynamic, able to move during the scene, and static stays still and can interact with dynamic bodies, which move. - Exactly, it's based on cannon.js, and pretty much, you see that it's kind of like another system that we are attaching to the scene. So pretty much, we have Networked A-Frame, we have Physics, and we can keep on attaching systems or developing our own systems to put into the VR scene. - And this is called Debug Mode, which should be the next slide? - Yeah. - Yeah, so it adds a red mesh over everything with collision detection. So you can see the invisible walls around the scene. - Yeah. This is a combination of the Physics, you can see the static bodies that are the walls, and we're generating, just, collision events every time that you get a sphere that's touching the panda, and that's adding to the score of every single player. And the final part, no, sorry, that's not the final part, another part, ingredient, of the VR experience is, how do you get the 3D models? So in this case, for the panda, what we did is we used Google Blocks. Who knows, here, about Google Blocks? Okay, so it's this VR experience that allows you to do modelling inside of a VR scene, so it's very low poly, and you can pretty much just play around with basic geometries, and you're starting to define and, let's say, extrude faces, take edges and vertices and kind of shape elements. So we created the panda using Google Blocks, it actually looks like this one, and if you want to use it in your VR experiences, then it's there to be downloaded. and then we put it into Blender in order to triangulate it and make sure that all the normals and the faces are pointing outwards. 'Cause sometimes when you're extruding and just playing around in blocks, you can leave faces looking inwards, and that is not going to look well in A-Frame. But pretty much, it's just, take your model, or any other thing which you have in Maya, in Fusion, in Blender, and just import it as a model. There are many things that you can create. - Yeah, not everything has to be food-related. - And the final one is Glitch! - There is more. There's still the magic that we have to add at the end. - True, right! - So as we said, we added some fish into this. And by that we meant we use Glitch. Glitch is a website which is an online ID, this is an example for Hello World in A-Frame, which is putting a plane down and putting three objects on top of it. Glitch is a really cool tool for prototyping or showing things because as soon as you type things, they're live online, and it gives you a link which you can just share with friends, and they can see your cool web app. And you don't have to play with WebVR, this works with all sorts of web tech. - Yeah, also, it's kind of useful if you're trying to do workshops, or just teach other people. - Or if you're coding from across the desks and you're editing each others' code. - Yes, indeed. And the final one, now, the final one, JavaScript. Which is pretty much the magic that makes it all work. If you wanted to add a custom script to A-Frame, then the way that we would do it is by adding a component. So basically, A-Frame is based on the entity component system. We could have entities which are pretty much just like blank boxes that don't have any behaviour or appearance themselves. Behaviour and appearance are defined by components, so I don't know, how many of you have used Unity here? Okay, so it's kind of like a similar system in the way that you have an object, and in this object you can put a C# or JavaScript into it and define, let's say, some sort of behaviour or appearance. And we also have the system. So systems can kind of be like more, scene-wide, so let's say about Physics, let's talk about the Networked A-Frame. In the case of the Physics, if we have one of the spheres that is just running outside of the area that we defined, and we are removing it from the DOM, and all of this can be done with the components. So the way that you would register a component is by finding a schema which might be, just, the default characteristics that you want to have. Let's say if you want to have a component that's a panda, so you just put "panda" into an entity, and you get the mesh, you get the behaviour, you get, already, the network, and you can do that and bundle it together. And of course there's some life cycle methods that are also good to take into consideration. Like init and tick that will run every single frame of the scene. Some of the things that you might want to do in the scene are related to manipulating the elements that are already in the DOM. A-Frame is done in the way that it's custom tacks, so you can manipulate it through the DOM. And you can do thinks like, with the query selector, get the scene, and in this case, we are starting the component, the system that's just responding to the spheres in the scene. So all of this can be done very easily, and you can see how it's just pretty much manipulating JavaScript. The A-Frame would kind of like putting it, kind of, like, building the scene, and then you would control and you would animate the scene with JavaScript. - Cool, so we get all of these, and we fuse them altogether. - You mix. - Yup, and you combine them all together -- - And then you put them in the weird part ... - Yeah, Glitch, A-Frame, JavaScript, Physics, Blocks and Networked A-Frame, and then we created Hungry Pandas! - And then you get Hungry Pandas, exactly. - And then each user, of course, sees what they see. - So this is what every, what each user is kind of seeing. You can see that in one they're seeing a sphere that is purple, and the other one, they're seeing a sphere that is white. This is because of the template we have defined for the spheres, so we have a template for the panda, we have another template for the spheres, and they're being synced. How many users would you say we could get? A lot? - A whole two users before it breaks. - No, I wanna say two users, but if you're building for phone and mobile experiences, you do have to make sure to -- - Yeah, like, test on devices, make sure they don't overheat, or ... - Yeah, basically, with this set of knowledge, you already have the possibility to build worlds in VR. - Yup, and use that power responsibly. - Yes, 'cause you can do a lot of weird things. - Yup, like, to that cat. - Yeah. - Poor cat. - So, we think that now you kind of should have an idea of what WebVR is. - We should probably check. How many people know what WebVR is, more or less, now? Oh, it's a lot of people! - I know, we don't have enough stickers ... - Oh! - So, okay. You should know, as well, how to get started with WebVR using A-Frame, how to connect WebVR experiences with the Networked A-Frame, how to create WebVR experiences online with Glitch, and share them, bend A-Frame to your will using JavaScript, and the pandas are quite competitive when they're hungry. Now, this is kind of like a very fast overview of the things that you can do in a web browser. If you have any questions, or you would like to get access to the code, the code is in GitHub. - Yep, it's on Glitch, and it's on GitHub. So if you follow that link, glitch.com/~hungry-pandas. - And if you have any other questions or you would like help developing -- - Just tweet at us or speak to us in person. - Yeah, cuz we're gonna be here. - Yeah, we'll give you stickers. - As well, if you want WebVR stickers, you're welcome.