Sessions is temporarily moving to YouTube, check out all our new videos here.

The Evolution Of Asynchronous JavaScript

Alessandro Cinelli speaking at Front-End London in September, 2016
Great talks, fired to your inbox 👌
No junk, no spam, just great talks. Unsubscribe any time.

About this talk

One of JavaScript’s strengths is how it handles asynchrony. Async is one of the most important and often misunderstood part of Javascript or any other language. We will see how dealing with asynchrony in JavaScript evolved over the years. It all started with callbacks… and it landed on generators!


Alessandro Cinelli: [00:11] Hi, everyone. Before starting the talk, I have to tell you something. I'm sorry, but, I'm not a front-end developer. [00:16] [background conversation] [00:16] [laughter] Alessandro: [00:18] But, I love you guys. [00:21] [laughter] Audience Member: [00:21] Yay! Alessandro: [00:23] That's why I try to come here as much as I can. Yeah, I know about React, about Angular. Sometimes I play with that. Also, with Backbones, but last time that I did something real on the front end, from A to Z, I was using these kind of libraries. [00:37] [laughter] Alessandro: [00:40] I was lucky enough, let's say four years ago, that I started being curious about Node.js and my previous experience, I was lucky enough to move a monolithic application written in PHP to Node.js Let's talk about evolution of asynchronous JavaScript. [00:59] Then I start with bling with Node.js, it's a very cool language and the JavaScript is awesome, but I was feeling a bit like a set Panda. In the sense that I was used to do stuff in PHP, sometimes in java back in the days. Some stuff were feeling a bit awkward. Also, how you deal with the asynchronous in general in JavaScript. [01:21] What's asynchronous? Basically, asynchronous, we can say that is where how you manage what happens now and what happens later. [01:27] The problem is that, from the code perspective when you are reading the code base, it's difficult as a beginner to understand how asynchronous stuff works in JavaScript, because you don't know when it's the later. It's difficult to express it and to also read the code. [01:45] Let's take a step back on how JavaScript works in general. Like many languages you have the stack and the heap, but you have also another thing in JavaScript that it's like...It's asynchronous, sorry. [01:58] [laughter] Alessandro: [02:03] You have a queue. Basically in JavaScript you fill up the stack with a message from a queue and in this message you have a function. Every time you have a new function, you also have a new message. Everything happens in completely isolation. [02:18] It's important to understand that because if you don't understand how JavaScript works behind the scenes, for me, it's very difficult to understand how asynchronous works. This queue is, we all know, it's the event loop. Basically, JavaScript is mono-thread, so you're waiting for a message. [02:36] When the stack is empty, you fill the stack and again, again, again. So we know that JavaScript is non-blocking. That's the cool thing about JavaScript itself and that's why it's also cool on the server-side with Node.JS, which is a perfect fit for web applications. This is a classic picture of how JavaScript works. [02:56] You have a queue with all the events, then you have this event loop that will execute one thing at a time because it's mono-thread. If you have an async call and your call is, let's say, an AJAX request, accessing the file-system, or a web API or whatever. [03:10] It will be delegated to the thread pool and you will not wait for the data to be ready to execute the next thing. Once the data are ready, we will fill up again the queue and so on and so forth. [03:30] That's why, for example, Node.JS is very awesome for web applications because you can handle easily the IO in an async fashion and seems that things are going faster and you can do multiple stuff. [03:41] But if you're having CPU intensive actions to perform, JavaScript and Node.JS in general is not a good tool. What I understood when trying to understand how JavaScript worked, the fact that JavaScript itself before ECMAScript 2015 didn't know anything about asynchronous because the JavaScript runtime itself doesn't know about async things. [04:10] So who is then dealing with async? It's the hosting environment itself. It's V8, or the browser or the engine in the browser that will manage the event loop and the queue that we saw before. I said until the latest standard. We will get back to that later and I will explain why. [04:29] Most devs, I think not just the beginners but in general, is very difficult to read the code when you have asynchronous stuff that are happening in the background because it's not that easy from the brain perspective to understand how stuff works. [04:49] As long as you're reading the code base, as long as you have something like a call to do asynchronous stuff, your brain goes nuts. Can you do that in JavaScript? Yes. Can you do an XHR request? An AJAX request in this way? Yes, you can do that but you're blocking. [05:06] Async is great, but can be hard in general. Why? Because we have a sequential brain -- most of us have a sequential brain. We're not really multitasker. Yeah, we can do some stuff like I'm doing now, like an Italian guy with my hands... [05:22] [laughter] Alessandro: [05:23] But I can just do one thing at a time. As soon as we have something that diverts a bit, in our brain, gets crazy, goes crazy. Let's take this example. Let's say that we have a checkout page or even something on the back. It doesn't matter. [05:40] Basically, you need to let the customer to do the checkout. You want to, first, load the user info, then the car items, then also the exchange rates because I don't know, even with the Brexit now, maybe you want to also sell there in Italy and you want to give the total amount in Euros, and you want to be precise about that, and then you calculate the total. [06:05] This is the idea of the checkout. Can we write something like this in JavaScript? We can't really do that because we are still blocking. Think about blocking, in general, as a pool. When you get the value, you immediately, basically with a sync [inaudible] request. Even if it's something that you, again, immediately, or assign in a value, you are, basically, pool a value. Give me that value. [06:35] This is important and we will see that a bit later. How can we deal with asynchrony in JavaScript? Of course, we'd callback. If you take the example that we had before, in order to deal with asynchrony, we need to pass callbacks, basically, everywhere. We'll end up with this code. [07:03] People start in the Internet and start screaming and say, "Oh, pyramid of doom." [07:07] [laughter] Alessandro: [07:07] Sorry. "Callback hell," but I think, people are just lazy in the sense, not in a good way. In the sense that it's easy to write code like this because you can do that immediately, but you can do better. You can avoid sometimes this callback hell. Let's take this very useful example. [07:30] Here, you have a JavaScript code for Node.js where you are reading a file to do a list, and you want to add an item, and then you want to save the file. It's easy to understand what this code does but you can start seeing a small callback hell, pyramid of doom here. How can you solve this? Easily, you can name your functions first and you can split them. [07:53] The code is still readable, and you can also reuse your functions, and you can avoid the pyramid of doom. When we have a sync callback, it's like having a push because we are waiting for something to give us a final value or an error or whatever. In this case, for example, the value will be pushed there. It's basically the difference between a pull and a push. [08:25] What are the main issues or drawbacks with using callbacks? The laws of counterflow, it's not that easy to follow the whole flow. The laws of error handling as well. If we go back to the example that I did before, this is a convention in a...when Node.js...Basically, you need to pass the error around. Otherwise, you're going to lose it. [08:48] There are also other techniques like using some events on [inaudible] . Still, it's a bit tedious to pass this error around. Other drawbacks of using callbacks -- sorry -- is the inversion of control or also called Hollywood Principle. "Don't call us, we will call you." [09:09] Let's take the example that we had before. We have these [inaudible] exchange rate and the [inaudible] exchange rates is coming from an external third party service because we don't want to handle that kind of stuff. We have an awesome library that this third party provides us. We just need to call that function and that's it. [09:32] What happens is, this function is not under our control. What happens, if this function calls...calculate total multiple times. What if it calls it...? Never calls it? It's not under our control so what can we do? We can solve maybe using some flags. [09:50] If we think about it, if we need to do that all the time, it's not a good way to handle things. What if it's never called, as I said? What if it called too early? What if it's called too late? How can you tell...? [10:12] [laughter] Alessandro: [10:12] How can you tell if something...? [10:14] [laughter] Alessandro: [10:15] How can you tell...? [10:15] [applause] Alessandro: [10:16] Thank you. How can't you tell if something is asynchronous? Even if you have a callback, it doesn't mean that something is asynchronous. You can have an [inaudible] for each, to accept callbacks but they are synchronous. The only way is to read the source. [10:31] Probably, maybe with a main function that you know, but in general, how can you tell if something is asynchronous or not? You have to read the code. Again, don't get me wrong. I love callbacks because callbacks are the fundamental unit of asynchronous in JavaScript. [10:46] The fact that lots of people are talking about how to deal with the asynchronous JavaScript, with asynchrony in JavaScript, it means that probably, they are not enough. Lots of things are happening in the language itself. Again, what if waiting for a value from asynchronous goal was easy as blocking as the code that I show you at the beginning? [11:07] We are getting there. Probably, most of you knows about promises. For who doesn't know what a promise is, basically, it's like a proxy that represents a value that is not necessarily known at the moment that the promise is created but it's a promise. You will get something. [11:27] The thing is that, you can also attach to a promise. You can associate handlers for, let's say, if you are getting a value, a successful value or a failure. In this way, promises lets asynchronous methods to return values like they were synchronous methods. [11:47] One of the cool thing is that our promises are always async, always. You don't have to bother to say, "Is this asynchronous or synchronous?" If it's a promise, it's always asynchronous. Remember when at the beginning that I told you, "Yeah, before ECMAScript 2015, JavaScript didn't know anything about asynchrony"? [12:08] Of course, in the tradition, the promise is inside the language and they have to assure that they are asynchronous. They need to do something. They introduce a new thing. It's called Job Queue. It's inside the language itself and unfortunately, it's not exposed as an API. [12:23] As developer, we can't use it. Basically, it's like a roller coaster. Instead of getting back in the queue after your ride on the roller coaster, you can still continue. JavaScript can be used in this feature to assure that a promise will always be asynchronous. [12:42] There will be always handled once. A promise have several status like pending. It's the initial state. Fulfilled, if you get any data, any values, so if everything went successfully. Otherwise, it can be rejected if there was any failure. [13:04] That only means that it's either fulfilled or rejected. Once you go from one state to another, to fulfill or reject it, you can't change the state. The state can't change. At least, you know you can solve the issue that we saw before with the retrieve exchange rate. [13:21] Another thing is that, a promise to be a promise, it has to be thenable. You need this method called then and it's the only way to access the eventual value that you will get or from the synchronous goal. Then itself, will accept two different handlers -- one for failure or one for a successful data that you are getting back from the asynchronous goal. [13:49] Another cool thing is that a promise can return a promise. You can even combine this then method and could combine promises. Another thing is also that you have a catch method that will catch the errors, any eventual errors that you'll get during the execution. [14:08] As one of my idol says, "Talk is cheap. Show me the code." Let's go back to the load user info example. Here, we are creating a promise and then we are doing our async code. As you can see, there is no...We are not getting redial of the callback. [14:28] The callback is still there but at least, it's handled nicely. If we go back to the previous example, that example at the beginning, you can write something like this. We are getting closer to what we wanted. The control flow...Yeah, sort of. We fixed that. [14:48] The Inversion of Control issue, fixed. By the way, Inversion of Control is a pattern. It's not something bad per se but it's something that we need to be careful when you are going to use it. The error handling...You can use the catch and everything's fine. Async is always async, so it's a win. [15:09] There's always a but because you can still...People say, "No, stop using callback. Use promises everywhere so you will fix the pyramid of doom issue," but you can have the same thing with promises. This is what I was running at the beginning, even today, sometimes. [15:25] Sometimes, it also happens that maybe you are in a hurry, and then you want to do some stuff quickly, and you end up running a code a bit like this. Even here, you can fix it instantly because the then will always return another promise so you can just easily put it in this way. [15:43] The fact that we are using the same data structures, we can easily get the beat of that and then we can end up to a code like this, please. Now, it seems that with the promises, we solved the issue of the control flow. People are starting using promise everywhere. That was what I was doing at the beginning. [16:03] It seems fine because you have your main code base with all these thens. Say, "Yeah, OK. Now I understand what this JavaScript code does." On the other side, you are losing the main meaning of the promises itself and you are still writing tons of code to wrap your callbacks. Your code becomes promises dependent. It will impact also the design in general. [16:29] The question is, to promise or to callback? If you have a library...Sorry, the pizza was good. [16:37] [laughter] Alessandro: [16:37] If you have a library, you can support both. Thanks to our awesome language where you can have the total anarchy everywhere. Basically, you can pass a callback. Even if you don't pass the callback, it doesn't matter because it's JavaScript. [16:53] What it can do, you always return a promise. If you have a callback, cool. Call the callback. Otherwise, just go on with the promise. Is there any other drawbacks with promises? They just return a single value. Of course, you can wrap a multiple values in an array on an object but still, it's just a single value. [17:15] A single resolution, in general, is bad for String. Also, on the front end, correct me if I'm wrong, you can't write something like this. If you write something like this, after the first click, you're fucked because the promise doesn't change the state. You need to do the other way around. [17:33] The click has to create the click, has to create a new promise every time. What about performances? Wow. For sure, promises are slower than callbacks because as we saw before, you are just wrapping the callbacks. 90 percent of the time, you won't feel it and it's not a real problem. [17:55] If people start to say, "Yeah, with promises, they're slow." Who cares? If your program is slow, it means that, probably, the issue is somewhere else. [18:05] Promises is not something, let's say, too new or something that you can ignore, because it's inside the language. Also a lot of APIs are using it. You can start using it even yesterday. [18:21] We were almost close to what we wanted in the beginning. Again, what if waiting were just these as blocking. Can I get to this kind of code? Sort of. We need to introduce another feature of JavaScript, generators. How many of you know about the generators? How many of you are using them? Liars. [18:44] [laughter] Alessandro: [18:49] What's a generator? A generator is a new type of function that doesn't behave like the other function in JavaScript. You don't have a run to completion behavior. Once you fire up the function in JavaScript, ciao, ciao, you can't stop it. [19:01] Generators, they behave in a different way. This is a generator. Very useful one, by the way. As you can see, the main difference that you have this star in front of the name of the function. Then you have this yield keyword. Let's see what happens executing this code. [19:18] If you call this function straight away, nothing will happen. You are basically constructing the iterator, because the generator will give you back an iterator. If we check what's inside -- if you want to start the iterator, we need to call next. [19:34] If we inspect the value, we will get that the value's undefined. You will not get an object with two properties. Value's undefined and done is false. It means that we are not finished. We did not finish iterating. Let's call next again. We are resuming the iterator. In this case, we'll get value equals 1, the X value, and done is true, so our iterator is done, basically. [20:05] With the yield, what we are doing here, we are pausing. AKA, we can say that we are blocking. On the Panda is a bit happier than before, because you can somehow try to....we can control our flow. You can use yield as many times as you want. [20:24] We said, generators, it's going to give me back an iterator. Why did they even call them iterators? Because iterators is just one side. The other side is observable. Another useful example, this time the bar function. In this case, we are using yield there. We are multiplying by X with the original value of 14. [20:49] Let's see what happens. We call bar, we call next, and again, value undefined, done false. Then, we call next, but in this case, we are passing a value free. If we check what's the value of the object, we will see the values 42 undone is true. [21:11] As you can see, we can block, we can pull values, and we can push values as well. I don't know if you get the idea, but basically, it's what we wanted to do at the beginning, right? Let's try to combine them with promises. [21:31] Promises will handle all the async stuff. Once the async alls are done, you will have the generator that will push the value back to the caller. Then we'll call the next promise with the next. We can end up writing code like this, which looks like almost what we wanted in the beginning. [21:55] Fine, are we happy? Yes, but this thing doesn't work in the sense that if your code get totaled in this way, nothing will happen, because you still need something that will manage and orchestrate your iterator saying, "OK, once the load user info promise is resolved, call the next one," etc. etc. [22:20] Of course, in JavaScript, we have tons and tons of JavaScript libraries every day and new frameworks every day. There are libraries that can solve this issue. One of those libraries is called [inaudible] . I don't know how to pronounce that correctly. [22:34] We'll do those stuff that I just told you, and manage the iterators, call the next one, you have to call next, etc. etc. and pushing the value. You can just pass the get total function that we declared in the beginning to call and everything will be fine. [22:51] It's not something that you can do straightaway with vanilla JavaScripts straightaway. It seems that in ECMAscript 2017, cross your fingers, we will get something that will help us. It's called async await that basically is the combination of the constant that I just explained, promises and generators together. [23:17] This is what it will look like in 2017. You have async function, get total, and even here it's good, because you are telling who is writing your beautiful code that this function is asynchronous. You need to also use another keyword. It's await to say, before going to the next thing, you need to wait for the value. [23:40] If the Panda was happy before, now he's super, super happy now, because we reached it to the point to what we wanted. Today you can start using async await feature using Babel, which is a transpiler. You can write the code with async await. Then these two will convert it to ECMAscript 6 or ECMAscript 5 standard. As usual, you install things with NPM. That's it. [24:16] There are also other ways to handle asynchronous code, asynchronous behavior in JavaScript, but I wanted to focus more on the feature inside the language itself. There are also other tools like Eregs, Streams, and Highland, whatever. [24:29] Callback, promises, async await, what shall we do? As usual, there's no solution. Just choose your concurrency model, because I think that the main thing that we...the most difficult thing that we have to do every day is to choose stuff. It's not just to learn how to use this language, this platform is, "Now, what should I use?" [24:55] I don't have an answer for that. You can just try all three of them. I just wanted to also say, publicly, big thank you to this guy. I don't know how many of you know Kyle Simpson. This guy helped me a lot. He wrote a series of books called "You Don't Know JavaScript". It's true, still true. They are open source on GitHub. You can even buy them on O'Reilly. [25:24] That's it. I'm Cirpo. I'm Italian, half Belgian. I'm dev leader of [inaudible] . I do backend stuff, sorry. Also I'm part of web devs and Codemotion that are two big communities in Italy that are organizing conferences, etc. That's it. Thank you.