Sessions is temporarily moving to YouTube, check out all our new videos here.

Readable Microservices, With Functional Programming

James Chow speaking at London Node User Group in November, 2016
Great talks, fired to your inbox 👌
No junk, no spam, just great talks. Unsubscribe any time.

About this talk

James Chow will talk about how the principles of functional programming align themselves with modern distributed architectures. Join him as he builds a fully functional microservice application using LNUG's new functional programming toolbox. James shows us how all the various parts of functional programming come together to create safe, efficient code that is easily readable and extendable.


I'm James and I'm another Java Script developer. Actually I'm actually DevOps at the moment but, yeah, so what I really want to do with this talk is to actually take some of the building blocks that Andreas and Stefano have actually described in theirs and try and show you a practical example. I've actually built a very small microservices application using some of the functional tools and some of the functional building blocks that they've described, yeah? So just in terms of that, I'm not going to read out the sentences of what a microservices is but in terms of purposes of this talk, we're just going to create a very small application which is going to perform just one...have one single unit of responsibility and perform one single operation. So one thing we have to think about when we're building a Microsoft application is how does functional help in this kind of sense? How do we want to think about how we want to build this application using some of the functional patterns? So what we do here is that we want to describe these as set of functional patterns here. So if we just describe one signal end point an application we could describe it as this. We can say it's basically a set of operations that we want to perform on some sort of incoming payload. We want to execute asynchronously because it's all time based and we want to have a defined response so here it's either success or failure. And be able to deal, in the application, deal with both of these responses in a certain way. So for this example application we're going to be building, it's going to be a very simple application, TalkFinder 2000, which we went out and bought a domain for. So it's going to be like we're just going to have a data store with a self-analog data. So it's going to be search for all all download talks and then also retrieve data from another data store which has all the speaker information about all those speakers and return all that into a result set. So if we think about it we can actually describe this as three separate operations. So the first one is we go find all the talks when given a search term. So if we get some sort of plain search text, we just want to search through our order elastic search, in our case, elastic search data store. Find all those talks and then go find speaker data about each one of those talks and then join them into a result set. So if you think about it, we can just describe this as three separate operations. So if you are at a very high level overview, I'm not going to actually put any code in these slides but what I've got here is basically a description of these as type signatures. So at the very top, we can actually think about this entire microservices as just simply one function. So here we're going to take some talk query which is some object with some search term inside it. And really we're just going to perform some asynchronous action however that may be composed and overturn the result set which is all the talks embedded with speaker data. So we can break this up into a set of operations and one of the first ones maybe we just go find the talks. So we go query some data store and return a list of talks back from that. So here if we look at a type signature, it takes that talk query and returns a future which resolves to a list of talks and then we have our second operation which is just a take. Those list of talks perform another query against our MongoQuery data store and embeds those speaker data into that list of talks. So you can see how we can compose and combine all these operations to finally fit the type signature of our complete service. So if I eventually give a quick overview of the data that we're dealing with here, so our talk objects are basically they're made of tight descriptions and speaker property which here is our Github handle and we have an array of speakers which have that handle property which matches to the same Github handle and all the information about the speakers. And this is the results that we actually want to end up with. So here it's talks embedded with speaker. All it does is they took the speaker object and just assigned it to the speaker property in that. So it's really just combining the two but we need to orchestrate this because these are two separate operations. So let's explore our first asynchronous function. So here we were going to do this and we need to do a post-transformation on the data that we get back. So findtalks, we look at the type signature, talk query, returns on the array of talks, our future type of array of talks. So here we just have our elastic search function, it takes that talk query but it returns it into a result set that's a bit obfuscated. This a result that you would get back from doing elastic search operation. So there's object and here deeply nested into the body is the actual talk values that we actually care about. So we need to get a hold of...for us to conform to this type of signature, we need to extract all that talk information from that object. And therefore we need another function that can deal with that. So here we have past results, here it would take the resolution of that future, it would take that object and then extract out the talk values. So we need to pluck all these values outside of this object. So we're going to end up having to compose all this and join that together at the end but I'll show you in the code when we do that. And then we have another asynchronous operation, this time it's going to be a pre-transformation. So basically it's going to be performed...we need to take some data, perform some passing on it and then actually feed that to an asynchronous function to actually get the values that we need. So find speakers from talks basically takes that talk array and then does asynchronous operation and turns a future type of all the speakers. So here we have a particular standard MongoQuery find so we're going to take that...sorry, sorry, this is the pre-transformation, yeah? So we've got a list of talks and what we really want to do is that we want to take that talk and pass it into an object that we can feed to a MongoQuery. So that function and then we want to actually take that speaker query object that we've generated and then pass that onto our Mongo client library to actually get the results that we care about. So here it just returns an array of speakers. And finally we want to join these two together. Again we have to conform to the type signatures that we were talking about. So here we're going to take that array of talks and then ultimately we're going to join these together and return the talks embedded with the speaker data. So we've pretty much we've got to join those asynchronous operations together and we can do it like this. So remember we've got our second asynchronous operation which takes the talks and returns the speakers and that's where our lovely innerjoin function, that fairly complex looking thing, came from before. But here this is how we actually joined these two arrays together. So here it's going we see it's kind of carried. It takes a string which is the property that was the speaker property on the talks. The string which is the property handle on the array of speakers and then we are waiting for the talks and the speaker arrays to be given as arguments before we actually turn the actual result set that we care about. So demo time, so here I'm actually going to just show you the code base. Don't worry, it will be really overly complicated and no one will understand it but I'll just show you anyway. So let's see if I can swipe past. There we go, hurray. So this is it. Massively complicated. So this is our entire search function. So if you were thinking about where do we do composure? We're doing it from right to left so here we're working from bottom to top. So at the very base of it you can see our search function right at the bottom including our type signature. So here we have the top query which returns the result set that we care about. So if we look at it, we're actually composing those two asynchronous operations that we talked about earlier. So if I work from top to bottom...sorry bottom to top, Christ, so findtalks is the first thing we do. So findtalks takes that talk query, it passes to the elastic search client library. Here I've just wrapped it so that it returns to future. If anyone asks later, I can show you how we wrap all that but I've got a client library there which just returns our future type which resolves to the talk values that we care about. And then we hear and we map over that. So what you're doing here is that you're mapping over the resolution of the future. So elastic search search is going to return your future which is the result set that we care about and then we're just going to map over that future and apply the function which is actually going to be applied against the resolution of that future. So here past results is using's now combining those two utility libraries that Andreas has covered and here's one, path and pluck. So path here is actually if you look at the type signature is going to extract the property hits hits value until it gets back the array of the source array that elastic search has got. And then we're going to compose that with pluck so that it extracts the property_swords which actually returns the talk values that we care about. So you can see that as you compose those, you actually end up fitting the type signature that we care about. And so if you look at findtalks, basically you can see how it conjoins the input arguments from one to the output arguments of the last one so that it fits that type signature. And then let's look at add speakers. So here this is where we can do some nifty partial application. So the first thing it's going to do, it takes the talks array and then from that we're going to pass it to findspeakers from talks. So that function is actually going off and is composed of two more operations, MongoQuery from talks and actually the Mongo client library to actually search in the speaker collection. So MongoQuery from talks takes an array of talks and if we look at the top MongoQuery talks and returns to speaker query. So here we're going to take that list of talks and we're going to pluck out the property of the speakers which was just those Github handles. And then we're going to pass that and we're going to generate a MongoQuery object. So here it's actually going to do a contains query. So he's going to say, "Look for all the speaker objects which basically contains where the handle value is matching any of these speaker Github handles that I've got here. And from that, we actually generate now the query object which we then pass to our Mongo library to actually do the asynchronous operation. So here it returns the future of that business speakers. So that returns it back to...if you look about add speakers again, so what we're going to get there is a list of a future type resolve, sorry, resolving to a list of speakers. But let's take a look at innerJoin this time. So remember if in innerJoin we took two properties and and an array of talks and array of speakers. So here we're actually we're partially uploading the first three arguments. So if we had partially uploaded the first three arguments, what we're expecting back is we're getting back the function which expects the last value...I'm sorry, one more argument which is the array of speakers. So here what it's going to do is it's going to map over that list of speakers which is expecting and then it's going to join those together. So here we can actually compose and partially apply just the talks so we're only expecting a list of speakers back from there and then we can actually join those over together and return the result set back. So if you go and so you see that is the last function that is actually part of the composed chain of our original one. So here, you can actually search and you can just...and the search is just all it's returning is a function which results to a future future type. And that's really our entire microservices describe all that complexity of all those asynchronous operations, all of that composed into one simple function that we can just export now and use however we feel free to use or like to use. So you can see that with this, it keeps our code really terse, really, really descriptive as well. We've broken down all our individual operations with very clear and concise naming convention for all of those. The type signature describes everything that all our input values and all our resolutions to all those values. And then we can see how we can compose and like Lego blocks, we can join them together and compose them all together to actually generate the operation that we need. So how do we actually end up invoking this? So remember what we return back from this whole chain is a future type. So if we go back, so if we can look at an express application, our handler is no more complex than this. So here, we've just imported that function, we've taken an input query which is basically wherever the query spring parameter we may have been passing this handler and we fork it. And here this is how we handle our status and our result set. We pass it back here. So with this, we basically push all the side effects, all the execution, all of it right to this handler. We don't perform any execution until this gets invoked. So our entire microservices is basically a description of all the operations that we want to do, but we don't have to actually care about how we handle all the error handling, exception handling until right at the end and when we actually care about actually wanting to deal with it. So here's where we actually end up managing all of that. So we can go off and chain all our futures and everything together but we don't actually have to do any exception handling until the point at which we really care about, it's when we want to return the error back to the user, whoever is consuming the microservices. So, I can give you...I've just displayed a demo application which is You can feel free to go to this site. We went and bought the domain pricey for it. But it's hosted on so don't brag it too much because I'll end up paying a big aid of your spill, but feel free to do it. So this is a lovely program with a beautiful font built in elm which I have no idea but I would have chosen the app but Andreas loves it and so we got it here. So here we have our, our lovely font fabrication. So here you can type any search term that you want and it's going do a search on elastic search. It's going to look through all descriptions, all the titles and give you the results sent back. So, well, we are all here listening tonight is functional programming so let's look for functional, yeah. And, yeah, and he came up and Andreas was really pissed that he was too quick because he had a lovely little spin on everything to wait for data. I'm sorry, I couldn't slow that down. It was like six milliseconds, I couldn't make it any slower for you but there you go. So here we have all our list of results so that's coming up so beautifully. You've got links to Github, so it has all the Github URLs that you might want to link to so you can hear or my code will be in my repo there. We'll put out later for you so that you can have a look at it just to know there's no tricks. We're not doing anything special with it. And we have our little handlers and everything down there and if you click more, you can actually see description. And it actually displays all that text. So all of that is being returned from our endpoints. And just I don't know if this will work, let me see if we can actually go to the endpoint itself. Yeah, and you just can see all the data being returned there. So really what I want to take away from this talk is that building a microservice application and use functional handles doesn't have to be overly complicated. What you get is that you get a very succinct, very terse code and you get a way of dealing with all your exceptions that you can deal with that right at the end. You can push that all to the edge and you don't have to handle any of that upfront and you can...and in terms of being able to test all of this, it's very, very easy to test. You can actually don't have to test your client libraries, you don't have to test any of that, you can just literally test all your functional, your operators, your functional transformations. And you can do that all in your unit testing and you don't have to mock anything or do anything and stab anything out. So, yeah, so here it's just showing that you can actually build very, very concise terse code from following your functional pattern rather than building very large monolithic applications. Okay, so that's it from me.