!= How to build a RESTful API - Sessions by Pusher

How to build a RESTful API

Laszlo Gyulai speaking at London Node User Group in October, 2016
Great talks, fired to your inbox 👌
No junk, no spam, just great talks. Unsubscribe any time.

About this talk

In this talk, Laszlo Gyulai explores the design and testing of REST APIs


[00:00] [music] [00:11] Welcome to my talk. I'm Laszlo. I'm here to talk about REST APIs. This is my first public speaking, by the way. I'm very excited to do it here. [00:21] [applause] [00:21] The best place to do it is [inaudible] . It's a London user group. Let's look at my talk outline. I'm going to talk about, in the first part, about the REST design. In the second part, I'm going to design a very simple API with some really cool open-source tools, reflecting on the previously discussed design. [00:52] What is a RESTful API? Some will say, "It's using HTTP requests to get put, delete, and post data." Others will also say, "It's about identifying resources with URIs." Some seasoned developers will shout at you that it's all about the links. You must use the links. [01:17] At the end some just may conclude that it's all about using HTTP properly. Some of these statements are wrong, some of these are partially true, but they are all missing the point. To be able to design a RESTful API, you first need to understand REST. Actually start with, what is REST not? [01:41] REST is not a specification. REST is not a standard. There is no official governing body behind it. There is no [inaudible] . This is one of the main reasons why REST and RESTful APIs, there are so many confusion and debates around them. It's all driven by one guy's thesis like a PhD dissertation plus best practices. [02:08] What is REST? REST is an architectural style. What is an architectural style? An architectural style is the architecture plus a set of constraints applied to the architecture to result a desired architecture. When we have the desired architecture we have an optimized system for the common use cases. [02:40] REST stands for Representational State Transfer. It was written down by Roy Fielding in his PhD thesis in the year 2000 where he described an ideal architecture, where he described the existing and modern Web architecture in an abstraction. [03:05] Representational State Transfer, the name was given to evoke an image of how a well-designed Web application works. Roy described this with a very simple example. Let's imagine a network of web pages that represents a virtual state machine. Each page represents a state. [03:37] First the user receives the first page resulting in the index state. Next the user progresses through the application by selecting a link being the next state transferred to the user. At this stage we end up with the transfer stage or state as the state of the application. REST is still not HTTP. [04:10] Of course even back in 2000, the Web was already running on HTTP. It was Roy and many fellow engineers, they were working hard improving it and this is where Roy described the existing architecture as an abstraction. However there are other protocols that are RESTful. [04:37] For example CoAP is one of the protocol that is a RESTful protocol for embedded systems like Internet of Things and it stands for Constrained Application Protocol. It uses minimal resources on both the devices and the network. Why use REST? I have an answer for this. I'm not sure if this is right answer, but this is what I come up with. [05:07] We use REST because the World Wide Web is a RESTful architecture. If you start building a non-RESTful API, then you will end up with a sub-optimal system, but it's important to note that sub-optimal system in terms of the architecture and not in terms of any other places like the client's side like front-end application and so on. [05:45] When is an API RESTful? This is very easy to answer. An API is RESTful if it behaves or acts under the REST constraints at all times. Let's talk about the REST constraints. REST defines six constraints to achieve the desired system architecture, the optimized architecture. [06:08] First constraint is client server. This is based on the separation of constraints principle, client can evolve independently from the server and the other way around as well. [06:22] Second, stateless. This states that the communication between client and server must be stateless and each transaction must include all the information necessary to complete the transaction. This has many advantages, and one of them is that the server doesn't have to remember the client state. It can serve more clients simultaneously using less resources. [06:58] Third, basically, this is an interesting observation that the most efficient network request is the one that doesn't use a network. We can already know what's the next one, is caching. If you build a REST API, it must not ignore caching. [07:19] The fourth constraint is all about the uniform interface. In order to have an efficient caching system in our network, in our system, the system elements must communicate via a standard interface. In a uniform interfaced network, information travels in a standard way. This has four rules to it to complete the constraint. I quickly just go through it. [07:52] Also, uniform interface is one of the most important part in REST, as well as when it comes to building your RESTful API. You actually practice it, rather than the previous ones. The previous three were already implemented even in 2000 in the Internet architecture. This one, it really, really relies on you building, your interface and using of course HTTP properly. [08:22] First, identification of resources. This means that any information that can be linked can be a resource like an image or a file, a document, even a concept like a person or another collection of resources. [08:44] Next is representations. Manipulation of resources happens through the representations. It can be an HTML document, the client can request an HTML document, it can request adjacent document, and it can manipulate the document via the representation. This is a very powerful concept because it abstracts obstructs the resource itself from its representation. [09:14] Third, the self-descriptive messages. This is very obvious because many of us can also relate all ready from HDP protocol that this one means that the desire state can be described in the message, so in the headers and in the other, for example, server response codes. [09:44] The fourth and the last rule is the hypermedia as the engine of the application state. Hypermedia engine of the application. If you remember my first slide about the representational transfer, it boils down that your application is driven by the links and with hyperlinks within hypermedia. They're just links. [10:09] As many of you can see, these concepts relate closely to HTTP. When an API is using HTTP properly, it takes a great step towards being RESTful. [10:23] The next constraint is layer system. This is a simple concept, but it is very powerful. In a layer system, clients and servers communicate in a network and intermediaries can be placed in between them for specific purposes such as caching. [10:45] The last constraint is code-on-demand. This constraint is optional and it allows clients to download client's side executables. The most popular example for this is front-end JavaScript, for example. This is the constraint that allows front-end developers to have jobs. [11:19] [laughter] Laszlo: [11:23] The next is case study. I'm going to sit down and demonstrate some open source API tools and try to reflect on some of the design concept I'm just talking about. Also, I introducing these tools that you can look at later on. [11:41] I'm taking a very simple API. As a requirement, I am setting out a service where users can list their code repositories. Here, we can already name to resources, user and repository and repo for short. [12:10] Resource conceptualization or resource mapping goes hand in hand with resource identification. Once we have the resource, we assign an identifier to it. Here, I can assign an identifier to the collection of resources such as /users and /repos. Then I can also identify a single resource via IDs. [12:40] My approach to my case study is documentation first. It is a very useful approach when it can comes to designing APIs. Documentation-driven design is a thoughtful process. It does help to think about concepts before start coding the actual implementation. [13:08] For documentations, we can either document our API or design it in plain English, or we can start using API documentation tools or description languages. There are some really good ones, some of them including RAML, Swagger and API Blueprint. Actually in the previous [inaudible] , we had a Swagger presentation. [13:37] I'm going to be using API Blueprint, but I'm not biased. It's really about choose whatever language or whatever tool set is best suited for your project or API. If you want something simple, look for something simple. If you like something really powerful, then you will have to look something...You have to have a learning curve or just get familiar with the ecosystem. [14:07] I don't have much time. I'm just going to quickly run through some of the examples about how you can design an API with some really cool open source tools. I'll be using and just demonstrate some concepts. It's going to be quick. You can stop me if something is very gray, but yes. I'm going to just sit down and then just point out a few things there. It could be useful for any of us. [14:35] [pause] Laszlo: [14:35] Get this screen up. Cool. I start out with a really simple project folder. I have a blueprint file. I'm using API Blueprint, which is a Markdown-like syntax written API description language. That's what it is, basically. It's based on Markdown and it's extended with its own language. [15:12] [pause] Laszlo: [15:16] This is my starting file. On the top, I have described the format. This is just API Blueprint-specific versioning, so it's not related to our API. Then I give it a title and the description for my API. I'm going to render this. [15:54] Because it's a language and it has its own syntax, you can use many kinds of renderers. These are just, again, open source project. I'm going to be using something called Aglio. Aglio will take my Markdown file, my API blueprint file and render it and serve it up as HTML. I'm going to go and just start Aglio. [16:19] [pause] Laszlo: [16:19] Start and open. [16:20] [pause] Laszlo: [16:20] Aglio is going to be running on port 3000. I'm running it with a server flag so it also serves up the HTML. That didn't work. OK, cool. I have the title and the description. This is my starting folder. [17:02] Next, I'm going to be describing the documentation. I'm going to just expand on it and then just work on it as we go along. In the next step, I'm going to describe our API route. API route is going to list API capabilities, so if I go here, you can see...Let me explain quickly what is it, what you can see here. [17:33] The group is a notation in the API blueprint. You can group related resources together. Then in the next header, I name a resource, I name it routes collection for a route. Then I can add actions for my resource with a sub-heading with...In the sub-heading, I can also define the method, the HTTP method for the actual action. [18:01] Below that, as a list element, I describe the response body with the HTTP response code, with the content type, and the actual body itself. What I'm doing is I'm just in the API root, I'm returning a list of links as the API capability. Sorry, I'm going to show it to...So you can see as well. [18:30] Here is the description of the resource. Here is the description of the method, of the action taken on the resource, and then the response. [18:44] When I run this, you can see that I already have an entry called the API root and then it has the /root and it gives us the application JSON content type header. Sorry, this is also down on the bottom. It gives me the response to 100 per header and the body itself. [19:19] This is the documentation that's being rendered by idea from the Markdown document. This is my example for the root resource. I'm going to take the next step, just quickly go through it. Now I describe the root. The next one is outlining the...Oops. Audience Member: [19:44] [inaudible] Laszlo: [19:44] Open that down. Cool. There you go. [laughs] Next, what I'm doing is the same thing over and over again. I group the resources as users, I describe different endpoints like /users, I add methods, get, post. This is just outlining what I will support. There is no responses, no request examples in there, but it's just outlining. [20:21] If I go back to my documentation, you can see that in the documentation, you can scroll through and you can verify with your co-workers that this is what you plan to do with the API. [20:36] The next step, I'm going to detail the user view. This is an interesting bit where you can just return a big JSON example if you want to. [20:53] API blueprint also supports a format that's called MSON and that's again just different way of describing the data structure, which is more flexible than JSON here, allowing mix-ins and includes so you can keep your document dry, extending on base data, for response data, for request data. I'm going to do that. [21:18] I'm going to go for the next step. I'll show you the MSON. There you go. Previously, we've seen the JSON document that's being returned. Here, you can see a specific data structure describing its own types. What's really useful about it is that I'll show you in a second that you can move this and separate it into data structures. [21:49] In the bottom, I just describe my data structure as user. I have the user response in one place and then I can just place, hey, list all users, return an attribute called an array of user. Then view user, return something, just the user. [22:10] The first one is returning an array of users and the other one is returning a user. This is all about just keeping your documentation dry. Not repeating yourself. The next stage, I think I'm running overtime, right? Audience Member: [22:33] Right. Laszlo: [22:34] Cool. [22:34] [pause] Laszlo: [22:43] Next. For the next stage, I just have a completed document. What I mean is I just went ahead and completed all the endpoints. You can see examples for it. [22:54] By the way, this is on my GitHub account so you can download it later. I can just rush through it now. If you want, you can have a look at in your own time. [23:06] You can see examples for request, response pairs, and including headers. You can specify locations. Then you can also add your attributes. [23:18] This, I just went ahead and completed it for the upcoming examples, which is about looking and testing. It is very useful to have a syntax-based documentation language because it's not just about rendering your documentation, it's also about, you can write any application you can think of based on your documentation. [23:48] One of them is mocking the API. Based on my examples, front-end developers can already go ahead and start writing their applications without me writing any actual server-side code. To achieve that, you can use many mocking applications that will read API blueprint and provide and API for us. I'm going to go ahead and just do that. [24:15] One of the modules is called Drakov. I've already instead it as a dependency here and I have added to my script running for the file that we're describing and we're setting it on Port 3000. Let me go ahead and do it. Run, run, run. Mock. [24:46] Drakov will list what it read out from the document, the supported endpoints and we can actually browse the mock server, going to 3001. There we go. We got back the root response. I'm just going to quickly go through with postman. I don't know how to make postman bigger. I can't. [25:23] With postman, I can just click through links, which is one of the RESTfulness about the API. I have the root resource. I can just go to users. From users, I can go and look at the user repo. This is what Roy meant about running your API through hypermedia or having the state as hypermedia. [25:51] It's not just mock servers that you can do. Based on the documentation, you can also do testing servers. Testing servers can be really useful. [26:09] They again read the documentation and then start making HTTP codes because against any specified endpoint, an arbitrary endpoint. What I'm going to do, I'm going to run the testing server against the mocking server. Let me do that quickly. Next. There you go. [26:26] [pause] Laszlo: [26:47] The mocking server is still running. The testing server is called Dredd. It is another open-source module and I just added to the scripts as well here. I'm going to go yarn test. It should run the test. Yes, yarn test, against the mocking server. All the test are testing so I'm just going to go run it again. You can see that Drakov is responding to the testing server. [27:35] Obviously, we have now the mocking server, front-end developers, or rather client developers. Service consumers can test our services without us writing any code, validating our ideas. Then we have our automated testing server that we can start testing tests. [27:55] Let's say I'm just going to go and I'm going to start my last step. I'm just going to go and say...I'm going to show you the first ugly API. What I'm going to do, I'm just going to write just enough code to pass the first test. I'm just going to hard code my response for the root, which is users and repos. I'm going to serve that back to the API. [28:27] This is the simplest example for testing the first test. I'm going to stop the mocking server and just make sure I...Yes, stop the mocking server and I'm going to start my actual server. This server is listening on 3002. [28:49] For Dredd, the testing application, there is a YAML configuration file, which has the arbitrary endpoint. It already has 3002 set to it for our real server. I'm just going to run Dredd and see if it passes the first test. Yarn. [29:15] Yes, if I go...I don't know how to show. As you can see on the top, I have one test passing and the other one failing. It's hitting my real API endpoint. This is how you can start validating your server when you start writing your code. [29:48] That concludes my presentation. I must give the credits to Roy Fielding. I do recommend for people who's interested in RESTfulness at APIs to read the thesis, the PhD dissertation. It's very easy to read. [30:05] It isn't much. It clears up a lot of misconceptions and interpretations that you can find on the Internet. You can find good books but the best is basically just go to the dissertation and read it. [30:21] The second one, I also recommend a design rule book from Mark Masse. That's actually a quite good interpretation of REST. Then you can also go to slice.com and then seek out my presentation slides. Also, there is a link on the slide for the code itself. [30:39] Thanks very much. Look me up on Twitter and medium.com. [inaudible] . [30:50] [music] [30:50] [applause]