Sessions is temporarily moving to YouTube, check out all our new videos here.

You Aren't Writing Enough Tests

Simon Emms speaking at Bristol JS in April, 2017
Great talks, fired to your inbox 👌
No junk, no spam, just great talks. Unsubscribe any time.

About this talk

We know this: people don't write enough automated tests. But with a few simple ideas, you can be getting 100% test coverage and good end-to-end coverage so you can write robust, enterprise-level software.


Simon: Good evening. As you can see, because you can all read. I knew he'd turn this back on me. What I really want to do now, now that I know that, is I want to connect up the Jenkins-Box at work, so that whenever somebody breaks the bill, this is the way I put the [inaudible 00:00:17] to it. Just go squawk, some other [inaudible 00:00:18]. That's what I really want to do. Right, so, testing, the dreaded part of being a developer. Testing is a little bit like exercise. [00:00:30] We all sort of know we should do more of it, but some of us don't actually do it as much as we should. The stuff that we will be focusing on tonight ... My background, I've been a contractor for about six years, no five, and I've been developing for just over 10 years. The stuff I'm going to be doing, or focusing on tonight, is restful APL JavaScript, which is my sort of bread and butter. But the principles of it can be applied to any part of [00:01:00] development. It doesn't have to be a gyroscope. You can do this in, the fundamentals of this, can be done in any language. How I got here tonight, aside from jumping in the car from Chapman, was about six months ago I started a new contract at a place in Birmingham. I was having a chat with the head of development, one of the more senior developers, and they were talking about co-quality metrics. That's really interesting to me. Steve, who's the head, [00:01:30] what are we going to do in terms of co-quality? How do we ensure that we have good hoses, maintainable? There were various things pushed about. I said, as well, that I think we should have 100% test coverage in your code. Harvel, who is the senior developer said, well he looked at me really shocked. He went, there's no way you'll do that. I went, I basically did this conversation, that we're going to have in the next sort [00:02:00] of half an hour, or so. About three weeks ago he said to me, you know when we said about doing 100% test coverage, I didn't think you'd be able to do it, but you proved me wrong. I'm going to bet you a pound that you aren't brave enough to go to Vancouver and speak at the No Jazz Conference there. So I submitted myself and I find out the next week I got accepted. So I thought I'd come and polish it up a bit by talking to you guys. Why do we write tests, and if anyone can shout out a few ideas about why we write tests. Speaker 2: [inaudible 00:02:31] Simon: [00:02:30] Refactoring the codes is a good one. There was one I heard over there. Speaker 3: [inaudible 00:02:35] Simon: Find [inaudible 00:02:36] that's fine, yeah. Those are sort of the main two ones. One I like, or I hear I a lot is, to prove my software works later on. The whole point of software is, that we are making an assertion and we want to go back later and prove that it still works. Refactoring, as well, that one shouldn't have come up, that last one. That was the big reveal. But [00:03:00] they also serve as my documentation. I'm going to go home. I'm going to forget. Go the pub with my mates. I'm going to talk to my wife, and she's going to tell me about things that I'm not really listening to, but I claim that I am. Then I'm going to come back to work in three weeks, and somebody is going to say, how does this work? I'll go don't know. Well, you wrote it. Well yeah, but I've done things since then. The third one that always makes me despair is my boss tells me to write them. The reason this makes me despair is not necessarily because it's not true. Because everyone faults at the start [00:03:30] of their careers [inaudible 00:03:31]. You must write bugs. We've got these metrics we need to keep. But the reason I say this first because I just get the feeling that people just don't get it. I think as a senior software engineer, and anybody who is in the same position as me, we have to sort of be in a position to sort of teach the junior developers why unit tests, and testing in general, is important. My general reason is why do we write tests is that they reduce the levels of uncertainty, can't even read, in our software to manageable [00:04:00] level. Now, I am of the opinion that software development, it is an art, we're all creatives. No matter whether you're from front end, back end, you do embedded stuff on machines. We are all creative. We are building something. Actually, I think that software should be beautiful, whatever that definition of beauty is. But more to the point, software engineering is a science. The degree that many of us got is Computer Science. The scientific method is [00:04:30] ... It's not the right answer. It's not anything else, but over time we learn the least wrong answer. That is the scientific method. [inaudible 00:04:41] because people will go, and they'll go, and they'll say, this is what we should do for science today. Then somebody will go and do some research. Then in a few weeks later actually no we were wrong three months ago, this is what we should be doing. Then most people don't understand what science is. As scientists, we are making sure that our software is better today then it was yesterday. [00:05:00] The more test we write the less room for error there is going to be in our software in the future. If we write line or code that can do two things, and we only test one of those things, it's got a 50% uncertainty, right from the off. Then if you extrapolate that with all the thousands and thousands of lines in your code that's just going to be ... We don't have any level of certainty in our code. [00:05:30] What counts as enough tests? Well, the obvious one I've already thrown at you. It's a nice roary thing, so it's a good thing to have. But it's 100% unit test coverage of our files. That is all the lines of code, all the functions, all the branches, and all the statements. There's 100% of absolutely everything. The other thing that I like as well is to have every major branch covered in our end-to-end tests. [00:06:00] One of things with an end-to-end test is ... It's a unit test is there to show that that particularly unit of code worked. It might be a small thing. It might be embedded deep down in the software. It might only be called once every three weeks, just for whatever reason. The application as a whole is written all the time. The unit test is there to say, this is a piece of software that works correctly, and the end-to-end [00:06:30] test is there to make sure the whole application works as you have defined as what is correctly. Just going back to the 100% unit test coverage, one of the reasons that I'm also a big fan of the 100% testing [inaudible 00:06:45], is that many years ago I was working a company up in Birmingham. One of the developers came over to me, and he said, I've seen the line of code and I think you wrote it. Okay, what's it say? It says, [00:07:00] if error phone you are a twat. Okay, yeah, I think that might have been me. What happened was I was doing that thing we've all done, where we go, ahhh, yeah this is what it should do. I'm just going to hack away, and I'll get rid of that line in minute. But I just want to make sure the whole thing works together. Yep, brilliant. Then that works, that works, that works, and then somebody comes over to you and disturbs you. You go make a cup of tea. You go for lunch, and you've left that line of code in, and you didn't remember to take it out. If you write a nice unit test, I can't guarantee [00:07:30] this, but I'm fairly confident that if you wrote in a unit test this should throw the error you twat. That you'd probably not write that. I know I almost certainly wouldn't. One test feature ... one test should be written for every new feature, or bug fix, that you submit as code request, or as a commit to the master branch, however, how you're doing it in your software. Now, the reason for that is [00:08:00] that if something already passes, and you go right I want to make sure this works. Oh, it passes already. I don't have need to write a unit test for that. What you're doing is you're continuing to have uncertainty, even if you haven't got any previously failing tests. So let's have a look at this why I think additional tests are useful. This is going to be fun. Right, so if you go on to Google and you put in [00:08:30] TDD code Kata, you'll end up coming to a website, and it gives you how to do testing and development. For the record, I'm not necessarily for or against testing and development. It has it's advantages. It has it's disadvantages. I hack away with the best of them, and I want to just make sure that it works. But this is a really good thing to go through, if you're interested. These are my unit tests here. So the first is that it should [00:09:00] add together 0 numbers. I'm going to write my calculator, and I'm adding together zero numbers. So the sum should be zero. Fine, no issue with that. It should add together one numbers. Well, so add two, the sums there for two. Again, not a problem. Should add together two numbers, two plus six is eight. Fine, that's the first step of the cutter. Now, because I am an efficient developer, and I'm brilliant. They'll believe anything. [00:09:30] What I've done here, I've used the odds value in ES6. I know that what zero, one and two numbers they'll just get put together, and I know that odds that reduce in that fashion will then add it all together, so that it does my sum. I've proven that. But then later on the spec changes, and we have to add together N-Number amount of numbers together. There's really a little test there, but [00:10:00] I've skipped it. Now, if I run that test as it was, well I know that it's going to add together N-Number of numbers, it's quite a difficult thing to say. I just put that in. But what I've don is, I've changed the definition. The definition was only for zero, one and two numbers. I actually ... In the first three tests I wrote the ... I did the specification wrong. Fortunately, it's come right later. So I need to prove that that still [00:10:30] works. I have to write that test. I still got 100% unit test coverage without that being in there, but by getting it in there I've reduced the level of uncertainty in my software. This is what everyone says to me when I say 100% test coverage. If you don't have 100% unit test coverage, let's have a look at a logarithm that [00:11:00] I've written. It's not particularly good. It's quite wet. The code is not brilliant, but it works. I wanted to have all these different methods in there. So I've gone to trace and trace two. There's actually eight different methods, nine different methods now I think. When I run all those unit tests, I get 100% test coverage. Then I want to add in test three, which I've put an error in there. It's going to end up with a infinite [00:11:30] loop. It's going to keep coming back, and it's going to fail. It's going to kill my application if I let that go through. But if I don't have 100% test coverage in there that never needs to get tested. So I have introduced a critical error into my software. Nobody knows about it. A 100% test coverage will save you from being called at 3:00 in the morning, and saying the software is broken. I don't always think actually for the record, that 100% test coverage [00:12:00] always is relevant. If your software is short life, and is of low importance, who cares. If for the record, you'll doing lots of adding. Your paid to work five days, and you want to go and get another job, because your job's not really ... Your not being stretched. As a software engineer, we still should be doing stuff of importance, either to a company, or to ourselves. Right, I'm going to come on to integration tests now. Because, the thing is, that everyone talks about unit tests, [00:12:30] and they go, well, those are the important ones. So lets go and do lots of unit test. Why do we need integration tests? This is why. Unit tests work as a door shift. You have to make sure ... The unit tests check the unit of codes are working. The end-to-end tests that the application [00:13:00] works. You see the thing is, if you write lots of different units of code there's no actual guarantee that they're actually talking to each other correctly. JavaScript is loosely typed with the exception of TypeScript, which has it's stronger interfaces. Although, it still can just be called a JavaScript. There's no way of guaranteeing in JavaScript that you're interfaced, that your calling matches the interface that you've just called. The integration tests will help you with that. If you can get good levels of integration tests [00:13:30] then you'll do that. It was a good test though. So for unit tests, tests it for atomicity. They're straight forward tests to private contracts. By private contracts what I mean is it tests the individual units of codes are talking correctly. The one thing in there, in fact there's two things in there, I really want to point out. Number one, a unit test is atomic, and I'm going to atomicity in a moment. But that's just a bit of a technical word that just have a remember. It never, ever, ever, ever, ever, ever, ever, [00:14:00] ever, connects to real service. That's what your end-to-end test are for. End-to-end tests, again, are not atomic. Again, I'm going to come back the to this. An end-to-end test doesn't test absolutely everything. Let's say that in your software you'll write in a data base connexion, and you don't want your data base ... If your data base goes down you don't want your software to die. You want it just to go, well, I can't do anything with the data base, but I'm going to return a message to you and [00:14:30] say, don't worry. It's fine. Can't do anything with the data at the moment, but you go and carry on with your life, see what else you do. That's really easy to simulate in a unit test. Because in a unit test you can stop things out, and you can pass in different sections to cover that, and that's fine. In an end-to-end test, how are you going to simulate that your data base has gone down automatically. It's very easy to do it manually, but it's really, really difficult [00:15:00] to do it automatically. There's no point in that. Do the major parts of the code. Just to re-integrate, unit tests do not ever connect to a network. It's not a unit test if they're connected into a network. Atomicity, this is quite a technical term. Atom is a Greek word, which means can't be split down. Obviously, the Greeks have never heard of Ernest Wilford, but that's neither here, or [00:15:30] there. What this does is, that it tests the underlying state of the data. Now, in a unit test, it's fairly straight forward because you are only testing your input of the function, and the output of the function. There's no underlying data in there because you're giving it everything it needs, and you're coming at it with everything it expects. So it's fairly straight forward to do that. In an end-to-end test, when you connect a real data base, what could end up happening is, [00:16:00] and I've had this before, where you've got two thousand end-to-end tests. End-to-end test number 746, it's job is to get some data. Check that data is correct. Then make sure, and it puts it to the end point correctly. Fine, not too difficult. But what you don't is that unit system number 534 that's created that data. So what you do is, you send off to your continued integration server. Your continued integration server [00:16:30] comes back with an error, fine. You see that 746 has died. You go okay, right, okay. I'm going to focus on only that unit test. I'm going to put the dot only on there. What happens? It passes. Then you've got to go through the previous 745 tests to work out which one has added that data, which has broken your test. There's some really good tests out there. There's one called AVA, A-V-A, which is actually a specific [00:17:00] test runner that designed for atomic events. It's all run in parallel, so it never tells you which order they're being run. It does it effectively in a random order. That's good for this type of testing. Because there's no dependency on anything previously, or anything afterwards. How do we achieve 100% test coverage. This is what I get asked [00:17:30] the most. I'm a believer in 100% test coverage. Test by contract, what does it mean, or does it mean anything? Again, it's testing by the interface. You passing something, you expect it to do something, and return a particular result. Let's have a look at some more code. It's really weird looking over the back of me. [00:18:00] Right. You all have your mouths closed at the moment. Right, here we go. Right, a very simple user store. It's not particularly well written, but it simulates a job. We're getting the Mongo Client from the Mongo DB library. We're finally using it in password. We pass the user then into the password. We connect the Mongo Client, and then we just make the call. So [00:18:30] we've gone into our users collection. We find one by using N password. Never actually do that with a password. You should always put a hash number, but this is just a demo, so it's fine. So that's how most people, I think, would write their software. All mostly straight forward. Let's see how it would look with a unit tester. Well, you can't really test it with a unit tester, I'll argue. Because the first thing you have to do is you have to make a data base connexion. Like the one we've got right now. Then we know here. So fine to use in a password. So [00:19:00] we go and test a random password, and we expect that to be done. But we've got to pass it in a data base connexion, the way it's written. There are certain tools like Proxy [inaudible 00:19:09] that can simulate. That can sort of replace that connexion. But it makes your software really flaky, and it's a real pain to use. Don't test like that. The one thing that I would do is a thing called dependency injection. Likes have a look at this. Dependency [00:19:30] injection, it's not like sequel injection, which is a bad thing. Dependency injection is a very good thing. We have largely the same code, except the difference between the previous methods on this one, which is a bad one, we are ... Our modules for export is an object, and then in here it's making a data base connexion. [00:20:00] In this one, we're exporting the function, and that function received the Mongo DB connexion. Here, we don't actually have to make the data base connexion. All we're doing is we're going ... Well, in both that data base we've got to do it as a function, and then we'll return exactly the same as before. All straight forward. The difference with this is that we can actually test absolutely every part of it. [00:20:30] All I need to do is, this is a stub of my Mongo. My Mongo is going to be [inaudible 00:20:39] is a bit dicey. It creates mocking objects of everything that you can really interrogate during your tests, and we pass it in here. So as I use the store, which is the previous file, and we can simulate a valid user. We can simulate no valid user. We can simulate [00:21:00] a connexion error. That's basically three things that the data base is good. You've got a good answer. You've got a bad answer, and you've got an error. I've got 100 different test codes in that file, and I made one change. All I did was export a factory function there. Factory function sort of came to prominence in the early 90s, a bit later actually than the early 90s, with a book called "Design Patterns" by the Gang of Four. Anybody who is a software engineer should have a copy [00:21:30] of that book. It is brilliant. It explains lots of different common ways of approaching software problems between objects. The factory function is, as I've just shown, is a way of you call a method, and it returns something in a known state. The known state is today's data connexion is returning the promise, and it's going to make that call to my Mongo connexion due to the advantage of the second [00:22:00] example of the dependency injection. It means I can reuse a lot more of my code. I don't have to keep making my Mongo connexion. It means I can do lots more advanced things. When you're doing things with really high traffic servers, you can do things like safer connexion pooling. Writing a good end-to-end test, start with some known data. This is the thing that you always have to start with. I mentioned it a few times if you have to miss a tea. Create lots of data. You've got ... You're testing the ... Everything works together. [00:22:30] We've got a data base connexion. If we just go back to our previous example, I want to make sure it works with a valid user, an invalid user, and with no user. I can't test it if it works with an error. It throws an error, but that's okay. It could mean the tests are covering that. But I can do that with my end-to-end test. The beauty of that ... If you don't do that with the dependency injection you can actually come back and it hasn't defined everything correctly. It goes back to the picture of the door. [00:23:00] Never rely on data that you create in a test. That's really key. It goes back to the atomicity. If you're setting some data in a test consider that data tarnished, never use it again. The reason you shouldn't use it again is because it is actually quite time consuming in an automated test to drop down your data base. To create your, if you're using MC SQL data base, to go [00:23:30] and create the Skeemer, and then to populate it. You just don't ... You don't need to do it for every single test. You've got two thousand. It's going to take hours to run. If you just sort of say, okay, start off with this known. I'll do my getters, and anything that I post to it I'll never use again. It's all fine. You're test will run in a few minutes, or a few seconds, hopefully. If you've got any third party services, and a client with this learned the hard way. If you've got any third party services, like you're doing something [00:24:00] you're going to post to Twitter. Stub it out. Have a mocked instance of Twitter. Use something like Swagger, or if you don't want to invest in Swagger, or something like that, then just create an express application that behaves in exactly the same way as Twitter. It does some known things. The reason you should not rely on a third party server is two things, right. Firstly, you've gotta have that AP ... You've actually then gotta have an account like that. Just something like Mail Good, that's going to cost you money. [00:24:30] That you don't want to do. Secondly, if it's sort of like Twitter then you'll have just the test just sending out reams and reams of data, which just looks unprofessional. Then ditch Postman it's a real important thing. I say ditch like an inverted comment because I find Postman a really useful tool. By ditch Postman, I mean, any test that you write the first time Postman, write it as an end-to-end test. If you do it as an end-to-end test [00:25:00] it's part of your arsenal. It makes sure that all your code works correctly, and you can keep going on, and on, and on. If you do it in Postman it's infemoral. Once you've closed your machine, unless you've tested it correctly and shared with all of your team and you've paid for the teams, you can't do it again. But if you put it as part of your end-to-end tests it's then part of the contract of your code. Right, so I'm going to show you a working example of my code. Right at the bottom of that link is [00:25:30] a link to all the slides, and to all the software that I've done. Which I've just turned off, which is really clever of me. So let's have a look. Let's have a look at a known, [00:26:00] right. This is using a library, which I wrote myself called Steeplejack. The purpose was it was to avoid me having to all the things that I was doing all of time, including the dependency injection, and the test coverage. This defines all my injection stuff by default. So [00:26:30] it means I've got lazy loading, which is one of things that I wanted to work on. I've got here, I define my dependencies externally. I named this one here sort of my public service, and called this my product [inaudible 00:26:40] and my product [inaudible 00:26:40], and product model. When I want to find my product ID, well that's nice and simple. I pass in my ID. I've passed in my product store at the top here. So that's then ... So that's dependency is then injected, and it just then either returns it's My Model, or not. [00:27:00] Let's have a look the unit test because their fairly simple. I test here that I've defined my injectors correctly, which then that's then part of my unit tests. I passed in here, I've got my store, so that's my Mongo connexion. I'm going to make sure it resolves with some result. My model here is going resolve with collection results. I [00:27:30] passed it in there. In this we could have tested by contrast. I don't actually need it to be a real example. It doesn't actually have to look like the data it's going to look like. If you're doing something with that data it does. But I'm not doing anything with that data, and that. So I'm just passing in, making sure I get it out. I'm making sure that my store is called once, with no argument. I'm making sure that my model is called with one argument. Because I don't care what's happening, this is just an example. Because somebody asked me last night about, well, surely you're testing too [00:28:00] much Simon. You've got everything in. Well, I'm not. Because all I'm doing here, in this one here, so there are no Rd function. It is making a data base connexion for each of those IDs. I happen to be using Lowdash, but I could perfectly easily just replace that with array prototype reduce. That wouldn't break my test because my test is to make sure that this Rd function is called three times with those values. That is a suite [00:28:30] test that we can build upon. We start off with that data, and we go on, we profit. We can go, and make dinosaurs, and do more fun things. Right, any questions. Yes. Speaker 3: What's your software suite? I recognised some things you [00:29:00] did with [inaudible 00:29:03]. What's your tools off the top of your head? Simon: My go to one is markItUp is my test runner. Chai is my assertions library. CINON is my stub. The reason I do that is because when I started doing OJS, four or five hundred years ago, that's what everyone was using. Of the things I've come across since there's never been anything I've gone and thought, I actually [00:29:30] need to change it for this. I'm partially tempted with AVA. I've explored that. The only one that I've looked at and thought, maybe. But for me personally, because I've sort of trained myself over five years to write atomic tests. I don't need that functionality. For somebody who is new, and sort of getting used that writing atomic tests, AVA would probably be a really good way of starting. Speaker 5: Is it working? Yeah, I wanted to talk about when you're faulting code, [00:30:00] and there's like lots of graphics. You've got some graphics there. But it's massive, and there is quite a lot of stuff going on in there, and you don't really know what's happening. Simon: Yeah. Are you part of D3, by any chance? Speaker 5: No, no, no, I'm faulting some like stuff, like doing triangles and stuff like that. Simon: Right, a very good question. My answer to that is I do back end, and I don't have to bother with that sort of thing. I had this conversation with a mate [00:30:30] of mine, because he is a front end developer, and because of my big mouth, as he would say. They had to end up doing 100% unit test coverage, as well. I wasn't very popular with that. Me and him actually worked with the D3. We sort of worked on a preliminary stub for D3. So what we had is, we had some helper files. Because the D3 was commonly used throughout the application, but we went, okay, we'll create a D3 stub [00:31:00] class as part of our helper tests. It's like a helper file inside suite of unit test. The idea then was that it made ... We could then test ... We'd make the right call to draw this particular diagram. It's a really difficult question, and aside from sort of like stubbing those sorts of things out. Because you're not testing ... The important thing to realise is that you're not testing that that application works correctly [00:31:30] by virtue of the fact that they published it. They are saying, we are going to make sure it works correctly. You just have to make sure that you're using it correctly. Speaker 6: [inaudible 00:31:39] actually buy D3 [inaudible 00:31:41] correctly. Simon: Yeah, it's a very difficult one that. I can see from your face that you don't think that I've answered the question. Speaker 5: No, I don't know. I don't have to spend ... If you ... I mean, the algorithm might be working on sort of complex sets. Once [00:32:00] you've got the visual working the same way then you know you're quite a long way to getting there. But then, afterwards, you've got ... Well, I don't know necessarily what it's supposed to be able to do. Where it's supposed to be able to fail, and where it isn't. If you're passing somebody else's code, you don't know ... Assumptions are made on t limitations of that library, for instance. Simon: Yeah, [00:32:30] when you say passing somebody else's code, are you like doing an MPM in store, and using the stuff that they've published, or are you sort of faulting it? Speaker 5: Faulting some job trip libraries to hack. Simon: Okay. Speaker 5: So borrow, and triangulations, and some other things, yes. Simon: I don't know, is the short answer. I'd be interested to have a look at that, and maybe we can sort of- Speaker 5: [inaudible 00:33:00] some problems. Simon: [00:33:00] Yeah, message me. Say hello. Anyone else. Speaker 7: Any other questions. Good, thank you very much. Simon: I got off lightly.