Unit testing IO in Haskell

The generally side-effect free nature of Haskell code makes it convenient to test. Haskell programs can interact with the outside world — otherwise they would be useless — but these side-effects are only possible in the IO monad. It is still important to test this code that performs IO, but we found it much more challenging. This post explains how we used mock IO monads along with monad typeclasses to solve this problem, while also improving our application code in the process.

Note that this post assumes intermediate knowledge of Haskell

Recently I began working on a Haskell Pusher HTTP library. This is a good example of the structure of many programs: there is a single location where an IO action is performed (in the case of this library making an HTTP request), and the rest of the code is pure logic that generates the arguments for the IO action, and processes the result. We will use the IO in this library as an example of something we want to test.

The Haskell Pusher library uses the following function to make HTTP requests. It is defined in the http-client library and returns an IO action to make a request and return the response as a lazy ByteString.

httpLbs :: Request -> Manager -> IO (Response ByteString)

Mocking the server

In order to test the IO code, we need to simulate the server that the library is interacting with, using pure code. Because we need to represent something that responds with data, we can model this using a reader monad. asking for data can simulate making requests to a server.

bash
1newtype MockServer m a = MockServer
2  { server :: ReaderT (Response BL.ByteString) m a }
3  deriving (Applicative, Functor, Monad, MonadTrans)
Copy

We defined _MockServer_ as a monad transformer so that it can be combined with other monads.

The idea is that we can “run” this reader monad with the particular response we want to send.

bash
1runMockServer
2  :: MockServer m a
3  -> Response BL.ByteString
4  -> m a
5runMockServer (MockServer s) = runReaderT s
Copy

Generalising the types

We now have a problem because the Pusher library code needs to type check with this mock server, as well as a real server (IO). The existing concrete type of IO a won’t do. The solution in many languages would be dependency injection: take the httpLbs function as a parameter. But the solution we came up with does not require the function to be explicitly passed in; instead we create a generic monad typeclass, that can represent both the mock and IO, to be used as the return type. This allows the caller of the library to pick the concrete type that the function should return. Lets’s see this in practice:

bash
1class Monad m => MonadHTTP m where
2  httpLbs :: Request -> Manager -> m (Response BL.ByteString)
3
4-- And the two instances we need:
5
6instance MonadHTTP IO where
7  -- The regular function from the http-client library
8  httpLbs = HTTP.Client.httpLbs
9
10instance Monad m => MonadHTTP (MockServer m) where
11  -- ask gets the environment from the reader monad;
12  -- i.e. what you "run" it with
13  httpLbs _ _ = ask
Copy

Using these typeclasses has a benefit beyond testing:

• It splits the IO type up into types that specifically reflects the action to be performed: for example MonadHTTP could be just for making HTTP requests, and MonadTime could be used for getting the current time
• It is easier to refactor because new concrete implementations can be swapped in by implementing the typeclass. For example, if you no longer wanted to perform an IO action, then you could simply make your pure code an instance of the monad typeclass instead.

Using the mock server in tests

Let’s see how we test the get function for performing HTTP GET requests in the Haskell Pusher library using the mock server.

First define the response the mock server will respond with

bash
1succeededResponse :: Response BL.ByteString
2succeededResponse = Response
3  { responseStatus = mkStatus 200 "success"
4  , responseVersion = http11
5  , responseHeaders = []
6  , responseBody = "{"data":"some body"}"
7  , responseCookieJar = createCookieJar []
8  , responseClose' = ResponseClose (return () :: IO ())
9  }
Copy

Then, using hspec, we can compare this with the result of running our get function (which uses httpLbs internally to make the request).

bash
1test :: Spec
2test =
3  describe "HTTP.get" $ do
4    it "returns the body when the request is 200" $
5      withConnManager $ \mngr ->
6        runMockServer
7          (get mngr "http://example.com/path" [])
8          succeededResponse
9        `shouldBe`
10          (Right $
11            A.Object $
12              HM.singleton "data" (A.String "some body"))
Copy

And so we can test IO in much the same way as regular hspec tests. Testing the similar post function would be very similar, and testing a failed response would involve defining a failedResponse and running the mock server with that instead.

When readers are not enough

In our example, the reader monad was enough to model the tests we wanted to implement; it was sufficient to hardcode a single response on a test-by-test basis. However, you may find that a real world system you are trying to mock requires more complex logic, for example, it may need to keep track of some state during the course of the test. In that case it may be more appropriate to use the more flexible state monad. In general, you will need to think carefully about what IO your system is performing, and exactly which monad(s) you will need in order to model it.

Pushing the boundaries of what can be mocked

The example we have shown in this post is a simple one. Things get much more complicated in some real world applications, depending on the kind of IO you wish to mock. For example, we have a much larger system that we have applied a similar testing strategy to. We found the trickiest kind of IO to mock was concurrency (specifically forkIO) and STM. It was possible, but it did mean we had to write a mock thread scheduler — a non-trivial amount of code.

An interesting future project would be to create a test harness that had built in support for mocking these common IO actions.

IO mocked! And other goodies

In general we found this approach to be a great way of testing IO in Haskell. Not only this, but as mentioned before, generalising the return types to typeclasses lead to other benefits. The caller now picks the concrete return type, which makes refactoring easier, and also lets the client of library code implement their own concrete return types. Another advantage is that the type signatures are more descriptive than just IO. Finally, like with most testing, we found that it forced us to structure our application code better, particularly in terms of separating IO from the pure logic.

I am still relatively new to Haskell, so feedback is always welcome. In particular I have read how free monads can be used to achieve similar things, and I would be interested to hear how that approach compares.

Further reading

• Porting the Pusher integration tests to Haskell
• 10 things I learnt diving in the functional programming deep end – with Haskell
• Useful online resources to learn Haskell