Testing Communicating Haskell Processes with QuickCheck and HUnit
In previous posts, I have explained the sort pump and a way to generate prime numbers. It would be good to get some assurance that those programs work correctly; we need to test them. The newest release of CHP (v1.4.0, released yesterday) contains a new Control.Concurrent.CHP.Test module that will help, and this post will explain how to use it.
Probably the most well-known Haskell testing framework is QuickCheck. QuickCheck takes a property for the output of a function. It then generates random inputs, feeding them to the function and checks that the property holds on the output. QuickCheck is a very good fit for testing our sort pump: our inputs can be lists of integers, and the property of the output is that it should be sorted.
An alternative approach to testing is to use something like HUnit which runs a specific prescribed test. HUnit is a good fit for our primes pipeline: we want to test that the numbers coming out are the prime numbers.
We will start by using QuickCheck to test the sort pump. QuickCheck version 1 only supported testing pure functions, which is not enough to test our concurrent programs. Thankfully, QuickCheck 2 supports testing monadic functions (even if the documentation is lacking), so we can use that via the new CHP function: propCHPInOut. As the documentation for the function says:
propCHPInOut :: Show a => (a -> b -> Bool) -> (Chanin a -> Chanout b -> CHP ()) -> Gen a -> Property
The first parameter is a pure function that takes both the input to the process and the output the process gave back, and indicates whether this is okay (True = test pass, False = test fail). The second parameter is the process to test, and the third parameter is the thing to use to generate the inputs (passing ‘arbitrary’ is the simplest thing to do).
The first parameter is straightforward — the output should be a sorted version of the input. The second parameter is the sort pump process. The third parameter could just be ‘arbitrary’, the default generator for lists of Ints, but it would be nice to make sure that the list is likely to contain duplicate entries, so we supply a customised generator:
test :: IO () test = quickCheck $ propCHPInOut (\input output -> output == sort input) sorterGrow gen where gen = do len <- elements [0..30] replicateM len $ elements [(-5)..5]
That is the complete code for testing our sort pump — and it passes. The propCHPInOut function (and its HUnit equivalent) is very useful for testing these single-input single-output stateless processes.
The primes pipeline is different as it has no input — it is a generator that just produces output. So we will instead use the more general testCHP function that works with HUnit. testCHP takes an item of CHP Bool and turns it into an HUnit test, so we have to do all the heavy lifting in producing that Bool that represents test success or failure. (In future it might be nice to provide support for HUnit asserts in a CHP program, interfacing nicely with the poison mechanism, but that’s not implemented just yet.) The key thing to be careful about is that if your process exits with poison, that is automatically counted as a test failure. So you need to make sure that if the test passes, it shuts down without passing on the poison — for this we can use onPoisonTrap rather than onPoisonRethrow:
test :: IO () test = do runTestTT $ testCHP $ do c <- oneToOneChannel liftM snd $ (primes (writer c) `onPoisonTrap` return ()) <||> (do ps <- replicateM 200 $ readChannel (reader c) poison (reader c) return $ ps == take 200 funcPrimes ) return () where funcPrimes = sieve [2..] sieve (p : xs) = p : sieve [x | x <- xs, x `mod` p > 0]
This code creates a channel to communicate on, and passes the primes process one end of it. Note that any poison thrown by the primes process will be trapped rather than passed on. At the other end of the channel, we read the first 200 numbers then poison the channel, before checking that these are indeed the first 200 primes (which we generate with the classic functional algorithm). If the primes process unexpectedly poisons the channel, the poison from the second part of the parallel composition will propagate to the top-level and thus fail the test — we don’t need any extra code to check if primes misbehaves by arbitrarily poisoning the channel.
This second test is an example that builds up and tears down the whole infrastructure with our own code, but it still is not too long. Just as with pure functions, QuickCheck is useful for testing random inputs to processes, whereas HUnit is useful for specific tests. It would also be nice to get Lazy SmallCheck working with CHP, but because of the way it throws errors for test failures it may be more difficult (and I haven’t checked if it can do monadic tests). These examples have worked with processes that do not carry any particular state — in future I hope to return to look at testing processes that do keep some state over time.