Unit Testing Grails Services that use Redis Without Stomping on Data

The Grails Redis plugin lets you use Redis as a store for all kinds of data. I find it especially useful as a compliment to a primary datastore. I use Redis for it’s caching and set operation strengths and let relational databases, and other NoSQL databases leverage their strengths.

One limitation of Redis is that there isn’t an embedded version of it that’s easy to spin up and shut down for tests (like H2 is for databases). This can make testing more difficult as your unit tests might stomp on the data that you want to keep in Redis when you’re doing a grails run-app. Your CI server might also have multiple sets of tests running at the same time that could hit race conditions if they’re all trying to make assertions against Redis at the same time.

The solution that I’ve come up with is to leverage Redis Database Numbers. Unlike a SQL database, Redis doesn’t have schemas to partition data. Instead it buckets data into various database numbers. By default, all operations are against database “0″. The default Redis configuration comes with 16 database buckets (numbered 0..15), but most people only use the first one (database 0). We can use those other buckets to hold our test data.

The factory class below creates a new Redis service that’s configured to point to a database number falling within testDatabaseRange (here 8..15). When you ask it for a new instance, it increments a key kept in database 0 and gives you back the modulus value that falls within your specified range. This lets multiple test servers operate against the same Redis instance and makes it so that they don’t use the same database at the same time.

package grails.plugin.redis.test

import grails.plugin.redis.RedisService
import redis.clients.jedis.JedisPool
import redis.clients.jedis.Protocol
import redis.clients.jedis.JedisPoolConfig
import redis.clients.jedis.Jedis

class TestRedisServiceFactory {

    public static Range testDatabaseRange = 8..15

    public static String DEFAULT_TEST_HOST = "localhost"
    public static int DEFAULT_TEST_PORT = Protocol.DEFAULT_PORT

    public static RedisService getInstance(Map configMap = [:]) {
        Map defaultedConfigMap = [
                host: DEFAULT_TEST_HOST,
                port: DEFAULT_TEST_PORT,
                timeout: Protocol.DEFAULT_TIMEOUT,
                password: null,
                poolConfig: [:]
        ] + configMap

        if (!configMap.database) defaultedConfigMap.database = nextTestDatabase(defaultedConfigMap)

        RedisService redisService = new RedisService()
        JedisPoolConfig poolConfig = new JedisPoolConfig()

        defaultedConfigMap.poolConfig.each { configKey, value ->
            poolConfig."$configKey" = value
        }

        JedisPool jedisPool = new JedisPool(
                poolConfig,
                defaultedConfigMap.host as String,
                defaultedConfigMap.port as Integer,
                defaultedConfigMap.timeout as Integer,
                defaultedConfigMap.password as String,
                defaultedConfigMap.database as Integer
        )

        redisService.redisPool = jedisPool
        return redisService
    }

    public static int nextTestDatabase(Map configMap) {
        Jedis jedis = new Jedis(configMap.host as String, configMap.port as int)
        assert jedis.ping() == "PONG", "Unable to ping redis server at ${configMap.host}:${configMap.port}, make sure you've got redis running"

        // cycle through redis databases so that we won't have collisions where redis tests for 2 CI jobs use the same database #
        Integer counter = jedis.incr("test:nextRedisDatabase")

        return (counter % (testDatabaseRange.max() + 1 - testDatabaseRange.min())) + testDatabaseRange.min()
    }

    private TestRedisServiceFactory() {}
}

Now you can create a unit test, and get full interaction with Redis, without worrying about deleting data you might want to keep:

@TestMixin(GrailsUnitTestMixin)
class MyServiceUnitTests {

    RedisService redisService = TestRedisServiceFactory.getInstance()
    MyService myService

    @Before public void setUp() {
        redisService.flushDB()  // ensure that we're clean on setUp so no one else pollutes us
        myService = new MyService(redisService: redisService)
    }

    @After public void tearDown() {
        redisService.flushDB() // ensure that we don't leave anything around to pollute anyone else
    }

    @Test void testRedisServiceUnitTest() {
        redisService.foo = "bar"
        assert redisService.foo == "bar"
    }

    @Test void testMyServiceWithRedis() {
        ... // setup, possibly using redisService to insert data into redis that we want as part of test setup

        myService.someMethod()  // some method that calls into myService's redisService

        ... // test assertions, possibly using redisService to assert that expected modifications have been made
    }
}

The only case where you’d have a collision is if you had more than testDatabaseRange concurrent tests running, or if one server ran through many quick tests, and another had a very slow test. The solution to that is to just increase the number of databases that Redis allocates at startup to something much larger (like 1024) and changing the testDatabaseRange to work with that new range.