Grails build-test-data Plugin Released!

2009/04/14

Creating maintainable test data is hard.

Often an entire object graph needs to be created to support the instantiation of a single domain object. This leads to either the cutting and pasting of that creation code, or relying on a canned set of objects that we’ve grown over time and maintained as the domain objects change. After a while, adding just one more Widget to that set of canned data ends up breaking tests just about every time.

There has to be a better solution, right?

Yep! Due to the “Power and the Glory of Grails”™, we have a lot of metadata at our fingertips about those domain objects. We know what validation constraints we’ve placed on our objects, and which objects depend on other objects to live.

Using this additional information, Joe Hoover and I have created a grails plugin called “build-test-data”. With it, you can create test data and omit the fields that you don’t care about for your test. The plugin takes care of populating the values required by constraints and persists your domain objects to the database.

This plugin is focused on creating test data for integration testing. If you’re doing unit testing, there are a number of mocking solutions out there (including a few ways to do it built right into groovy). Unit testing only goes so far though, and if you want to ensure that your code works when it’s hooked up to a real database, with real transactions, saving, triggers, etc, you’re only going to get that from an integration test.

Installation of the plugin is easy. Just run this grails task in the root of your project:

grails install-plugin build-test-data

When the plugin is installed, most domain object should be able to be instantiated and persisted out of the box.

// automatically creates/saves a Book with all required fields populated
def book = Book.build()

Once installed, all you have to do is call the new build() method on your domain class and you’ll be given a valid instance that has been persisted to the database.

It will populate the required fields as defined by the constraints and it has the ability to deal with almost all of them.

A couple of them, such as custom validations and regular expressions, are very difficult to work with so if a sample value that we try doesn’t get past validation, we’ll throw an error.

Build Test Data Example

Here’s a quick example. Say you have the following domain objects:

// grails-app/domain/Author.groovy
class Author {
    String firstName
    String lastName
    static hasMany = [books: Book]    
}
 
// grails-app/domain/Book.groovy
class Book {
    String title
    Date published
    BigDecimal price
    static belongsTo = [author: Author]    
}

Remember that grails properties are required unless nullable is explicitly set to false, so all of those properties are required.

If you’re testing a BookPurchasingService and the method you’re calling needs a Book, normally you’d need to also create an Author to go with it. Here’s how you might normally create your test data without the build-test-data plugin:

void testPurchaseBookService() {
	def author = new Author(firstName: "first", lastName: "last")
	author.save()
	Book tenDollarBook = new Book(author: author, title: "title", published: new Date(), price: 10.00 as BigDecimal)
	tenDollarBook.save()
 
	def result = service.purchaseBook(tenDollarBook)
 
	//... assertions about expected result of service call
}

You need to create an Author just to create the Book, even though your service only cares about the Book. If the Author class ever adds an additional required field, you not only need to update all of your Author test classes, but all of your Book test classes as well.

Additionally, there’s all kinds of “noise” in your test class that makes it more difficult to understand exactly what it is you’re trying to test.

If you’ve got the build-test-data plugin installed, you don’t need to worry about any of that. Creating sample data is easy and you can specify only the parameters that you actually need specific values in to exercise your test. The plugin will fill in the rest and persist it to the database automatically.

void testPurchaseBookService() {
	def result = service.purchaseBook( Book.build(price: 10.00 as BigDecimal) )
 
	//... assertions about expected result of service call
}

Behind the scenes, build-test-data populates the Book.title and Book.purchased fields, it also creates a new Author with a firstName and lastName and saves all of that to the database.

This test is much cleaner as you only see the things that actually matter to the test and none of the administrative flotsam and jetsam needed to set things up.

In the repository on the bitbucket website, there is a test “bookstore” grails application that has a fairly complex domain model that exercises the build-test-data build method and it’s ability to serialize an object graph.

Check out the full set of documentation on the bitbucket build-test-data wiki and let Joe and I know what you think of it.



There are 28 comments in this article:

  1. 2009/04/14Chad say:

    Nice work guys – glad to see you guys were able to get this released. Look forward to using it! thanks, chad.

  2. 2009/04/15faenvie say:

    what advantages does this approach have
    compared to the grails-fixtures-plugin ?

  3. 2009/04/15tednaleid say:

    @Chad Thanks! It took a bit longer than I was hoping to get it out, but I think it’ll be useful. I’m using it on my current project so that helps drive development :).

    @fanvie Good question!

    Fixtures start out well, but I think they degrade over time as the size of your project gets bigger.

    There are 3 big advantages that I think build-test-data has over grails-fixtures.

    1. To create a fixture, you need to specify the entire valid object graph. This means that it takes more time to create a valid fixture to use in your test. Because it takes more time people tend to reuse the same fixtures in multiple tests causing dependency issues between tests. You need a 3rd Widget in the fixture for your new test, but adding that breaks all the tests that were expecting only 2 Widgets in the fixture. Otherwise, they copy and paste a huge object graph just to create a new Widget which has it’s own set of problems.

    Build-test-data avoids this because it’s easy and lightweight to create another dataset that’s specific to your particular test case. There’s no interdependency between tests.

    2. Fixtures are static and this leads to fragility. If you add another required field to your Widget, this will break all of the fixtures that instantiate that Widget, even if the Widget isn’t the object your test is directly using, it just happens to be in the required dependency graph for an object you do need.

    Build-test-data automatically fills in any missing constraints, so your tests will continue to pass unless the code under test is requiring specific values in the new fields. If it is, you just need to modify the tests that are directly affected.

    3. Fixtures separate the definition of your data from the actual test code. This can make the tests harder to understand if you aren’t intimately familiar with the data that the fixture is creating. The attempted fix is to try to give the fixture a meaningful name, but that’s not always possible to the level of detail that you need to grok your testing data. (i.e. I see that we’re asserting that we got 3 widgets back for this service call, but why? What’s special about those widgets over the other widgets?)

    Build-test-data is small and quick, so it lets you keep your test data definition right in your test rather than separated away in another file.

    I’m starting a page on the wiki to compare build-test-data to other test data creation patterns and to answer questions like these (it’s not quite complete yet).

  4. 2009/04/15Doug say:

    Hi,
    Thanks for the plugin. This is exactly what I am looking for to add quick dummy data in development env.
    One question though, do you create the values randomly? I will check out the plugin later today.
    Doug

  5. 2009/04/15tednaleid say:

    @Doug

    It depends a little on the constraint and the class of the object, but for the most part the values generated are the simplest thing that could possibly work. For string values, the first thing that’s tried is the name of the variable (with truncation or padding if there are length requirements). That kind of thing.

    It’s possible to use the TestDataConfig file to override how a property gets generated at a

    I’ve been thinking about some sort of wildcard approach (similar to how grails filters work) that would let the user override entire sets of properties, but that’s not in there yet.

    If you have other ideas on how values should be generated, I’d be interested to hear them.

  6. 2009/04/16Feng say:

    Nice work, this is what I need in TDD.
    I use java.sql.Date in my project.
    Can I ask a feature? When build an Author, I hope to claim how many books.

  7. 2009/04/16tednaleid say:

    @Feng Thanks, I’m glad you think it’ll be useful.

    I’ll make sure to add java.sql.Date support in the next version, for now you should be able to just provide the date values as parms to the build method.

    For the feature request, that sounds like a good idea and I’ll add it to the list of enhancements. I’m assuming you’re talking about doing something like this:

    def a = new Author(books: 3)

    Where books is a hasMany on author. The plugin looks at the value, sees that it’s a number rather than a list of books and instantiates 3 books automatically.

    That should be doable and I think it’d be a worthwhile addition. Right now you’d have to do something like this:

    def a = Author.build()
    3.times { Book.build(author: a) }
  8. 2009/04/22Chad Small say:

    Hey Ted,

    I finally hit my first heavy need for this plugin – took a bit, but I ran into this integration test around object graph transactional saves. Do you know ‘why’, as you state it in this post — “Remember that grails properties are required unless nullable is explicitly set to false, so all of those properties are required.”

    If no constraints are put on the class, why must every property be initialized on a new. For example, based on your example above, if this is done (all props included in new)

    void testSaveGraphFine() {
    def author = new Author(firstName: “first”, lastName: “last”)
    .addToBooks(new Book(title: “title”, isbn: “123″, published: new Date(), price: 10.00 as BigDecimal)).save()

    def foundBook = Book.findByTitle(“title”)
    assertEquals “expect to find book”, “123″, foundBook.isbn
    }

    Life is good because all of the props are included in newing up the objects in the graph. HOWEVER, if you have the same test, but take off a property, say price…

    void testSaveGraphBoom() {
    def author = new Author(firstName: “first”, lastName: “last”)
    .addToBooks(new Book(title: “title”, isbn: “123″, published: new Date())).save()

    def foundBook = Book.findByTitle(“title”)
    assertEquals “expect to find book”, “123″, foundBook.isbn
    }

    Grails is not happy – but I’m wondering why – to distinguish blank from nullable? Hoping you can shed some light.

    Whatever the reason, a good reason to use the build-test-data plugin ;)

    thanks,
    chad.

  9. 2009/04/22tednaleid say:

    Hey Chad,

    If I’m understanding your question correctly, I think the answer might be kind of unsatisfying: unless otherwise specified, fields are required because that’s how it was programmed :).

    You can actually see that grails adds the nullable constraint if you inspect the info you get from the Domain Class:

    def printAppliedConstraints(className) {
        println "$className applied constraints: "
     
        def dc = grailsApplication.getDomainClass(className)
     
        dc.constrainedProperties.each { name, constrainedProp ->
            println "\t$name -> ${constrainedProp.appliedConstraints*.toString()}"
        }
    }
     
    printAppliedConstraints("Author")
    printAppliedConstraints("Book")

    Prints:

    Author applied constraints: 
    	books -> [org.codehaus.groovy.grails.validation.NullableConstraint@803f5b[true]]
    	firstName -> [org.codehaus.groovy.grails.validation.NullableConstraint@e7311f[false]]
    	lastName -> [org.codehaus.groovy.grails.validation.NullableConstraint@4794eb[false]]
    Book applied constraints: 
    	title -> [org.codehaus.groovy.grails.validation.NullableConstraint@41b7f9[false]]
    	published -> [org.codehaus.groovy.grails.validation.NullableConstraint@dc1fdb[false]]
    	price -> [org.codehaus.groovy.grails.validation.NullableConstraint@e3e87c[false]]
    	author -> [org.codehaus.groovy.grails.validation.NullableConstraint@c247e7[false]]

    Showing that even though we didn’t specify it, Grails actually puts the nullable constraint on every object. I’m guessing that they thought it was a better default than not having it nullable, maybe because the database exception that’d get thrown for non-null fields that weren’t supplied isn’t as nice as the error that grails gives.

    (This isn’t the case for Lists (as you can see with the Author.books being nullable=true)

    Blank is a little different than nullable. You can have a field that is nullable, but not allowed to be blank (basically if a value is provided, it can’t be a zero length string, but if one isn’t provided, that’s ok).

    Let me know if that didn’t answer the question you were asking.

  10. 2009/04/24igor say:

    Hello, Ted.

    Thanks for great plugin. I want to vote for adding joda-time support.

  11. 2009/04/24tednaleid say:

    Thanks Igor, the next version will make it easy to support Joda time as well as any arbitray classes that a system might need.

  12. 2009/05/9Nice say:

    Why can’t I use build() in my unit tests? It says ‘no signature method found’

  13. 2009/05/9tednaleid say:

    @ Nice. You can’t use build() in unit tests for the same reason that you can’t do just about anything else without creating a mock version of it in unit tests.

    When running unit tests, plugins aren’t started up (which is why they run quicker). You also don’t have a connection to a database in a unit test.

    Built-test-data is designed for use in integration tests. I talk about this on the build-test-data wiki home page.

    You’ll either need to change your test to an integration test or else mock out what you’re doing in your unit test.

  14. 2009/05/22Szymon Polom say:

    Nice plugin! I have two questions:

    1. Is there a built-in way to respect unique constraints with the plugin? E.g. I have an email field that has to be unique.
    2. Is there a way to delay the instance saving? E.g. I have domain classes with a hasMany relationship and a minSize > 1.

  15. 2009/05/22Szymon Polom say:

    Btw, your Wordpress doesn’t redirect back correctly after posting a comment.

  16. 2009/05/22tednaleid say:

    @Szymon

    1. Currently unique constraints aren’t supported. There are a couple of ways to deal with this in the plugin’s current state. Either pass in your own unique value to the build() method for the field that needs to be unique, or you can use the config file to specify a set of values. There’s an example of how you can generate 20 unique values on the sample code page on the bitbucket wiki.

    In the config file, you can create an array that contains unique values for a field, and as new instances of that object get created, unique values will be handed out.

    In the future, I plan to support closures directly for generating config values rather than needing to do the screwy thing with Expandos.

    2. To delay saving, there’s an optional 2nd parameter you can pass to build, if it’s false, it won’t save the object.

    Also, the minSize constraint is actually supported in the current version of the plugin so you might not need to do this.

    Thanks also for the report about the wordpress redirection issue. Can you let me know what browser you were using? I haven’t had any problems with firefox 3 on OSX, but I’d like to see if I can track it down.

  17. 2009/05/30Rob Fletcher say:

    Really interesting plugin. I’m having some trouble with enum and joda-time properties, though. The plugin seems to think they are domain classes then barfs trying to persist them. Is there a way to get this to work? I’m using 0.1.2

  18. 2009/05/30tednaleid say:

    @Rob

    There was a fix for enums in 0.1.2, so if you’re running into problems with them, please file an issue on the bitbucket repo with a sample test case if possible.

    Joda-Time support is going to be in the next version (along, I believe, with support for any non-domain objects that have zero arg constructors).

    For now, you can either just supply those values to your call to the build() method (if they’re on the primary object you’re building). Alternatively, you can use the TestDataConfig file to return values.

    The semantics of it need a little work though. These issues are the kinds of things that are being worked on for the next release.

  19. 2009/05/30Rob Fletcher say:

    Even when setting a joda-time property via TestDataConfig or directly in the build call I get the error “Identity property not found, but required in domain class [org.joda.time.DateTime]“.

    The enum works so long as I specify a value either in TestDataConfig or the build call.

  20. 2009/05/30Rob Fletcher say:

    Additionally the joda-time property I’m trying to set is dateCreated which GORM would assign automatically on save so it might be worth getting the plugin to ignore that property even if it’s non-nullable (regardless of whether it’s a joda-time type or a regular java.util.Date).

  21. 2009/05/31tednaleid say:

    Thanks for the replies Rob. I think your idea about not setting the dateCreated (as well as dateModified) by default is a good one.

    I’ve created an issue on bitbucket to make this change. It’ll be in the next release.

    I’ll take a look at the enums too, I thought the code was such that the first enum value was picked for an enum item.

  22. 2009/06/2Szymon Polom say:

    Thank you for your responses tednaleid.

  23. 2009/06/2Szymon Polom say:

    Regarding the redirection, it happens to me with the newest Firefox 3 on both Vista and Linux.

  24. 2009/06/15Ted Naleid » Grails plugin build-test-data 0.2.1 released say:

    [...] learn more about the basics of the build-test-data plugin, see this blog post and check out the Basic Usage and Sample Code wiki pages. To install, just run the install-plugin [...]

  25. 2009/06/15Andrés say:

    Ted thank you for your time on developing the plugin and on answering in such a complete way all of the questions people have had.

    I have been looking for something like this for quite a lot time now because creating all of the object graph for complex apps is quite boring. Im eager to try your plugin and I hope to come back and give you some feedback about my experience =) Thanks again!

  26. 2009/06/15tednaleid say:

    Thanks Andrés! I’ll be interested to hear what you think of it.

  27. 2010/03/1Madhav say:

    I installed plugin and created Functional test case. But data is not persisting in data base . I see test results are success and no error being thowing out.

    Can anyone help me whats going on ….

  28. 2010/03/1tednaleid say:

    @madhav test cases normally get rolled back after they’re done so the database isn’t polluted with results. If your tests run successfully, that’s really all that matters :).

    If you’re just trying to generate some data to save in your database, you could do it in a “grails console” or even in your BootStrap.groovy file so it’s available when the app is actually running.

Write a comment: