Modularizing Groovy Config Files With a Dash of Meta-Programming

| Comments

This is a continuation of a sporadic set of blog posts about some practical uses for groovy metaprogramming.

Groovy Config DSL Overview

In Java, configuration is normally done with properties files. They’re kind of a pain because they’re inflexible, don’t allow executable code, and don’t easily provide a heirarchy.

Groovy greatly improves on Java by building in support for a simple configuration DSL using the ConfigSlurper.

To use it, just instantiate a new ConfigSlurper and give it the name of a groovy script file that you’d like it to parse.

// SampleConfig.groovy, a simple groovy script
person {
    firstName = "Ted"
    address {
        city = "Minneapolis"
    }
}
    
// TestConfiguration.groovy
def configObject = new ConfigSlurper().parse(SampleConfig)

This creates a ConfigObject, which is a subclass of a Map and can be similarly accessed using dot notation

println configObject
// prints: [person:[firstName:Ted, address:[city:Minneapolis]]]

assert "Minneapolis" == configObject.person.address.city

ConfigSlurper is built into groovy and is heavily used for Grails configuration. Grails is a “convention over configuration” framework, but anytime you want to modify the conventions, you’re likely going to be touching a config file. Grails plugin authors also often use config files to drive plugin behavior.

In larger projects, these config files can start to get quite large and unwieldy.

How ConfigSlurper Does It’s Thing

DSLs in groovy, like the ConfigSlurper DSL, work by intercepting calls to methods and property setters. If you look at the internals of the ConfigSlurper.parse(Script script, URL location) method (the big one at the bottom), you’ll see that it heavily messes with the metaClass of the configuration script by creating custom versions of invokeMethod and getProperty.

Intercepting Methods

When you have a config script that says:

person {
    firstName = "ted"
}

What you’re saying is call the “person” method and pass the { firstName = “ted” } closure to it.

Normally, this would blow up because you haven’t defined a “person” method. With the modified metaClass, invokeMethod instead adds “person” as a key in the config map and then runs the closure associated with person. Any subsequent method calls or properties in that closure are assigned as the value in the map to the “person” key.

Intercepting Property Assignments

Property assignment (like firstName = “ted”) works a little differently. Every script has something associated with it called a binding. The binding holds script-wide variables that aren’t explicitly declared. When parsed by ConfigSlurper, calls that would normally set variables in the script’s binding are monitored and inserted into the map at the appropriate place in the stack.

So with the modified metaClass and binding the config above turns into this when slurped:

[person:[firstName:ted]]

If we had explicitly declared the name property, it wouldn’t get in the binding, and wouldn’t be part of the resulting config map.

person {
    def firstName = "ted"
}

// result: [person:[:]]

Modularizing Config Scripts

I got to thinking about how nice gsp files are to manage with the ability to include other gsp fragments. It would be great if we could do the same kind of thing with config files. That would let us better manage large config files. It would also let us define external config files that we pull in at run time, or config templates that we want to override just a piece or two out of. All from inside the config file itself.

Something like:

// SecurityConfig.groovy
security {
    includeScript( SecurityDefaults )
    active = true
}

// SecurityDefaults.groovy
username = "mysql"
password = "sekr1t"     
url {
    port = 2112    
}

Loading SecurityConfig would have everything in SecurityDefaults and also set active = true:

def configObject = new ConfigSlurper().parse( SecurityConfig )
    
println configObject
// [security:[username:mysql, password:sekr1t, url:[port:2112], active:true]]
    
assert configObject.security.active == true
assert configObject.security.url.port == 2112

A Solution

Getting the same functionality in config files ended up being a bit more difficult a problem to solve than I first expected. A big part of that was due to some misconceptions I had about how the ConfigSlurper was working as a DSL, as well as what exactly a groovy Script was and how it differed from a regular Groovy class.

It turns out that when a groovy script is compiled, it’s contents are placed inside a concrete implementation of the abstract Script.run() method. So this:

// MyGroovyScript.groovy
person {
    firstName = "Ted"
}

Is functionally equivalent to this:

class MyGroovyScript extends Script {
    def run() {
    person {
            firstName = "Ted"
        }
    }       
}

So one way to solve the problem is to create an abstract subclass of script and implement the “includeScript” method. All “includeScript” needs to do is override the metaClass and binding of the child script so that it uses the same metaClass and binding as the parent script.

public abstract class ComposedConfigScript extends Script {
    def includeScript(scriptClass) {
        def scriptInstance = scriptClass.newInstance()
        scriptInstance.metaClass = this.metaClass
        scriptInstance.binding = new ConfigBinding(this.getBinding().callable)
        scriptInstance.&run.call()
    }
}

ConfigSlurper modifies the metaClass and binding before loading the parent class to keep track of things as the script executes, it doesn’t care who’s actually calling the methods.

This lets us do exactly what we want, the only limitation to this approach is that the parent script needs to explicitly state that it’s a class that extends ComposedConfigScript and implements the run() method:

// SecurityConfig.groovy
class SecurityConfig extends ComposedConfigScript {         
    def run() { // normal contents of a config file go in here
        
        security {
            includeScript( SecurityDefaults )
            active = true
        }

    }
}

// SecurityDefaults.groovy
username = "mysql"
password = "sekr1t"     
url {
    host = "example.com"
    port = 2112    
}

// OtherClass.groovy
def configObject = new ConfigSlurper().parse( SecurityConfig )
    
println configObject
// prints: [security:[username:mysql, password:sekr1t, url:[host:example.com, port:2112], active:true]]

Using this technique lets us modularize our config files. This helps us both better organize things as well as decouple pieces of information that don’t necessarily belong together (like log4j info and website url information).

I had tried a number of other techniques that would let me use includeScript without declaring an explict class (like using a DelegatingMetaClass for the Script class and static import) but wasn’t able to get them working to my satisfaction. Those solutions are great for most cases, but break down with all the meta-magic that’s already happening with the ConfigSlurper.

Another option that could be considered, would be to patch the ConfigSlurper implementation to directly allow includeScript functionality. I’d be happy to submit a patch that does this if people are interested in this functionality.

Comments