I write my own custom iterators all the time and name them something descriptive. This makes the code much more readable. Rather than trying to decipher what a for loop is trying to do, we wrap up all of that iteration logic into a meaningful name and we cleanly separate that iteration from the processing that we’re doing with each element.

This kind of design is a core concept in Uncle Bob’s Clean Code, one of my favorite programming books in the last few years.

This example iterates over a collection and calls the passed in closure until we hit a value greater than 5.

def eachUntilGreaterThanFive = { collection, closure ->
    for ( value in collection ) {
        if ( value  > 5 ) break
        closure(value)
    }
}
 
def a = [1, 2, 3, 4, 5, 6, 7]
 
eachUntilGreaterThanFive(a) {
    println it
}

prints:

1
2
3
4
5

This code makes it obvious what the iterator is doing (looping till we hit a condition) as well as what will happen with each element iterated over (print it out).

For a real life example, I had a need to iterate over a list of values and where I needed both the current object as well as a peek at the next object in the list.

Doing this is Java is a bit of a pain, but groovy makes it easy to write and (hopefully) to read, we can also add it directly onto the Collection metaClass so that it’s available for all of our Collection instances:

Collection.metaClass.eachWithPeek = { closure ->
    def last = null
    delegate?.each { current ->
        if (last) closure(last, current)
        last = current
    }
    if (last) closure(last, null)
}

These test cases show that as we iterate through the collection, we can see the current item and peek at the next one (if any). If the collection is empty, we don’t execute the closure it, and if we’re at the end of the list there isn’t anything to peek at:

[].eachWithPeek { current, peek ->
    assert false // shouldn't get here, nothing to iterate through
}
 
[1].eachWithPeek { current, peek ->
    assert current == 1
    assert peek == null  // only 1 element, nothing to peek at
}
 
def results = []
[1, 2, 3, 4, 5].eachWithPeek { current, peek ->
    results << [current, peek]
}
assert results == [[1, 2], [2, 3], [3, 4], [4, 5], [5, null]]

By default, grails uses ~/.grails as the working directory. If you’re doing branchy development, you can run into problems with this if you’ve got plugins installed in one branch that aren’t in the other. Having a unique directory per branch prevents you from having to run grails clean all the time.

Here’s a quick shell script that changes the grails working directory to have the branch name as a suffix if your source is contained in a mercurial repository (ex: the default branch would be ~/.grails_default and the 1.0 branch would be ~/.grails_1.0). If your application is not in a repo, it just uses the regular ~/.grails directory.

#!/bin/sh
HG_BRANCH=`hg branch 2>/dev/null`
GRAILS_SCRIPT=$GRAILS_HOME/bin/grails
 
if [ $HG_BRANCH ]; then
	GRAILS_WORK_DIR=`echo ~`/.grails_$HG_BRANCH
	echo "** grails working directory: $GRAILS_WORK_DIR"
	$GRAILS_SCRIPT -Dgrails.work.dir=$GRAILS_WORK_DIR $@
else
	echo "** default grails working directory"
	$GRAILS_SCRIPT $@
fi

Just save this script as “grails” and put it in your PATH before the $GRAILS_HOME/bin directory (also make sure that you’ve defined $GRAILS_HOME). I have a ~/bin directory that’s the first thing in my PATH.

If you use the grails-debug command, you can repeat these steps for that, just change GRAILS_SCRIPT to $GRAILS_HOME/bin/grails-debug.

This same technique could easily be modified to be used for other source control systems such as git or subversion.

I’ve made some enhancements to them in the interim that have made them even easier to use.

The most important alias is gtaf, which is short for “grails test-app” for failed tests.

It will search through your test output directory and look for any tests that failed. If it finds any, it will rerun only those tests. Otherwise, it will rerun all tests. That makes it easy to just use gtaf all the time. If any tests fail, it will open them up using Console.app.

If you’re not on OSX, or would like to use something else to view the failed test logs, just modify the testlog alias to do something different.

There are alternative versions of it to only run failed integration tests (gtaif – Grails Test App Integration Failed), failed unit tests (gtauf – Grails Test App Unit Failed) or attach a debugger to the tests as they run (gtadf – Grails Test App Debug Failed).

These aliases, plus most of the rest of my zsh setup, is available on bitbucket. If you haven’t used zsh before and are still using bash, I suggest switching over for the numerous benefits that it gives you.

Here’s the section of zshrc_general that has the grails testing aliases, just add this to your .zshrc/.bashrc and make sure to uncomment the appropriate GRAILS_TEST_LOG_DIRECTORY export if you’re not running grails 1.2 yet:

# grails > 1.2
export GRAILS_TEST_LOG_DIRECTORY=target/test-reports
 
# grails < 1.2
# export GRAILS_TEST_LOG_DIRECTORY=test/reports
 
# after grails-test if there were ERROR messages, you can open those logs with the Console.app using this
alias testlog='for F in `grep -lE "FAILED|Caused\ an\ ERROR" $GRAILS_TEST_LOG_DIRECTORY/plain/*.txt`; do echo ">>> opening" $F; open -a Console $F; done;'
 
# grails-debug-suspend doesn't exist by default, it's a copy of grails-debug with the suspend flag changed to "y" so that
# we can attach a remote debugger before it proceeds past startup
 
# aliases where you can optionally pass in a set of tests to run (or no argument to run all tests in that group)
alias gta=grailsTestApp
alias gtad=grailsTestAppDebug
alias gtau=grailsTestAppUnit
alias gtaud=grailsTestAppUnitDebug
alias gtai=grailsTestAppIntegration
alias gtaid=grailsTestAppIntegrationDebug
 
# aliases that will rerun any failed tests (or all tests if there aren't any failed tests)
alias gtaf=grailsTestAppFailed
alias gtadf=grailsTestAppDebugFailed
alias gtauf=grailsTestAppUnitFailed
alias gtaudf=grailsTestAppUnitDebugFailed
alias gtaif=grailsTestAppIntegrationFailed
alias gtaidf=grailsTestAppIntegrationDebugFailed
 
 
function grailsTestApp() { grailsTest grails "" $1 }
function grailsTestAppFailed() { grailsFailedTests grails "" }
 
function grailsTestAppDebug() { grailsTest grails-debug-suspend "" $1 }
function grailsTestAppDebugFailed() { grailsFailedTests grails-debug-suspend "" }
 
function grailsTestAppUnit() { grailsTest grails -unit $1 }
function grailsTestAppUnitFailed() { grailsFailedTests grails -unit }
 
function grailsTestAppUnitDebug() { grailsTest grails-debug-suspend -unit $1 }
function grailsTestAppUnitDebugFailed() { grailsFailedTests grails-debug-suspend -unit }
 
function grailsTestAppIntegration() { grailsTest grails -integration $1 }
function grailsTestAppIntegrationFailed() { grailsFailedTests grails -integration }
 
function grailsTestAppIntegrationDebug() { grailsTest grails-debug-suspend -integration $1 }
function grailsTestAppIntegrationDebugFailed() { grailsFailedTests grails-debug-suspend -integration }
 
function grailsFailedTests() {
	FAILED_TESTS=''
	for F in `grep -lE "FAILED|Caused\ an\ ERROR" $GRAILS_TEST_LOG_DIRECTORY/plain/*.txt`; do
		FAILED_TESTS="$FAILED_TESTS `echo $F | sed -E 's/.*TEST-(.*)Tests.txt/\1/g'`"
	done;
	echo "failed tests: $FAILED_TESTS"
	grailsTest $1 $2 $FAILED_TESTS
}
 
function grailsTest() {
	echo "Running: $1 test-app $2 $3 || testlog "
    time $1 test-app $2 $3 || testlog
}

I believe that they’ll also work fine as-is in bash, though I haven’t tested them.

Combining the unix “watch” command with the mysql “show processlist” command gives me what I’m looking for. A quick, self-updating status of the current state of the database.

watch -n 5 --differences "mysql -u username -psekrit -e 'show processlist'"

Shows something like this:

Every 5.0s: mysql -n 5 -u username -psekrit -e 'show processlist'                                   Thu Jun 18 05:25:14 2009
 
Id      User    Host       db      Command Time    State         Info
3141    admin   localhost  mydb    Query   34978   freeing items SELECT id, type, active, email FROM user WHERE email
3146    admin   localhost  mydb    Sleep   0                     NULL       
24876   root    localhost  NULL    Query   0       NULL          show processlist

Where the “time” column is the number of seconds the query has been running, and Info holds the actual query (you can use “show full processlist” to see the full query).

Watch is a nice little linux util that runs a command every “n” seconds (it defaults to 2 seconds). If you use the –differences switch, it will highlight the differences between one update and another. I use watch for all kinds of monitoring activities, such as watching a directory to see a file grow in size as it gets transfered.

Here’s a quick shell function that you can add to your .bashrc/.bash_profile/.zshrc to use on arbitrary hosts:

function mysqltop() {
    MYSQL_OPTS=$@
    watch -n 5 --differences "mysql $MYSQL_OPTS -e 'show processlist'"
}

Then just pass in any creds/host info you need like a normal mysql command:

mysqltop -u ted -psekrit1 -h example.com -P 3307

If you’re on linux, you probably already have “watch” installed. If you’re on OSX, you probably don’t, but you can get it quickly through macports. Install macports, make sure “port” is in your path and run:

sudo port install watch

There is also a command called mytop that you can get which looks like the same thing, but prints out the processlist details with some nicer formatting and a little extra information. It’s in macports, but it has a number of dependencies including mysql so if you didn’t install mysql through macports, you might want to stick with what I have above or get it another way.

(EDIT: updated with bash function)

I found it painful to manually eject each individual drive in the finder so I threw together this quick AppleScript to eject all the disks.

tell application "Finder"
	eject (every disk)
end tell

Just open up /Applications/AppleScript/ScriptEditor.app and paste that in. Then choose “Save As” and pick “Application”. That will compile the script and create a .app file that you can click on to run, or you can put it in your path and execute it there.

I think the same kind of script could be created with the command line “diskutil eject” command, but this seemed cleaner as I wasn’t able to come up with a generic way to figure out which disks were “ejectable” and which weren’t. AppleScript is able to figure that all out for you.

I’ve created a new bitbucket repository to hold my shared zshrc configuration. You can get it for yourself by cloning the repository:

cd ~
hg clone http://bitbucket.org/tednaleid/shared-zshrc/

If you don’t have mercurial installed, you can just grab the compressed archive of your choice from the downloads page.

I’d suggest cloning/unzipping into your home directory, but you can put it wherever you want.

There’s an install.sh script in there that will create a new ~/.zshrc file. If you have an existing .zshrc file, just move it out of the way and run the install script (it won’t let you whack an existing file).

./install.sh

That created .zshrc file will load up all of my shared settings and aliases from wherever you have the shared-zshrc directory. It also will look for a host specific file where any sensitive info, or host dependent info can be added. I normally put things like my change directory and ssh aliases in here.

I probably created around half of these files myself, but I couldn’t tell you for the life of me where the majority of the other half came from. This file has accreted over the years like barnacles on a ship.

Some of these commands originally started out in either a tcsh file (or was it ksh?), I eventually moved to bash, and then a couple of years ago, a coworker showed me the wonders of zsh and I haven’t looked back.

If you’re still using bash, give zsh a try. Changing your shell is as easy as running this command:

chsh -s /bin/zsh

ZSH Specific Stuff

Most of the zsh shell configuration magic is in zshrc_compinstall. Some features include:

• shared history across shells
• command spelling correction
• vastly improved tab completion of commands (including keyboard navigation of potential matches)
• file globbing that makes finding files easy (try ls **/*.txt to recursively search for all txt files, much easier than find . -name "*.txt"
• automatically using pushd for cd commands, use “dh” to see a history of your directories, use cd -N to go back N directories in your history (ex: cd -3)

Useful Aliases

Some of the most useful aliases in zshrc_general include:

Grep through Aliases (ag)

I often forget what some of my less often used aliases are, but I know what commands they use. I’ve set up “ag” (alias grep) to help me search through my aliases and find the right one:

% ag python
webshare='python -m SimpleHTTPServer'

Grepping History (gh)

Great for finding commands in your history that you can only remember part of.

  gh grails
  ...
  582  grails create-app testDates
  584  grails create-domain-class Foo
  585  grails generate-all Foo
  587  cd grails-app
  591  grails generate-views Foo
  592  grails run-app

Global Alias – GV – to “grep -v”

Grep is great for finding things, but often you find too much and need to slice stuff out. If you use “grep -v” that finds all lines that don’t match a pattern. The global alias “GV” that makes this even easier.

   grep -ri foo

If that finds too much stuff (say catalina log files and svn files that you don’t care about), just slice them out with GV:

   grep -ri foo GV svn GV catalina

That will remove any lines that have “svn” or “catalina” anywhere in them (including the file name).

I think that rc files tend to be very personal things, what works for me likely won’t work for you exactly as it is. But I also love to look at other people’s rc files to grab the things that look interesting. Feel free to poke through mine and grab whatever strikes your fancy.

There’s a ton of other stuff in there that I use every day, including sections specific to version control (svn and hg) as well as groovy/grails shortcuts. Feel free to poke around and ask me any questions about what something does.

One of the most common use cases is to search a string for a regular expression pattern. If a match is found, then do something with the matched value.

Currently in groovy, the recommended way to do this is to create a matcher and then use indexes to work with any matches that might be found:

assert "10292" == ("New York, NY 10292" =~ /\d{5}/)[0]

If you try to do that on a string that doesn’t actually match the regular expression, you’ll get an IndexOutOfBoundException. To be safe, you need to check matcher.find() (not “matches” as that requires the entire string to match!) to see if the string is actually in there:

def m = ("New York, NY" =~ /\d{5}/)[0]
def zip
if (m.find()) {
   zip = m[0]
}

It also has inconsistent behavior if the regular expression happens to have capture groups in it. Then it returns an array containing the match and the capture groups, forcing you to index into that array to actually get the match you want:

assert "c" == ("foo car baz" =~ /(.)ar/)[0][1]

Groovy has already added closure aware replace method to the String class. The patch adds a complimentary find method to string that will return the string matched by the closure without needing to worry about matcher objects and array indexes.

You can either call it without a closure to get the full found match back (even if it has groups in it):

assert "10292" == "New York, NY 10292".find(/\d{5}/)

It safely returns a null if the match isn’t found, which can clean up boilerplate safety checks quite a bit. The user can check for null using groovy truth if they want to:

def zip = "New York, NY".find(/\d{5}/)   // returns null
 
if (zip) { ... }

If you want to work with capture groups, or manipulate the value, you can pass a closure to the find method that will be passed the full match as well as any capture groups (just as the collection based regular expression methods work):

// no capture groups, only the match is passed to the closure
assert "bar" == "foo bar baz".find(/.ar/) { match -> return match }
 
// one capture group
assert "b" == "foo bar baz".find(/(.)ar/) { match, firstLetter -> return firstLetter }
 
// many capture groups, all passed to the closure after the full match
assert "2339999" == "adsf 233-9999 adsf".find(/(\d{3})?-?(\d{3})-(\d{4})/) { match, areaCode, exchange, stationNumber ->
    assert "233-9999" == match
    assert null == areaCode
    assert "233" == exchange
    assert "9999" == stationNumber
    return "$exchange$stationNumber"
}

If you think this would be a valuable addition to groovy, you can vote for it in JIRA.

function switchGrails() {
    echo "Switching to groovy version: $1"
    echo "Switching to grails version: $2"
    sudo rm /usr/local/{groovy,grails}
 
    sudo ln -s /usr/local/$1 /usr/local/groovy
    sudo ln -s /usr/local/$2 /usr/local/grails
    echo "Done!"
    ls -latr /usr/local/{groovy,grails}
}
 
alias g104='switchGrails "groovy-1.5.7" "grails-1.0.4"'
alias g11rc2='switchGrails "groovy-1.6.0" "grails-1.1-RC2"'

You can create your own aliases like the ones above to switch to the groovy/grails combinations that you happen to be working with.

Just stick the code above in your .profile/.bashrc/.zshrc file and restart your shell to make the aliases available.

This function assumes that you’ve got you’ve got groovy and grails installed in your /usr/local directory and that you use a symlink at /usr/local/groovy that $GROOVY_HOME is pointed to and /usr/local/grails that $GRAILS_HOME is pointed to. If those assumptions aren’t correct for you, you’ll have to tweak the script.

It also uses “sudo” as it assumes that /usr/local is owned by root and not by your logged in user. If you’ve chowned the directory to yourself, you can remove the sudo (and the need to enter your password).

Now it’s easy for me to switch between different projects by just typing the appropriate alias:

pollux% g104
Switching to groovy version: groovy-1.5.7
Switching to grails version: grails-1.0.4
Done!
lrwxr-xr-x  1 root  wheel  23 Mar  8 19:55 /usr/local/groovy -> /usr/local/groovy-1.5.7
lrwxr-xr-x  1 root  wheel  23 Mar  8 19:55 /usr/local/grails -> /usr/local/grails-1.0.4
 
 
pollux% g11rc2 
Switching to groovy version: groovy-1.6.0
Switching to grails version: grails-1.1-RC2
Done!
lrwxr-xr-x  1 root  wheel  23 Mar  8 19:56 /usr/local/groovy -> /usr/local/groovy-1.6.0
lrwxr-xr-x  1 root  wheel  25 Mar  8 19:56 /usr/local/grails -> /usr/local/grails-1.1-RC2

There are over 30 code samples in then article and I wanted to make sure while writing and editing that all of the code samples ran exactly as they appeared in the article text. Also, when you download an issue of GroovyMag, you get a zip file that has a PDF and a set of code “listing” files for each article. Each listing file contains a snippet of groovy code that appeared in the issue.

I decided to write a couple of simple groovy scripts to keep things DRY, and to ensure that my edits didn’t break anything. The first script extracts code listings out of my draft article and saved them to individually numbered listing files. The second script executes each of the listing files and reported success or failure for each. Sort of a poor man’s JUnit for writing articles.

Both of these scripts leverage regular expressions and the techniques that are outlined in GroovyMag.

I wrote the article using the markdown syntax. If you haven’t seen markdown before, it’s a wiki-like syntax that is very easy to read and write, and can also be converted into a variety of formats easily, including HTML.

In markdown, code blocks are simply lines that are separated from other lines by at least one blank line (or the start of the file) and are indented at least 4 spaces.

This is the script that I used to parse through the markdown file. It uses a regular expression to find all of the code blocks in the markdown and then writes each of them to it’s own listing file. Listing files were to be named consecutively from listing_1.txt through listing_n.txt.

#! /usr/bin/env groovy
INPUT_START_OR_BLANK_LINE = /(?:\A|\n\n)/
FOUR_SPACES_OR_TAB = /(?:[ ]{4}|\t)/
CODE = /.*\n+/
CODE_LINES = /(?:$FOUR_SPACES_OR_TAB$CODE)/
LOOKAHEAD_FOR_NON_CODE_LINE = /(?:(?=^[ ]{0,4}\S)|\Z)/
 
// this regular expression will find all of the consecutive code lines in a markdown file
// in a markdown file, if the line starts with a tab or at least 4 spaces, it's a code line
// slightly modified from one in markdownj
// see: http://github.com/myabc/markdownj/tree/master/src/java/com/petebevin/markdown/MarkdownProcessor.java
MARKDOWN_CODE_BLOCK = "(?m)" + 
                      "$INPUT_START_OR_BLANK_LINE" +
                      "($CODE_LINES+)" +
                      "$LOOKAHEAD_FOR_NON_CODE_LINE"
 
def removeOldListings(dir) {
    dir.eachFileMatch(~/.*listing_\d+\.txt/) { file ->
        println "Removing $file"
        file.delete()
    }   
}
 
 
def createListings(file) {
    listingNumber = 1
    (file.text =~ MARKDOWN_CODE_BLOCK).each { full, codeBlock ->
        def listing = new File("listing_${(listingNumber++).toString().padLeft(3,'0')}.txt") 
        println "Creating $listing"
 
        // groovy's String.eachLine skips blank lines, but we want these in our source 
        // to make things more readable so we'll make our own eachLine
        (codeBlock =~ /.*/).each { line ->          
            // each markdown code block comes back with a tab or 4 spaces at the beginning, strip those off
            def matcher = (line =~ /$FOUR_SPACES_OR_TAB(.*)/)
            if (matcher.matches()) {
                matcher.find { fullLine, code ->  listing << "$code" }
            } else {
                listing << "$line\n"
            }
        }
    }
}
 
removeOldListings(new File("."))
createListings(new File("article.markdown"))

Once I had all of those listing file, I used this script to execute each of the listing files and report whether any problems had occurred during the execution. It uses the “eachFileMatch” method groovy adds to the File object, which you can give a regular expression pattern so that you can iterate over a targeted subset of files to process.

#! /usr/bin/env groovy
def executeListings(listingDir) {
    listingDir.eachFileMatch(~/.*listing_\d+\.txt/) { listing ->
        print "Executing $listing..."       
        try {
            new GroovyShell().evaluate(listing)
            println "Success!"
        } catch (java.lang.AssertionError e) {
            println "ERROR!"
            e.printStackTrace()
        }
    }   
}
 
executeListings(new File("."))

Just like a good set of unit tests, these scripts gave me the courage to make edits to my article, without needing to worry if I was breaking something or forgetting to make a change. It’s a different kind of meta-programming when I can use regular expressions to help me write about using regular expressions :)

The one problem with this script is that if the files already exist, a prompt will come up after ~10 seconds or so (after the grails environment bootstraps) asking you if you want to Overwrite everything:

grails generate-all "*" 
...
~10 seconds pass
...
Generating views for domain class Baz ...
File /foobar/grails-app/views/baz/list.gsp already exists. Overwrite?y,n,a

This was a bit of a pain as I’d often kick the script off, get distracted and then come back to the shell with that prompt still waiting for me to tell it what to do. I’d rather just start working with my shiny new scaffolding.

The easy solution to this is simply to pipe the answer you want into the grails command:

echo "a" | grails generate-all "*"

Doing that will pipe the “a” into the grails command so that when the prompt finally comes up, it knows that it can continue regenerating all of my scaffolding.

It seems simple enough after I figured it out, but I thought it could save some other people time when they’re hacking around with templates and know they want to regenerate all of them.