def fibList = [1, 1, 2, 3, 5]
 
fibList.each { println it }  // prints all of the numbers in the list
assert fibList.any { it == 3 }
assert fibList.every { it > 0 }
assert fibList.collect { it - 1 } == [0, 0, 1, 2, 4]
assert fibList.findAll { it > 1 && it < 5 } == [2, 3]
assert fibList.find { it > 1 } == 2
assert fibList.inject("fib: ") { str, val -> str << val }.toString() == "fib: 11235"

That’s really nice if you’re working with raw lists and maps, but what if you have a class that doesn’t extend list or map? How hard is it to empower that class with the groovy iteration methods? If this were Java, you’d likely need to implement an interface with these methods (and throw a “not implemented” exception for those you didn’t feel like taking the time to implement).

Since it’s not Java, but groovy (and you’ve read the title of this blog post :), you know it’s easy!

All you have to do is put a public method on your class called iterator() that returns an Iterator.

If you have a member variable that has an iterator you can expose, you can just use that:

class EvenFoo {
   private numbers = [2, 4, 6, 8, 10]
   Iterator iterator() {
       return numbers.iterator()
   }
}
 
def evens = new EvenFoo()
assert evens.each { println it } // prints even numbers 2 through 10
assert evens.findAll { it > 5 && it < 9 } == [6, 8]
assert evens.any { it == 4 }
assert evens.every { it > -1 }
assert evens.collect { it } == [2, 4, 6, 8, 10]
assert evens.inject("evens: ") { str, val -> str << val }.toString() == "evens: 246810"

You can also use the power of groovy to implement a subset of the Iterator interface using a map (just define hasNext and next). Return that, and it’s just as good as a regular Iterator from groovy’s perspective (duck typing rocks!):

class EvenFoo {
   def i = 0
   def max = 10
   Iterator iterator() {
       i = 0
       return [ hasNext: { i < max }, next: { i += 2 } ] as Iterator
   }
}
 
def evens = new EvenFoo()
assert evens.each { println it } // prints even numbers 2 through 10
assert evens.findAll { it > 5 && it < 9 } == [6, 8]
assert evens.any { it == 4 }
assert evens.every { it > -1 }
assert (evens.collect { it }) == [2, 4, 6, 8, 10]
assert evens.inject("evens: ") { str, val -> str << val }.toString() == "evens: 246810"

This is a contrived example, but I’ve run into a number of situations where I want to iterate over something that isn’t a list or map. It can be especially useful in lazy loading situations (such as a file/directory scanner or a SAX parser).

My home directory has a bin folder in it that’s continually getting new jigs added to it, and my zshrc file is an ever-expanding list of quick shell functions.

Recently, I’ve been doing a lot more work with Mercurial as the team that I’m on switched from Subversion a couple of months ago on our Grails project. The initial transition was a little difficult for some people, but I think just about everyone is pretty happy with the transition now that we’ve made it.

Something that has helped everyone get comfortable with more complex Distributed Version Control System like Mercurial has been the distribution of shims and jigs amongst the team. I thought these tips might be useful to others as well.

Configuring Mercurial

After installing mercurial (the easiest way is to just get the appropriate binary package for your OS), you’ll want to get it configured.

A good place to start is with someone else’s configuration file. Here’s my my current ~/.hgrc configuration file (with the username replaced):

[extensions] 
fetch=
extdiff=
color=
hgext.graphlog=
hgshelve=~/Documents/workspace/hgshelve/hgshelve.py
rdiff=~/Documents/workspace/rdiff/rdiff.py
hgext.mq =
 
[diff]
git = 1
 
[ui]
username = YOUR NAME <YOUR_NAME@YOUR_COMPANY.com>
 
[extdiff]
cmd.kdiff3 =
cmd.chdiff =
opts.chdiff = --wait
 
[merge-tools]
kdiff3.args = $base $local $other -o $output

In it, you’ll see a number of useful extensions, including:

• hg fetch - automates the pull/update working copy/merge/commit merge steps
• extdiff extension - Mercurial allows you to set up external tools to do diffs and merges. I’m using 2 different graphical merge tools for different purposes
• hg chdiff - Changes.app for just comparing two files in a diff. Changes is a beautiful shareware diff tool for the mac, it unfortunately doesn’t do a 3-way merge.
• hg kdiff3 - For merging, (and some diff situations), I need the power of a 3-way merge tool and I’ve found kdiff3 to be the best one that I’ve used. It’s an open source, free, cross platform 3-way merge tool. Using a 3-way merge tool is imperative for getting your merges correct.
• color extension - just colorizes some output including the status command
• hg glog - an ascii graphical log of the commit tree (enhanced “hg log”)
• hg shelve - simplified patch management that allows you to “shelve” changes without committing them locally, for instance if you want to do a fetch, I asked about this on Stack Overflow
• hg rdiff - remote diff, allows you to compare a file checked into your local repo with a file in the remote repository, I also asked about this on Stack Overflow
• mq extension - mercurial queues, a more advanced topic that I don’t think anyone else at my workplace uses, but there’s a lot of power in mqueue if you look into it. The Mozilla guys have a really nice introductory tutorial to using this plugin.

Setting up KDiff3 so you can do 3-way merges

The simplest way to get KDiff3 is to download the binary from the main website (click on the download link and then choose the binary appropriate for your platform).

For the mac, you’ll just get a zip file with a KDiff3.app file in it that you should drag to your /Applications directory. Then, you need to make sure that the actual executable in the application is in your path. I just create a symlink in my path to the executable inside the application package:

ln -s /Applications/kdiff3.app/Contents/MacOS/kdiff3 /usr/local/bin/kdiff3

You might not have /usr/local/bin in your path by default. If you can execute kdiff3 --help on the command line, you’re ready to go.

Setting up the “shelve” extension

The easiest way to get the shelve extention set up is to simply clone the project that holds the python script to your local filesystem:

hg clone http://freehg.org/u/tksoh/hgshelve/ hgshelve

Then just make sure that the path in your ~/.hgrc file points to the correct location of the hgshelve.py file in the cloned repository.

You can test that it’s working by just executing:

hg help shelve

If you don’t get an error message, it’s set up right and you can start using it.

Setting up the “rdiff” extension

Setting up the rdiff extension is similar to the shelve extension above. Just clone the repository:

hg clone -r c3ac1c497bf0 http://hg.kublai.com/mercurial/extensions/rdiff

We clone an older revision currently as the tip in that project uses some unsupported stuff that isn’t in mercurial 1.0.2, if you’ve got a newer version of mercurial, you might be able to get the tip of the rdiff extension and have it work.

Then make sure that your ~/.hgrc is pointed to the right path for the rdiff.py file in the repo you just cloned.

It’s a little quirky in that it actually modifies the existing “hg diff” command by detecting if the first parameter is a remote URL. If it is then it will diff that file against your tip file in your local repo (not the working copy). This as the remote repo is first in the arguments, it’s the reverse of what I’d expect, but you can pass “–reverse” to the hg diff command to switch that around.

I could see these being potential enhancements to the extension, but for now, I can work around them with a bash/zsh shell function in my startup file. It does a temp checkin of my working copy (held by the mercurial transaction so it can be rolled back), executes the reverse diff, and then rolls the transaction back to return things back to the way they were:

hgrdiff() {
    hg commit -m "temp commit for remote diff" && 
    hg diff --reverse http://my_hardcoded_repo $* &&
    hg rollback      # revert the temporary commit
}

And then call it with:

hgrdiff <filename to diff against remote repo tip>

Other Mercurial Jigs and Shims

Here are some other jigs and shims that I’ve found useful.

Jig to Get a List of Files With the Same Status

One jig that I’ve gotten a ton of use out of, is a modification to the normal “hg status” command. Normally, “hg status” returns a list of all files that are different in the working directory when compared to the tip if the repository. In the output of status, each file is printed with a character in the first column that signifies the status of that file. Ex:

% hg status
M build.xml
A src/groovy/Foobar.groovy
A src/groovy/Quux.groovy
? src/groovy/Bar.groovy
? src/groovy/Baz.groovy

Some of the most common statuses are:

• ? - not tracked, unknown to the source control repository
• M - modified, this file is different in the working copy than on the file system
• A - added, source control is aware of this file, but it hasn’t yet been committed
• R - removed, this file has been marked for deletion on the next commit
• ! - missing, this file is not present in the working copy but is in source control

Often, I’ll want to do something with all of the files in a particular status, but I can’t easily do that as they’re mixed in with the other file types. Also, the status flag at the beginning of the line prevents me from piping the status into another command.

I’ve come up with this shell function to allow me to filter lists of files (it actually works for subversion too, so there are 2 aliases). Just paste this into your bash/zsh startup file:

vcst() {
	# print out all of the files with a passed in status flag (M - modified, A - added, ? - unknown, etc) (default ?)
	# expects first parameter to be the version control command (likely svn or hg)
	STATUS='\?'
	if [ -n "$2" ]
	then
		STATUS=$2
	fi
	$1 status | egrep "^$STATUS" | awk '{print $2}'
}

alias svnst='vcst svn'
alias hgst='vcst hg'

Then you can use the “hgst” command to see all the files in a particular status, one per line. By default, it shows files that aren’t tracked (”?“), but you can pass a parameter that specifies another flag type if it’s other than “?“.

To see the list of files that aren’t in source control, just use:

% hgst
src/groovy/Bar.groovy
src/groovy/Baz.groovy

If you want to delete all of the files that aren’t in source control, all you have to do is send the results of that command to rm:

rm `hgst`
# removes src/groovy/Bar.groovy and src/groovy/Baz.groovy

If you want to edit all of the files that have been added in TextMate:

mate `hgst A`
# opens src/groovy/Foobar.groovy and src/groovy/Quux.groovy in TextMate

If you want to do a diff and see only the files that have been modified using kdiff3:

hg kdiff3 `hgst M`
# would show a diff of build.xml vs the working copy

Shim to See a List Incoming and Outgoing Files

When I’m about to push changes out to a remote server, or pull changes down from that server, I often want to know what files will be affected by this action. Surprisingly, there isn’t an easy built-in way to do this.

There is the hg incoming -v command, which will print out a log format of all of the different changesets. But I don’t really care about all the changesets, I just want to know which files will be affected. Here’s a sample run of incoming with 3 changesets:

% hg incoming -v
comparing with /Users/ted.naleid/temp/foo
searching for changes
changeset:   1:5c5f71bda234
user:        Ted Naleid <ted.naleid@carol.com>
date:        Tue Nov 25 20:30:51 2008 -0600
files:       src/groovy/Foobar.groovy
description:
initial commit of foobar.groovy
 
 
changeset:   2:fe4a615747af
user:        Ted Naleid <ted.naleid@carol.com>
date:        Tue Nov 25 20:31:36 2008 -0600
files:       src/groovy/Bar.groovy src/groovy/Foobar.groovy
description:
committing 2 files
 
 
changeset:   3:21e3d26bc3eb
tag:         tip
user:        Ted Naleid <ted.naleid@carol.com>
date:        Tue Nov 25 20:31:54 2008 -0600
files:       build.xml src/groovy/Baz.groovy src/groovy/Foobar.groovy src/groovy/Quux.groovy
description:
3rd commit of files

That’s not very useful as it’s hard to parse out the actual files in that list (and see only the unique list).

I use the following shim (just put in your bash/zsh rc file) to make this simple:

# look for lists of files in piped output, sort the unique set of them and print them one per line
lf() {
    egrep "^files:" | awk '{for (i=2; i<=NF; i++) print $i}' | sort | uniq 
}
 
alias ic="hg incoming -v | lf"
alias og="hg outgoing -v | lf"

Once this is installed, you can simply use the og (outgoing) or ic (incoming) aliases to see what you’ll affect if you actually do a push or a pull.

Here’s the output using the incoming shim on same changeset above:

% ic
build.xml
src/groovy/Bar.groovy
src/groovy/Baz.groovy
src/groovy/Foobar.groovy
src/groovy/Quux.groovy

That makes it quick and easy to see the real things that you’ll affect by your actions.

Enhancing Your Shell Prompt With Mercurial Info

A couple of other commands that I have in my zshrc (they should also work in a bash startup file):

hgtarget() {
    hg_root=`hg root 2>&1 | egrep -v "$abort:"`
   	if [ $hg_root ]; then
    	if [ -f $hg_root/.hg/hgrc ]; then
        	hg_target=`cat $hg_root/.hg/hgrc | egrep "^default =" | sed 's/\(^default = \(http:\/\/\)*\)\(.*@\)*//'`
        	echo "$hg_target"
        fi
    fi
}

hgbranch() {
	hg_root=`hg root 2>&1 | egrep -v "$abort:"`
	if [ $hg_root ]; then
		hg_branch=`hg branch`
		echo "$hg_branch"
	fi
}

These commands will tell you the path to the default push/pull target for your repository (hgtarget) as well as the current branch that you’re working on (hgbranch) in your repo.

I’ve added these to my shell prompt so that when I’m in a mercurial repository I’m more informed of the context that I’m working in. My prompt is a little complicated (lots more going on there besides this), but there are a number of good tutorials out there for modifying yours.

Under the covers, Mac OSX uses a tool called “ipfw” (IP firewall) that allows for all kinds of fancy traffic shaping and piping. It’s a very unix tool though and it’s not the easiest thing to use if you’re not familiar with parsing through man pages.

There is a GUI called WaterRoof (http://www.hanynet.com/waterroof/) that makes things a little better, though it still isn’t the most intuitive thing.

Here are the steps to slow your connection down using WaterRoof.

0. This is easiest if you have your browsers all shut down. It makes it easier to identify the right connection, though it’s not required.

1. Load up WaterRoof, you’ll see the main control panel:

2. Click on the “Net Connections” page to see all your current active connections:

3. Start up firefox and go to the website you want to be slow (in this example, a QA box on a local intranet running the app on port 8080):

4. Click update in the WaterRoof “net connections” page so it will now show the connection to your new server (in this example “qa01app01:8080″). It lists the IP, but because you don’t have any other browsers open, it’s easy to identify the connections to port 8080 (or port 80 if you’re going over the normal http port). Alternatively you can use a tool to identify the IP of the machine you’re hitting (such as dig/telnet) and then find it in the list.

5. Highlight any one of the connections to port 8080 (as in the image above) and click “Block or limit selected connection” to get to the screen below. In that screen, enter a number in the “Bandwidth” box under “Limit connection BW”. I entered “3” to slow things down to 3 Kb/second (regular modem speed is normally between 3-7 Kb/second, depending on the wind). Then hit the “limit” button. You’ll need to enter your password.

6. This will create a “pipe” and 2 “rules” (one for incoming, one for outgoing to the selected server) to limit the bandwidth (the top 2 rules in the image below). The 3rd rule in the image below is the default one that is always there (“allow ip from any to any”).

You can now go back to your browser and hit “cmd-shift-R” to reload without cache and things should be much slower.

To get back to normal, you’ll want to delete the 2 rules and the pipe that were created (highlight each and hit the “-”).

This is an easy, repeatable way to slow down your connection on demand. There are quite a few other things that can be done with ipfw/waterroof. If you get a little deeper into it, it becomes pretty easy to put together little scripts that are doing all sorts of traffic shaping and forwarding on your mac.

I just asked a fairly detailed question about working with mercurial. Something that wasn’t (obviously) covered on the mercurial wiki, through googling, or in the mercurial handbook, and got a quality answer back in only 8 minutes. That’s fantastic for a general purpose development website and is a great start after only being open for a couple of weeks.

They’ve started up an interesting reward/karma system over there where they award badges (similar to achievements on Xbox360) for a bunch of different positive behaviors. A nice little token system that’s paired with “power-ups” at different point levels. It’s easy to get the first few badges with just a little participation on the website, and I can see how certain personality types that are common in engineers would get big rewards out of collecting these.

It’s an alpha Firefox plugin that’s attempting to be a command line for the internet. It reminds me a little of yubnub, but quite a bit more powerful as it’s available on every page and is context sensitive.

Essentially, it has a set of built in commands (that you can add to an extend) and it’s aware of the current browser context, so if you have something highlighted, it can act on that subset of the current page.

Previously, I’ve been a heavy user of Firefox smart keywords, which allow you to assign aliases to bookmarks and type the aliases in the location bar. I’ve created smart keywords that allow me to search wikipedia, amazon, imdb and the dictionary. Ubiquity has all of these, plus a lot more built-in.

I’ve only been using it for a little bit, and there are some rough edges, but I think that there is quite a bit of promise here as well and thought that there might be a few other keyboard jockeys out there that would appreciate what Ubiquity is trying to do.

The first issue was that I’m running ssh on a non-standard port and the repository does not reside in my home directory.

The Mercurial Documentation on using SSH does document how to use mercurial with unusual ssh locations, but it’s not obvious in the “hg -v help clone” documentation. You will want to use a ssh:// format URI and also need to make sure to use a “//” if you want to use an absolute path to your repository.

This example shows how to ssh to port 9292 on example.com as user “ted” and get to a repository that’s located at /export/repos on that server.

hg -v clone ssh://ted@example.com:9292//export/repos

If you don’t use a “//” it assumes that the location of the repository is relative to the user’s home directory.

The “-v” (verbose) parameter is also useful for debugging what’s going on as it will show you the SSH command that it’s running under the covers.

The second issue was that after entering the above command and typing in my password, I’d see this error:

remote: zsh: command not found: hg
abort: no suitable response from remote hg!

This seemed strange as I knew I had mercurial installed on my remote computer (that’s how the repository got there in the first place). I’ve got mercurial installed under /usr/local/bin which is a directory that’s added to my PATH environment variable in my .zshrc file (I’m using the ZSH shellinstead of the default bash shell that most people use on OSX).

It turns out that under the covers, mercurial is executing a command like this:

ssh ted@example.com -p 9292 "hg -R /export/repos serve --stdio"

When ssh is directly executing a command on a server instead of logging in, that’s called “non-interactive” mode and it doesn’t source your .zshrc file (I’m assuming that a similar thing happens in bash). To fix this for ZSH, you can create a .zshenv file that WILL get executed by both interactive and non-interactive shells commands and put your PATH in there.

I’m assuming that a similar thing happens in a bash shell (and other shells as well), so if you’re seeing that the “hg” command can’t be found, make sure that you can execute a command like this:

ssh foo@bar.com "hg help"

If it says “command not found: hg”, then you don’t have the correct path for a non-interactive shell.

The presentation starts with the pros and cons of distributed systems and gives a brief overview of the top 3 most popular DVCS systems:

• Git
• Bazaar
• Mercurial

I go into the details of why I chose to use Mercurial and describe some common usage patterns for people used to using Subversion.

I then did a live-coding session where I created a quick grails application, added it to a new repository, cloned the repository and pushed changes back and forth to show how mercurial handles merging and history.

I also suggest how developers can use mercurial as a “super client” to enable much of this power while still working on a team that uses subversion.

Overall the presentation was well received, and many of the developers seemed excited about the potential in a DVCS like mercurial. I know a number of them downloaded mercurial this afternoon and gave it a shot. I also spoke with our build guy and one of our architects who both seem interested in switching over.

If you’re interested in installing mercurial, you can easily download binaries, or else you can go to the main mercurial website and compile it from source.

Here’s the presentation (without notes):

| View | Upload your own

I’ve also uploaded a version of the presentation with full notes on slideshare. Adding the notes makes it too small to read in their flash app, so I suggest downloading the presentation if you want to see them. The notes also include all of the detail in the live coding session (though you have to have grails 1.0.2 or better installed to get the same stuff I did).

I’d be interested to hear from others who have convinced their teams to switch from using a centralized repository to a distributed one like Mercurial.

I actually came to Groovy/Grails after working with Ruby/Rails for about a year. Both languages and framworks have their strengths and it’s nice to see cross-pollination of good ideas going both ways. From the comments, it sounds like Ruby 1.9 is getting some of the features that we’ve had in Groovy for a while.

One thing that I’ve missed from Ruby is the strong category support (called Mixins in Ruby). I’ve been reading that the 1.6 version of Groovy is much stronger in this area, and I’m looking forward to trying it out to see how it matches up.

I used to think that the regular expression support in Groovy was lacking compared to Ruby, but after a little digging, I found that it’s just about as easy and powerful as Ruby is.

A couple of other things that I miss from Ruby:

• package management through RubyGems (yes, we can use maven/ivy/etc, but it’s really no where as easy as having this at your fingertips:

  sudo gem install foo

• multiple assignment of variables on the same line (though this will be available in groovy 1.6)

  a, b = "foo", "bar"

• block regular expressions using %r{...} - these make it much easier to document really long regular expressions rather than one huge string
• array.first and array.last to safely retrieve the first or last element out of an array.

  Yes, I can use ?.getAt, but it’s not as descriptive

  def array = ["foo", "bar", "baz"]
  array?.getAt(0)   // returns "foo"
  array?.getAt(-1) // returns "baz"

• Being able to use ||= to instantiate a variable if it doesn’t exist and set it to a default value if it doesn’t already have a value (though the Groovy “Elvis” operator gets pretty close).

  This example instantiates val if it doesn’t exist and gives it the default value of “start_val” and then appends “foobar”, if val does exist, it will simply append “foobar” to it.

      (val ||= ["start_val"]) << "foobar"

• Having the .irbrc file auto load ruby code when going into irb (the interactive ruby shell. Having a .groovyrc file that gets loaded when going into groovysh or groovyConsole would be great to attach custom enhancements to the Groovy language.

I know that some of this stuff can be added relatively easily through MOP, but it’s nice to have it right at your fingertips when you fire up a groovyConsole to try something out. The Groovy core team has been good about continuing to implement things from other languages that are useful and make sense for Groovy and overall, I’ve been a happy programmer for the last year that I’ve been using Groovy and Grails.

Some people, when confronted with a problem, think “I know, I’ll use regular expressions!”
Now they have two problems. — Jaime Zawinski

There is a common and well-earned aversion in the Java world to regular expressions. Prior to Java 1.4, regular expressions weren’t even part of the core language. Post 1.4, using regular expressions is still a painful task of working with Pattern and Matcher objects. Lots of typing is involved to make anything happen. It’s difficult enough that most Java devs don’t end up using them enough to actually remember how to read a regular expression, and they need to dig up the JavaDocs (or cut and paste an old example), every time they want to use them.

This aversion has persisted into the Groovy community to a level that I haven’t seen in other dynamic scripting languages like Ruby, Python, and (obviously) Perl.

The current regexp docs that pop up when doing a google search are all outdated and don’t use any of the best techniques that are available in the groovy 1.5.X and 1.6-beta code that is now available. The recent Groovy Recipes book doesn’t have an entry for regular expressions in the index, and I was unable to find a single example of a regular expression in the entire book.

This is unfortunate because Groovy makes using regular expressions much easier than in Java. Under the covers, you’re still working with the same old Java Pattern and Matcher objects, but the Groovy syntax and additions to those classes are pleasant to work with.

String Escaping with Slashy Strings

Groovy adds a new type of string escaping, Slashy Strings, that can be used to make your regular expressions easier to read. Forward slashes around text create String objects, just like quotes do. Unlike quoted strings, you don’t have to escape backslashes with another backslash in a Slashy String:

assert java.lang.String == /foo/.class
assert ( /Count is \d/ == "Count is \\d" )

You can also use groovy expressions Slashy Strings, just like double-quoted GStrings:

def name = "Ted Naleid"
assert ( /$name/ == "Ted Naleid" )
assert ( /$name/ == "$name" )

There isn’t anything specific to regular expressions with Slashy Strings, but many regular expressions use shorthand character classes such as \d (digit), \s (non-whitespace character), \b (word boundary) etc. The JavaDocs for Pattern actually has a nice reference for regular expression character classes if you’re not familiar with them.

Groovy Regular Expression Operators

Groovy adds 3 new operators

• “~” - used before a string and it will cause the string to be compiled to a Pattern for later use

// \b means word boundary, [A-Z] means any capital letter, + means one or more
// so this matches any string of one or more capital letter with a word boundary (non-word character) on either side of it
def shoutedWord = ~/\b[A-Z]+\b/

• “=~” - Creates a Matcher out of the String on the left hand side and the Pattern on the right.

def matcher = ("EUREKA" =~ shoutedWord)  
assert matcher.matches()         // TRUE
 
def numberMatcher = "1234" =~ /\d+/  
assert numberMatcher.matches()   // TRUE

• “==~” - Returns a boolean that specifies if the full String matches the Pattern

assert "1234" ==~ /\d+/    // TRUE
assert "FOO2" ==~ /\d+/    // FALSE!!!

Enhancements to the String Class

In Groovy, the String class has been enhanced with a few “replace*” methods that allow you to leverage regular expressions. These methods originally come from the Matcher class, but attaching them directly to String puts them right at your fingertips.

replaceFirst will replace the first substring matched by a regular expression within the specified String:

assert "Green Eggs and Spam" == "Spam Spam".replaceFirst(/Spam/, "Green Eggs and")

replaceAll will replace all matching substrings within the specified String:

assert "The armor was colored silver" == "The armour was coloured silver".replaceAll(/ou/, "o")

There is an alternate version of replaceAll that takes a closure for the second parameter. This is especially useful in the situations where you want to manipulate the matched value, or groups within the match to dynamically determine the replacement text.

For example, if we wanted to be able to turn a dashed phrase (”foo-bar”) into a camel case word (”fooBar”) we can’t just remove all dash characters, we also need to make the first letter after the dash capitalized (the “B” in “fooBar”).

To do this, we can use a regular expression that captures the first letter after a dash in a group using parenthesis.

 
def dashedToCamelCase(orig) {
	// regular expression is a dash, followed by parenthesis that form a group where we hold the word's first character
    orig.replaceAll(/-(\w)/) { fullMatch, firstCharacter -> firstCharacter.toUpperCase() }
}
 
assert "firstName" == dashedToCamelCase("first-name")
 
assert "oneTwoThreeFourFiveSixSevenEight" == dashedToCamelCase("one-two-three-four-five-six-seven-eight")

Using the version of replaceAll that takes a closure gives us a chance to manipulate the first character of the word and capitalize it. This closure is always passed the full matched text of the regular expression as the first value, and then any groups as subsequent values.

Here we modify a phone number and keep the area code group, but replace the exchange and station number with hash marks:

assert "612-###-####" == "612-555-1212".replaceAll(/(\d{3})-(\d{3})-(\d{4})/) { fullMatch, areaCode, exchange, stationNumber ->
    assert fullMatch == "612-555-1212" 
    assert areaCode == "612"
    assert exchange == "555"
    assert stationNumber == "1212"
    return "$areaCode-###-####"
}

Enhancements to Collections

Groovy also makes significant additions to what you can do with Collections. In addition to each, collect, inject, etc, there is a regular expression aware iterator called grep that will pass each item in the Collection through a filter and return a subset of items that match the filter. We can use a regular expression as a filter:

// regular expression says 0 or more characters (".*") followed by the string "bar" that is at the end of the string ("$")
assert ["foobar", "bazbar"] == ["foobar", "bazbar", "barquux"].grep(~/.*bar$/)

You can achieve the same thing with findAll but it takes a little more typing:

assert ["foobar", "bazbar"] == ["foobar", "bazbar", "barquux"].findAll { it ==~ /.*bar$/ }

Working with Matchers

As we’ve seen, using the =~ operator will return a Matcher object. Many of the existing regular expression examples on the web work by treating the Matcher as a list and getting the first (zero-based) element out of the list:

def matcher = "foobazaarquux" =~ "o(b.*r)q"
assert ["obazaarq", "bazaar"] == matcher[0]
assert "bazaar" == matcher[0][1] // get the first grouping of the first map

This is a little fragile as matcher[0] will throw an error if there was not actually a match. Calling matches() doesn’t help as matches only checks if the regular expression matches the WHOLE string:

("foobazaarquux" =~ "o(b.*r)q").matches()  // returns false!
("foobazaarquux" =~ ".*(b.*r).*").matches()  // returns true, ".*" matches 0 or more chars of any type

You can check getCount() to see how many matches there were for some safety:

def m = "foobar" =~ /quux/
if (m.getCount()) {
	// example won't get here as "quux" doesn't exist in "foobar", the count is 0
        println m[0]
}

A groovier way to work with Matchers leverages collection iterators and the built in closures that Groovy provides to them. Matcher supports the iterator() method and with that, gets everything else that any groovy List or Collection would have, including collect, inject, findAll, etc.

def paragraph = """
    Lorem ipsum dolor 12:30 AM sit amet, 
    consectetuer adipiscing 1:15 AM elit. 
    Nunc rutrum diam sagittis nisi 9:22 PM.
"""
 
def HOUR = /10|11|12|[0-9]/
def MINUTE = /[0-5][0-9]/
def AM_PM = /AM|PM/
def time = /($HOUR):($MINUTE) ($AM_PM)/
 
assert ["12:30 AM", "1:15 AM", "9:22 PM"] == (paragraph =~ time).collect { it }
 
assert ["12:30 AM", "1:15 AM"] == (paragraph =~ time).grep(~/.*AM$/)

A limitation of the iterator-based methods is that they don’t give you access to the individual groups (hour, minute, am/pm), just the full matched string (”12:30 AM”). The each method is more powerful because as it iterates through, it passes the full match as well as each of the individual groups into the closure.

("foo1 bar30 foo27 baz9 foo600" =~ /foo(\d+)/).each { match, digit -> println "+$digit" }
 
// result:
// +1
// +27
// +600

Another example (using the paragraph and time Matcher from above) showing how to pretty print all of the timestamps:

 
(paragraph =~ time).each {match, hour, minute, amPm -> 
    println "$hour:$minute ${amPm == 'AM' ? 'this morning' : 'this evening' }"
}
 
// result: 
// 12:30 this morning
// 1:15 this morning
// 9:22 this evening

Regular expressions are a powerful tool that Groovy makes as accessible as any other top-tier scripting language. Using techniques to break more complicated regular expressions into their component pieces can make them much more readable (as in the time example above).

If you’re doing any sort of string processing beyond a simple contains or split, regular expressions in groovy can turn mountains of Java into a couple of lines of code.

Why only 3? Because people don’t learn a couple of pages of keystroke combinations at a time. But with only the 3 of the best keystrokes, there’s a good chance I’ll get you hooked and you’ll seek out some more.

1. Incremental Search Through Command History - ctrl+R

Most people know that pressing the up arrow at a prompt will go to the previous command, and subsequent up arrows will move backwards in history.

That works fine for commands that you just typed. What if you had checked out the grails trunk a week ago and wanted to do it again, but couldn’t remember the exact repository name. Just type ctrl-R and then a part of the command that you can remember, such as svn co. As you type it will search backwards through your command history for a command that matches. When your comamnd appears, just hit enter to execute.

svn co http://svn.codehaus.org/grails/trunk/grails/ grails
bck-i-search: svn co_

If there were multiple commands that match the pattern that you typed, either type more of the command to get more specific, or hit ctrl+R again to find the next most recent command that matches the pattern.

2. Delete Last Word - ctrl+W

There are a number of great movement and deletion commands (almost all of them originating from emacs keybindings). But ctrl+W is the one I use more often than any other. It deletes the previous word

On the mac, one other benefit of knowing these keybindings is that they work in just about every cocoa based application as well as a few other apps. For example, in Firefox, ctrl-W will work on the location bar to make it easy to delete parts of an URL. Once these commands become part of your muscle memory, you’ll find yourself using them all over the place.

3. Insert Last Argument of Previous Command - option+. (period) or esc+.

How many times have you wanted to do 2 things with a file or directory in succession? It happens all the time to me and this is my favorite command that I’ve never seen anyone else use.

For example, if I want to do a quick update of one of my domain classes from subversion:

svn up grails-app/domain/Author.groovy

and then edit it in TextMate, all I have to do is type

mate

then press option+. and it autocompletes to:

mate grails-app/domain/Author.groovy

There is a word designator shortcut, “!$“, that also means “last parameter of previous command”, but with ctrl-. you actually get to inspect and edit the parameter easily.

Also, on OSX, esc is the default “meta” key, but you can (and should) go into the terminal preferences and check the “Use option as meta key” checkbox on the Keyboard tab.

If you’re interested in learning more about terminal history commands, I highly recommend The Definitive Guide to Bash Command Line History.