Ted Naleid

Gradle is a great build tool. It’s easy to download and install. If you’re on a mac and have homebrew, it’s as easy as:

brew install gradle

It’s very easy to use with a little experience, but I find having a good starting place to base your work from can help.

Here’s a quick function that I’ve got in my .zshrc to bootstrap a new groovy gradle project in the current directory (it should also work in in a .profile/.bash_profile/.bashrc).

function newgradle() {
    echo "Creating files for new gradle project"
 
    cat <<EOF>.gitignore
*.un~
*.iml
*.ipr
*.iws
build
.gradle
EOF
 
    cat <<EOF>build.gradle
apply plugin: 'groovy'
apply plugin: 'idea'
 
repositories {
    mavenCentral()
}
 
dependencies {
    groovy 'org.codehaus.groovy:groovy-all:1.8.6'
    groovy 'org.apache.ivy:ivy:2.2.0'
    testCompile 'junit:junit:4.10'
}
 
task createSourceDirs(description : 'Create empty source directories for all defined sourceSets') << {
    sourceSets*.allSource.srcDirs.flatten().each { File sourceDirectory ->
        if (!sourceDirectory.exists()) {
            println "Making \$sourceDirectory"
            sourceDirectory.mkdirs()
        }
    }
}
 
idea {
    project {
        jdkName = '1.6'
    }
}
EOF
 
    gradle createSourceDirs
 
    git init
    ls -a1 && find src    # list all created assets
}

It creates a build.gradle file ready to work with java and groovy projects, including IDEA integration (just execute gradle idea).

This gives you all of the tasks necessary to compile, jar, test, and distribute your code. For more information, check out the gradle docs on the java, groovy, and idea tasks.

It also creates all the necessary source directories for you and initializes a new git repository (with starting .gitignore file) for you to save your work.

You can easily tweak the build.gradle or .gitignore files to fit your needs. If you don’t use git, you can either delete those lines, or subsitute the equivalent lines for the source control tool you use. These are just a good starting place for me.

Here’s the sample output from the script above:

% mkdir testapp
% cd testapp
% newgradle                                                                   
Creating files for new gradle project
:createSourceDirs
Making /Users/tnaleid/Documents/workspace/testapp/src/main/resources
Making /Users/tnaleid/Documents/workspace/testapp/src/main/java
Making /Users/tnaleid/Documents/workspace/testapp/src/main/groovy
Making /Users/tnaleid/Documents/workspace/testapp/src/test/resources
Making /Users/tnaleid/Documents/workspace/testapp/src/test/java
Making /Users/tnaleid/Documents/workspace/testapp/src/test/groovy
 
BUILD SUCCESSFUL
 
Total time: 2.344 secs
Initialized empty Git repository in /Users/tnaleid/Documents/workspace/testapp/.git/
.
..
.git
.gitignore
.gradle
build.gradle
src
src
src/main
src/main/groovy
src/main/java
src/main/resources
src/test
src/test/groovy
src/test/java
src/test/resources

Then you’ve got all this gradle functionality ready to use:

% gradle tasks                                                           
:tasks
 
------------------------------------------------------------
All tasks runnable from root project
------------------------------------------------------------
 
Build tasks
-----------
assemble - Assembles all Jar, War, Zip, and Tar archives.
build - Assembles and tests this project.
buildDependents - Assembles and tests this project and all projects that depend on it.
buildNeeded - Assembles and tests this project and all projects it depends on.
classes - Assembles the main classes.
clean - Deletes the build directory.
jar - Assembles a jar archive containing the main classes.
testClasses - Assembles the test classes.
 
Documentation tasks
-------------------
groovydoc - Generates Groovydoc API documentation for the main source code.
javadoc - Generates Javadoc API documentation for the main source code.
 
Help tasks
----------
dependencies - Displays the dependencies of root project 'test'.
help - Displays a help message
projects - Displays the sub-projects of root project 'test'.
properties - Displays the properties of root project 'test'.
tasks - Displays the tasks runnable from root project 'test' (some of the displayed tasks may belong to subprojects).
 
IDE tasks
---------
cleanIdea - Cleans IDEA project files (IML, IPR)
idea - Generates IDEA project files (IML, IPR, IWS)
 
Verification tasks
------------------
check - Runs all checks.
test - Runs the unit tests.
 
Other tasks
-----------
cleanIdeaWorkspace
createSourceDirs - Create empty source directories for all defined sourceSets
 
Rules
-----
Pattern: build<ConfigurationName>: Assembles the artifacts of a configuration.
Pattern: upload<ConfigurationName>: Assembles and uploads the artifacts belonging to a configuration.
Pattern: clean<TaskName>: Cleans the output files of a task.
 
To see all tasks and more detail, run with --all.
 
BUILD SUCCESSFUL
 
Total time: 4.871 secs

On a recent grails project, we’re using a git repo that was originally converted from a SVN repo with a ton of large binary objects in it (lots of jar files that really should come from an ivy/maven repo). The .git directory was over a gigabyte in size and this made it very cumbersome to clone and manipulate.

We decided to leverage git’s history rewriting capabilities to make a much smaller repository (and kept our previous repo as a backup just in case).

Here are a few questions/answers that I figured out how to answer with git and some shell commands:

What object SHA is associated with each file in the Repo?

Git has a unique SHA that it associates with each object (such as files which it calls blobs) throughout it’s history.

This helps us find that object and decide whether it’s worth deleting later on:

git rev-list --objects --all | sort -k 2 > allfileshas.txt

Take a look at the resulting allfileshas.txt file for the full list.

What Unique Files Exist Throughout The History of My Git Repo?

If you want to see the unique files throughout the history of your git repo (such as to grep for .jar files that you might have committed a while ago):

    git rev-list --objects --all | sort -k 2 | cut -f 2 -d\  | uniq

How Big Are The Files In My Repo?

We can find the big files in our repo by doing a git gc which makes git compact the archive and stores an index file that we can analyse.

Get the last object SHA for all committed files and sort them in biggest to smallest order:

git gc && git verify-pack -v .git/objects/pack/pack-*.idx | egrep "^\w+ blob\W+[0-9]+ [0-9]+ [0-9]+$" | sort -k 3 -n -r > bigobjects.txt

Take that result and iterate through each line of it to find the SHA, file size in bytes, and real file name (you also need the allfileshas.txt output file from above):

for SHA in `cut -f 1 -d\  < bigobjects.txt`; do
echo $(grep $SHA bigobjects.txt) $(grep $SHA allfileshas.txt) | awk '{print $1,$3,$7}' >> bigtosmall.txt
done;

(there’s probably a more efficient way to do this, but this was fast enough for my purposes with ~50k files in our repo)

Then, just take a look at the bigtosmall.txt file to see your biggest file culprits.

Purging a file or directory from history

Use filter-branch to remove the file/directory (replace MY-BIG-DIRECTORY-OR-FILE with the path that you’d like to delete relative to the root of the git repo:

git filter-branch --prune-empty --index-filter 'git rm -rf --cached --ignore-unmatch MY-BIG-DIRECTORY-OR-FILE' --tag-name-filter cat -- --all

Then clone the repo and make sure to not leave any hard links with:

git clone --no-hardlinks file:///Users/yourUser/your/full/repo/path repo-clone-name

You can use this command from the parent directory that contains your git repository and it’s clone to see how much space each of them take, and how much you’ve shrunk the repo in size:

du -s *(/)     # add the -h flag to see the output in human readable size formats, just like ls -lah vs ls -la

With these commands, I was able to reduce the file size of our repo with a few thousand commits down below the size of the checked out repository (more than an order of magnitude smaller). I only removed old binary files, we still have full history for all code files.

There are a few very nice looking, mac-like diff tools for OSX (Kaleidoscope and Changes come to mind), but none for doing “real” merges. By this, I mean real, 3-way merges with all of the information you need in front of you.

There are no good-looking, “mac-like” merge tools, but if you swallow your pride there are a few different options for 3-way merges, including Araxis Merge ($$$!), DiffMerge, DeltaWalker, and FileMerge which comes free with XCode.

I’ve tried them all, and find them all confusing. They all tend use a 3-pane display to do the merging with your file in the left pane, the file you’re merging in the right pane, and the messy half-merged file in the middle.

That’s not enough information.

A 3-way merge actually has four important sources of information: Read the rest of this article »

Having a Clojure REPL (Read Eval Print Loop) right inside Vim makes it easier to test ideas, get documentation, and explore your code. There are a few hoops that you need to jump through to enable it, but the payoff is worth it.

Read the rest of this article »

I’m a very happy customer of Dropbox. It allows painless syncing of files across multiple computers without extra features to complicate it. The top rated answer on Quora to the question “Why is Dropbox more popular than other programs with similar functionality?” sums things up perfectly.

One of my favorite uses of Dropbox is to sync almost all of the non-machine specific configuration files and directories in my home directory across all my OSX computers (currently my iMac, MacBook Air, and my work laptop).

Doing this lets me make a configuration change to one computer and have it almost instantly available on any other computer without any manual steps.

This is especially important for my zshell and Vim configurations as I’m always tweaking those, but it’s also helpful to have my Documents, Downloads and Pictures shared.

I have a folder in my Dropbox directory called home, I use a script called link.sh to automatically create symlinks in my home directory to the things I’ve got stored in Dropbox.

Dropbox/home currently has these files and directories in it:

.ackrc
.dbvis
.groovy
.gvimrc
.hg
.hgignore_global
.ssh
.subversion
.vim
.viminfo
.vimrc
.zshenv
Desktop-starling.local/   # unique Desktop for my MacBook Air
Desktop-kestrel.local/    # unique Desktop for my iMac
Desktop-thrush.local/     # unique Desktop for my work MacBook Pro
Documents/
Downloads/
Pictures/
bin/

My Dropbox/home directory also has a shell script in it called link.sh:

#! /usr/bin/env bash
cd $(dirname $0)
 
function linkFile() {
    LINK_TO_NAME=$2
    if [ -z $LINK_TO_NAME ]; then
        LINK_TO_NAME=$1
    fi
    if [ -a $HOME/$LINK_TO_NAME ]; then
        echo "**** Found existing $LINK_TO_NAME, skipping..."
    elif [ -h $HOME/$LINK_TO_NAME ]; then
        echo "Already symlinked $LINK_TO_NAME, skipping..."
    else
        echo "Linking $1 to $LINK_TO_NAME"
        ln -s $PWD/$1 $HOME/$LINK_TO_NAME 
    fi
}
 
 
for F in $(ls -a1 | grep -v link.sh | grep -v Desktop | egrep -v "^..?$" | egrep -v "^.*un~$" | grep -v .DS_Store); do
    linkFile $F
done
 
export HOSTNAME=$(hostname)
 
if [ -d "Desktop-$HOSTNAME" ]; then
    linkFile "Desktop-$HOSTNAME" "Desktop"
else 
    echo "Unable to find Desktop-$HOSTNAME to link to Desktop"
fi

What the script does is:

1. cd into the directory that the script is located in (it only symlinks files in the same directory)
2. list out all of the files and directories in the same directory as the script
3. filter out the things we don’t want to link (like ., .., the link.sh script itself, etc)
4. For all of the files/directories that pass the filter, call linkFile to create a symlink in the current user’s home directory as long as there isn’t already a file or a symlink there
5. Then look for a file called Desktop-$HOSTNAME where $HOSTNAME is the name of the current machine and create a ~/Dropbox symlink to it if it’s found.

It should be safe and non-destructive and only create symlinks when there isn’t anything else there with the same name.

I didn’t have my Pictures, Documents, and Downloads in my Dropbox for quite a while and was able to get away with the free 2GB plan. I recently upgraded to a paid Dropbox plan as I wanted those directories shared as well (though I exclude a couple of them from my work MacBook Pro).

For “special” directories like Desktop, Pictures, Documents, and Downloads, I needed to use sudo rm -r [dirname] to remove it before I could create the symlink (BACKUP THE DIRECTORY FIRST).

I’ve been using this for over a year, and haven’t noticed any apps that care that those directories are symlinks.

Also? I have used this shell script many times on my systems, and I think it’s safe, but PLEASE backup before using it, or deleting any directories. An adult crying is not a pretty sight :).

I’m currently on a project that has a couple of different apps that are using different versions of grails that need to be run concurrently. Switching a symlink no longer fit the way I needed to work so I came up with a couple bash/zsh aliases that are smart about the version of grails for the current directory.

These aliases work for both the grails as well as the grails-debug commands (for attaching a remote debugger).

If there is an application.properties file in the current directory, we can find the current version of grails for the app.

If there isn’t an application.properties file in the current directory, the script just defaults to whatever version of grails you’ve already set up as your default through the standard $GRAILS_HOME environment variable. You can use the grails symlink switching aliases that I created previously to easily move this between versions.

alias grails="execute_grails_version grails"
alias grails-debug="execute_grails_version grails-debug"
 
function execute_grails_version() {
    GRAILS_CMD=$1
    shift
    if [ -f application.properties ]; then
        export GRAILS_VERSION=`grep app.grails.version application.properties | sed -E 's/.*=(.*)/\1/'`
        export GRAILS_HOME="/usr/local/grails-$GRAILS_VERSION"
        echo "application.properties found, using \$GRAILS_HOME of $GRAILS_HOME"
    else 
        echo "application.properties NOT found, leaving \$GRAILS_HOME as $GRAILS_HOME"
    fi
 
    if [ ! -d $GRAILS_HOME ]; then
        echo "ERROR: Unable to find \$GRAILS_HOME directory at $GRAILS_HOME"
        exit 1
    fi
 
    echo $GRAILS_HOME/bin/$GRAILS_CMD $*
    $GRAILS_HOME/bin/$GRAILS_CMD $*
}

UPDATE: There are a few situations where aliases aren’t available (or are a pain to get available) such as when code is being executed as part of another application rather than from the command line. To get around this, these scripts (created by a co-worker of mine, @sjurgemeyer) could be put in your PATH, ahead of your $GRAILS_HOME/bin and used instead of the aliases above:

grails:

grails-version grails $*

grails-debug:

grails-version grails-debug $*

grails-version:

GRAILS_CMD=$1
shift
if [ -f application.properties ]; then
    export GRAILS_VERSION=`grep app.grails.version application.properties | sed -E 's/.*=(.*)/\1/'`
    export GRAILS_HOME="/usr/local/grails-$GRAILS_VERSION"
    echo "application.properties found, using \$GRAILS_HOME of $GRAILS_HOME"
else 
    echo "application.properties NOT found, leaving \$GRAILS_HOME as $GRAILS_HOME"
fi
 
if [ ! -d $GRAILS_HOME ]; then
    echo "ERROR: Unable to find \$GRAILS_HOME directory at $GRAILS_HOME"
    exit 1
fi
 
echo $GRAILS_HOME/bin/$GRAILS_CMD $*
$GRAILS_HOME/bin/$GRAILS_CMD $*

At the end of gr8conf today there were quite a few door prize giveaways. Winners were picked using a printout with attendees listed in (I’m assuming) random order. The guys running the lottery were going down the list and calling off names.

This was right after my talk on using Redis with Groovy and I thought to myself, “this is a perfect example of where a quick redis script could automate this and make it a bit more groovy”. So I threw together this script in about 15 minutes:
Read the rest of this article »

The grails console has a number of disadvantages that make it a little clunky to use:

• You can’t attach it to a running grails instance, you need to run grails console from scratch
• You can’t pass it parameters, such as the name of a script you’ve saved
• It’s a swing app, which is always a little wonky in it’s keybindings and it’s behavior
• It’s not <insert your chosen editor>

I got tired of these limitations so I decided to do something about it. I put together a groovy script that uses HTTPBuilder to POST groovy code to a running Grails app that has the grails console plugin installed.

If you’re running this in any non-development environment, you’ll want to ensure that it’s behind some form of authentication. The most popular grails security plugin is Burt Beckwith’s spring security core, so this script comes with built-in support for that.

This script makes it easy to call from any editor that allows you to execute scripts (pretty much every programmer’s editor on the market). At the bottom of the post, I show how to integrate it with Vim (my editor of choice), but it’d be just as easy to script it from TextMate, JEdit, IntelliJ, Eclipse, etc. Any editor that can take the file you’re currently editing and pass it to a shell script.

Here’s my Vim session with a grails script at the bottom and the output from that grails script in another window at the top.

Read the rest of this article »

If you’re developing on the mac using redis and want it to start automatically on boot, you’ll want to leverage the OSX launchd system to run it as a User Daemon. A User Daemon is a non-gui program that runs in the background as part of the system. It isn’t associated with your user account. If you only want redis to launch when a particular user logs in, you’ll want to make a User Agent instead.

From the command line, create a plist file as root in the /Library/LaunchDaemons directory with your favorite text editor:

sudo vim /Library/LaunchDaemons/io.redis.redis-server.plist

Paste in the following contents and modify it to point it to wherever you’ve got redis-server installed and optionally pass the location of a config file to it (delete the redis.conf line if you’re not using one):

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
	<key>Label</key>
	<string>io.redis.redis-server</string>
	<key>ProgramArguments</key>
	<array>
		<string>/usr/local/bin/redis-server</string>
		<string>/optional/path/to/redis.conf</string>
	</array>
	<key>RunAtLoad</key>
	<true/>
</dict>
</plist>

You’ll then need to load the file (one time) into launchd with launchctl:

sudo launchctl load /Library/LaunchDaemons/io.redis.redis-server.plist

Redis will now automatically be started after every boot. You can manually start it without rebooting with:

sudo launchctl start io.redis.redis-server

You can also shut down the server with

sudo launchctl stop io.redis.redis-server

Or you could add these aliases to your bash/zsh rc file:

alias redisstart='sudo launchctl start io.redis.redis-server'
alias redisstop='sudo launchctl stop io.redis.redis-server'

If you’re having some sort of error (or just want to watch the logs), you can just fire up Console.app to watch the redis logs to see what’s going on.

UPDATE: This script has been merged into the Grails 1.3.6 release so it’ll be part of core Grails with the next version of grails. You’ll only need to get it manually for versions earlier than that.

A while ago, I created a grails script that allowed the execution of a groovy script within the grails context. Using this gant script gives you access to all of the grails domain objects, hibernate session, and other plugins that you have installed into the app, all from the command line.

It had rotted a bit in the grails 1.3 days (I’d last updated it for grails 1.2), but finally got around to updating it to work with the latest version of grails (currently 1.3.5).

I’ve created a mercurial repository to hold a test project for the script so that it’s easy for me to verify that it works with the latest version of grails.

If you don’t have mercurial installed, you can just get the raw version of the file and save it in the “scripts” directory of your grails application.

Then you can create a groovy script that you’d like to execute (often to bootstrap data, or run some periodic process) and execute it with:

grails run-script path-to-script/scriptName.groovy

I often create my groovy scripts within a “grails console” and get them to work there, then save them for later execution by “run-script”.

You can see it in action by cloning the sample project out on bitbucket and executing the sample script I have in it:

hg clone ssh://hg@bitbucket.org/tednaleid/grails-run-script
cd grails-run-script
grails run-script userScripts/createBook.groovy

Which shows:

Welcome to Grails 1.3.5 - http://grails.org/
Licensed under Apache Standard License 2.0
....<more grails output>...
Running script userScripts/createBook.groovy ... ...
it worked!
Script userScripts/createBook.groovy complete! ...
Application context shutting down...
Application context shutdown.