Tag Archives: D3

Data-Driven Documents, a JavaScript visualization library

Adventures with Twitter in Groovy

Have you ever wanted to get a better sense of an event’s commentary on Twitter? For example, a subset of your following may be attending an event in your periphery but making interesting enough comments to warrant a closer look.

Well, I’ve had these thoughts on an increasing basis lately so I thought I’d dive into the Twitter stream a bit to see what I could learn.

Fortunately, I was aware of a gist to query Twitter by @pmonks, and this Groovy script became the foundation for my exploration.

Since I don’t normally program in Groovy, I had to setup Groovy on my MacBook Pro. (To be clear, I went for “get this to work” rather than “accomplish your configuration with all best practices incorporated.” I’m sure there may be better techniques where Groovy setup on MacOS is concerned, and I thank you in advance for any comment-based pointers.)

  1. Visit the Groovy website and understand basic Groovy installation instructions.
  2. Downloaded the binary release (Zip) of Groovy 1.8.6 (i.e. latest available version of the stable branch at the time of this post).
  3. Extracted Zip contents and moved (via Terminal) them under my up-to-date Java distribution:
    prompt$ sudo mv ~/downloads/groovy-1.8.6 /usr/share/java

    (I’ve seen apparent best practice guidance to create a symlink that allows you to access Groovy without referencing the version number, but I’ve skipped that here.)

  4. Edit your .profile file to point your environment to your new Groovy installation. (If you don’t have a .profile file, read and follow this.)
    prompt$ open /Applications/TextEdit.app .profile

    Add (or confirm) the following lines in your .profile file:

    JAVA_HOME=/Library/Java/Home; export JAVA_HOME
    GROOVY_HOME=/usr/share/java/groovy-1.8.6; export GROOVY_HOME
    PATH=$GROOVY_HOME/bin:$PATH; export PATH

    Save your .profile file, exit TextEdit, close all open Terminal windows, exit Terminal and open a new Terminal window to enable .profile changes to take effect.

  5. Confirm that Groovy is ready to act on your script:
    prompt$ groovy -v

    My confirmation looked as follows:
    Groovy Version: 1.8.6 JVM: 1.6.0_31 Vendor: Apple Inc. OS: Mac OS X

For this particular script, queryTwitter.groovy, you need to provide a peer twitter4j.properties file that will provide to Twitter your username and password. Once that’s ready,

prompt$ groovy queryTwitter.groovy %23interesting_event

The results are ready to be nested within a table element, if you want to produce HTML for display as-is. However, you could also modify the script to render JSON output and pursue alternative visualizations (e.g. rate of tweets over time, geolocation of individuals in the tweetstream, word clouds for the event, influencer status, etc.)–something for D3.js.

D3 (Data-Driven Documents)

Examples of D3-based visualizations

“D3.js is a small, free JavaScript library for manipulating documents based on data.” -http://mbostock.github.com/d3/

Since I’ve been trying to spread the word with colleagues about this powerful, expressive visualization technology, I thought it might be worth sharing similarly here.

The following tutorials have been helpful in ramping folks up on D3:

D3’s creator, Mike Bostock, has published a white paper on D3 (Stanford Visualization Group) and presented D3 during the most recent W3Conf (browser-based slides).

A couple of spotlights on D3 include one by Flowing Data and another by SVG Magazine.

Flowing Data also posted an application of D3, “How Americans Spend Their Day” (background context).

If this whets your appetite, visit the D3 wiki for a whole lot more goodness (documentation, additional tutorials, example visualizations, etc.).

Update 8/30/2012: Mike Bostock posted his 3/5/2012 D3 workshop given at VIZBI 2012. Also since my original post, I discovered D3 for Mere Mortals and Christophe Viau’s Try D3 Now post.