Tom Morris

Jay Kreps: Scala Macros: "Oh God Why?" →

Folding in Ruby and Scala

I’d like to think I’m no slouch at functional programming, but I came across a nice little example today of where Ruby’s functional programming features weird me out: inject.

Consider a perfectly simple example of where functional programming would be useful: summing an array of objects of the same type that all have a property or method that returns an integer value.

Here is how one might do it in Java (or C, PHP or another language without the benefit of higher order functions):

That’s easy enough. You might need to deal with what happens the length value is nil, and you might need to handle exceptions. Whatever.

But, of course, functional programming is here to make your life better. You shouldn’t need to write the same foreach-accumulator code every time for every different property. So, imagine if we had an array of objects that had five different properties that are all integers: length, height, weight, smelliness. We want to sum them all, or average them all or whatever. This is where higher-order functions become so useful.

In Scala:

Okay, now imagine we’ve got something similar in Ruby. The code transliterates reasonably well.

Two nitpicky things here about syntax:

1. The map syntax is simpler in Scala: I’ve always found “and-colon” in Ruby to be a rather ugly and unreadable construction. I understand it alright, but I’m not confident that Ruby newbies understand it. (But, then, lotsa Ruby newbies don’t grok functional programming: I certainly didn’t.)
2. The word “inject” is a bloody strange word for a well-understood concept: every other language with fold-left calls it fold-left (or foldLeft or foldl or fold or something similar involving the word “fold”). “inject” is eminently forgettable.

I could, of course, monkey patch in “fold_left” as an alias to inject.

Here is where it gets more interesting for Scala though. The type system lets you do type-dependent monkey patching.

Consider the generic concept of folding or reducing values: it is simply a process where you turn numerous values of the same type into a unified result of a particular type. So, summing is a folding operation, as is producing a mean, a mode, a median etc.

Enter Scala implicits.

Now one can do something like this…

How does this differ from just monkey patching ‘sum’ into Ruby’s Array class? It only applies to lists made exclusively of one particular type: Ints. Implicits let you target specific types in a way that Ruby monkey patching doesn’t. This is pretty powerful, and yet another reason to learn to love static typing.

You can call sum on List[Int] but not on List[String] (unless you define a new class that implements a sum method and an implicit conversion function for List[String]). Without one of those, you’ll get a type error if you call something like the following:

Think about the power of this: programming often boils down to doing some repetitive action to a list of values of the same type. In Ruby, you can monkey patch the individual class, but think about how you can make code more readable by adding methods to both the objects and the collections of objects, without those modifications spilling out and affecting collections of objects of other types.

I think this is a real example of where Scala’s static type system combined with implicits and functional programming allows for a far more expressive way of coding.

A common idiom in C# is to use null to mean the absence of a value. However, null is also used to indicate an uninitialized value. This duality can lead to confusion and bugs. If you use the option type, there is no question what the value represents.

Things I’m currently quite excited about

There’s a few tech things I’m pretty excited about at the moment.

Firstly, Scala is now at the point where it is actually productive. Scala 2.9.0 RC1 has just come out and it’s got freaking parallel collections in the standard library. Finally, I can stop looking so longingly at Clojure. As far as I can tell, you’ll be able to do something like this:

Array(1,2,3,4,5) par map (_ * 2) seq

And it’ll make an array, turn it into a ParArray, map it, and turn it back into a sequential Array. That’s pretty cool.

Then there’s Scalatra. Scalatra is Sinatra but for Scala. And the setup is very simple. Once you’ve written the app, you get a WAR file, which you can upload to…

Elastic Beanstalk. Finally, Amazon have delivered a Heroku-like experience. This is really exciting for hacking stuff together. For 2¢ an hour, you can host JVM apps on an EC2 micro-slice, but without having to touch the operating system: you literally go to the website, pick a subdomain and upload a JAR file. The Hello World I wrote in Scalatra was about 5Mb.

Completely unrelated: WebGL. Holy moley is that exciting. If you are running Chrome or Firefox 4, go and ogle this. It’s beautiful. I’m not nearly clever enough to do 3D programming, but just imagine a world of composable 3D objects with URIs and a universe of RDF-esque ties between them. Virtual objects roaming the one Web intermingled with thoughts and data in some kind of bizarre interlinked Platonic ether. That was kind of the dream of VRML, right? And is sort of the reality that Second Life has. But we could go further.

Imagine Wikipedia with WebGL: you click a little link in Wikipedia, and the article fades slightly only for the objects the article is representing to appear in front of you. You can rotate them, download them, throw them around in physics demonstrations, attach them to your avatar. But then each part of the WebGL model could have RDF metadata attached: this particular set of polygons is supposed to represent a particular type of material. You click and all the other virtual objects made of iron or copper or polyurethane or semi-transparent blue plastic or plutonium appear before you. As you are exploring Wikipedia, you copy a few items you find into your personal inventory, a sort of 3D version of your web bookmarks. Then you come across the Wikipedia article on nuclear reactors. And you are a curious sort, so you start seeing what happens if you chuck all sorts of strange things into the nuclear reactor. What happens if you drop this 3D model of Thomas Jefferson in to the reactor? Suddenly, our mashups have stopped being dumping crime figures on Google Maps and they’ve become shockingly real world and Grand Theft Auto-ish.

What else? Open Election Data. This one’s a simple one: a new link in the Web of Data, joining data.gov.uk up with dbpedia. Haters gonna hate, right, but making it so that when one dataset is talking about “the Monster Raving Looney Party”, they use an identifier that is linked to all the other representations of the same thing is the absolute start of the Web of Data. We may have to have these proxy projects to sit as intermediaries between governments and more direct community-created resources like Wikipedia/Dbpedia. This is an interesting start.

One final thing: Wikimedia. I’ve only got back into Wikipedia recently, and there’s loads of fun stuff we can do with the data. I’ve got a list of hacks as long as my arm. I may work on one tomorrow at Rewired State.

HackDiary: Start actually doing something

Tonight I reaffirmed a simple truth often discovered by programmers: keep it simple and just get on with it.

I’ve got a project I’ve been working on for a month or so now. It’s a web project in Scala. The actual web front end is extremely minimal: in terms of interface, we’re talking old-school Google here. A box where you enter a string, and a web page you get in response. No sessions or other complexity.

But once you enter said string, a lot of parsing and magic has to go on behind the scenes. And more importantly than that, the project is going to be open source and have an API that other programmers will have to write to.

I’ve been basically fucking around with the project for a month or so. I’ve been modelling the API and drawing fancy diagrams and doing way too much design up front. But unless this actually works, it’s mental masturbation. And mental masturbation is a lot less enjoyable than physical masturbation.

Yesterday, I got Scalate to run on Elastic Beanstalk in about ten minutes. Inspired by this, I decided to “throw one away” with my existing code and start afresh.

I took the same code I’d used to produce the Scalate HelloWorld, changed all the project names and class/package names, added some more HTML to the get("/") method, and rewrote the basic logic of the application.

Getting the web front-end working took about half an hour, and it’s taken about another hour to get the basic logic of the back-end plugin architecture working. But now I actually have some working code. This is good. In total, I’ve written about 72 lines of code including HTML, but excluding the web.xml file.¹ In total, I’ve created four separate compilation units (basically source files), a total of four classes/singletons (one of which is just an abstract class to define the plugin interface).

Now, the previous incarnation of the same project had about ten compilation units, but that probably meant about 15 or 20 classes/singletons. I couldn’t quite decide on the API. But because there was no actual working code, it was all masturbation.

There’ll be bits I can pull into the new version. There’s at least two classes I can just copy straight in because I’m happy with them. But a lot of it I can throw away.

There is such satisfaction in getting things to work, and it provides such a good base to work on. Once the code actually does something, you have something to build on. You can build one or two plugin modules and show some real-world results. You can then abstract it all away a bit, refactor and start adding new features.

That I can sbt package and put it up on Beanstalk in a few minutes means I can show someone a prototype if, you know, hypothetically, I were trying to get funding (cough).

Here is my prescription if you have a project where the code compiles just fine but doesn’t actually do anything useful.

mv project/ project-old/

mkdir project/

cd project

Now get on with it by building the simplest prototype you can. Now your mental masturbation can be turned into actually building features, and you can transfer your masturbation energy back to physical masturbation.²

I know, I know. I’m probably the fifth person to give this lecture in the last hour. But O(1) bird in the hand is worth O(2ⁿ) in the bush, right?³

If you want to go to things like Hack Days and HackCamps, this important skill of actually doing something is a profoundly important necessity. One very good way of learning to do that is to try something new. You’ll naturally want to get it working in order to shortcut your own impatience.

I tried Visual Studio 2010 with .NET 4 on Windows 7 the other day because of the thought of building Windows apps. I don’t know C# very well, but the curiosity about the platform kind of nudges you into actually doing something small to see if it works. And, what do you know, in about half an hour, it’s actually possible to build a simple Windows app from scratch. That’s pretty cool. It’s such an easy win, and it prompts you into a sort of change in perspective: if you can build a Windows app in half an hour, or a Mac app with MacRuby too, or put up a Java web app in ten minutes, you raise your consciousness of what is actually possible. Doing things makes you aware that you can do things.

Yeah, yeah, insert appropriate cynicism if you want. Actually, don’t. Fuck you. This is my HackDiary. I can be non-cynical and point out the bleeding obvious. Fuck the cynicism. I’m gonna end on a nice cute puppy picture.

Actually achieving tangible things in programming makes me happy. As do puppies.

I finally got around to trying Elastic Beanstalk.

It’s shockingly easy and may be my new favourite toy. I can now build simple web apps in Scala and test them on Amazon’s Web Services platform ridiculously quickly.

I built this Hello World in about ten minutes using Scalatra, a Scala version of Sinatra. Most of that time was spent setting up an SBT project with the relevant dependencies, downloading the dependencies and so on. The actual code is ridiculously simple: you just make a class that inherits from ScalatraServlet, and add:

get("/") { "Hello World" }

That’s pretty damn easy.

Compile it with sbt compile

Test it locally with sbt jetty

Package up a WAR file with sbt package

Then you just go to the Elastic Beanstalk console, create a new application, upload the WAR file, answer a few simple questions about what kind of EC2 server you want and stuff, and a few minutes later, it’s running.

Beanstalk basically gives you a load-balanced Tomcat setup that scales up and down as needed.

Yes, it’s Hello World. It’s not very impressive. But the fact that you can get Hello World running in a fairly new, Hacker News-hyped language like Scala and deploy it to Amazon in ten minutes is impressive and means that the dream of super-easy lightweight Java deployment might finally be here.

As someone (potentially, depending on time) with an open source Scala web app in the pipeline, this pleases me greatly.

HackDiary: Scala now more comfortable than Ruby?

A few years back, Python was the first programming language I’d pull out of the toolbox. It was the second programming language I learned that is actually still used (BBC BASIC, AMSBASIC, PHP, then Python). Then I learned how to use Ruby, and while it has some shortcomings compared to Python, it has some real advantages as a hacking language. The Perl regex heritage was one, as was the functional programming constructs like map and filter, and the REPL.

But tonight, I think I may have reached the point where the balance has tipped towards Scala.

The problem was simple. I had a text file consisting of times in the format HH:MM. I wanted to get an idea of the distribution of those times across the day. I opened a Ruby REPL:

a = "17:05
16:53
16:29
16:13
14:19
14:14
14:09
14:08
14:02
13:39
13:38
13:36
13:28
13:26
13:26
13:25
13:11
13:08
12:57
12:44
12:40
12:37
12:32
12:21
12:11
12:02
11:45
11:28
11:25"

Err, now what. I guess I need to reverse it. Okay.

a.reverse!

No, no, I shouldn’t be using bang methods. Stateful is hateful or some other Haskelly slogan. Naughty me.

a = a.reverse

I know. I’m fussy.

Err, now what. I want to group ‘em.

a.methods.sort

What? Ruby doesn’t have a grouping method? It must have. I’m not writing a for loop. It’s 2011. Fuck that.

Tab over to Google Chrome.

Wait, no Internet connection.

Open up a Scala shell. Paste the same lot in with some Python-style triple-quotes around them. Yeah, Scala has that. That’s pretty much one enormous reason to use it as opposed to Java, right? REPL assigns them to res0

var times = res0.split("\n").reverse.toList

There we go. A List[java.lang.String]. Scala’s immutable lists rock.

And then doing the data mangling is pretty easy. Type times., press tab, look down the method list. The Scala REPL puts all the methods in an alphabetical, pretty-printed list—like ls does in the bash shell.

“groupBy”. That looks right. I haven’t used this before. Better check the Scaladoc. Oh, wait, offline.

cp /usr/local/Cellar/scala/2.8.0/src/scala-library-src.jar ~/tmp/scaladocs/
cd ~/tmp/scaladocs
unzip scala-library-src.jar
scaladoc scala/collection/immutable/List.scala
open index.html

(That’s for Scala on OS X installed using Homebrew. The location of scala-library-src.jar will vary on other systems.)

Click around. Look at List. Here we go “groupBy”. What’s it tell me?

def groupBy[K](f: (A) ⇒ K): Map[K, Repr]

That’s pretty cryptic, right? It makes sense eventually but it takes time to learn how to read it. It says that it takes a function (f) which takes something of any type and returns a key (K), then gives you a back a Map[K, Repr]. Basically you hand it your list, and a function which takes each member of that list and produces a key from it. Then it gives you back a map with the key and all the members that represent that key.

times.groupBy(x => x.split(":").first)

Now I got a map of like “12” with a list of all the stuff that begins with “12” and it’s in, you know, res1 or something

res1 toList

Now we’ve got a List[(java.lang.String, List[java.lang.String])] in res2. For Scala-newbs, that’s a List of a 2-tuple made up of a String and a list of Strings.

So, where were we?

res2 sortBy(_._1) foreach {x => println(x._1 + ": " + x._2.length + x._2.map(x => "#").mkString)}

And here’s the result:

11: 3 ###
12: 8 ########
13: 9 #########
14: 5 #####
16: 3 ###
17: 1 #

Not the prettiest graph, but, you know, it’s a good start. It wouldn’t be very hard to turn this kind of data into a Google Charts API graph and open it in your web browser. Yeah, this is the JVM: there are ways. Horrible, evil ways.

The sort of code I’ve written isn’t pretty. It won’t win any awards from those clean code people. But that’s not the point. Nobody’s REPL code is pretty: that’s why it’s REPL code, not bloody production code.

The point is quick hacks and data processing. How does Scala do for that? Let’s see: type safety, Java libraries, a ridiculously awesome collections library, a permissive and enjoyable syntax (especially for higher-order functions—filter { _ > 5 }, anyone?). The only thing I really miss is some of the Perl inheritance in Ruby ($_, regex syntax). There’ll be tasks that Ruby will probably be better for, and I’ll probably still write a lot of scripts in Ruby (a large chunk of my life consists of crappy little Ruby scripts talking to each other, and typically filling up my inbox with a bunch of shit when they don’t). Scala may have tipped over into being the first thing I type into my terminal when starting to hack.

Or you could take another reading: all you need to do to get me to use your programming language is build a better collections library and a better REPL. I’m just fickle, a higher-order function whore (as opposed to a functioning whore of a higher order). If that’s so, great. Get on with building better REPLs and better collections libraries for your language and you’ll get my total, unerring loyalty until someone builds something better.

Dean Wampler on the State of Scala: 2.8, Concurrency, Functional Programming →

On learning functional programming

Functional Scala: Introduction is a pretty good introduction to functional programming.

For me, learning functional programming was a very strange thing because I did it almost completely unconsciously. I learned Ruby, where using a for loop rather than an each method is just not done very often. Then you start using closures for callbacks, event loops and the like, and it all just naturally falls into place without much thinking at all. I found with Ruby, learning functional programming wasn’t like sitting down and learning functional programming, any more than if you use Java/C/whatever you sit down and think “I want to learn imperative programming”.

When you learn functional programming through Ruby, you don’t think “gee, I’m passing around a closure which is a function without a name and is a piece of data and I’m not modifying state and bla bla”. No, it is much more subtle and corrupting! You just see “oh, each is like for. When I want to write a for loop, I write an each loop.” And you don’t learn the theory behind it at first, until it suddenly clicks what is going on. And then you abstract a bit away and do a each(&:foo) call and you see the power of it.

Then you sort of think, “well, all the crazy kids on Hacker News are hyping this up like it is the second coming of Jesus, I better have a look and see what I haven’t understood”.

And you go and read a tutorial on functional programming and you read all these comments about how radically different it all is, and you read code samples in Haskell with super-tight tail-call-optimized recursive Fibonacci implementations or how much more awesome qsort is in OCaml or F# or Scala or whatever.

And, yes, if you are a mathematically-minded person writing the sort of code where one wants to write code that looks and feels like a maths textbook, that’s fine. But it never quite seems real.

What nobody seems to ever show is an example of a large programme: I’m not talking about an enormous 50m line Java-esque monstrosity with heaps of AbstractFactoryFactory crud. No, it’d just be nice if someone were to explain how one goes from the ‘look at this sexy syntax and type inference magic’ through the ‘wow, higher-order functions are great’ to actually writing useful stuff in these languages.

This is rather why I like Scala. Although there are pesky things like the backwards-incompatible changes to the collections framework between 2.7.7 and 2.8.x, and the frankly bloody annoying mismatch between inheritance and auxillary constructors which can really make refactoring painful, the learning curve isn’t too steep and the path one takes to learning Scala is littered with what basically amount to free cakes. You can start writing Scala like Java,¹ and you’ll get the benefit of it just having a nicer syntax. You get to write better OO code: you can use traits, case classes and the like. You get nice toys like scala.xml and the pimp my library pattern (is it a pattern now? This is basically Ruby’s monkey patching adapted for static typing aficionados).

But then you slowly start getting seduced with vals and closures and that apply method, and before you know it your software is filled with singletons with complex apply functions that return functions that return functions which return functions which return immutable data structures. And you’ve snuck all that into some monstrous Java enterprise system, and nobody really knows because it is just a war file. That’s the theory anyway.

I’d love to hear other experiences of what it has been like to learn functional programming or suddenly realising that despite not knowing big words like ‘referential transparency’, you already know FP.

Hack diary: Readability golf

Day of the week is a simple little problem but allows for some really good code-golfing (at worst) and sensible code improvements. I’m not wild about the Scala solution though. It uses for-loop comprehension which hurts my head. I’m okay with list comprehension in Python in some circumstances, but I find that stringing together higher order functions is much more readable than list/loop comprehensions.

Here’s a cleaned up version of the for-loop version that uses an anonymous function instead of being in a main method:

(start: Int, end: Int) => 
  for (year <- start to end;
       date <- Some(new GregorianCalendar(year, DECEMBER, 25));
       if date.get(DAY_OF_WEEK) == SUNDAY)
  yield(year)

The logic here feels smushed together. Creating the array feels like a separate step from filtering them on the basis of the conditional. This is very efficient to write if you only have to do it once and there is no plan to ever refactor it.

Here is how it looks if you swap the for-loop comprehension for higher-order functions:

(start: Int, end: Int) => (start to end) filter {
  (x: Int) => new GregorianCalendar(x, DECEMBER, 25).get(DAY_OF_WEEK) == SUNDAY
}

For both, you have to import the Calendar and GregorianCalendar libraries from the Java class library:

import java.util.{Calendar, GregorianCalendar}
import Calendar._

The reason I prefer the higher-order function version is simple. It clearly separates the construction of the array (technically, the immutable inclusive range – scala.collection.immutable.Range.Inclusive with scala.collection.immutable.Range.ByOne) from the filtering of that range. One could abstract this out into two separate functions: one which constructs the range and the other which checks whether in a particular year December 25 falls on a Sunday.

Once you’ve finished refactoring it into separate functions and wrapped it all in a singleton object, you get something like this:

object YuletideRule {
  import java.util.{Calendar, GregorianCalendar}
  import Calendar._

  def christmasFallsOnSunday(year: Int) = {
    new GregorianCalendar(year, DECEMBER, 25).get(DAY_OF_WEEK) == SUNDAY
  }

  def checkYearRange(start: Int, end: Int) = {
    (start to end) filter christmasFallsOnSunday
  }

  def printYearRangeReport(start: Int, end: Int) = {
    checkYearRange(start, end) foreach println
  }
}

I think most people can understand this, even if they aren’t Scala or Java programmers. You could make it more readable for non-Scala programmers by explicitly adding the return type annotation on the christmasFallsOnSunday method so they know it returns a boolean.

Similarly, those not used to functional programming might find the infix-with-no-punctuation syntax a bit strange. You can change checkYearRange(start, end) foreach println to something more explicit like checkYearRange(start, end).foreach(println), but I find that the more Scala I do, the more this Haskell-style syntax is appealing. That said, I find I can only tolerate it in small amounts, and it certainly looks strange when I see lots and lots of it piled up.

Abstract Type Members versus Generic Type Parameters in Scala →

If anything will prevent Scala from mainstream adoption, it is reading stuff like this. I’m sure it all makes sense. Just not to me. It rather makes me want to move back into the slow lane and write in dynamic languages.

Notes on Play from LSUG

Just archiving my tweets from the London Scala User Group this week: @vigosun gave a talk about Play and this is what I had to say.

@vigosun is coming from PHP: standard Java frameworks (Spring) not welcoming. [tweet]

play is not the “traditional JEE stack” [tweet]

Play is a platform, “not just a glue framework”. “Usability first”! [tweet]

Play features: stateless, RESTful, no-compile, efficient error reporting, built-in test framework, templates [tweet]

In short, Play is everything that makes Rails cool but for Java and Scala. [tweet]

the MVC is basically the same in Play as it is in Rails. [tweet]

the model in Play is rich, not anaemic. [tweet]

testing in play: unit, functional, Selenium, coverage (Cobertura) and fixtures using YAML. Easy for rails users! [tweet]

backgrounding: scheduled jobs, bootstrapping, suspendable reqs using futures [tweet]

interesting modules: CRUD (like Django admin, Rails scaffolds), AppEngine, MongoDB. [tweet]

support for i18n looked good in Play. [tweet]

IDE and editor support for play includes Textmate. [tweet]

play is David Pollak’s second favourite Scala web framework. Hehe. ;-) [tweet]

scala in play was hard: wanted to keep API interop, but also get benefits of the language. [tweet]

benefits of scale for play: 2.8, scalatest, scalate, REPL and Akka integration. [tweet]

@vigosun is demoing building a basic app in Play using scala and eclipse. Play comes set up with H2 like Rails does with sqlite. [tweet]

I spy annotations: my least favourite java language feature. ;-) [tweet]

class Foo extends Model - how it bloody well ought to be. [tweet]

object Foo extends QueryOn[Foo] [tweet]

damn, the Model system in Play looks shockingly simple. [tweet]

the templating language that is begin used with Play looks pretty awesome. [tweet]

not sure about the REST cred of Play: will have to look into accept and so on. [tweet]

“Start using Play! Java now” [tweet]

play is the only java framework that doesn’t induce despair in me. [tweet]

Will I try Play? Yeah. Looks like it might be pretty good. I’d like it if you could use it for non-web projects too! I mean, the simple setup of database stuff, for instance.

Rubyist propaganda. When I can connect to a database in a way that won’t be embarrasing around the dynamic folk, it may be time for some Scala propaganda posters.

Scala dev diary: wherein traits become the new mock objects

I’m having a lot of fun playing around with Scala. It really is a truly wonderful programming language. Scala for Hackers had a recent surge of popularity with links from Hacker News, Reddit, Twitter and so on. I’ve put most of my thoughts in there, but I wanted to share my thoughts on two of them for you. Both relate to the intersection of the things I like about Ruby and the things I’m not so wild about in Javaland.

First thing is mock objects - or rather, fake objects. I don’t usually like making mocks, stubs and fakes. It feels suspiciously like too much work. Really. The only thing I really want from that whole class of mocking, stubbing and faking is a way to replace expensive, non-deterministic or otherwise temperamental calls with fake calls. That means that when the object calls out to the database or across the network - calls which may fail due to the database being down or the network being congested or whatever - my tests should just be able to presume that they are fine.

I played around with Mockito. It should be pretty simple, right? In Ruby, I have RSpec and Mocha. In Scala, I have Specs and Mockito. And Mockito is good for a lot of things including spying - if you want to see whether certain methods get invoked and so on, that’s fine. But I wanted something simpler. I just want a fake object for those pesky network and database calls. Enter traits!

Ruby has mixins, right? Well, Scala hs traits. And because Scala lets you define code anywhere - more like Ruby and less like Java - you get some real nice benefits. Check this out.

First, we define ourselves a class. I’m calling it Foo, of course. But we should imagine it is some kind of web service. It uses a method call called ‘http’ to call out to the web and get something we want. To make life simple, let us imagine it returns us some XML data. Okay. Simple enough. So, here is Foo.

It’s a pretty dumb class. Obviously, it needs the full Java makeover - some Javadocs, ten trillion import statements, a few cryptic annotations and to declare that it implements lots of interfaces. We’ll leave that for another day.

Because we want to unit test Foo. But we don’t want that method ‘getFromApi’ to call out to the network. In our test file, we create a trait called FooMocker which inherits from Foo.

The FooTest object should be pretty simple to understand for anyone who has used something like RSpec or another BDD-style testing framework (JBehave, NBehave and lots of other things with the word ‘Behave’ or ‘Spec’ in the name). Ignore that the test isn’t very well written. That isn’t the interesting bit. The key bit in the FooTest object is the ‘mustBe’ bit. For those who haven’t read Scala before, line 9 has no special compiler magic - it just switches between the two different call syntaxes because (despite what the Pythonistas tell you) there being more than one way to do something can make it more rather than less clear. Heh.

So, line 9 is just doing an ‘assert equals’ test. But check line 8. Here we are instantiating a new Foo object but crucially with the FooMocker. FooMocker contains an override for Foo. Again, this is like Ruby’s use of module mixins, but on an object. Being able to say “create a new X with this trait Y” is really quite powerful. And it is type-safe too. You can’t add FooMocker to anything but Foo or something that inherits from Foo.

Traits are really quite an interesting beast: for Rubyists, the best way to think about them is that they provide mixin inheritance. If you are coming from Java, think about them as super-duper interfaces. What makes them a super-duper interface? Well, they are an interface that can be mixed in at instantiation time rather than class definition time. And you can optionally include implementation or leave them abstract, or any mixture of the two. Really, traits are amazing. Go read Scala for Java Refugees Part 5: Traits and Types or Scala for C# programmers, part 1: mixins and traits. And then read Scala’s Stackable Trait Pattern.

Testing is pretty easy then. Specs and some mixture of Mockito and faking fake objects using traits gives me a pretty nice testing setup. I can even do some concurrency testing with Specs using the ‘eventually’ construct. And with the choice of Scala’s actors model, Akka, HawtDispatch and the Java concurrency model, that’s pretty flexible too.

I’ve ranted about Maven before: sbt really is Maven done right. If you are doing Scala, just use sbt. That is all that needs to be said about build tools. I like XML more than most people and I can’t stand XML configuration files.

The giant elephant that is left is database abstraction tools. Databases are boring, and I can’t for the life of me find any way of making them interesting. If I have to, I’ll use PostgreSQL. But I’ve spent the last 24 hours doing nothing but getting frustrated with databases and the sheer crapness of the abstractions we have built on top of them. I have a very simple task: I am pulling some XML off the ‘net every so often (not an RSS/Atom feed - a custom thing from some .NET app) and I want to store the data inside for future reference. I am not bothered whether it is stored as XML or whether I explode the XML into some relational tables. I don’t even object to using some fancy NoSQL thing. If all else fails, I may just cache the XML on the filesystem - the consequences of losing the XML files is minimal and the performance isn’t an issue. But I figure I want to know how to do database abstractions easily.

In Ruby, I have access to three quite nice ORMs. There’s obviously ActiveRecord which is built into Rails. There’s also DataMapper. And there’s even the super lightweight Sequel. Sequel is particularly lovely as it seems optimized for very lightweight data hacking and can be used in ‘shell script’ type scenarios.

Look at the Sequel homepage. It has clear documentation of how to get started on the homepage. ActiveRecord isn’t quite as friendly - it is in an RDoc frame - but still, it has example code right there on the homepage. DataMapper and Sequel have clear documentation that shows you how to go from basically a blank sheet of paper to a working database connection.

Now have a look at Autumn, a Python ORM. Or SQLObject or Storm’s tutorial.

People have told me to look at Spring. I’ve looked at the web page and I have no idea what it is. Apparently, it is the leading platform to build and run enterprise Java applications. Right. I want a piece of software to bridge between my object-oriented code and my database. Building? That’s done by my compiler. Running? That’s the JVM’s job. Enterprise? No thanks.

Okay, what about Hibernate? Relational Persistence for Java and .NET sounds like what I want. So I go to the Getting Started page. Note the filename: quick-start.html. Quick start! I like that. Okay, what do I have to do?

Make sure the appropriate JDK is installed. Hibernate Core only requires JDK 1.4 at the time of this writing, but to make use of JPA, annotations, etc you will need to use JDK 1.5 or JDK 1.6.

Okay, I’m running JDK 1.6. OS X comes with JDK 1.6 so that’s good. JPA? I’m a newbie, but I do know that stands for Java Persistence API. I look at Wikipedia and I spot all the words that people have warned me about: annotations, XML descriptor files, Enterprise JavaBeans, EJB containers, JSR 220. No, I just want to talk to a database. I just want a quiet life. Blue pill! Blue pill! I don’t want to follow the white rabbit down the rabbit hole. Haven’t got time for this.

Make sure the appropriate Maven version is installed.

No. I’m not playing if I have to use Maven. But, you know, the documentation could be lying to me. I’ve been told this kind of thing before. So, screw the prerequesites. Let me just get to the documentation. A few clicks later and I’m here.

Let’s see. Using HSQLDB. That’s fine. Maven dependencies. Urgh. It is not a requirement to use Maven. Hold on. Why on the quick start page does it say that I need to make sure I’ve got the appropriate Maven version installed if it isn’t a requirement that I use Maven? Then it says I have to create a hibernate-mapping XML file. Gah. No. And then I need to make a hibernate-configuration file which is another XML file. hibernate.cfg.xml. Then mvn compile.

XML configuration files? No. Annotations are out too because of the Scala 2.7 nested annotations problem. Annotations seem to only shunt the problem along from XML into being in a sort of pseudo-extended limbo. Annotations are a metadata format as far as I’m concerned - why am I storing key details about how my data gets storied in an annotation? What if I decide that I want to use another JVM language? Where is the portability? As Ola Bini says: Java Annotations are NOT a way to implement DSLs, and will never be.

The sad thing is that there is lots of good stuff hidden away in Hibernate. I’m sure of it. Loads of people rave about it! Given the design constraints of the Java language, culture and toolchain, I’m sure Hibernate is actually pretty awesome. I’m sure there are enterprise developers out there who have fought tooth-and-nail to use Hibernate rather than something even more complex.

And so it is with the rest of the ORMs I looked at. MyBatis? XML configuration files. This feeling of ennui eventaully sets in. This kind of complexity is not sane. It’s a sign something is fundamentally wrong.

So I started looking at Scala native ORMs. And there’s a few to choose from here.

scala-query looks very nice. Have a look at some of this test code. It does look very nice, but it isn’t really finished yet. tbje seems to have taken it over just in terms of keeping the pom.xml up to date. Using it currently means compiling it yourself, building your own JAR file and using that. And it isn’t particularly compatible with 2.7, so you’ve got to pretty much use the bleeding edge 2.8 RC6 to use it.

I tried it out in a blank sbt 2.8.0 RC6 project and I just got very, very cryptic exceptions. Like this.

So, what else is there to explore? Well, the next thing I’m planning to try is Lift’s ORM layer, lift-mapper. You can apparently use it without using the rest of the Lift framework - you just pull the bits out of the tutorial that do what you need. It isn’t quite as pretty as the scala-query code samples and certainly not as simple to get one’s head around as the ORMs used in dynamic-land (man, Sequel is just so lovely). Lack of things like method_missing make it so things like ActiveRecord’s crazy find_by_whatever type methods just aren’t easy. But the fact that it is running inside Lift and Lift is running inside things like Foursquare gives me some confidence that it might actually work! Heh.

There is an alternative, now, of course. The various NoSQL stores. While everyone in the Web 2.0 world is using NoSQL stores for performance and scalability, I don’t see why they can’t scale down in the other direction - to the sort of “prosumer” scripting language crossover market that Scala is partly aiming for. And, well, there has been some nice libraries for NoSQL stores. Cassandra and MongoDB are supported by Akka - there’s a guide here. The sort of thing I’m doing would fit pretty well into the CouchDB model. And there are some CouchDB libraries in Scala: scouchdb by Debasish Ghosh, Lift has the lift-couchdb module and the slightly cryptic but always awesome Databinder Dispatch has a CouchDB module - see this Specs test to see usage.

That’s an interesting prospect: using a language where it is much easier to connect to a Cassandra or CouchDB store than it is to connect to an RDBMS. As someone who can see the benefit in using both traditional RDBMSes and the new range of NoSQL stores, this disparity makes me slightly uneasy. I haven’t looked into using an XML store either. Despite my repeated complaints about XML configuration files, I have no objection to XML itself. Something like eXist may do the job. Problem I have with all these things is that they don’t scale down. The nice thing with Rails’ sqlite integration, and the great thing in theory about HyperSQL/H2 in Java is that it is supposed to be moronically easy to get going. Simple persistence still feels like it isn’t there yet, even if a lot of the Scala pieces are coming together. The future of the JVM isn’t in more XML configuration files and JavaEE and EJB type stuff. It can’t be, can it? If it is, a lot of Rubyists and other converts from dynamic-land may find ourselves in a bit of a pickle.

One last thing. I use sbt. You should use sbt. sbt sbt sbt. Yay sbt. Everyone loves sbt. Okay. Now, if you want to find Java libraries, a lot of the time mvnrepository.com is where you go. It tells you version numbers, repository details, all that good stuff. And, even more conveniently, for Maven, Ivy and Groovy users, it has copy and paste samples. When at Science Hack Day last weekend, I asked Premasagar Rose (@premasagar on Twitter) from Dharmafly if he could put together a Greasemonkey script for us SBT users. And he has! It’s up on userscripts.org. Greasemonkey has always been available for Firefox, and there is support and plugins for Greasemonkey in Opera, Google Chrome, Safari and even Internet Explorer. With this, if I hear about a Java library I can just search in my browser with something like “mvn jawesome” and it’ll take me to a page about jawesome, then I click through to the version I want and can copy some of sbt’s libraryDependencies syntax and paste it into my project file. One less time when I have to deal with XML configuration files - and it means I can try out libraries much quicker. Thanks, Premasagar!

Dustin Sallings explains why he doesn’t use Maven. Maven 2 might be the least pleasant piece of software I have ever used. For Scala users, there is sbt. For almost anyone who uses Java-based languages, there is Buildr. They rock pretty hard and require no XML configuration at all.