Category Archives: Scala

Giving talk about Swarm in Seattle next month!

I’ve been asked to give a keynote at IEEE P2P’09 in Seattle in September, and I’ve decided to talk about Swarm. Swarm is my proposal for a new way to build applications that allow you to scale them across multiple servers in a way that is almost transparent to the programmer.

In preparation I’m currently working hard to develop a prototype that makes use of the upcoming delimited continuations support in Scala 2.8. My talk will be at 9:30am on Wednesday 9th September.

You can read more about Swarm here.

Rediscovering Standard ML

Back when I was studying computer science in Edinburgh (1995-99), we all learned the ML programming language, specifically Standard ML, which was developed in my CS department in the 70s.  Even though I think most people on my course quickly decided they didn’t like it, it grew on me as I became more familiar with it (and it seemed pretty simple relative to Prolog, which we had to learn in our AI courses).

I hadn’t used it since I left university, but given the recent resurgence of interest in higher-level programming languages, with features like type inference, such as with Scala and F#, I decided it was time to indulge in a bit of nostalgia, and refresh my memory.

At university we had a book called ML for the Working Programmer, which I remember being pretty good, but I’d long-since lost it, so I went onto Amazon and purchased it again.

I then downloaded Standard ML of New Jersey, a popular and powerful SML compiler/interpreter. I had previously considered Caml, an alternate dialect of ML to Standard ML, but I couldn’t really get used to it, and I liked the fact that SML supports continuations, a feature useful for building highly concurrent applications.  Furthermore, there is MLTon is an extremely powerful “whole-program” SML compiler that reputedly produces extremely efficient code.

So what does SML look like? Here is a simple recursive function to count the elements in a list:

fun count([]) = 0
| count(h::t) = 1 + count(t);

SML relies on pattern matching. The first line tells SML to return 0 for an empty list. The second line will only be reached if the first line doesn’t match, and will only match if the list has at least one element. It splits the list into its first element, called ‘h’, and the remaining elements, called ‘t’ using the ‘::’ operator. It then returns 1 plus the number of elements in the tail.
After typing this into SML’s interpreter, SML produces this:

val count = fn : 'a list -> int

This is the type inference at work. Without specifying a single type, SML has inferred that the function count takes a list of type ‘a (pronounced “alpha”) which can represent any type, and returns an integer.  Note that SML has correctly inferred that the type of the elements in the list passed to count is irrelevant for the purpose of counting them.

Lets try something else, a sum function to add up the elements in a list:

fun sum([]) = 0
| sum(h::t) = h + sum(t);

The general structure is similar to count, but this time SML infers different types:

val sum = fn : int list -> int

As expected, it has inferred that the list must be a list of integers.

SML is a powerful language, and I’d recommend it for anyone keen to broaden their knowledge of programming languages.

Unlike Haskell it is “impure”, meaning that functions can have side-effects. Some people seem really hung-up on this idea of purity, but so far as I can tell it is more of a fetish which makes life more complicated, and with few practical benefits (whenever I ask people to explain what is so desirable about purity, the response is generally rather theoretical or hand-wavy).

I also think its nicer than Scala in some ways, for example in ML you can do pattern matching on function definitions, rather than having to use match. And of course, its way more mature than Scala.

If you are going to try it, I suggest using Emacs together with sml-mode.

A contrarian perspective on Scala

As you will notice if you read through my last few posts, I’ve been exploring Scala, a relatively new programming language that combines most of the nice features of languages like Ruby and ML, with the ease of deployment of Java.

I’ve been evaluating it with respect to a new project I’ve been pondering, and have concluded that its not ready yet. There are two key reasons.

The first is that the implementation of Scala’s standard libraries is still unstable. Within days of first discovering Scala, I’d found my first bug, a rather fundamental flaw in Scala’s JSON parser. This suggests to me that Scala’s standard API lacks even basic unit testing (this despite the existence of a variety of seemingly powerful testing technologies, such as Specs). I filed a bug for this problem here.

Its hard to justify basing a project on a language where the standard libraries cannot be relied upon to work. I’m afraid that Scala really need to establish some better quality control for the code that makes it into the standard library.

Secondly, the tool support isn’t there. This may seem like an unfair criticism, but I strongly suspect that most potential Scala users are Java refugees, and said refugees are accustomed to powerful IDEs like Eclipse, with features like code completion. You get addicted to features like this, and regardless of its merits, a language that requires you to forego them is starting with a big negative.

Scala has two Eclipse plugins, a more stable one without proper code completion (which crashes on me about once a day), and a less stable beta plugin with code completion, which crashes more often. This beta is more promising, but development seems to have stalled since its last release a few months ago, and the developer is working hard to lower people’s expectations. The bottom line is that neither is production quality, and they will waste your time with mis-reported errors and regular crashes.

Scala has incredible potential, it is the natural migration path for those frustrated with Java’s verbosity and inflexibility, but its not ready for mainstream usage yet. Anyone that opts to use it for a large project today is thinking with their heart, not their head, and is jeopardizing their project.

Scala: The best of both Ruby and Java

Many programmers love Ruby, they just can’t get enough of it. Its probably one of the most aggressively evangelized languages since Java first came on the scene. They generally cite features that include a flexible and extensible syntax, closures, and how concise and expressive the code can be.

For example, you can create a Map (Ruby calls them “hashes”, even though a hashtable is only one way a map can be implemented) with a simple syntax like:


numberMap = {"one" => 1, "two" => 2, "three" => 3}

The Java equivalent of this is far more verbose:


Map<String, Integer> numberMap = new HashMap<String, Integer>();
numberMap.put("one", 1);
numberMap.put("two", 2);
numberMap.put("three", 3);

The Java version does have one advantage though, which is that we’ve told the compiler that numberMap uses Strings for keys, and Integers for values. If we try something like:


numberMap.put(4, "four");

The compiler will complain, or if you are using an IDE like Eclipse, you will be told there is a problem almost the moment you type this. This is called “static typing”, or “compile-time type checking”. Ruby’s approach is called “dynamic typing” or “run-time type checking”.

In Java you must painstakingly tell it the type of each variable you use, and this is one of the main reasons Ruby is more concise and easier to read than the equivalent Java.

Java has another advantage, which is that it is far faster than Ruby. In some benchmarks Ruby requires over 15 times as long to perform a CPU-intensive task as Java (and this has caused real life problems for some Ruby users).

So where does Scala come in? Lets look at the map example in Scala:


var numberMap = Map("one" -> 1, "two" -> 2, "three" -> 3)

You will notice that it looks very similar to the equivalent code in Ruby, but there are some important differences. In particular, just like Java, the Scala compiler knows that numberMap uses Strings as keys, and Integers as values. Unlike Java, you didn’t have to tell it this, it just figured it out for itself! This is called “type inference”.

This means that if I try adding a new key-value pair to numberMap, but using an Integer for the key, and a String for the value, Scala will complain as soon as you try to compile it (or your IDE will warn you immediately). With Ruby, it is only likely to cause a problem when you run your software and try to retrieve this key and value from the Map and get an Integer and a String, rather than the String and the Integer that you were expecting.

It is hard to overemphasize what a timesaver compile-time type checking can be, it eliminates a whole class of bugs that would otherwise occur at runtime, and Scala brings you the benefits of this, without the verbosity.

Scala also has a bunch of other nice Ruby features, which Java lacks, including closures, and a very malleable syntax that makes it well suited for “domain specific languages”. It has all of these features, and combines them with the benefits of static typing.

But there is more. When you compile some Scala code, it is compiled to Java class files, and can be run on a Java Virtual Machine just as if it was compiled Java code. This means that Scala can go wherever Java can go, it can also use any of the vast number of libraries that have been built for Java, and it can take advantage of today’s very efficient Java runtime environments. Because of this, compiled Scala is almost as fast as compiled Java!

Scala is still quite young. Its own libraries are impressive (especially the scala.actors library), but I’ve found one or two bugs already (in the JSON parser lib), suggesting that its still somewhat immature.

Nonetheless, for anyone yearning for the advantages of both Ruby and Java in one language, you should definitely take a look!