Learning Scala: Compiler Surprises

Scala seems to be getting a lot of publicity lately. I’m glad for that, because the more I learn about it, the more I realize that it’s a really great language and needs more attention. Some other bloggers have already written introductory articles and talked about various aspects of Scala, but maybe I’ll have something to add. I’m planning on using Scala alongside Java in personal projects, but at the moment I’m just learning and experimenting, writing short pieces of code to try out the various language features.

Coming from Java programming, even after learning the basics and perhaps some of the advanced concepts of Scala, one can run into some really surprising compiler error messages because of the way Scala works. I ran into one of these issues and thought this would be an interesting way to illustrate some features of Scala. Note: for clarity, I’m using type annotations in this blog entry where I would normally leave them out.

After executing the following code, it’s natural to expect that i will have the value 13, as we’re used to in Java, C++ and other languages:

var i: Int = 1
i += 10 + 2

In Scala, the above code will actually produce a compiler error:

error: type mismatch;
found : Int(2)
required: String

My first reaction: what the hell? There’s nothing to do with Strings here, there are only Ints! It’s easy to get sidetracked looking for the wrong answers if this kind of error happens in some larger context with type inference involved. But this comes from still thinking in Java or C++ mode, seeing operators and assignment operators. Lets switch to the Scala mode of thinking, and try follow what the compiler does with this code, if not exactly.

Firstly, operator precedence in Scala is defined by the first character of the operator name, so += and + actually have the same precedence. The exact rules can be looked up in ScalaReference.pdf or other documentation. So from left to right, += is applied first. And since operators in Scala are actually methods of the operands, this translates to

(i.+=(10)).+(2)

Now we are already starting to see part of the problem — the parentheses ended up not where we wanted, so in this case, we actually need to use explicit parentheses in Scala, while we would not in many other languages: i += (10 + 2), which expands to i.+=(10.+(2)). Well, that was quick, we have solved the problem! But the original error about requiring String is still a mystery. Lets continue until we get to the bottom of this.

As said above, operators in Scala are methods, but of course the assignment operator = is an exception — otherwise you couldn’t assign anything to a new variable. Operators ending with = are treated specially as well — they are “expanded to assignments if no other interpretation is valid” (as quoted from ScalaReference.pdf). If the left operand has the method +=, then that method is called, otherwise l += r will be expanded to l = l + r (actually it’s a bit more complex, but for simplicity lets assume that this is the only rule). For the Int class, the method += does not exists, so i.+=(10) is expanded to i = i.+(10).

The full expression is now: (i = i.+(10)).+(2)

Lets assume that the first part of the expression — including the assignment — is correct (it is). So now we are left with trying to add 2 to the result of the assignment. Assignments in Scala return the value () of type Unit — which is analogous to Java’s void. So the rest of the problem lies in this simplified expression:

().+(2)

The Unit type doesn’t have a + method, and this causes the compiler to look at implicit conversions to other types that do define this method. That’s right! Scala lets you define your own implicit conversions between types. And more interestingly: they are scoped. Some of these conversions are already defined in the Predef object. All members of Predef are implicitly imported into every Scala source file, so the conversions defined there are always in scope. On with our example: since Unit (as all other types in Scala) is a subtype of Any, the compiler finds the only usable implicit conversion to a type with a + method:

implicit def any2stringadd(x: Any) = new runtime.StringAdd(x)

Finally we are seeing something about String! What this line does is define a conversion method that converts from Any (and its subtypes) to StringAdd. A call to this method can be inserted implicitly by the compiler where a StringAdd is needed, but an object of another type is given. The method name is arbitrary, but naming it like this is a convention. Lets look at the StringAdd class:

final class StringAdd(self: Any) {
def +(other: String) = self.toString + other
}

This class takes Any as a parameter and defines a method + that converts the class parameter to a String using the toString method and then concatenates it with the argument. Together with the implicit conversion, this effectively adds a String concatenation method to every type, making it easier to write expressions like this:

var s: String = 1 + " minute"

Instead of the somewhat silly looking things you can do in Java:

String s = "" + 1 + " minute";
// or
String s = String.valueOf(1) + " minute";
// or, for those who still refuse to believe that the javac compiler actually does optimize string concatenation to the same as below:
String s = new StringBuilder().append(1).append(" minute").toString();

Update: I was wrong about this — 1 + " minute" does work in Java as well, as szeiger pointed out in the comments.

Back to Scala… the compiler will insert the implicit conversion, expanding 1 + " minute" to any2stringadd(1).+(" minute"), and if we inline that method for illustrative purposes (as far as I know the compiler doesn’t actually do this), we get new StringAdd(1).+(" minute"). Similarly, our problematic expression becomes:

new StringAdd(()).+(2)

And finally the reason for the original error is visible to us! We are trying to pass an Int to a method that takes String as an argument. Fully expanded, the original code looks like this:

new StringAdd(i = i.+(10)).+(2)

While this is what we intended:

i = i.+(10.+(2))

So, there are definitely some surprising differences when starting to use Scala, but I am not seeing these as problems most of the time — just differences caused by awesome (and occasionally less awesome) design decisions. Certainly differences that need getting used to, but I think it will be worth it.

P.S. This issue was actually brought up in the Scala users mailing list today, but I hadn’t read it when I wrote this post. It turns out that it’s already recorded in the Scala issue tracker as ticket #92. Personally, I don’t have a strong opinion on whether this should be fixed or not, but if you do, you could leave comment on that issue (you must register first, though).

Advertisements

6 thoughts on “Learning Scala: Compiler Surprises

  1. Actually, the string concatenation is just as easy in Java:

    String s = 1 + ” minute”;

    Chained “+” operations are always considered as a whole and if there is at least one operand of type String, a string concatenation is performed.

  2. I just checked the first edition of the JLS. It’s always been the same but it turns out my earlier description was wrong, too. In fact, ‘+’ operators are evaluated strictly from left to right with no look-ahead to detect string concatenation. But string concatenation occurs when *either* the left or right hand operand is a string. That’s why the 1 + ” minute” -> “1 minute” example works. “foo” + 1 + 2 also gives the expected result “foo12” but 1 + 2 + “foo” gets evaluated as (1 + 2) + “foo” and gives the result “3foo”. The somehwat ugly workaround “” + 1 + 2 + “foo” enforces a string concatenation so you get “12foo”.

    I don’t think I have ever encountered such a situation (with 2 non-string objects at the beginning of a string concatentation) in all my years of Java programming.

  3. Pingback: Learning Scala: Performance Impact of Implicit Conversions « Villane

  4. Pingback: Scala: for a/vs while « Villane

  5. Pingback: help me learn french

Leave a Reply

Please log in using one of these methods to post your comment:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s