Thursday, December 3, 2009

String interpolation in Scala

When I tried Scala for the first time, one of the first missing language features I noticed was string interpolation (you guessed it, I was using Ruby at the time). Of course, this is a small convenience rather than a major language feature and usually String.format and java.util.Formatter are pretty good substitutes. Still, string interpolation comes now and then in Scala discussions and one has to wonder if Scala's powerful language extension facilities can emulate a feature like this.

It turns out you can get reasonably close:

trait InterpolationContext {
class InterpolatedString(val s: String) {

def i = interpolate(s)

implicit def str2interp(s: String) = new InterpolatedString(s)

def interpolate(s: String) = {
val sb = new StringBuilder
var prev = 0
for (m <- "\\$\\{.+?\\}".r.findAllIn(s).matchData) {
sb.append(s.substring(prev, m.start))
val matchString = m.matched
var identifier = matchString.substring(2, matchString.length - 1)
try {
val method = this.getClass.getMethod(identifier)
} catch {
case _: NoSuchMethodException =>
prev = m.end
sb.append(s.substring(prev, s.length))

object Test extends InterpolationContext {
val a = "3"
def main(args: Array[String]) {
println("expanded: ${a}; not expanded: ${b}; more text".i)

How does this work? If you call the i method on a String, it will force implicit conversion to a InterpolatedString, similar to how the r method works for converting to a Regex. The interpolate method uses reflection to find a method with the name equal to the identifier delimited by "${" and "}". This works both for vals and defs due to Scala's adhering to the uniform access principle. If the delimited string is not a valid identifier or a parameterless method with this name doesn't exist, it is not substituted, but stays as it is.

How do you use it? Just extend or mix in the InterpolationContext trait in the class where you want this to work, and then call i on the Strings where you want interpolation to work. The InterpolationContext trait serves two purposes- first of all, it imports the implicit conversion to the interpolated string, and second it provides access to the methods of the current object via reflection.

The limitations of this method are that interpolation only works on member vals and defs of the current object only. I rather like this, because you know you can't accidentally construct a String from user input in an html form like "Name: ${System.exec(\"echo echo pwned >> .bashrc\")}". Also, interpolation doesn't work for private members as well as local variables. Finally, you have to both mix in or extend the trait and call a method on every String (even though it's a one-letter method). This is not too bad, because you can control the scope where interpolation works and therefore avoid and debug nasty surprises easier.

I don't see this as a pattern, which can be used in real-life projects, and I don't see the use of implicits here worthy of emulation. Nevertheless, trying to copy features from other languages is instructive about the limitations of your current language of choice and the tradeoffs you get for these features. Last but not least- even if it's a gimmick feature, implementing it was fun.


Nilanjan Raychaudhuri said...

Good stuff.
I tried little more scala tricks on your version

Let me know what you think

Vassil Dichev said...

@Nilanjan Using an extractor in your version is a neat idea. It could also be developed further to decompose the match patterns into first and rest, similar to how it works for lists.

I realize my solution wasn't very functional or idiomatic Scala, but I wanted it to work on 2.7 without a lot of modification.

As Daniel Sobral mentioned, my version could be further improved if there was a better replace function on a match:

The initial version was using scala.reflect.Invocation, which made the reflection code a bit shorter, too.

Kota said...

Interesting! But I think that your solution of String interpolation is a little ugly. To achieve the same effect, using XML literal is straightforward:

object Test {
val a = "3"
def main(args: Array[String]) {
println(<t>expanded: {a}</t>.text)

Vassil Dichev said...

@Kota I know my solution might not have much practical value. Using XML literals is cute, but there are several things to have in mind:

* escaping of &, >, < could ruin your day
* you can't defer the construction of the XML literal and the expansion, as it happens with the InterpolatedString
* Constructing the XML element and especially calling text is moderately inefficient- converting toString actually passes through Utility.escape, which needs to check character by character and append one character at a time. For one-off literals you might not care much for performance.

Chas Emerick said...

Nice post. FWIW, I took my own shot at the string interpolation problem, but for Clojure:

String Interpolation in Clojure

I'm definitely a Clojure partisan at this point, but it's interesting compare/contrast what each language offers in the way of extensibility.

Vassil Dichev said...

@Chas That's a great post! Certainly puts things in perspective. I can't argue with your rhetoric about Scala, because it's a matter of taste :)

In Scala, you cannot match the power of macros with standard techniques, but you mentioned correctly that compiler plugins can help here.

At any rate, I find that the use of implicits, at least here, is not more magical than macros. Since we expect the same type after the conversion (String), the implicit will only be triggerred by the "i" method, so it is transparent where it's applied.

Daniel said...

I'm happy to report that replaceAllIn now accepts a function String => String. Also, there's now a replaceAllMatchDataIn, accepting a function Match => String.

Vassil Dichev said...

@Daniel Cool, that will make the code much more compact. Thanks for pushing this feature request!

MonkeeSage said...

Here's a version that works with 2.8.RC1 or better:

  def interpolate(s: String) = {
    "\\$\\{(.+?)\\}".r.replaceSomeIn(s, m => {
      try {
      } catch {
        case _: NoSuchMethodException => None

Pretty nice. :)

Jason Wheeler said...

I took the example in the first post by Nilanjan Raychaudhuri and updated so that it could interpolate properties on objects (ex: ${}, ${obj.members.size}, etc).

I realize that Scala 2.10 will include string interpolation but for anyone who can't upgrade immediately this might help.