[Mulgara-dev] Language discussion

[Paul Gearon]

Hi everyone,

I'm interested in adding a new language to Mulgara, and I'd like to
solicit some feedback please. I don't want to replace anything here. I
just want to add new option to the system.

First, the rationale for this.....

== Rationale ==

Over time I've been writing code that is more and more "functional" in
design (as opposed to "imperative"). The point of this has been
reliability, and theoretically readability. Unfortunately, Java is not
suited to this at all, leading to some horrible constructs,
particularly when generics get involved. The package
org.mulgara.util.functional has been working well, but using it is not
as simple as it should be. For instance, the simple operation of
mapping a list of commands to a list of results (either Answer or
String objects) is currently being written like this:

  results = C.map(cmds, new Fn1E<Command,Object,Exception>() { public
Object fn(Command cmd) throws Exception { return
cmd.execute(connection); } });

Instead, this should look more like:
  results = cmds.map(cmd | cmd.execute(connection))

(I intentionally didn't use a specific language syntax here, but you
should get the idea).

Type information is starting to get out of hand as well. Generics have
offered a lot of great documentation and reliability to the codebase,
but Java's implementation has problems with complex constructs (it
can't always deal with "extends" correctly) and with redundancy. For
instance, after adding generics to one map (that was completely
undocumented) we got:

  Map<ConstraintElement,Map<ConstraintElement,Map<ConstraintElement,List<ConstraintElement>>>>
varArgsByModel =
HashMap<ConstraintElement,Map<ConstraintElement,Map<ConstraintElement,List<ConstraintElement>>>>();

This is actually simplified a little, but the more typesafe construct
ran into inferencing issues with <? extends ....>.


== Comparison ==

The JVM is a major strength for Mulgara, and I have no desire to
change this, but there are several other viable languages that could
address some of these issues, and I've been looking at them for a
while. My criteria, in approximate order of importance has been:
- Runs on the JVM.
- Stable
- Language contains closures.
- Seamless integration into Java codebase (easy to call into and out of).
- Performance.
- Popular (so others can work with it)
- maintains type information

The languages I've looked at are:
- JRuby
- Groovy
- Jython
- Scala

There are others, but these ones all meet the popularity requirement,
and there is a limit to the number of new languages I can look at. :-)

I came close to JRuby, especially given the optimizations and tight
JVM integration that the JRuby team were able to achieve when they
started working at Sun. However, the inconsistencies in the language
(lambda vs. Proc, begin/end vs. {}, etc), along with the lack of type
information made me uncomfortable with it for this application. Groovy
is a similar language, but with trivial Java integration. However, I
never felt it was as mature and efficient as JRuby has become (Ronald
may disagree here), and also has similar type issues.

I relied more on reviews when it came to Jython. It has been around
the longest, but also the language I've spent the least time with. The
main impression I get from the community is that this is a good system
for doing transitional work, but ultimately you want to commit to
either Java or Python.

But in the end, the language I've decided I want to go with is Scala.
This is the language that has been getting the most press lately,
particularly on programming blogs and in Twitter. In particular, I was
seeing a lot of positive comments from people who I respect a lot. I'm
still not an expert in it, but the work I've done with it so far has
been *very* productive. Stability appears to be very good: basically,
if it compiles it will run correctly, and I've yet to see something
fail to compile. Performance is fine, which appears to be a concern
with some of the other languages. Interoperability with Java is
trivial. While it wasn't a specific criteria, I was also impressed
with the consistency of design - once you know the rules, special
cases are almost nonexistent. So it's looking good on most fronts.

The thing that really stands out for the language is the type
inferencing. This means you have all the benefits of a strongly typed
language (and I'm not going to argue them here, but I totally buy into
it), but without the coding overhead present in Java. Type information
can often be fully inferred, which can make a lot of coding look as
type-free as Ruby code. But the full type system is still present. For
instance the Map example I gave above would use both explicit typing,
plus some inferencing to look like this:

val varArgsByModel =
Map[ConstraintElement,Map[ConstraintElement,Map[ConstraintElement,List[ConstraintElement]]]]();


== Comments? ==

OK, so now you've seen my rationale, I'm interested in soliciting
other opinions.

Do people think it's a good idea to introduce a new language? A bad
idea? Are you ambivalent? Remember, a new language will be operating
alongside Java, and not replacing anything. Also note that all the
systems I've looked at will introduce new jars to the lib directory.
(Scala would introduce 10MB of jars)

If you agree that a new language should be added (or you don't mind if
one is), then which language do you want to see? What features of your
preferred language do you think are compelling?

My own preference is for Scala (as explained above). Have I missed
anything about this system that you think I should take into account?
Do you have criticisms about Scala in general? (or any other language)

Regards,
Paul Gearon