We do hiring interviews on Open Source Fridays. Most of a candidate's time is spent on pair programming, whiteboarding, and giving a presentation to the team. Here is an example question from a whiteboarding session:

Background: Mark Watson has proposed a simple solution for packaging Java libraries to be IDE-friendly:

users of these libraries (mostly just me :-) only want the highest level APIs to show in popup completion lists and the entire set of implementation classes remain invisible. The solution is easy: a public API class with implementation classes in the same package with package-ony (i.e., neither public or private) visibility.

Mark is cleverly combining the existing visibility mechanisms, plus creating an extra class, to trick IDEs into exposing the right methods for IDE completion.

To generalize this solution, we would need to

• create a named abstraction that solves the problem
• hide the "how" details behind the new abstraction

In this case the new abstraction we need is a custom visibility scope. Java provides four scopes: public, private, protected, and {package}. We want to add new scopes with names like TopLevelAPI. One can imagine many ways to do this in Java. Here are a few obvious ones, building on existing idioms:

• Define an annotation, e.g. public @TopLevelAPI myMethod()
• Define an external class that extends Java reflection. (The JavaBeans API is a perfect example of this. JavaBeans create a custom class member type, property, that augments the existing constructors, methods, and fields.)

In Ruby, you could create a custom visibility scope using object-oriented design. Pick a base class for all classes that need the custom scope. Define a class method named for the scope, e.g.

def self.top_level_api_instance_methods
  # TODO: return array of top level methods
  # TODO: declarative helper?
end

You could reopen Class to provide this feature for all classes. (Rails provides a good example of a domain-specific custom visibility scope, with a slightly different implementation. The assigns are a special scope for controller fields that are automatically visible to the view.)

Questions:

1. The three approaches outlined above would all require IDE collaboration. Compare and contrast the approaches from the point-of-view of an IDE vendor charged with supporting them.
2. The approaches above also require that developers mark certain methods as TopLevelAPI. Compare and contrast the approaches in terms of the work a developer does to take advantage of them.

We are hiring. If you are interested, send us your thoughts on the questions above, or post them in the comments.

The response to my earlier post on testability metrics has been, well, a little negative. I think that the strident tone of the conversation (for which I bear much of the blame) has artificially widened the gap of opinion between me and some people I respect.

So I am going to start again, and fill in the gaps from the earlier post. Along the way, I will answer the interesting questions raised in the comments.

Java is naturally untestable, and that ain't news

Let's start at the very beginning, with the idea that Java is naturally untestable. What does the word natural mean here? That if you use Java naturally, taking advantage of the various features of the language when they are appropriate to the task at hand, you will end up with untestable code. This is not news to anybody, least of all experienced Java programmers. Here are a few quotes from the Java community. First, on the Testability Explorer home page:

The metrics are a calculation of the skill of the development team in making their classes testable.

From the Spring Framework mission statement:

Testability is essential, and a framework such as Spring should help make your code easier to test.

And from the Guice homepage:

You will still need to write factories in some cases, but your code will not depend directly on them. Your code will be easier to change, unit test and reuse in other contexts.

If I can paraphrase these quotes: Easily testable code doesn't just happen naturally in Java. A team must develop skill in testability. In fact whole frameworks have been created, counting testability among their primary objectives.

Java can be tested

Of course, many Java programmers can and do write testable code. As a Java programmer, I learned to avoid certain idioms that are not testable, or to use a framework to do it for me. But this (in my view) isn't natural. I have made the point elsewhere that a language should be richly consistent. If "testability" is a core feature of the language, then testability should be combinable with all the other core features. There should not be a laundry list of special cases of the form: "Don't do X or you can't have Y."

In many other languages, testability is almost free!

The Testability Explorer defines testability in terms of two things that you should avoid:

• unmockable complexity
• global mutable state

As in the earlier post, I will focus exclusively on the first of these: unmockable complexity. (Global mutable state is a discussion for another day.) Also, I am not going to worry about the complexity part. I use cyclomatic complexity measures in both Java and Ruby code, and find them to be decent predictors of code that a human reviewer would find objectionably complex.

What remains then, is the definition of unmockable. The Testability Explorer considers code unmockable if it cannot be overridden or injected, which it defines as (some subset of) constructors, statics, and privates.

By this definition, Ruby has no unmockable complexity:

• Ruby's "statics" are polymorphic to begin with, so they can be overridden just like instance methods. (In fact, they are instance methods.)
• Ruby constructors can be overridden or simply rewritten.
• Ruby's reflection model will let you publicize private members when needed in a test.

Several commenters asked for a patch so they could try out LOW CEREMONY mode for themselves. Easily done: set the unmockable complexity number to zero across the board. I did it by refactoring MethodInvocation.computeMetric.

What do the testability metrics really tell us?

When I figured out how Testability Explorer calculated its metrics, I knew immediately what it would "prove" -- it is one of those metrics whose outcome is driven by its assumptions. If unmockable complexity really is a primary enemy of testing, then the Testability Explorer will quickly "prove" that you should either

1. Use a Dependency Injection (DI) framework (in Java)
2. Use a low-ceremony language that makes mocking trivial

If you want to introduce Spring or Guice to a Java shop that still doesn't believe in DI, Testability Explorer is your friend. It will probably "prove" that your code base isn't testable. Or you can use it like I did in the original blog post to "prove" that Ruby projects are intrinsically testable.

What does the Testability Explorer actually prove? Not as much as one might hope. With the right tool support, the unmockable features of Java become mockable. And even a green project may still be hard to test. This is a problem with metrics in general. Once you understand what they measure, it is all too easy to game the metric, often unintentionally. My colleague Jason Rudolph is giving a talk called How to Fail with 100% Test Coverage that explores this problem.

At Relevance we include two metrics related to testability in our continuous integration builds: rcov test coverage and flog score. Test coverage doesn't measure ease of testing, but insisting on high coverage makes it painful to have complex code. Flog measures method complexity. Unlike Testability Explorer, we treat all complexity as a testability issue.

Java has better metrics, nya nya

Java beats Ruby in support for collecting both of these kinds of metrics. If you want to heckle a Rubyist in a language war, here's some ammunition:

• Ruby's coverage tool only provides line coverage, while Java has tools like EMMA that can give you branch coverage within a single line.
• Java-oriented complexity tools (like pmd) are much more elaborate than flog, and can provide all kinds of specific advice above and beyond a simple numeric score.

A few more responses to commenters

To the heroic defenders of Ant: I didn't offer any of my own opinions on Ant. I merely reported the numbers from Testability Explorer, and the conclusions that project implies with its color scheme. The only reason I even picked Ant was that it came first on the page. :-)

To the attackers of JRuby: I agree that JRuby is hard to test. I have already looked into it, contributed code, blogged about it, and given conference talks on the subject. One interesting positive aspect of JRuby, which in my opinion dwarfs the negative metrics, is that the JRuby committers are almost always online in the JRuby IRC. They will answer your questions, and they are working to make the project better.

Finally: I am not a Ruby zealot. In ad hominem attacks, please refer to me as a Lisp weenie wannabe.

...and the Java community is collecting hard data to prove it.

UPDATE 5/22/08: Please read this first/instead. It is a much clearer statement of my ideas. I am leaving the original unchanged below, of course.

The Testability Explorer is an open source project that lets you measure the testability of Java code. This is an interesting idea: a metric not of direct quality, or of testing, or of test coverage, but of ease of testing. Presumably code that is easy to test will get tested, and will therefore be of higher quality, other things being equal.

How does it work? By calculating two proxy measures for the difficulty of isolating code in a test:

• Non-mockable total recursive cyclomatic complexity. (Rough translation: difficulty caused by code that cannot be stubbed out for test purposes.)
• Global mutable state.

These measures are then aggregated at the class level, and converted into one of three simple assessments: "excellent", "good", or "needs work". These class-level assessments then roll up to a color code score for an entire project. You can see the color codes for a bunch of Java projects on the Testability Explorer home page.

The overall approach may not be perfect, but for the moment, I'll take the approach as given and ask this question: "How would the metric apply to code written in a lower-ceremony language like Ruby?" I recompiled the Testability Explorer, adding a new CEREMONY flag. HIGH ceremony assumes Java as it is, and LOW ceremony assumes a hypothetical Java that is more like Groovy or Ruby.

I then ran Testability Explorer in HIGH CEREMONY mode on ant.jar. Here are the results, elided for brevity.

       Analyzed classes:   751
   Excellent classes (.):   497  66.2%
        Good classes (=):    28   3.7%
  Needs work classes (@):   226  30.1%

Having 30.1% of classes in the "needs work" category earns the Ant project an ugly orange splotch on the Testability Explorer home page. I then ran the same test again, in LOW CEREMONY mode:

        Analyzed classes:   751
   Excellent classes (.):   732  97.5%
        Good classes (=):    15   2.0%
  Needs work classes (@):     4   0.5%

Wow. If ant.jar were written in LOW CEREMONY Java, it would earn a happy green spot on the Testability Explorer home page. And in fact, ant.jar is already written in LOW CEREMONY Java. I didn't change ant.jar, I changed the language.

What does this mean? Maybe nothing. But here is one story you might tell:

• It is difficult to write testable code in Java. Even some solid, successful open source projects score poorly.
• Testability problems are baked into the Java language.
• Making code testable (at least in the terms defined by the TE project) does not have to be difficult for developers. With some languages, testability happens naturally.

Congratulations to the Java projects that are in the green on Testability Explorer. It isn't impossible, but it sure is hard work.

Folks in the Java community know a ton about scalability. So when they needed to connect at JavaOne last week, they turned to a scalable social networking platform.

Open source projects always need extra minds. Does your hiring process contribute to open source? It could, and it's easy to do.

Part of our hiring process is a day-long visit that includes a lot of pair programming. To make the day valuable for all involved, we want to work on interesting problems. Open source is made to order:

• There are no NDA or confidentiality issues
• If you want, it is easy to agree on (and study) projects in advance

We also do whiteboard sessions, where interviewees and Relevance folk work together to solve design problems. In the past, I haven't made a specific effort to find open source projects for these sessions. This morning I was reading about a Guice/Wicket integration issue, and it occurred to me that open source issue trackers are a goldmine of interview opportunities.

The next time you encounter a complex issue in an open source project, file it away for your next job interview. If you make interesting progress, contribute it back. It doesn't have to be code! Good analysis and documentation can be a huge help to the next developer on a project.

Oh, and if you are interviewing at Relevance in the next few months, be ready for this interview question:

There is an interesting Wicket/Guice integration pitfall documented on the Wicket wiki and as issue WICKET-1130. If you were free to change anything (e.g. Wicket, Guice, Java, the value of Pi) how would you solve this problem? How would your answer differ if you could change only Wicket?

Low ceremony languages are surging in popularity. Developers are discovering the pleasure of working in a language they can mold to meet their problem. For developers new to these languages, the problem is often where to start. There are so many! Should it be Python? Perl? Ruby? JavaScript? Lua? Lisp? Groovy? Smalltalk?

Despite differences in syntax, these languages share important similarities. For example, they tend to make it easy to create, pass around, and invoke functions, even at runtime. If you have lived long in the kingdom of nouns, this is certainly an interesting change.

But if you are choosing a language to learn, you are probably more interested in how the various languages are different. The place where the low-ceremony languages differ most is in their support for metaprogramming. Unfortunately, good information on this topic is hard to come by. I am always on the lookout for good books on the metaprogramming features of languages, and Groovy finally has one (third). Part Three of Venkat Subramaniam's Programming Groovy is devoted entirely to Groovy's metaprogramming features. Sweet.

I won't tell you which language to use (unless you drink a Lagavulin or two with me), but if you are considering Groovy, read part three of Programming Groovy.

Over the last few weeks I have repeatedly linked to Ola's post about the stable layer. I didn't take the time to go into detail, and I trusted that people (if they wanted to) could follow the link and understand what Ola was talking about.

Well, that didn't work so well. Most responders clearly did not understand Ola. A few informed me that I didn't understand Ola. :-) So I am going to make a clean break, and lay out my own argument in more detail. What follows are my views about how layered architecture affects language and platform choice. First, some ideas that are hopefully uncontroversial:

• Good design is layered.
• Leakage between layers should be minimal.
• Features within a layer should be orthogonal, and should not have to be re-implemented in higher layers.
• All kinds of programs benefit from this kind of layering, including languages, libraries, frameworks, and application code.

A small leap:

• The lowest layers are the most important.

This might seem obvious. All other layers depend on the lower layers, so a problem at the bottom affects a lot of code. But if you are working several layers higher, problems at the bottom are part of the air you breathe. The air may smell terrible, but you are acclimated and don't notice.

A big but uncontroversial leap:

• Java, the VM, is a good VM for the bottom layer.

This is uncontroversial because the majority has chosen. And they are right to do so: the Java VM is well-specified, widely implemented, carefully optimized, and supported by a huge array of tools.

A mistake:

• Java, the language, is a good language for the bottom layer.

Noooo! Java is a high-ceremony language. At every turn, Java enforces a high busy-work/real-work ratio. Specifically:

1. Java's checked exceptions bloat code, make components harder to use and maintain, and lead to tons of boilerplate code, each line of which is a bug-in-waiting.
2. Java's new operator/constructors cannot pick a return type. The amount of code that exists only to work around this is staggering. Two entire cottage industries have sprung up to deal with this single issue: factory patterns and dependency injection.
3. Java has no metaprogramming features to automate common tasks such as field accessors, standard constructors, and simple delegation.
4. Primitives, functions, and classes are not first-class objects, leading to huge code bloat to deal with these types specially.
5. Java's core reflection and interception capabilities are clunky, requiring tons of bolt-on technologies to make them workable, including AOP, annotations, and code generators.

That's a pretty big stink, but if you are used to it you probably can't smell it anymore.

The net result of these problems is that bottom layer code written in the Java language will be bloated and difficult to maintain. These problems multiply if we use the Java language for higher layers as well. What should we do?

Keeping the VM, avoiding the language

For better or worse, Java is already the bottom layer for many businesses. A complete rewrite is impossible, so we need an approach that lets us continue to use our existing Java code. There are two obvious choices:

• Use a framework to hide Java's most glaring flaws, and continue to use the Java language for development. The most popular option here is Spring. Spring provides framework-level fixes for several problems in Java: dependency injection, unchecked exception wrappers, and a powerful AOP capability, to name a few.
• Use a better VM language. There are lots of choices, including Clojure, JRuby, Rhino, and Groovy. All of them can interoperate nicely with existing Java code.

My opinion, based on extensive experience with both options, is that the "better VM language" approach is better than the "fix Java with a framework" approach.Smaller is better.

Some advice

In the past few weeks, I have been approached by several organizations to advise them on platform decisions. Every organization is different, but here are some guidelines to consider.

• Your team matters far more than your language. Pick a platform at random, then take your platform analysis budget and spend it finding good team members and helping them get better.
• Your process matters far more than your language. If your team is not delivering real business value on a regular, repeating timeframe, stop worrying about your platform and start worrying about things like estimating, agility, testing, and continuous integration.
• The static/dynamic languages debate is a red herring. The Java language's problem is ceremony, not static typing. Use whatever combination of static and dynamic typing works for you. [1]
• There can be more than one! The Java VM simplifies the interop and deployment story. So quit trying to decide, and try a few different JVM languages.
• The hot new JVM languages have different syntaxes, but similar features. They all solve the problems with Java that I enumerated above. Throw a dart at the wall, pick one, and get started coding.
• Beware "Use the right tool for the job." This is true, but useless without context, and it is becoming the weapon of choice for pundits who write no code. Be a polyglot, but also be articulate about why tool X is the right fit for job Y.
• Stop writing plain old Java code. Groovy obsoletes plain old Java. We ought to just say "Java 7 = Groovy" and move on.

Keep an open mind. Try several approaches. Judge your choices by how easy they would be to unmake or adapt. Have fun!

Notes

[1] In the past I have had a lot to say about static/dynamic typing. I realize now that I was trying to talk about ceremony. I am still worried about the same problems, but I think I now know them by more accurate names.

During the recent Rails/Grails kerfluffle, one of the memes that came up over and over again was that Grails had a specific feature which Rails lacked and that it was a Big Deal™. Specifically, Grails defines a service layer with automatic transaction demarcation which allows you to remove complex domain manipulation code from your controllers, leaving them to deal with loading resources and redirecting to views.

As a thought experiment, we set out to find out what would need to happen to enable this in a Rails app. By the end, we’d written the plugin we are releasing today: SimpleServices. It amounts to about 12 lines of code, but that isn’t really a boast, as we’ll see in a minute.

Before we look at SimpleServices, we should discuss what makes up the service layer in Grails:

• Services are classes which are, by default, singletons and are assumed to be stateless.
• Their methods are wrapped in a transaction by default, though you can turn this behavior off (not sure why you would want to, but whatever)
• Services are injected into controllers via standard Spring dependency injection methods
• Services are singletons by default, but can be declaratively scoped to one of many other scopes: request, session, flash, etc. This would enable stateful services.
• “Transactions”, in this case, are full-fledged container-managed transactions, which can involve database transactions, messaging system transactions, and any other resources that expose a transactional API and can be enlisted in a container-managed transaction.

When we wrote this plugin, we were attempting to get out the most robust implementation of the default use case, which is:

• singleton, stateless services
• ActiveRecord transactions
• encapsulation of service creation details (obscuring their instantiation API)
• encapsulation of complex domain logic, outside of your controllers

When you install the plugin, you can create a folder in RAILS_ROOT/app called “services”. Within this folder, you can define one or more service classes. A service class has a name that ends with “Service” and derives from Service::Base. The plugin causes these to be auto-loaded in the traditional Rails fashion. Each method in the service is automatically wrapped in a transaction, and any errors raised within the method will cause the transaction to rollback.

Within your controllers, you can declare which services you will be using in your actions. For each service you declare your need for, a method will be added to the controller in the form of ”#{service_name}_service” which hides all the service instantiation code.

class AccountController < ApplicationController

  services :account, :user, :security

  def update
    base_account = Account.find(params[:base_id])
    target_account = Account.find(params[:target_id])
    account_service.transfer(base_account, target_account, params[:amount])
  end
end

You aren’t forced to use services from controllers, either. If you want the declarative support, but in your models, you just include SimpleServices::ServiceInjection in whatever class you want. For example, you could include it in ActiveRecord::Base and have it available for all your models. Likewise, you can create instances of services directly, without declarative support, by calling AccountService.instance.

Known Issues

• The plugin has a dependency on the Aquarium gem. In order to provide the wrapping, we needed a good aspect library. (Aspectr has fallen on hard times, and Aquarium seems to have a lot of activity and a very simple API.) Just install the gem before using SimpleServices.
• We don’t provide support for scoping the services yet. It seems that most people use the stateless singleton model, and this was a straightforward case to implement. We’ll start looking into scoping the services next, and we’d love to hear from anybody who can make a good case for, say, a stateful, session-scoped service. I honestly can’t think of a good one.
• We are limited in this 0.1 release to single-database ActiveRecord transactions. We are looking ahead to enlisting the transaction manager when you are running in JRuby, but that will be in a future release.

We’d love to hear if people in the Rails community find this useful. We are playing around with it in a couple of apps, and it does have the benefit of cleaning up the controller codebase and providing zero-thought transaction support. Yay, services! But, we’d love to hear from others on this, and on how it could be improved.

Good luck, and happy servicing.

Graeme has responded to my previous post. Let's start with where (I think) Graeme and I agree: The enduring difference between Rails and Grails is that Rails is built on Ruby, and that Grails is built on Groovy and Spring. Most of the other differences will be competed away, to the benefit of both frameworks.

So it comes down to this: In which layer should services and frameworks be built?

• If you think that most services and frameworks should be implemented in the dynamic layer (and above), then you agree with me, and probably prefer Rails' architecture over Grails'.
• If you think that most services and frameworks should be implemented in the stable layer, then you agree with Graeme, and probably prefer Grails' architecture over Rails'.

This only matters if you are primarily driven by architecture, i.e. you are Jane. Joe's worries are entirely different. I expect Grails to have significant growth and success helping Joe.

At the end of this post I will tell you whether to use Rails or Grails for web development. But first, let me tell a tale of two decision makers:

1. Joe has a problem to solve. The problem is specific, the need is immediate, and the scope is well-contrained.
2. Jane has a problem to solve. The problem is poorly understood, the need is ongoing, and the scope is ambiguous.

How should Joe and Jane think differently about software platforms?

1. Joe's platform needs to be mainstream. It needs to offer immediate productivity, and the toolset should closely match the problem. Also, Joe doesn't want to climb a learning curve.
2. Jane's needs are quite the opposite. Jane needs flexibility. She needs glue that doesn't set. She needs a way to control technical debt (Joe doesn't care.)

For my part, I am interested in Jane's problems. (And anyway, Joe often discovers he is actually Jane midway through projects.)

So how does this affect platform choice? If you are Joe, you care about specific details about what a toolset can do right now. Most of Graeme's Top 10 reasons are in the "Right here, right now" category. This is true regardless of whether you think he is right. (Sometimes he is, sometimes not.)

My advice to Joe: Know exactly what you need, and then pick the platform that comes closest to solving it out of the box. Depending on Joe's needs, either Rails or Grails might be appropriate (or neither!). A particular point in Grails' favor would be an established team of Spring ninjas.

If you are Jane, you care more about architecture. I mean this term in two senses:

1. Architecture: the decisions you cannot unmake easily.
2. Architecture: the constraints on how you think and work.

If you are Jane, you care about how and why the platform was assembled, because you are likely to have to adapt it quite a bit.

Most of the commenters on my earlier post (and Graeme in his addendum) correctly identified the real architectural difference between Grails and Rails. Rails builds on Ruby, while Grails builds on Groovy and Spring. Rails wins this architecture bakeoff twice:

• Ruby is a better language than Groovy.
• Spring does most of its heavy lifting in the stable layer, which is not the right place.

My advice to Jane: Rails over Grails.

Nathan seems to think that Java is the Duplo of Programming Languages. He is beyond Java, of course:

After learning Ruby, various functional languages, and even C, I’d rather program in any of those than Java.

...But he thinks Java makes a great first language:

The thing that makes Java such a good first language is that the particular abstractions it provides are the ones that underlie the higher-level abstractions that are provided by higher-level languages... The most basic abstractions Java uses – variables and static methods - are a good basis because they correspond roughly to intuitions developed from algebra and other areas of human experience.

Apparently people who learn Java first think that static methods are a more basic abstraction than, well, functions.

Java as a first language is a bad idea. It misses what makes Java great, and why Java will still be great in five years.

In theory, a language like Ruby or Groovy is ideal for writing unit tests for Java. Mocks and stubs are easy and natural, and extending the test vocabulary into the problem space is easy. In practice, there are problems. I have found that testing you end up having to be very skilled in both the test language and the target language, plus understand the marshalling rules between them.

This new library should help.

I am about to give a new Refactotum talk at the Gateway NFJS show, this time refactoring Java code in JRuby. You can download the slides here.

Along the way the talk generalizes Bob Lee's continuous tax idea, and shows how it can be applied at a project planning level.

This apples to oranges comparison is sure to enrage partisans of various technologies. The authors' inept analysis doesn't match the graph at all, but the graph sure is interesting, if you believe it.

JRuby was already easy to download and build. After this week's installment of This Week In Refactoring, it is a tiny bit easier. Plus a new bug is found, by accidentally squashing it.