...and the Java community is collecting hard data to prove it.
UPDATE 5/22/08: Please read this first/instead. It is a much clearer statement of my ideas. I am leaving the original unchanged below, of course.
The Testability Explorer is an open source project that lets you measure the testability of Java code. This is an interesting idea: a metric not of direct quality, or of testing, or of test coverage, but of ease of testing. Presumably code that is easy to test will get tested, and will therefore be of higher quality, other things being equal.
How does it work? By calculating two proxy measures for the difficulty of isolating code in a test:
- Non-mockable total recursive cyclomatic complexity. (Rough translation: difficulty caused by code that cannot be stubbed out for test purposes.)
- Global mutable state.
These measures are then aggregated at the class level, and converted into one of three simple assessments: "excellent", "good", or "needs work". These class-level assessments then roll up to a color code score for an entire project. You can see the color codes for a bunch of Java projects on the Testability Explorer home page.
The overall approach may not be perfect, but for the moment, I'll take the approach as given and ask this question: "How would the metric apply to code written in a lower-ceremony language like Ruby?" I recompiled the Testability Explorer, adding a new CEREMONY flag. HIGH ceremony assumes Java as it is, and LOW ceremony assumes a hypothetical Java that is more like Groovy or Ruby.
I then ran Testability Explorer in HIGH CEREMONY mode on ant.jar. Here are the results, elided for brevity.
Analyzed classes: 751
Excellent classes (.): 497 66.2%
Good classes (=): 28 3.7%
Needs work classes (@): 226 30.1%
Having 30.1% of classes in the "needs work" category earns the Ant project an ugly orange splotch on the Testability Explorer home page. I then ran the same test again, in LOW CEREMONY mode:
Analyzed classes: 751
Excellent classes (.): 732 97.5%
Good classes (=): 15 2.0%
Needs work classes (@): 4 0.5%
Wow. If ant.jar were written in LOW CEREMONY Java, it would earn a happy green spot on the Testability Explorer home page. And in fact, ant.jar is already written in LOW CEREMONY Java. I didn't change ant.jar, I changed the language.
What does this mean? Maybe nothing. But here is one story you might tell:
- It is difficult to write testable code in Java. Even some solid, successful open source projects score poorly.
- Testability problems are baked into the Java language.
- Making code testable (at least in the terms defined by the TE project) does not have to be difficult for developers. With some languages, testability happens naturally.
Congratulations to the Java projects that are in the green on Testability Explorer. It isn't impossible, but it sure is hard work.
Comments
Congrats, by lowering the standards for your language of choice you made it look like its easier to test while actually only proving that your “relevance” is exceedingly low.
Are you seriously claiming that because a tool that was written more than five years ago (ten maybe?) is not very testable, Java code is not testable?
You Ruby zealots crack me up.
Why are you so obsessed with attacking Java with fallacious arguments? I’ll agree that Java has some built in features which can make testing difficult (“static”), but it’s not hard to write testable code. Guice is green, and, for the most part, I wasn’t even able to use Guice itself to build it. Ruby has its share of features you should avoid, too, not to mention it makes it a lot easier to write difficult to maintain code over all. Using the word “ceremony” to describe type information makes you sound like a religious zealot who doesn’t actually understand the technical benefits. I know you know how to use IntelliJ. What’s the deal? I like Ruby, but it’s not the answer to everything, just like Java isn’t.
Because global, mutable state and high cycolmetric complexity is impossible in Ruby. Next up, somebody puts in an “OMG PONIEZ” flag that makes everything hit 99.9%, and says that it is due to the benefits of static languages. That would be at least as useful as what you have here.
“It is difficult to write testable code in Java.”
This post did it. Anyone who is hiring you must be out of his mind. Other than that Bod said it all.
Peace -stephan
I don’t see how your choice of language effects testability. Dynamic languages allow you to reach in and change the definition of any method at runtime. Dynamic languages fans suggest that as a result, testability practices such as Dependency Injection are not relevant and that dynamic language is inherently more testable. However code is global state and you reaching in and changing the global state produces all the same problems as global state in Java except worse since you are not only changing global state of data but also global state of code. So I simply don’t buy this argument. To me changing the definition of method at runtime falls into the same level of testability as subclassing in tests and overriding a method. TDD fans frown on this practice, for good reason, but it is no different then changing method definition at runtime. Dynamic languages are more terse which is a great plus, but I don’t see how you can build a large scale many developer codebase out of dynamic languages. Static typing is wordy but it is a level of information which is very useful in large many team projects. So everything needs to be weight in proportion.
1) Make controversial blog title 2) Ridiculously try to make that claim plausible 3) ... 4) Profit!
In America, you grind a ruby. In Soviet Russia, Ruby grinds you!
Why is it that most Rubyists seem to never get tired of blogging about Java? If it is all that great, why cant those people just leave us alone with our old static and supposedly “untestable” language/platform?
Wow. It’s a rare feat to make a case against Ruby programmerrs by attacking Java, but you have provided such a salacious and moronic argument here that you’re showing the world how stupid your language war really is… and how unbelievable shallow and moronic your “science” can be.
Good job.
Ruby developers should really get a life.
Stu,
I’m a little confused with your “CEREMONY flag.” Could you explain how this altered the calculation? Or even post a patch so that others could take the same analysis approach to other jars?
Thanks, Jonathan
Whats sad is that I bet you’ll happily take on Java gigs for your sweatshop. If you’re so pleased with your chosen environment and are so convinced of its success, why not stop trying to sell Java consulting services?
Gawd – you Ruby lovers continue to astound me. Now you choose to use “high ceremony” and “low ceremony” flags on the most deeply entrenched Java build tool – Ant. More than 90% of Java shops continue to use Ant [I guess the remaining share must be maven or other tools]. Ant users span from mom-and-pop Java shops to Fortune-50 companies. Java and the plethora of open-source tools available for Java are too deeply entrenched worldwide. Using mumbo-jumbo like “high ceremony” and “low ceremony” is not going to make Ruby any more relevant than it already is. The software business is all abuzz with dynamic languages – and no doubt there will be an impact, but Java is here to stay folks. Recent awareness about agile practices has made Java developers very conscious about making their code testable.
So I thought about this some more. Could you explain to us what exactly did you change in the way the testability-score is computed. As far as I can tell, you changed the equation and now claim that since the change you did creates smaller number hence the ANT is more testable. Hmm… That is kind of like saying that I live 10 miles from work and hence I will drive in miles since 10 is less then 16 kilometers. Unless you publish the equations or what exactly you changed and then rerun it on all projects to get relative numbers, I think changing the measuring stick in the middle of the game is kind of pointless. :-)
OK, to add fuel to the flames, I find it very interesting that JRuby has some of the worst testability numbers out there. http://testabilitymetrics.appspot.com/report/jruby/jruby/1.0RC3 Perhaps you could look into why that is? I think it would make a great Blog entry.
Great keep up the good work putting Ruby in the trash where it belongs :)
I see what you did there ruby boy.
Obvious troll is obvious.
As a person who actually tried to integrate Ant with.. some other tool… and could confirm that “solid, successful open source project” has many architectural flaws (i.e. lot of stuff hardcoded) and not so good source code quality.
Regardless, I can’t see how a person can /seriously/ try to extrapolate this to “Java Is Naturally Untestable…”