If you are coming to the Refactotum session at RailsConf today, you can download the overview material here. We will also have electronic materials available in the room.

Aaron Bedra, our resident ninja, just saw the publication of his Rails Security Audit PeepCode book. We’re excited for Aaron personally, and pleased to see the release of some concise, useful documentation on the kinds of threats and mitigations that apply to Rails developers. Check it out, and of course, big ups to topfunky for publishing it and being a generally hoopy frood.

Just about the whole Relevance crew is at RailsConf this weekend, so if anybody wants to chat with Aaron about the book, just look for the nerds in the thinkrelevance.com t-shirts and we’ll find Aaron for you. You won’t be able to find him yourself; he’s a ninja.

Next Thursday, we will be presenting the Refactotum session at RailsConf, and showing people how to contribute to open source Ruby and Rails projects. To prime the pump, I am trying to find 101 possible refactorings between now and then. You can follow along in real time on Twitter here.

To stay within the Twitter limits, I am experimenting with different ways to fit code review comments into 140 characters. Suggestions welcome. I am numbering the proposed refactorings, so if you decide to do one of them you can refer to the number.

Caveat lector: In order to find 101 refactorings I am reading the code quickly. Some of my ideas will be bad, and should be rejected.

The Refactotum Project has two objectives:

1. Show developers how to contribute to open source projects.
2. Encourage healthy code through refactoring

To perform a refactotum exercise, you need to set aside four hours, and perform the following steps:

• Select an open source project.
• Download the project and run the tests.
• Find a bite-sized problem to solve.
• Spec out and implement your solution.
• Create and submit a patch.

The four-hour timebox is very important. The number one enemy of contributing to open source is inertia. (You are fighting that battle by reading this.) The number two enemy is overambition. Start small.

Selecting an open source project

Start by selecting an open source project. You do not necessarily have to know much about the project, but you do need to know the programming language the code is written in! We will give examples written in Ruby, but the ideas are general and can be used with any language.

Here are some Ruby projects that are easy to get started with:

• Mephisto
• Typo
• Radiant

Download the project and run the tests

The next step is to download the project and run the tests. Most projects will give you at least three different options:

1. Download a compressed dump of the project source
2. Anonymous source control checkout (no commit access)
3. Developer source control (commit access)

You will want the anomymous source control checkout. Having a source control checkout should guarantee that you are working on the latest, greatest code. Since you are contributing to this project for the first time, commit access is not an option. (You have to earn that.)

To get the repository

• For a Rubyforge project: Go to the project homepage, and then click on the "SCM" menu for instructions.
• For a github project: Go to the project homepage, and copy the "Clone URL" link. Then execute git clone {clone url}

Many Rails applications rely on implicit loading of gems, so you may find that you don't have all the gems you need to run the tests. If you see a message of the form

no such file to load -- foo

your best bet is to gem install foo.

Find a bite-sized problem to solve

Pick something small and simple. You'll be amazed at how quickly a small change grows into something more ambitious. Here are some easy ways to brainstorm ideas for a Rails application.

• Take advantage of being a beginner. If you had any trouble getting the project tests to run, others will too. Solve this problem first. This can often be as simple as a README change. Patches that help new developers get started are especially welcome. Project maintainers are blind to these issues, because they have internalized important bits of HOWTO knowledge that are not documented anywhere.
• Fix any tests that are failing.
• If the project publishes a TODO list, see if there are any small tasks that you can do quickly.
• Run rcov on the test suite. Pick a single method that is missing code coverage. Write a test that provides coverage (and fix any defects you encounter). If the application does not inlude an rcov rake task, contribute one.
• Run flog on the application. Review the methods with a flog score higher than 80, and find one that can be refactored for simplicity. (You probably want to avoid methods with a flog score higher than 150, as these will likely blow out the four hour timebox.
• Run tarantula on the application. Add tests for any inputs that Tarantula generates that cause HTTP 500 errors, and then fix the errors.
• Review the code in application_helper.rb. This is often a hiding place for code that needs refactoring.
• Scan the log for warning messages, and fix them.
• Look for logic in the view templates, and extract it into helpers.
• Look for controller methods that do model work, and extract into models.

Spec and implement your solution

Write specs, or tests, that demonstrate the code you plan to write. Then write the code to pass the tests. Look at the other tests in the project and follow their lead.

Create and submit a patch

Most open source projects welcome well-organized patches. (If they don't, fork 'em.) Make sure you make it easy for a busy maintainer to understand your patch.

• Include complete test coverage for your changes. (Use rcov to make sure.)
• Make sure all the other tests still pass.
• Give your patch a descriptive name. Describe what it does and why.
• Assign your patch an accurate severity and priority.

Most importantly, read the project instructions for contributors, and follow them. Here are a few examples:

• Contributing to Rails
• Contributing to Rubinius
• Contributing to JRuby

Pitfalls

There are several ways to stumble during your first open source contribution. Keep an eye out for these warning signs:

• Do not shave yaks. If your small change triggers big ideas, don't chase them down (yet). For example: maybe you are not git-taculous yet. If not, then pick an svn-based project for your first refactotum. Git is way cool, and you should learn about it, but not while trying to do something else.
• Do not be opinionated. When you are in someone else's project, code to their idioms. You can be opinionated later, once your contributions have earned credibility. Maybe you love rspec. But if the project prefers Test::Unit tests, then so do you. Once you have contributed a few patches, they are a lot more likely to entertain your suggestion of moving to rspec.
• Do not introduce library dependencies. To make your patch acceptable, you want to minimize the potential objections. Requiring a new library can be a big-impact change. Deployment may change, users may need to review the license, etc. If you are working on a refactotum where the obvious solution drags in a new library, pick something else. The time to propose a new library is after the initial success.

Let us know how it goes. Happy refactoring!

We do hiring interviews on Open Source Fridays. Most of a candidate's time is spent on pair programming, whiteboarding, and giving a presentation to the team. Here is an example question from a whiteboarding session:

Background: Mark Watson has proposed a simple solution for packaging Java libraries to be IDE-friendly:

users of these libraries (mostly just me :-) only want the highest level APIs to show in popup completion lists and the entire set of implementation classes remain invisible. The solution is easy: a public API class with implementation classes in the same package with package-ony (i.e., neither public or private) visibility.

Mark is cleverly combining the existing visibility mechanisms, plus creating an extra class, to trick IDEs into exposing the right methods for IDE completion.

To generalize this solution, we would need to

• create a named abstraction that solves the problem
• hide the "how" details behind the new abstraction

In this case the new abstraction we need is a custom visibility scope. Java provides four scopes: public, private, protected, and {package}. We want to add new scopes with names like TopLevelAPI. One can imagine many ways to do this in Java. Here are a few obvious ones, building on existing idioms:

• Define an annotation, e.g. public @TopLevelAPI myMethod()
• Define an external class that extends Java reflection. (The JavaBeans API is a perfect example of this. JavaBeans create a custom class member type, property, that augments the existing constructors, methods, and fields.)

In Ruby, you could create a custom visibility scope using object-oriented design. Pick a base class for all classes that need the custom scope. Define a class method named for the scope, e.g.

def self.top_level_api_instance_methods
  # TODO: return array of top level methods
  # TODO: declarative helper?
end

You could reopen Class to provide this feature for all classes. (Rails provides a good example of a domain-specific custom visibility scope, with a slightly different implementation. The assigns are a special scope for controller fields that are automatically visible to the view.)

Questions:

1. The three approaches outlined above would all require IDE collaboration. Compare and contrast the approaches from the point-of-view of an IDE vendor charged with supporting them.
2. The approaches above also require that developers mark certain methods as TopLevelAPI. Compare and contrast the approaches in terms of the work a developer does to take advantage of them.

We are hiring. If you are interested, send us your thoughts on the questions above, or post them in the comments.

Bad Ruby is coming to the Atlanta Ruby meetup on June 11.

I should also mention that we have a growing list of Relevance Tech Talks. Let us know if you would like to have one of these talks at a conference or user group near you.

The response to my earlier post on testability metrics has been, well, a little negative. I think that the strident tone of the conversation (for which I bear much of the blame) has artificially widened the gap of opinion between me and some people I respect.

So I am going to start again, and fill in the gaps from the earlier post. Along the way, I will answer the interesting questions raised in the comments.

Java is naturally untestable, and that ain't news

Let's start at the very beginning, with the idea that Java is naturally untestable. What does the word natural mean here? That if you use Java naturally, taking advantage of the various features of the language when they are appropriate to the task at hand, you will end up with untestable code. This is not news to anybody, least of all experienced Java programmers. Here are a few quotes from the Java community. First, on the Testability Explorer home page:

The metrics are a calculation of the skill of the development team in making their classes testable.

From the Spring Framework mission statement:

Testability is essential, and a framework such as Spring should help make your code easier to test.

And from the Guice homepage:

You will still need to write factories in some cases, but your code will not depend directly on them. Your code will be easier to change, unit test and reuse in other contexts.

If I can paraphrase these quotes: Easily testable code doesn't just happen naturally in Java. A team must develop skill in testability. In fact whole frameworks have been created, counting testability among their primary objectives.

Java can be tested

Of course, many Java programmers can and do write testable code. As a Java programmer, I learned to avoid certain idioms that are not testable, or to use a framework to do it for me. But this (in my view) isn't natural. I have made the point elsewhere that a language should be richly consistent. If "testability" is a core feature of the language, then testability should be combinable with all the other core features. There should not be a laundry list of special cases of the form: "Don't do X or you can't have Y."

In many other languages, testability is almost free!

The Testability Explorer defines testability in terms of two things that you should avoid:

• unmockable complexity
• global mutable state

As in the earlier post, I will focus exclusively on the first of these: unmockable complexity. (Global mutable state is a discussion for another day.) Also, I am not going to worry about the complexity part. I use cyclomatic complexity measures in both Java and Ruby code, and find them to be decent predictors of code that a human reviewer would find objectionably complex.

What remains then, is the definition of unmockable. The Testability Explorer considers code unmockable if it cannot be overridden or injected, which it defines as (some subset of) constructors, statics, and privates.

By this definition, Ruby has no unmockable complexity:

• Ruby's "statics" are polymorphic to begin with, so they can be overridden just like instance methods. (In fact, they are instance methods.)
• Ruby constructors can be overridden or simply rewritten.
• Ruby's reflection model will let you publicize private members when needed in a test.

Several commenters asked for a patch so they could try out LOW CEREMONY mode for themselves. Easily done: set the unmockable complexity number to zero across the board. I did it by refactoring MethodInvocation.computeMetric.

What do the testability metrics really tell us?

When I figured out how Testability Explorer calculated its metrics, I knew immediately what it would "prove" -- it is one of those metrics whose outcome is driven by its assumptions. If unmockable complexity really is a primary enemy of testing, then the Testability Explorer will quickly "prove" that you should either

1. Use a Dependency Injection (DI) framework (in Java)
2. Use a low-ceremony language that makes mocking trivial

If you want to introduce Spring or Guice to a Java shop that still doesn't believe in DI, Testability Explorer is your friend. It will probably "prove" that your code base isn't testable. Or you can use it like I did in the original blog post to "prove" that Ruby projects are intrinsically testable.

What does the Testability Explorer actually prove? Not as much as one might hope. With the right tool support, the unmockable features of Java become mockable. And even a green project may still be hard to test. This is a problem with metrics in general. Once you understand what they measure, it is all too easy to game the metric, often unintentionally. My colleague Jason Rudolph is giving a talk called How to Fail with 100% Test Coverage that explores this problem.

At Relevance we include two metrics related to testability in our continuous integration builds: rcov test coverage and flog score. Test coverage doesn't measure ease of testing, but insisting on high coverage makes it painful to have complex code. Flog measures method complexity. Unlike Testability Explorer, we treat all complexity as a testability issue.

Java has better metrics, nya nya

Java beats Ruby in support for collecting both of these kinds of metrics. If you want to heckle a Rubyist in a language war, here's some ammunition:

• Ruby's coverage tool only provides line coverage, while Java has tools like EMMA that can give you branch coverage within a single line.
• Java-oriented complexity tools (like pmd) are much more elaborate than flog, and can provide all kinds of specific advice above and beyond a simple numeric score.

A few more responses to commenters

To the heroic defenders of Ant: I didn't offer any of my own opinions on Ant. I merely reported the numbers from Testability Explorer, and the conclusions that project implies with its color scheme. The only reason I even picked Ant was that it came first on the page. :-)

To the attackers of JRuby: I agree that JRuby is hard to test. I have already looked into it, contributed code, blogged about it, and given conference talks on the subject. One interesting positive aspect of JRuby, which in my opinion dwarfs the negative metrics, is that the JRuby committers are almost always online in the JRuby IRC. They will answer your questions, and they are working to make the project better.

Finally: I am not a Ruby zealot. In ad hominem attacks, please refer to me as a Lisp weenie wannabe.

My reaction to Brad Ediger's Advanced Rails is a simple "thumbs up." If you are a serious Rails developer, you should read this book. Here's a few tidbits I liked:

• Very nice coverage of ActiveSupport. Developers who learn Rails first, and Ruby later (or never) often do not discover some of the goodies here.
• There is a security reason for HashWithIndifferentAccess. Do you know what it is?
• Nice explanation of how instance_exec works.
• Overview of several real-world approaches to configuration management: Plugins and gems aren't everything -- also covers RaPT, piston, and decentralized version control.
• How to test plugins. While Rails itself makes testing easy, writing tests for a plugin is confusing unless you have a thorough understanding of the Rails environment.
• Enumerator. It's like Enumerable, but tailored to you.
• Using proxy objects to deprecate ivars.
• A broad-ranging chapter on security, including password hashing, securing error messages, whitelisting and blacklisting, session attacks, XSS, CSRF, canonicalization, SQL injection, and tainting.
• Pragmatic ImageMagick. (Try saying that with a straight face.)
• A nice explanation of REST that goes beyond getting started (routes and scaffolds) to some interesting issues (various options for managing ETags).

There is very little not to like:

• The book endorses Ferret. Don't go there.
• Ediger seems to prefer vi to emacs. Sigh. The truth will set you free, my friend.

If you already know a lot of Rails, this book will be a quick read. But I bet you will have at least a half-dozen ah-hah moments, which is a good number. Read it.

Aaron and I are trying to get Jruby on Rails on DB2 to actually work. So far, we have been wrestling with the following issues:

• broken rake tasks
• schema dump and quoting
• remove index
• unexpected column metadata

We are driving our effort by porting existing projects that work with other databases, and seeing what breaks. We are not DB2 experts, so a lot of this is conjecture+test. We would love to join forces with anyone interested in getting the DB2 driver to first-class status. Give us a shout.

I will be the closing keynote speaker at RubyNation. I'll be beating my current favorite drum: Ending Legacy Code In Our Lifetime. (And yes, the Ruby community has legacy code. If we don't change our ways soon, we will end up with a lot more of it.)