As people spend more time with Ruby 1.9, JRuby, and Rubinius, we are seeing a lot more benchmarks. It has been a while since we published any metrics, so I thought now would be a good time to summarize our recent experience on some real projects. We chose to measure some different things, however:
- DPC. Developer Productivity, as judged by the Client. This is the percentage of iterations where the client felt that the developers were more productive in Ruby than they would have been with their second choice "Platform B."
- DPR. Developer Productivity, as judged by Relevance. Like above, but in Relevance's opinion.
- DP5. Is Developer Productivity in the Top 5 list of things we would like to have more of? In other words, a project with a high DPC and DP5 means that the client is happy with Ruby, but would be even more happy if Ruby was, um, Rubyer.
- CS5. Is Code Speed in the Top 5 list of things we would like to have more of?
Here's the numbers. The trends are not subtle.
DPC DPR DP5 CS5
Project A: 100% 100% 100% 0
Project B: 100% 100% 100% 0
Project C: 100% 100% 100% 0
Project D: 100% 100% 100% 0
At this point, even caring about Ruby's language performance would be a premature optimization at the business process level. Ruby's runtime performance is a non-issue for a broad spectrum of applications. In fact, I believe that all of our customers would be even happier with a language that ran 50% slower than Ruby, if it also made the development team a mere 10% more productive.
This mirrors my experience with Groovy while working on a large project – large in terms of complexity, lines of code and calendar time. At the end of the day, for a large class of applications, code maintainability, code expressiveness, programmer productivity, etc…. trump how fast a contrived benchmark executes something a 20 million times in a tight loop. It an amazing “aha” moment when a person goes back and looks at a complex piece of code that was written 9 months previous and you can almost instantly grok what its doing; or tell a customer “I can quickly make that change for you; would like to wait a minute and confirm its ways what you want”; or with a technical business analyst, show them a block of code and say “is this what you meant”. These things translate to real value for the client.
tell that to the customer when you find yourself hacking C after you tossed to wall because of a performance issue. only part of the applications, or scenarios fit to your claim.
Can you post details on the kinds of applications you’re using—how much they are required to scale (initially and expected), and at least generally what the projects do?
It’s very true that many web applications have a known & limited user base (like, “all of our salespeople in the field” or “customers of our appliances who need support”, etc.), and minimal processing in code, and in those cases benchmarks of one piece (the programming language used for the business logic) are fairly irrelevant.
BUT that’s not true of all web apps—you might have a much higher potential userbase, and you might have much more intensive processing requirements per-user.
So if you don’t describe your projects at least a little, this kind of review is only useful to you.