Can Rails Scale? Twitter Raises Some Questions

by Reuven Lerner - May. 02, 2008Comments (9)

Ruby on Rails is, as we've written on this blog before, one of the current darlings of the open-source world. Rails, which lets you create sophisticated Web applications relatively easily, is now the tool of choice for many Web application programmers.

I have personally been developing Web applications for about 15 years, and I have been moving all of my consulting work over to Rails in the last 2-3 years, both because of the added productivity, and because of the enjoyment I get from the work. It reminds me of the excitement I had building Web applications in Perl, back in the early 1990s.

One of the biggest issues that has dogged Rails from the beginning is the question of speed -- and with it, the question of scalability. Everyone agrees that Ruby programs execute slowly relative to many other languages. The question is whether the combination of Ruby's slowness, along with the intelligence of the Rails platform, makes it an unviable option for high-profile, "enterprise" applications.

Of course, it's almost always possible to throw more hardware at the problem, in this case by adding additional Web servers running Rails. So the issue is not whether you can make Rails scale for large-scale applications, but how many servers you will need, how much you'll need to think and worry about the network architecture, and how many people you'll need to run things. As Joel Spolsky wrote 18 months ago, "if you become The Next MySpace, you'll be buying 5 times as many boxes as the .NET guy down the hall."

There were thus a number of reactions to the news that microblogging-messaging startup Twitter would be moving away from Ruby on Rails. This news comes just a week after Twitter's chief architect left the company, making it easy to draw the conclusion that the two events were somehow related.

From Twitter itself, Evan Williams wrote that (a) it's not true that Twitter is moving away from Rails, and (b) "lots" of Twitter code isn't in Rails anyway. TechCrunch reported that despite this denial, Twitter is telling people that it is moving away from Rails.

Of course, it's possible that both of them are telling the truth. For example, Twitter might still be using Rails for displaying messages and other elements of the Web application. But for the heavy lifting, such as sending messages, it might have abandoned Rails long ago.

No matter how you slice it, though, Williams's comment would seem to indicate that Twitter isn't using Rails for its most mission-critical software, presumably because it doesn't scale well enough. Does this affect the usage of Rails? Does this mean that people with grandiose plans should look elsewhere?

Not necessarily: First of all, there are a lot of people and companies with a vested interest in seeing faster, more scalable implementations of both Ruby and Rails. Sun has poured a great deal of money and effort into the JRuby project, and it may well be that the most popular version of Rails in the future will run on the Java virtual machine (JVM).

But the other argument that benefits Rails is one that has been tossed around since the early days of Perl, which was often criticized for being slower than C. In those days, we would say that Perl might be slower than C, but that a company's most expensive resource is it's programmers' time. It's OK for a program to run in twice the time, if it takes a programmer half the time to write it.

Obviously, there are times when this is not the case, and languages like C exist for that precise reason. But if a company needs to buy twice as many servers, while using three fewer programmers, that might turn out to break even, or even put the company slightly ahead.

Would you use Rails in a mission-critical environment?



Khürt Williams uses OStatic to support Open Source, ask and answer questions and stay informed. What about you?



9 Comments
 

So you are saying that any application can scale, it's just to add servers. Well, that's really not what scaling is about now is it. Anyway. Good article.

0 Votes

The problem here wasn't a Ruby, or for that matter even a RoR problem per se ... it's just that sticking a db into the middle of what is fundamentally a message routing problem just doesn't make sense.

More here and here.

0 Votes

It is true that you do want to go with a tried and true testing environment when writing code, and Rails hasn't been tested under very heavy load too often, but a lot of the hot spots can be identified and fixed with redesign and mo' hardware, I guess, instead of building skills and expertise in another language.

Also, as Twitter says that a bunch of their services are sitting outside Rails, but that just means you need more tools and skills to develop, manage and maintain...

While true that it takes just a short while to get up and running, and you want to consider that, you definitely do want to bake some sense of scale in there. If you are doing things on Internet scale, you definitely do want to consider that! I'm sure Evan knows a thing or two about scaling, so I'm sure they thought about and evaluated some of that for what it was good at doing.

0 Votes

I am from India, a RoR programmer,I am new in the field and still learning.I am joining a company as RoR programmer.But frankly telling u guys that these Twitter related articles are scaring me. Pls tell me am I investing my time in something which is going to die soon. Shud I continue with rails or shud I start looking for other technologies

I am doing my masters in IT. RoR programmer will be my first job. so probably I can switch.

0 Votes

@Andree: Rather than saying "Rails doesn't scale," I would say that (a) Rails doesn't scale as well as other platforms right now, (b) a great deal of time and money are being invested in making Rails (and Ruby) more scalable, and (c) the trade-off between programmer efficiency and program-execution efficiency is often a tough call.

Java and PHP also had problems at the beginning, and both have managed to become highly scalable. I have no doubt that the same will be true for Rails in the coming years, if only because of the investment that Sun and others are making in JRuby -- which, being based in Java, is as scalable as any other Java-based system.

I think that the Rails community is right to take scalability seriously. I think that we would be wrong to panic, or to abandon Rails, unless you need your system to be uber-scalable tomorrow, and you can find good programmers who don't cost a lot.

0 Votes

I happened here by accident, but can't let a statement in the original post go unanswered.

Where, in today's world, is there a "trade-off between programmer efficiency and program-execution efficiency"? You're joking right? The equation these days is: Slow execution = 0 customers. And zero customers means no revenue to pay programmers.

I don't care how efficient my programmers are if the product they're turning out runs like molasses. And, I especially don't care if I'm off-shoring the code where a team of Chinese coders can optimize execution at a fraction of what a single U.S. coder earns.

The OS community needs to wake-up if you are considered a thought leader.

0 Votes

Anon - I think you raise an interesting point, but I think you are being extreme with your definition of "program-execution efficiency". If it is slow-as-molasses, of course - it is a no-brainer - you have nothing. However, we all know that the closer you get to machine code, the faster your program is going to run, and if you run a Java app, even with built in JIT and other optimizations, it is never going to be as fast, or as tight as a well-written native app. But, there is a reason people adopt Java over C for several apps, and it ain't Sun's marketing. Some languages and paradigms are just faster to get up-and-running with. RoR is the case-in-point here. Sure, there is a loss in performance, but is it enough to warrant you to take longer in getting your app ready? Depending on your app (and, there are several), it might NOT make sense to go for that extra efficiency in performance while trading off time to develop and time to market.

If your program runs slow as molasses, you are being cheated, no matter what! However, if you can get 80% of the way there with 20% of the time and materials, is that a tradeoff you would make? Well, depends on what you are doing, but most would, especially, if you can beef up your compute resources with price-dropping (but management escalating) hardware.

I think YOU need to wake up and realize that not all tasks require 100% efficiency if it takes 200% of the time!

0 Votes

It would seem that this is an issue that keeps arising again and again. In my opinion any language scales as well as the people who use are technically capable. Yes it may be slower than some languages but it also provides some pretty cool features that would allow you to spread over many machines and shard data much easier than others would allow (specifically the framework allows/disallows this). It also really depends on what the application is meant to be doing. If you require a lot of intensive processing of data then I would personally go down the Java et al route and do a little mix and matching to find the best tools for the job. For creating web apps in an extremely fast time is what Rails is all about. I have written an article before looking into the pros and cons of rails so take a look if you want at Rails Scales (Or Does it?)


0 Votes

That link above is meant to be http://www.landoweb.com/2008/07/13/rails-scales-or-does-it.


0 Votes
Share Your Comments

If you are a member, to have your comment attributed to you. If you are not yet a member, Join OStatic and help the Open Source community by sharing your thoughts, answering user questions and providing reviews and alternatives for projects.