Planet Autopackage

Hongli Lai: Ruby Enterprise Edition website “launched”

Tue, 13 May 2008 21:16:29 +0000

http://www.RubyEnterpriseEdition.com/ has been “launched”.

Next hint to be revealed on May 20, 2008.

Hongli Lai: Rails Escapes the Ghetto, Phusion Passenger now used on Dreamhost

Tue, 13 May 2008 20:13:28 +0000

Please help us with modding/digging these up.

Hongli Lai: Building native Linux packages for Passenger is now possible

Tue, 13 May 2008 17:44:42 +0000

We’ve made some modifications to Passenger to make Linux packagers’ lives a bit easier. Also, we’re looking for volunteers who wish to maintain packages for their distribution.

See http://blog.phusion.nl/2008/05/13/building-native-linux-packages-for-passenger-is-now-possible/ for details.

Hongli Lai: Passenger (mod_rails) 1.0.5 released

Wed, 07 May 2008 19:22:44 +0000

Passenger 1.0.5 has just been released. Please read the Phusion corporate blog for the announcement and upgrade instructions.

Hongli Lai: Interview by AkitaOnRails

Wed, 07 May 2008 10:53:13 +0000

AkitaOnRails had recently interviewed us. The interview covers our company’s background and information about Passenger, among other things. He asked us quite some interesting questions, so we recommend you to read the interview.

Hongli Lai: Does Ruby have a distribution problem?

Tue, 06 May 2008 20:33:26 +0000

Luke Kanies wrote an article, in which he says that Ruby has a distribution problem. It criticizes Rails applications for vendoring a lot of stuff, and criticizes RubyGems for not being able to handle native package dependencies, among other things.

I beg to disagree. The described problem is not a pure Ruby problem: it’s a general software distribution problem. I’m concerned that people would use this as another “reason” to oppose Ruby, even though it’s not specific to Ruby.

My response

I posted a reply to Luke’s article, and it was as follows:

Luke, I don’t really understand what you’re expecting from the Ruby/Rails community. You’re dealing with cross-platform distribution issues. That’s hard by its very nature. I was a developer in the Autopackage project (www.autopackage.org), a software packaging system that works across multiple Linux distributions. I’m currently a developer of Passenger (www.modrails.com). We’ve run into many of the issues that you mention here.

If I understand it correctly, you’re claiming that vendoring stuff is bad because:

You cannot vendor everything (glibc, web browser, etc).
As someone else has mentioned, vendoring stuff creates potential security issues. This is not unlike static linking in C/C++ projects.

On the other hand, vendoring stuff does have benefits:

Guaranteed compatibility. Suppose you rely on GTK 2.2. On a faithful day, GTK 2.2.5 was released, but accidentally introduced a regression, and now your application fails left and right. Uh oh. If you vendored GTK then that wouldn’t have happened. This is not a theoretical possibility: GTK 2.2.15 or something actually broke AbiWord. (I’m not saying that vendoring GTK is a good idea, as GTK is quite large. I’m just pointing out a benefit of vendoring.)
Less installation hassle. Not all platforms have good packaging systems. As far as I know, Debian-based distros are the only ones. RedHat-based YUM repositories tend to be quite small compared to Debian’s APT repositories. MacOS X and Windows don’t have native package management systems at all. If your app is cross-platform, then vendoring stuff is a lot easier for both the developer and the end user.

Vendoring stuff is not only common in the Java world, but also in the Windows and MacOS X world. Windows apps tend to bundle all their dependencies (with the exception of obvious stuff, such as Internet Explorer). How many games have you seen that bundle DirectX? Actually I’d say that not vendoring stuff is only common in Linux, and languages that have strong ties to Linux, such as Perl. Here’s where Debian’s package management system and huge package repositories really shine.

There’s almost no common ground in the world of package management. We at Autopackage had to invent our own dependency resolution mechanism (similar to APT) because there’s no lowest common denominator, even amongst Linux distros. RubyGems is probably created for the same reason: not all Ruby-supported platforms have (decent) package management, so they just wrote their own. Autopackage experimented with native package management integration (i.e. being able to use the system’s native package manager to resolve dependencies) but that proved to be much, much harder than initially expected, and up until today that feature still isn’t finished. So I don’t think you can reasonably expect the Ruby community to do something about this. It’s not a pure Ruby problem: it’s a general software distribution problem.

As for being FHS-compliant: I can only say “don’t bother” if your application is cross-platform. It seems that only hardcore Linux users care about that. Outside the Linux world, FHS is being criticized by pretty much everyone. Windows and OS X users complain that application files in Linux are scattered everywhere, instead of being self-contained. Scattering files isn’t a problem in Linux because of package management, but it is a problem in all other platforms that don’t have decent package management. I’ve found that being FHS-compliant is more trouble than it’s worth.

So how do you solve this problem? I don’t think it’s possible to come up with a general silver bullet solution. So that leaves the following choices to you, the developer:

Create a native package for every platform that you support, i.e. .deb for Ubuntu/Debian, .rpm for RedHat, another .rpm for Mandriva and SuSE because their package names are different, a .exe for Windows, .dmg for MacOS X, .tgz for Slackware, .??? for Solaris, etc.
Write a cross-platform installer. Passenger/mod_rails chose this option because it’s a lot easier than the first one. Passenger depends on Ruby packages (Rails, fastthread, Rake, etc.) as well as native packages (GCC, Apache, APR). It checks whether all dependencies are available, and if not, it tells the user how to install those dependencies. We’ve put platform autodetection and Linux distro autodetection code in the installer. So on Debian/Ubuntu it would tell you to run “apt-get install apache2-prefork-dev” while on Fedora/RHEL/CentOS it would tell you to run “yum install apache-devel”. We’ve found that this approach works extremely well.

Finally, we at Autopackage fully recognized the pros and cons and vendoring/static linking. Autopackage recommends the following: dynamically depend on stuff that are common, but vendor/static link stuff that are uncommon. We believe that this is a good trade off between the pros and cons. Passenger follows this recommendation as well: we vendor the Boost C++ library. Few people have Boost installed, and when they have it installed it often isn’t the version that Passenger requires. Installing Boost is a huge, huge pain on MacOS X. In this case, the benefits that vendoring Boost gives us outweight the cons by far.

On the other hand, Apache is fairly common, and easy to install on most platforms. Rake, fastthread, etc. are also easy to install because of RubyGems. That’s why we dynamically depend on those things instead of vendoring them.

So it all boils down to making the right choices and correctly balancing the pros and cons of vendoring. There’s no silver bullet.

Gunnar Wolf’s follow-up

Luke’s article was quickly followed by a blog post by Debian developer Gunnar Wolf:

By using Ruby Gems, you dramatically increase entropy and harm your systems’ security.

To this, I say nonsense. It’s pretty well-known that Debian, and Linux distros in general, dislike foreign packaging systems, regardless of their merits. I see quite a lot of conversations on #rubyonrails @ irc.freenode.net that are somewhat like this:

Person A: hi, I’m using Debian/Ubuntu/some-other-Debian-derived-distro. I typed “gem install rails”. But when I type “rails foo”, it says “command not found”. what’s going on?
Person B: type “gem update –system && gem install rails”
Person A: wow, it worked! thank you!

It is painfully obvious that Debian did something to RubyGems. I was bitten by this very issue as well. Debian’s RubyGems package places binaries in /var/gems (or something like that, I don’t remember the exact location) instead of /usr/bin. Fine, I understand that they don’t want foreign packages to pollute /usr/bin, which is a managed directly. But the least they can do is adding /var/gems to $PATH by default, something which they didn’t. As a result, many people who installed Rails via Debian thought that Rails is broken, when it’s actually Debian who crippled RubyGems.

“Increase entropy”? I don’t even know what this is supposed to mean in the context of Ruby software distribution.
“Harm your systems’ security”? As I’ve already stated, vendoring has both pros and cons. If one dynamically depends on a library, then it means that the system administrator and library author are responsible for security updates, but it also means that a security update can potentially break the application. By vendoring stuff, the responsibility of security updates is mostly shifted to the application developer. This is a trade-off: there’s nothing wrong with it, and whether it’s the best thing to do depends on the situation. Gunnar however has an extremely purist view, along the lines of “if you don’t agree with us then you’re an idiot, regardless of the circumstances”.

It seems more like that Gunnar’s words are carefully picked, with the goal of creating knee-jerk reactions. Debian said pretty much the same thing about Autopackage in the past, and now they’re doing it again with RubyGems.

Luke, Gunnar, will you realize that all you’re doing is ranting about a problem, without offering any solutions? And no, creating native Debian packages is not a solution, the world isn’t comprised of just Debian.

Hongli Lai: Wanted: Passenger testers using 64-bit OS X and 64-bit Linux

Mon, 05 May 2008 11:19:46 +0000

Passenger 1.0.2 fixed MacOS X compatibility. Unfortunately, the OS X fix broke compatibility with 64-bit Linux. The 1.0.4 release was supposed to fix it for both platforms. Before we released 1.0.4, we had an OS X user test it, and the test result was positive. But soon after the release we got bug reports from people, telling us that 1.0.4 broke OS X compatibility.

The main problem lies in the file descriptor passing code. We’re struggling to get it right:
1. The original code was copied from Ruby. Apparently that didn’t work well on 64-bit MacOS X. This also means that file descriptor passing on 64-bit MacOS X Ruby is also broken, not just Passenger.
2. There is generally very little documentation about file descriptor passing. We found a bunch of code samples, but it turns out that all of the ones that we’ve tried have portability problems. The developer man pages don’t provide a lot of information on file descriptor passing.

We want to get this right, but we don’t have any 64-bit machines at all. So we’re looking for people who can help us with testing Passenger on 64-bit OS X and 64-bit Linux. We’d like to have at least 3 people from both categories. If you’re interested, please send a message to our discussion board, or join #passenger on irc.freenode.net.

Thank you.

Hongli Lai: Phusion Passenger (mod_rails) version 1.0.2 released, and more

Tue, 29 Apr 2008 21:43:15 +0000

Passenger version 1.0.2 has been released. We’ve finally finished our corporate blog now, so visit http://blog.phusion.nl/ for the release announcement, overview of changes, upgrade instructions, and more.

Now that we finally have an official corporate blog, people can stop complaining about my blog template. Note that our corporate blog’s template isn’t finished yet: it still needs some polish.

Hongli Lai: The hidden corners of Passenger

Sat, 26 Apr 2008 19:15:11 +0000

There are a few technical achievements in Passenger that we haven’t actively marketed. But we’d like the community to be aware of them, so we’ve written this blog post.

Unix domain sockets

Not too long ago, Thin announced support for Unix domain sockets. This gave Thin an incredible speed boost. Switchpipe soon followed, with alpha support for Unix domain sockets.

It has always surprised me that the Ruby/Rails web containers didn’t support Unix domain sockets until early 2008. Unix domain sockets aren’t exactly rocket science or exotic: the X Window System has used them for client-server communication on localhost for as long as I can remember. Database systems such as MySQL prefer to use Unix domain sockets on localhost as well. It’s also well-known that Unix domain sockets are faster than TCP sockets.

There’s one down side to Unix domain sockets though: you have to remove them. Usually, server processes remove them during exit. But if the system crashes, or if those processes crash, then the socket files are never removed, and the system administrator will have to do that manually. Ouch, maintenance overhead.

Passenger uses Unix sockets extensively, and has done so since the first release. But there’s one difference compared to Thin and Switchpipe: Passenger uses Unix sockets in the abstract namespace (as opposed to on the filesystem) whenever possible. Abstract namespace Unix sockets have filenames, just like regular Unix sockets, but they do not appear as files on the filesystem. This means that after a reboot, or if Apache crashes, no stale files will be left on the filesystem. This way, the system administrator doesn’t have to manually remove stale files if something went wrong.

Passenger strives for a concept that we call “zero maintenance”. We believe that software should Just Work(tm), and system maintenance overhead should be as low as possible. The system administrator shouldn’t have to worry about stuff like stale files, if he doesn’t have to. Ideally, the system administrator should be able to forget that he ever installed Passenger at all. The usage of abstract namespace Unix sockets is one of the mechanisms that we use to achieve that goal.

PID files (or the lack thereof)

Mongrel and Thin write so-called PID files to the filesystem. PID files contain the PIDs of background processes, and are used for shutting down those processes. But if the system crashes, or if those background processes crash, then the PID files will never be removed, and they become stale PID files. Early Mongrel versions refuse to start if there are stale PID files, forcing the system administrator to remove them manually. Passenger on the other hand doesn’t use PID files at all. If one shuts down Apache, then all Rails processes are shut down as well.

So what happens if Apache crashes? The Rails processes will exit as well. We use so-called “owner pipes” in Passenger, pipes that are shared between Apache and the Rails processes. Owner pipes are essentially a mechanism for reference counting processes. If Apache crashes, then the owner pipe is automatically closed. The Rails processes will detect this, and will shut down gracefully.

Hongli Lai: Passenger (mod_rails): community response, donations, InfoQ interview

Tue, 22 Apr 2008 20:45:42 +0000

Community response: one week after the release

It has been a little more than one week since the first public release of Passenger. In this one week, lots and lots of people have blogged about us, including even O’Reilly. The feedback has been overwhelmingly positive!

A few nice quotes from the community:

“As many of you have already done, I hopped on board to see how it worked and was amazed on how easy it was to get up and going.” — O’Reilly
“So far I’m happy to report that, as advertised, it’s dead easy to use… and the performance seems solid.” — zerosum.com
“Mod_rails seems to be a surprising stable and finished product, and this for a first public release! I’m anxious to check out their Ruby Enterprise Edition.” — Handermann.be

Thanks for the praise, everybody.

“Enterprise licenses” (a.k.a. donations)

One may wonder why our “Enterprise”/Donation page was taken down so soon. It’s because people have been very, very generous with regard to donating to us. In fact, so generous that Paypal decided to block our account under the guise of “anti-money laundering investigation”. But everything has been cleared up now, so the donations page is once again online.

If you’d like to help us survive the coming winters, or just want to support Passenger’s development, then please consider getting an “enterprise” license.

InfoQ interview

We’ve been interviewed by InfoQ. This interview elaborates some things about the name and the license. It also elaborates our upcoming memory optimizations for Passenger/Ruby, which allows them to use less memory. More information about this soon, honestly.

On a side note, we’re preparing the release of Passenger 1.0.2. It will soon be available.

Hongli Lai: Phusion Passenger Enterprise Licensees, check your mail

Wed, 16 Apr 2008 19:02:31 +0000

Hi there,

Our apologies for the later than usual update, but things are getting really busy here at Phusion and this has been the first opportunity that we’ve been able to blog about the latest developments again! First off, we’d like to take the opportunity to ask all enterprise licensees to check their mail as they’re bound to find something nice there

Also, the next batch of enterprise licenses will become available very soon. We just wanted to let you guys know that we’re still alive and are working around the clock on some very interesting projects for our clients. Regardless of this, we want you guys to know that we’re still actively involved in the development of Phusion Passenger and Ruby Enterprise Edition and for those who want to know a bit more about Phusion Passenger and Ruby Enterprise Edition, we’ve recently done a mini interview with the fine people at infoq, and we encourage you to go check it out over here.

Even though we try to stay as actively involved as possible, we would also like to encourage the community to submit patches for the improvement of Phusion Passenger. We’re confident that we can make Passenger even better together (wow, that even rhymes so it must be true right? ;-)).

Also, it has come to our attention that some of you guys have expressed a concern about the memory use of Phusion Passenger and we have reason to believe that the memory wasn’t always measured in an accurate way. This is for the greater part the result of a well known problem with the innacurate display of memory usage by ps and/or top.

We’ll try to write an article on this as soon as possible where we will address this ‘issue’ with the appropriate measuring tools as well. Hopefully, this should set the record straight and take away any reluctance to try out Phusion Passenger as we’re pretty confident that Phusion Passenger uses a lot less memory than some people seemed to have inferred.

We’d like to leave it at this for now, and we’ll keep you guys posted and would love to stay in touch with you guys :-).

Cheers,
Hongli Lai
Ninh Bui

Hongli Lai: PHUSION PASSENGER (MOD_RAILS) PUBLIC RELEASE, HECK, IT’S ABOUT TIME :-)

Fri, 11 Apr 2008 19:19:47 +0000

After weeks of teaching it hardcore computer science kung-fu, we’ve finally reached a stage in that we think our little baby is now all grown up. It is no longer solely dependent of our care, and from this moment on, Phusion Passenger is available for the masses.

Phusion Passenger has been the result of many many many (did I say many? I mean MANY) hours of hard work by both Hongli and I ( Ninh ), but it couldn’t have been made possible without the help of our beta testers. In particular, we’d like to thank Dallas of Dreamhost for being so patient and we’re looking forward to seeing Passenger being fully integrated in your hosting plans. Also, we’d like to thank Alex of Twitter for trying his best to test Passenger in time.

Also, we’d like to thank the Ruby on Rails core team for their insightful feedback and support, and in particular, we’d like to thank David Heinemeier Hansson, Jeremy Kemper and Pratik Naik for this.

Yukihiro Matsumoto (matz), thank you for the kind words on our project, it really kept us motivated.

Ryan Bates of Railscasts, thank you so much for doing the screencast on Phusion Passenger. We’re looking forward to meeting you in person at Railsconf and being able to thank you properly Depending on the paint we’re allowed to carry with us, shaking a green hand may be involved

Antonio Cangiano, thanks for the coverage. It was when we saw that a technical evangelist at IBM had written about us that it truly hit us that we were doing something important.

Russel Norris, you really should not underestimate the importance of your great sense of humor for an open source project :-D.

Since we’re from the Netherlands, we’d also like to thank our fellow dutchmen Stefan Fountain of Soocial and Thijs van der Vossen of Fingertips for their support on this project.

Also, thank you Weyert de Boer for being a great friend and helping us streamline the markup of our website. Even though we haven’t been able to complete it yet, we’re confident that we’ll be able to finish this with your help and we thank you for this.

Lastly, we’d like to thank Hans Scholten of the University of Twente for offering to help us on writing a scientific paper on our work with Ruby Enterprise Edition and Phusion Passenger.

To you the reader, we hope that you’ll enjoy using Passenger and we thank you for your patience. We’ll be seeing you guys at Railsconf USA 2008 (and probably Apachecon USA 2008 as well! )

Be sure to check us out there, since we’ll be doing a talk on Phusion Passenger (mod_rails) and our garbage collector optimization work on MRI. The talk material will be based on our scientific paper to be published on eprints, so the tech savvy among you guys will be in for a treat :-).

For science and all things pragmatic, this was Hongli Lai and Ninh Bui of Phusion, The Computer Science Company, wishing you a good weekend as we’ll be seeing you guys monday again. We’re going to take that well-deserved time off now ;-).

P.S. Even though we’re currently getting swamped with clients, that shouldn’t stop you from contacting us for business enquiries. As a matter of fact, we’re planning on expanding our team, so if you’re a computer scientist / developer located in the Netherlands with a lot of work experience and interested in working with us, please contact us as well

Hongli Lai: Phusion Passenger Screencast 2.0 (With Ryan Bates, 100% Scary Hand Free), plus something more…

Fri, 11 Apr 2008 14:52:47 +0000

Ninh wrote this a while ago:

“Lastly, we’d like to take the opportunity to thank Ryan Bates of Railscasts for creating an excellent screencast on Passenger, and you’ll be able to see this for yourselves at the public release of Passenger this week. Needless to say, it totally kicks the *censored* out of our initial screencast :-D”

Are you guys ready?

Hongli Lai: Phusion Passenger (mod_rails) update

Tue, 08 Apr 2008 19:05:18 +0000

So you guys are probably still waiting for the release of Passenger, and I’m pleased to say that we’ve been able to include the test results from Dreamhost (thanks Dallas :D) in the first public release that you’ll see within this week. (Hold on just a little longer guys , with a bit of luck, Twitter’s results will come in as well prior to the release of Passenger this week).

Instead of writing an elaborate blog post, I’ll keep it concise for the time being as we’re currently getting swamped at Phusion.

Today, after planning to do this for quite a long time now, we went to go see our Operating Systems/Computer Architecture teacher, Ir. Hans Scholten to discuss the possibilities for receiving ECTS for our work on both Passenger and Ruby Enterprise Edition (our ruby branch containing our garbage collector optimization).

From this, it became clear that our work wasn’t something trivial and Hans elaborated on the possibilities for publishing this work scientifically. Needless to say, we’re more than thrilled about this. After all, we’re “Phusion, the computer science company” right?

All kidding aside though, we do intend on taking Hans up on his offer to write a scientific paper on our work, seeing as this is just too great an opportunity to ignore. With his help, we’re more than confident about the quality of its contents.

Lastly, we’d like to take the opportunity to thank Ryan Bates of Railscasts for creating an excellent screencast on Passenger, and you’ll be able to see this for yourself at the public release of Passenger this week. Needless to say, it totally kicks the *censored* out of our initial screencast .

To all the people that have submitted a voice sample, we’ll be crediting you on the screencast section of the Passenger as well .

Hang on in there for just a little longer people , we’re almost there!

Thanks,

Hongli Lai
Ninh Bui

Hongli Lai: Robustness comparison between Phusion Passenger, Thin, Ebb and Mongrel

Mon, 07 Apr 2008 09:19:29 +0000

We’ve written a robustness comparison between Phusion Passenger (mod_rails), Thin, Ebb and Mongrel. See Ninh’s blog for the article.

Hongli Lai: The moment you’ve all been waiting for (and so have we)

Fri, 04 Apr 2008 17:35:50 +0000

Here it is: the moment you’ve all been waiting for.

Hongli Lai: An update on Passenger (mod_rails) related stuff

Thu, 03 Apr 2008 12:26:30 +0000

Since we’re almost nearing the weekend, we thought we’d give you guys a little heads-up on the things we’re currently doing.

As you may have recalled from several days ago, we’ve asked for your voice in the most literal sense of the word for our new (series of) screencasts that we’re planning on doing.

We’re pleased to say that a lot of you guys out there have been sending in your voice samples, and to our great surprise, it wasn’t just guys who were sending in their voices :D. More on that later though, and for now, we’d like to again take the opportunity to thank you all.

Among those submissions, we’ve also received one from Ryan Bates which you may already know from his excellent Railscasts website. Greatly impressed by the quality of his previous work, we’re pleased to announce that Ryan Bates has agreed on doing the first ‘official’ screencast on Passenger. This screencast will be based on the latest beta that we’ve provided him with: don’t even bother asking him to leak it though, as he has assured us that such attempts will be in vain ;-).

But what about all those submitted voice samples? Well, we’re currently planning on doing a demo reel on Phusion as a company as well, and needless to say, we’re going to need a voice over for this as well.

It is for this reason that we’ve decided to take the people who’ve submitted their voice samples into consideration for this, since you all sounded more than suitable for this. Needless to say, this is only possible if you’re willing to do this, and we’ll be sure to contact you guys on this soon. Due to our busy schedules however, we may not be able to contact each and every one of you, and we ask for your understanding on this.

Whichever the case may be, we will however be crediting each and everyone of you on the Passenger screencast site.

Regarding the demo reel, we’re currently in talks with several video production companies and individuals for helping us create this, and we’ll probably be able to share some more information on this with you guys soon.

Concerning Passenger, we’ve decided to increase our sample space by accepting some additional large Ruby on Rails hosts for beta testing, in the hopes of being able to release it to the public even sooner. If you’re attending Railsconf like we’re planning on doing, then you’ll probably be able to ask them about their findings on Passenger as well

Also, you may have read our article on the performance of Passenger compared to other popular solutions out there. However, we haven’t touched the subject of stability/robustness yet, and since Passenger was designed for this as well, we’ve also tested this on the other popular solutions. We’ve been able to infer some interesting results, and we’ll be posting a detailed article on this as soon as we’re done doing administrative work for Phusion.

Lastly, even though we’re getting swamped at Phusion, this shouldn’t stop you from contacting us in case of a question/fanmail ;). We’d be more than happy to receive/answer them

With kind regards we are, your friends at Phusion,
- Hongli Lai
- Ninh Bui

Hongli Lai: Introducing Ruby Ain’t Hypertext Preprocessor (RAHP)

Tue, 01 Apr 2008 17:58:26 +0000

By the time of this writing, a lot of people have been claiming that Passenger will cause a revolution in the Ruby on Rails community. And even though we agree with them to a certain point ( ), we believe that one should always have a trump card in case all else fails. We’ve been working on our trump card for quite some time now in great secrecy.

Today, we’re pleased to announce this trump card to the world: Ruby Ain’t Hypertext Preprocessor (RAHP).

What is RAHP?

RAHP is a Ruby parser written in PHP, which will enable you to run all your Ruby applications within a PHP environment! This was possible due to the fact that Ruby and PHP are both Turing Complete.

Since PHP still enjoys more support among webhosts than Ruby on Rails, we’ve decided to combine the best of both worlds with RAHP. And as you may already have heard from a friend, PHP scales really well in Enterprise environments, unlike Rails applications.

But wait! It doesn’t end there! Using Zend optimizer we’ve been able to infer a tremendous performance boost, and we think we may need to retract our previous benchmark post on Ruby vs PHP. :$

Long story short:

RAHP will enable you to run your Rails applications pretty much on every webhost out there!
Because RAHP is based on PHP, your Rails applications will be able to use the thousands of existing PHP libraries with ease!
It is common knowledge that PHP is extremely scalable (unlike Rails). So by the transitive property, your Rails applications will become scalable as well, with the help RAHP on Rails!

Come to think of it, we’re considering to discontinue our work on Passenger in favor of RAHP. We’ll have to sleep on it though.

Stay tuned!

Hongli Lai: Matz on Passenger

Tue, 01 Apr 2008 17:38:08 +0000

Recently, Yukihiro Matsumoto (a.k.a. Matz, the creator of Ruby) blogged about Phusion Passenger (mod_rails) on his weblog. Since our Japanese is a bit rusty, i.e. non-existant , Matz was kind enough to mail us a summary in English from which we quote:

It is often said that Rails is weak on deployment; PHP runs fairly fast just by uploading scripts. Rails is slow on development mode, and requires restarting on production mode (and bit complex to configure). modrails might be the answer for it.

Needless to say, coming from Matz, this made us feel real warm and fuzzy from inside

Hongli Lai: Benchmark: Passenger (mod_rails) vs Mongrel vs Thin

Mon, 31 Mar 2008 14:04:51 +0000

Since the initial release of the teaser screencast, a lot of people have asked about Passenger’s (a.k.a. mod_rails) performance compared to Mongrel and Thin. Here’s the benchmark that you’ve all been waiting for.

In this benchmark, we compare the following software:

System specification:

Intel Core 2 Duo, T5300 @ 1.73GHz
2 GB RAM
Ubuntu Linux 7.10
Apache 2.2.4 (Ubuntu package)
Ruby 1.8.6 (Ubuntu package)

Test applications:

Typo 5.0.3. This is an out-of-the-box setup with only a hello world post. Uses semi-static caching.
Petstore revision 360, an application that the JRuby guys use for benchmarking. At first it wouldn’t start in Passenger because of a bug Petstore: they forgot to add require 'forwardable' in app/models/cart_items.rb. I added this line myself.
Eldorado 0.9.1. This is an out-of-the-box setup with only a single forum and a single post added.

Typo and Eldorado both use SQLite 3. Petstore uses MySQL, because for some reason Petstore performs very badly when SQLite 3 is used.

People have also recommended Lovd By Less as test application, but I couldn’t get it working (database setup failed).

The applications, when served by Passenger, are setup at the following test domains:

http://typo.test/
http://petstore.test/
http://eldorado.test/

Both the Mongrel cluster and the Thin cluster are setup at http://mongrel_cluster.test/. (Only either Mongrel or Thin is running during the benchmark, not both at the same time.)

And of course, we always use the production environment.

Backend configuration:

Default Apache configuration, as provided by Ubuntu.
Passenger: RailsMaxPoolSize is set to 10. This means that Passenger will never spawn more than 10 Rails application instances. Note that Passenger is compiled without optimizations, so if you install Passenger yourself and benchmark it, you might find that Passenger is slightly faster.

Mongrel cluster: a cluster of 10 servers for each test application, behind mod_proxy_balancer, using default settings:

ProxyPass / balancer://mongrel_cluster/
ProxyPassReverse / balancer://mongrel_cluster/
<Proxy balancer://mongrel_cluster>
	BalancerMember http://127.0.0.1:3000
	BalancerMember http://127.0.0.1:3001
	BalancerMember http://127.0.0.1:3002
	BalancerMember http://127.0.0.1:3003
	BalancerMember http://127.0.0.1:3004
	BalancerMember http://127.0.0.1:3005
	BalancerMember http://127.0.0.1:3006
	BalancerMember http://127.0.0.1:3007
	BalancerMember http://127.0.0.1:3008
	BalancerMember http://127.0.0.1:3009
</Proxy>

Thin cluster: same configuration as Mongrel cluster.

Benchmark: Mongrel clusters

First we benchmark Typo served by Mongrel clusters. Because the actual application code is loaded by Mongrel during the first request, we do not want to count the first request’s time. So we run the following command to “warm up” the Mongrels, and discard its result:

ab -n 1000 -c 100 http://mongrel_cluster.test/

Next, we test 10,000 requests with 100 concurrent users:

ab -n 10000 -c 100 http://mongrel_cluster.test/

Result:

Concurrency Level:      100
Time taken for tests:   32.628480 seconds
Complete requests:      10000
Failed requests:        0
Write errors:           0
Total transferred:      54890000 bytes
HTML transferred:       51480000 bytes
Requests per second:    306.48 [#/sec] (mean)
Time per request:       326.285 [ms] (mean)
Time per request:       3.263 [ms] (mean, across all concurrent requests)
Transfer rate:          1642.83 [Kbytes/sec] received

We repeat these steps Petstore and Eldorado.

Petstore:
(for Petstore we benchmark the URI “/shop/viewCategory.shtml?category=DOGS” because it’s written in the INSTALL file)

Concurrency Level:      100
Time taken for tests:   109.786026 seconds
Complete requests:      10000
Failed requests:        0
Write errors:           0
Total transferred:      40750000 bytes
HTML transferred:       37680000 bytes
Requests per second:    91.09 [#/sec] (mean)
Time per request:       1097.860 [ms] (mean)
Time per request:       10.979 [ms] (mean, across all concurrent requests)
Transfer rate:          362.47 [Kbytes/sec] received

Eldorado:

Concurrency Level:      100
Time taken for tests:   70.889388 seconds
Complete requests:      10000
Failed requests:        0
Write errors:           0
Total transferred:      41130000 bytes
HTML transferred:       35530000 bytes
Requests per second:    141.06 [#/sec] (mean)
Time per request:       708.894 [ms] (mean)
Time per request:       7.089 [ms] (mean, across all concurrent requests)
Transfer rate:          566.60 [Kbytes/sec] received

Benchmark: Thin clusters

The benchmarking steps for Thin are the same.

Typo:

Concurrency Level:      100
Time taken for tests:   25.795470 seconds
Complete requests:      10000
Failed requests:        0
Write errors:           0
Total transferred:      54910000 bytes
HTML transferred:       51480000 bytes
Requests per second:    387.66 [#/sec] (mean)
Time per request:       257.955 [ms] (mean)
Time per request:       2.580 [ms] (mean, across all concurrent requests)
Transfer rate:          2078.78 [Kbytes/sec] received

Petstore:
(for Petstore we benchmark the URI “/shop/viewCategory.shtml?category=DOGS” because it’s written in the INSTALL file)

Concurrency Level:      100
Time taken for tests:   96.874308 seconds
Complete requests:      10000
Failed requests:        42
   (Connect: 0, Length: 42, Exceptions: 0)
Write errors:           0
Non-2xx responses:      42
Total transferred:      40630350 bytes
HTML transferred:       37547028 bytes
Requests per second:    103.23 [#/sec] (mean)
Time per request:       968.743 [ms] (mean)
Time per request:       9.687 [ms] (mean, across all concurrent requests)
Transfer rate:          409.58 [Kbytes/sec] received

Eldorado:

Concurrency Level:      100
Time taken for tests:   71.311031 seconds
Complete requests:      10000
Failed requests:        5
   (Connect: 0, Length: 5, Exceptions: 0)
Write errors:           0
Non-2xx responses:      5
Total transferred:      41137070 bytes
HTML transferred:       35518568 bytes
Requests per second:    140.23 [#/sec] (mean)
Time per request:       713.110 [ms] (mean)
Time per request:       7.131 [ms] (mean, across all concurrent requests)
Transfer rate:          563.34 [Kbytes/sec] received

Benchmark: Passenger

The benchmarking steps for Passenger are the same.

Typo:

Concurrency Level:      100
Time taken for tests:   25.131980 seconds
Complete requests:      10000
Failed requests:        0
Write errors:           0
Total transferred:      55200000 bytes
HTML transferred:       51260000 bytes
Requests per second:    397.90 [#/sec] (mean)
Time per request:       251.320 [ms] (mean)
Time per request:       2.513 [ms] (mean, across all concurrent requests)
Transfer rate:          2144.92 [Kbytes/sec] received

Petstore:
(for Petstore we benchmark the URI “/shop/viewCategory.shtml?category=DOGS” because it’s written in the INSTALL file)

Concurrency Level:      100
Time taken for tests:   104.428742 seconds
Complete requests:      10000
Failed requests:        0
Write errors:           0
Total transferred:      40300000 bytes
HTML transferred:       36700000 bytes
Requests per second:    95.76 [#/sec] (mean)
Time per request:       1044.287 [ms] (mean)
Time per request:       10.443 [ms] (mean, across all concurrent requests)
Transfer rate:          376.86 [Kbytes/sec] received

Eldorado:

Concurrency Level:      100
Time taken for tests:   72.587018 seconds
Complete requests:      10000
Failed requests:        0
Write errors:           0
Total transferred:      41660000 bytes
HTML transferred:       35530000 bytes
Requests per second:    137.77 [#/sec] (mean)
Time per request:       725.870 [ms] (mean)
Time per request:       7.259 [ms] (mean, across all concurrent requests)
Transfer rate:          560.47 [Kbytes/sec] received

Summary

Here’s a nice summary of the requests per second, in a table and in a graph:

	Typo	Petstore	Eldorado
Mongrel	306.48	91.09	141.06
Thin	387.66	103.23	140.23
Passenger	397.90	95.60	139.27

Conclusion

I was pleasently surprised by the results. During Passenger’s development, our performance baseline was Mongrel behind mod_proxy. According to earlier (simple and naive) tests, Passenger performed similar to Mongrel. I expected Thin to be a bit faster than Passenger. These new benchmarks however suggest Passenger is faster than Mongrel, and is on par with Thin.

But you know what they say: there are lies, damn lies, and statistics. Your hardware and software is different, so if you benchmark it yourself, you’re likely to get (slightly?) different results.

Hongli Lai: Wanted: native English speaker for Passenger (mod_rails) screencast

Sat, 29 Mar 2008 16:32:51 +0000

I have good news and bad news.

Let’s start with the good news. Passenger (mod_rails) is almost ready for release. The code is as good as done (actually, it has been as good as done for well over a week now). The website is almost done. We’re just awaiting final test results from remaining beta testers.

The bad news: lots of people have commented about our current screencast, in particular about the now-infamous “scary green hand”. Being the perfectionists that we are, we’d like to redo the screencast before we release Passenger. Neither Ninh nor I are native English speakers, so we’re looking for a native English speaker who can help us with the voice-over part of the screencast. Preferably, someone with enough ‘gravitas’ in their voice, or in the case of a female voice, the ‘kindness’ should come out dripping from out of our audio speakers.

Would you like to volunteer? If so, please provide us with an audio sample. You can do this by either replying in this post, or by emailing us at “hongli at plan99 dot net” or “ninh dot bui at gmail dot com”! Thanks in advance.

Hongli Lai: Good Rails app for benchmarking?

Fri, 28 Mar 2008 00:52:48 +0000

Does anyone know a good Ruby on Rails app that I can use for benchmarking Passenger (mod_rails)? I don’t have any apps that I’m allowed to use for this purpose. Mephisto and Typo are out because they both use page caching, which would make the benchmark pointless.

Hongli Lai: Passenger and other Ruby frameworks

Thu, 27 Mar 2008 11:11:48 +0000

Traffic for our Passenger (mod_rails) preview site has increased three fold ever since RailsEnvy and weblog.rubyonrails.org (thanks David) linked to us. We’ve received quite some emails from interested people. One of the questions that we receive most is:

“What about other Ruby frameworks, like Merb, Camping, etc?”

There is the following saying: jack of all trades, master of none. At the moment, our goal is to make Passenger a great deployment system for Ruby on Rails, one that is matched by none in terms of ease of use and low maintenance overhead.

But some people ask:

“What’s wrong with mod_ruby?”

For one, look at its website. Last release was from 2006. I’m not sure whether it’s still maintained anymore.
Nobody (or almost nobody) uses it in production. This fact alone says a lot.
The Rails wiki page for mod_ruby lists a pretty bad limitation:

mod_ruby uses a shared interpreter per Apache process, which means that multiple Rails applications on the same Apache setup with mod_ruby would share the framework classes. This doesn’t work well with the Rails model of using class-level configuration attributes, so it’s considered unsafe to use mod_ruby and Rails with more than one application running per Apache setup, because different applications may start sharing the classes.

Granted, these are implementation problems and not conceptual problems, so they can be fixed. So some people ask:

“Why didn’t you just make a solid mod_ruby implementation?”

First, we must agree on the premise that Rails is a more popular framework than other Ruby frameworks. Wait, hold your flame thrower, this is not the reason.

Next, try thinking from the point of view of a new Rails developer. He’s introduced to lala-land: Don’t-Repeat-Yourself, convention over configuration, stuff that’s supposed to Just Work(tm).

Then he deploys his Rails app and finds out that he has to setup a Mongrel cluster (1 command but usually with 3 or 4 options that he’ll have to look up), configure Apache to proxy to the cluster (about 5 or 6 lines that he’ll have to setup), configure init.d/daemon tools/monit/god/whatever so that his Mongrel cluster is started at system startup, etc.

Oh, and you need to repeat this for every single Rails app.

Ouch, is this the proper Rails experience? I don’t think so. This is Passenger’s philosophy: deployment should be in line with the Rails philosophy. In other words, it should be brain-dead simple. Just imagine the money that you’ll save on health care for your system administrator if he doesn’t have to worry about all that stuff anymore.

Of course, the Rails philosophy doesn’t prevent deployment of other Ruby framework apps from being easy. So what’s holding us back? Well, here’s a few things that Passenger does but mod_ruby doesn’t:

Remember the screencast that we showed you? You can deploy a Rails app to a virtual host without any Ruby-specific or Rails-specific configuration! From this fact one can deduce that Passenger somehow automatically detects Rails applications (and yes, this autodetection can be disabled). Autodetection heuristics are framework-dependent. We’ll need different heuristics for each framework.
Passenger preloads framework code and application code in order to reduce startup time and to reduce memory usage. Preloading logic is framework-dependent.
Look at Troubleshooting obscure errors with mod_ruby. Notice the word “obscure errors” - errors should never be obscure or hard to track!
We’ve seen many people failing to deploy Rails apps because of errors during application startup. Lighttpd’s FastCGI implementation redirects those errors to /dev/null. Not good, this fact bit me 2 years ago — when I was deploying my first Rails app — and made me wonder why the thing wouldn’t work. Mongrel prints errors either to stderr or to its log file. But people who are new to Rails deployment don’t know that they should be looking in the log files (or that there are log files in the first place). Plus, looking into the log file is yet-another-step. Wouldn’t it be great if you can see startup errors right in the browser?

Passenger provides user-friendly error pages. If a Rails app failed to start, then Passenger will tell you what happened. A few possible error pages are:
- Whether a (specific version of a) system-wide Rails framework failed to load. Passenger will even tell you the correct command to (re)install it.
- Whether a vendor’ed Rails framework failed to load.
- Whether the app failed to start because of a syntax error, or a missing gem. Passenger will suggest solutions.
- Whether a database error occured during startup. Passenger will tell you to check your database.yml and to run ‘rake db:migrate’.
Here are previews of typical error pages:

As you can see, a lot of these error pages are framework-dependent.

As you may have noticed by now, we’ve put a lot of work into making sure that deployment is dead-easy. This is only possible if Passenger has knowledge about the frameworks themselves. We have a tight schedule and we only have 2 developers working on Passenger. Rails is more popular so it has more priority. We’re working around the clock to get this thing released.

Or we can get rid of the autodetection/preloading logic and just provide generic configuration options. But would you really want that? By no means should we force the system administrator to configure things. By default, stuff should Just Work(tm) out-of-the-box.

Hope this clears things up.

Hongli Lai: What’s the story morning glory?

Wed, 26 Mar 2008 20:55:54 +0000

Thought I’d give you guys a little update on where we currently stand (yeah, we’re still alive ;)). As you may have already know, we’ve announced Phusion Passenger (a.k.a. mod_rails) last week and the reaction has been overwhelmingly positive!

Our beta testers now not only include several of the Ruby on Rails core team, but also some of the largest Ruby on Rails webhosts you may know of. Needless to say, we’re very thrilled about this, and in particular, we’d like to thank Pratik Naik, David Heinemeier Hansson and Derek Perez for their insightful feedback and support. Also, we’d like to thank community members such as Russel Norris for keeping the conversations fun. With this kind of support, we’re convinced of the quality of Passenger, since a lot of love has gone into making this.

Again, we’d like to express our gratitude to all for supporting us on this. It’s important to note that for an open source project, it’s things like these that keep developers like us going on.

And while we’re on the subject of ‘going on’: we’ve been working around the clock the past few days to get things done as fast as we can. While Passenger is nearing completion and its website still needs a bit of work, it still remains uncertain if we’re able to obtain a preview/announcement slot at Railsconf . We’d like to let you guys know however, that we’re putting a lot of effort into making this happen, since we really believe that Passenger has the potential to set of a revolution in the Rubyverse by possibly allowing more people to enjoy Ruby on Rails.

For now we ask you for your patience and we’ll try to keep you guys posted on a regular basis regarding our activity.

With kind regards we are,

Hongli Lai
Ninh Bui

- Phusion.nl

Hongli Lai: Making PStore reaaaally fast (and stable)

Wed, 26 Mar 2008 18:04:08 +0000

PStore is a library in Ruby that “implements a file based persistance mechanism based on a Hash”. Ruby on Rails up until version 1.2 uses PStore as the default mechanism for storing sessions. But it’s pretty well-known that PStore “sucks”: people say that it’s slow, causes file corruptions, etc. This is one of the reasons why Rails 2.0 uses the cookie session store by default: it’s faster and needs less maintenance at the cost of a few (easy-to-avoid) caveats.

If you look at the Mongrel FAQ, then you’ll see that 3 items are devoted to telling you that PStore is the work of satan and that any sane web developer should ritually burn it. But this particular sentence caught my eye:

“Other things that can cause big pauses are:
- …
- Locking files wrong. Multiple processes locking files is a delicate thing to do. “

Locking files wrong? How can that possibly be? File locking, just like mutex locking, isn’t really rocket science.

Note that PStore’s RDoc advertises itself as transactional:

“# The transactional behavior ensures that any changes succeed or fail together.
# This can be used to ensure that the data store is not left in a transitory
# state, where some values were upated but others were not.”

I decided to take a look at its source code. What possibly could have gone wrong?

Well, I’m not sure what is wrong with it, but the relevant code, PStore#transaction, is a bit messy. I had a hard figuring out what it is exactly doing and why, but I figured that it does these things:

It locks the file.
It reads and unmarshals the file.
It generates an MD5 checksum of the file.
It runs the transaction block, then generates an MD5 checksum of the new contents of the file. The file is only written to if the MD5 checksum or the size of the new content doesn’t match that of the original file.
It writes new data to a temp file, then renames that to the original file (or at least, I think that’s what it does; it seems to use 2 temp files files). File renames are atomic on Unix but not Windows. This is why on Windows, one needs to implement file recovery as well. PStore seems to have some code for recovery, but it’s unclear how well that works. It doesn’t match the algorithm given on MSDN.

It’s not clear whether PStore has locked everything correctly. Oh, and PStore isn’t thread-safe (though it is reentrant).

PStore is usually used for writing small amounts of data, so calculating an MD5 is definitely not worth it - just write the file already!
Writing to a temp file ensures atomicity on Unix, but it adds another system call, and system calls are expensive. If one is able to open the file for writing-and-appending, then writing to it shouldn’t raise any errors except in rare conditions, such as out-of-disk-space conditions or hardware errors. Furthermore, I’ve never seen anybody using PStore for anything other than for session data and other not-so-important stuff, so performance is more important.
Aah, Windows…. All the recovery code just to work around the fact that Windows doesn’t support atomic file renames, cause a lot of performance loss.

So I rewrote PStore. The code is now easier to read and is faster. And as far as I know, everything is locked correctly so there shouldn’t be any concurrent issues.

By default, it doesn’t try as hard to ensure file integrity because harddisk I/O errors are very rare, but if file integrity is really an issue, then you can set pstore.ultra_safe_transactions = true to enable it. This option only has effect on Unix though: as of now, I haven’t bothered writing complex recovery code for Windows.

This is the benchmark program that I used:
http://pastie.org/170997

The results are as follows:

As you can see, PStore has become 2 times of even 3 times faster, depending on whether is ultra_safe_transactions is enabled! But let’s see what kind of effect it has on a dummy Rails 2.0 app that uses the PStore session store:

Dummy Rails 2.0 app, using original PStore library: 170.22 requests/sec
Dummy Rails 2.0 app, using my PStore library: 196.72 requests/sec

A small performance increase. (Though the cookie session store is still faster: 221.74 requests/sec.)

You can download it here:
http://pastebin.com/f163702f2

If you want your Rails app to make use of it, simply save it as “lib/pstore.rb”. For maximum stability, Rails’s PStore handler should be modified to ignore any errors encountered during unmarshalling.

One of these days I’ll send a patch to ruby-core so that this can be merged back upstream. But for now Passenger (a.k.a. mod_rails) has priority.

Hongli Lai: Rails deployment a solved problem? Just upload & it works?

Thu, 20 Mar 2008 20:55:12 +0000

A video says more than a thousand words, so just click on the video link below.

http://www.modrails.com/

Is this for real?

Yes people, this is real! We at Phusion have been developing this for a while now, and it’s almost done. Currently we’re in private beta testing phase. Our beta testers include several Ruby on Rails core developers, and so far the feedback from our testers is overwhelmingly positive.

Stay tuned for more news!

Hongli Lai: Memory usage comparison: Rails 1.2.6 vs 2.0.2

Wed, 19 Mar 2008 10:08:27 +0000

Wait, hold your horses!

When measuring memory, people usually measure the wrong thing. There are 3 things that one can measure:

1. Virtual memory size: All modern operating systems - probably including yours - use virtual memory. The memory that an application sees is not the same as your physical memory. Even if you only have 256 MB RAM installed, all applications on your system still see 4 GB RAM, and can use more than 256 MB RAM (on x86, that is). See the Wikipedia entry.

Even if you only have 256 MB RAM, the virtual memory size reported by a process can still be 4 GB! This is because processes can map files, hardware and other resources into its memory address space, without really loading them into physical memory. This is complicated by the fact that processes can share memory with each other. The VM size is quite meaningless when measuring real memory usage.

2. Resident Set Size (RSS)

This is the amount of the process’s memory that’s currently in physical memory. At first, this seems to be a good candidate for measuring real memory usage. But parts of the process may be swapped to the hard disk. The RSS also seems to include things like shared memory, which we clearly don’t want to measure. It’s unclear whether the RSS also contains kernel buffers and caches, which we also don’t want to measure.

3. Private dirty memory

This is the best measurement. It does not count shared memory. This blog post explains it all. The private dirty memory is easy to measure on Linux (I’m using Ubuntu 7.10), but I’m not sure how to do it on other operating systems.

4. Ruby heap information

3 is usually the best measurement - but not for Ruby! In the past I’ve explained how Ruby’s heap is implemented. Ruby allocates several heaps, and the heaps contain equally-sized slots in which Ruby objects are stored. If we measure the private dirty memory (that is, assuming fork() is never called), we’ll be effectively measuring the total heap size, including any free heap slots. Free heap slots are reused by Ruby, so they really shouldn’t be counted.

We want to count the number of used heap slots. Usually, there’s no way to do this. But I maintain a private Ruby branch (which will be released soon) with statistics information. We’ll use that for memory measurement. (My GC patch also provides statistics information, though the version I’m working on provides more accurate statistics.)

People interested in details should read a good book about operating systems. I recommend “Operating System Concepts” by Silberschatz et al. My “Operating Systems” classes professor has Dutch slides on the subject.

The measurement

After a cold start with Mongrel 1.1.3, the private dirty memories were as follows:

Rails 1.2.6: 25.5 MB
Rails 2.0.2: 19.7 MB

Nice! A 6 MB memory reduction after a cold boot!

Though, as I’ve said earlier, these numbers don’t really mean much. The application code (i.e. controller and model code) haven’t been loaded into memory yet. And Ruby doesn’t garbage collect until its heap is full. So let’s find out what happens after 3000 requests and a garbage collection run. I added these actions to PostsController:

def gc
    headers["Content-Type"] = "text/plain"
    GC.start
    render :text => ObjectSpace.statistics
end

And this route:

map.connect '/gc', :controller => "posts", :action => 'gc'

Then I ran, for both apps:

ab -n 3000 http://localhost:3000/
links http://localhost:3000/gc

The memory usages were:

Rails 1.2.6: Private dirty memory:28.1 MB
Total heap size: 14,702 KB
Free heap space: 7,789 KB

Rails 2.0.2

Private dirty memory: 20.9 MB
Total heap size: 8,059 KB
Free heap space: 2,098 KB

Nice! I didn’t expect this, but apparently Rails 2.0 uses a lot less memory!

[EDIT: Some people have contributed a chart. Thanks.]

Hongli Lai: Performance comparison: Rails 1.2.6 vs 2.0.2

Tue, 18 Mar 2008 18:52:14 +0000

Mike Hearn: moving to wordpress.com

Wed, 12 Mar 2008 18:26:44 +0000

I moved my blog to mikehearn.wordpress.com - I got tired of having to maintain it myself. If you notice any problems let me know.

Hongli Lai: RSpec patches

Thu, 28 Feb 2008 23:39:28 +0000

I totally fell in love with RSpec. Although I’m not really into BDD, I’ve found that RSpec makes my tests easier to read and easier to organize.

But their website really is a problem. Its navigation is pretty confusing. At first, I didn’t know that there are 3 more levels of pages under “Documentation” - I thought that the first page I see under “Documentation” is the only documentation available. This is because the navigation bar changes in ways that I think people would typically not expect. And their RDoc documentation looks horrible because it uses the default RDoc template.

Because they’re experimenting with Git since today, I decided to check out their sources and see whether I can contribute. I’ve redesigned their menu bar, and although it’s not perfect, it should be better than it currently is. Compare the current navigation with my redesign.

I’ve also submitted a patch for generating the RDoc documentation with my “Horo” RDoc template (which is based on the Jamis RDoc template). A sample can be found at: http://izumi.plan99.net/rspec-horo/

Hopefully these patches will get accepted.

Hongli Lai: Using custom RDoc templates in Ruby gems

Mon, 25 Feb 2008 19:40:21 +0000

I think that the default RDoc template looks horrible. The colors are not good at all, and the top frames waste a lot of space. On the other hand, the Jamis RDoc template looks a lot better.

Today I ran into a problem: RubyGems can generate RDoc documentation for projects, but how do I force it to use the right template? The Jamis template installation instructions don’t work with the Rake RubyGems task. Eventually I found out that I had to do this:

Rake::RDocTask.new do |rd|
	...
	rd.template = "./doc/template/jamis"
	...
end

And put ‘jamis.rb’ into the folder ./doc/template/

Mike Hearn: singularity: part 3

Tue, 12 Feb 2008 19:45:06 +0000

Last time, I talked about the basics of Singularity: SIPs, manifests, connections, and software isolation. Let’s have a look at how this applies to driver development.

Drivers are a Big Deal for us, because:

About 85% of all Windows crashes are caused by bad drivers
An analysis of Linux showed that driver code is 7 times more likely to contain bugs than the rest of the kernel
When drivers crash, they take the whole system with them, thus losing the users work and - by extension - users trust.
When drivers contain bugs, they can sometimes be used for privilege escalation. It only takes one privilege escalation for the bad guys to win.

So getting drivers right would be a huge win, both for reliability, and for building a foundation on which we can get security right too.

drivers in singularity

Because Singularity is a type system based micro-kernel, a driver is simply a set of Sing# classes (Sing# is C# with extra bits), shipped as verifiable MSIL bytecode images. It runs as a SIP, like any other program does. Obviously, using C# instantly solves a large class of common bugs - pretty much any exploit with “overflow” in its name is solved by using these languages.

You can’t represent the specialized instructions needed to control hardware in safe MSIL, so we need a workaround. This comes in the form of an unsafe DLL which provides objects abstracting the hardware control instructions. This DLL (Singularity.DriverRuntime) is a part of what Microsoft Research call the “trusted computing base”, but that makes it sound like it’s to do with Palladium and DRM, which it isn’t. It’s easier to think of this DLL just as logically part of the kernel, even though it doesn’t run in the context of the main kernel program. These classes have names like IoPortRange, IoIrqRange and so on, and provide a simple OO interface to programming hardware.

Singularity won’t let you construct objects of these types yourself, you have to use a set of special static method calls. And you can’t invoke them directly either. It seems we’re stuck - how do we get a reference to such an object, if we can’t construct them ourselves? And what’s the point of being so awkward?

The answer is that Singularity requires you to annotate your code with metadata, describing what hardware your driver is for and what resources it needs. This is done in the usual .NET fashion, with attributes that decorate your classes and fields. At install time, this metadata is used to build an XML manifest, describing what resources your code needs, and is input to a compile time transform, which fills out an empty method you provide. This transform is itself a part of the kernel, and is applied when you install the driver - it is what adds the protected call to the hardware abstraction DLL.

In this way, Singularity ensures that it has accurate declarative metadata on what exactly the driver needs. Because you can’t get hold of the relevant objects and thus program the hardware unless you define accurate metadata (and this is enforced by the system), the metadata is guaranteed to be accurate. That, in turn, makes it easy to solve problems like avoiding driver conflicts and figuring out the order in which drivers should be loaded at bootup.

the hole in the plan

This would all be fine and dandy, if it weren’t for The Catch (why is there always a catch?). The catch is this - any driver that can access a DMA capable device can subvert the type system by overwriting arbitrary areas of physical memory.

Direct Memory Access is an optimisation that lets hardware devices directly write data into RAM, bypassing the CPU. It’s very useful and speeds things up a lot, especially when moving lots of data around like with graphics cards, hard disks, network cards and so on. Unfortunately, DMA not only bypasses the CPU but also the MMU, thus a driver that can control a DMA capable piece of hardware can control the contents of memory. In a regular operating system that doesn’t matter, because such drivers are running in kernel mode anyway and are fully trusted. But for Singularity it’s a problem.

Fortunately for us, DRM comes to the rescue! Yes, I’m serious. Newer CPU architectures being developed by Intel and AMD feature something called, appropriately enough, an IOMMU. This does for hardware devices what the regular MMU does for the CPU - regulates memory access by mapping DMA read/writes through a set of page tables. By configuring the IOMMU, you can stop hardware from fiddling with memory. This feature was originally intended to stop people breaking into kernel space by plugging in physical pieces of hardware, which apart from the obvious downsides for DRM manufacturers can also be used to do things like unlock screens. And of course it protects computers from buggy hardware, which is not such an uncommon thing.

It’s also mighty handy for us, because it closes the hole that would mean DMA-aware drivers could break out of their type-verified jail.

how it helps security

Before you can make something secure, you need a threat model. What exactly do we want to stop?

I’m going to skip over that here and mumble something about stopping malware and viruses. It’s vague and not that useful, but it’ll help illustrate these examples.

Back in 2001 the idea of Microsoft restricting what software you could put on your system seemed grotesque. They were just coming to the tail end of the the browser wars, broadband was just beginning its rollout and the general problem of botnets had not yet reared its ugly head. The potential for abuse seemed way too high, the benefits way too low. Despite that, drivers were known to be such huge reliability problems that a signing program was instigated …. Microsoft would provide you with a unit test suite, and if it passed, that suite would sign the driver for you. The test suite checked for many common problems and didn’t require anybody to hand over their code to Microsoft. Seemed like a good idea, right?

Well, of course it was controversial. Worse, it didn’t necessarily work. Rather than pass the tests, some manufacturers cheated. Rather than risk a schedule slip, or pay the $250 verification cost, some moved the mouse and clicked OK on the unsigned driver warning for you. D’oh.

If we’re willing to be a bit despotic and require signing that cannot be disabled for some things, can we do better? Yes, but it would suck to require every driver to be signed. We can avoid this in the most common cases.

For instance, thanks to the declarative metadata, we can now statically inspect a driver and say, well, gee, this driver will only be activated for devices of class “sound card”, and it doesn’t use DMA, and it doesn’t read or write any files (we know this because we can statically verify that it can’t get any connections to the filesystem server). What can this driver do, except mess up the sound card? Not a whole lot, actually. It could still hose the box by misprogramming the hardware, but such bugs are rare. But it can’t blue-screen us, because it’s not running in the actual kernel. If it crashes, the worst that happens is audio is interrupted until the driver restarts and programs re-establish their connections (assuming we write software to handle driver crashes of course - more on that later).

Generally, we can define some set of privileges (in Singularity this is equivalent to defining some set of connections a SIP can access) which might be dangerous, and only require signing for those drivers. What might trigger such a requirement? Accessing system files or configuration is an obvious case. Being a network driver is another case (how to DoS a target without falling foul of the firewall? Become a network driver). Fortunately, i
t’s rare that drivers need access to anything other than the hardware and their own configuration data, and network drivers are typically uninteresting enough that the default drivers suffice for most people. Thus the majority of drivers can be safely written, distributed and installed without Microsoft ever needing to be involved whilst preserving the security of the system.

Isak Savo: Sweet new tool

Mon, 28 Jan 2008 20:10:32 +0000

Most experienced computer users will agree with me when I say that typing something is often quicker and easier than navigating a deep menu structure. I hate the standard windows start menu with a passion, which means I was quite happy when I found the nice open source tool called Launchy. Now just a quick <alt>+<space> away, I can search an indexed db of my launchers. That's on windows...

On Linux, I've been using a combination of terminal (requires you to know the binary name, plus tab completion only matches beginning of strings in bash), panel launchers and the deskbar applet.

Unfortunately, the deskbar applet lacks the sex appeal of Launchy, and it's always crowded with results I seldom want (yeah, I know I can configure it and add/remove plugins and what-not, but it still lacks sex appeal ;-) ).

Via planet gnome the other day, I found a new player in town - Gnome DO! It's a slick little app, very similar to Launchy on Windows, but slightly on stereoids. By default, you just type something and it'll run it for you when you hit enter. But for things which has multiple actions, there's also an "action" section which can be reached by hitting <tab>. So if I type "movie", it will display an icon showing my movie folder with default action to open it in nautilus. But right now, I don't want to do that so I hit <tab> and enter "terminal" giving me the option to open a gnome-terminal in that folder. Slick.

Now if someone could add SnagIt capabilities to the screenshot tool, I'd be an even happier Linux user. :-)