Passenger 1.0.5 has just been released. Please read the Phusion corporate blog for the announcement and upgrade instructions.
Passenger 1.0.5 has just been released. Please read the Phusion corporate blog for the announcement and upgrade instructions.
AkitaOnRails had recently interviewed us. The interview covers our company’s background and information about Passenger, among other things. He asked us quite some interesting questions, so we recommend you to read the interview. 
Luke Kanies wrote an article, in which he says that Ruby has a distribution problem. It criticizes Rails applications for vendoring a lot of stuff, and criticizes RubyGems for not being able to handle native package dependencies, among other things.
I beg to disagree. The described problem is not a pure Ruby problem: it’s a general software distribution problem. I’m concerned that people would use this as another “reason” to oppose Ruby, even though it’s not specific to Ruby.
I posted a reply to Luke’s article, and it was as follows:
Luke, I don’t really understand what you’re expecting from the Ruby/Rails community. You’re dealing with cross-platform distribution issues. That’s hard by its very nature. I was a developer in the Autopackage project (www.autopackage.org), a software packaging system that works across multiple Linux distributions. I’m currently a developer of Passenger (www.modrails.com). We’ve run into many of the issues that you mention here.
If I understand it correctly, you’re claiming that vendoring stuff is bad because:
On the other hand, vendoring stuff does have benefits:
Vendoring stuff is not only common in the Java world, but also in the Windows and MacOS X world. Windows apps tend to bundle all their dependencies (with the exception of obvious stuff, such as Internet Explorer). How many games have you seen that bundle DirectX? Actually I’d say that not vendoring stuff is only common in Linux, and languages that have strong ties to Linux, such as Perl. Here’s where Debian’s package management system and huge package repositories really shine.
There’s almost no common ground in the world of package management. We at Autopackage had to invent our own dependency resolution mechanism (similar to APT) because there’s no lowest common denominator, even amongst Linux distros. RubyGems is probably created for the same reason: not all Ruby-supported platforms have (decent) package management, so they just wrote their own. Autopackage experimented with native package management integration (i.e. being able to use the system’s native package manager to resolve dependencies) but that proved to be much, much harder than initially expected, and up until today that feature still isn’t finished. So I don’t think you can reasonably expect the Ruby community to do something about this. It’s not a pure Ruby problem: it’s a general software distribution problem.
As for being FHS-compliant: I can only say “don’t bother” if your application is cross-platform. It seems that only hardcore Linux users care about that. Outside the Linux world, FHS is being criticized by pretty much everyone. Windows and OS X users complain that application files in Linux are scattered everywhere, instead of being self-contained. Scattering files isn’t a problem in Linux because of package management, but it is a problem in all other platforms that don’t have decent package management. I’ve found that being FHS-compliant is more trouble than it’s worth.
So how do you solve this problem? I don’t think it’s possible to come up with a general silver bullet solution. So that leaves the following choices to you, the developer:
Finally, we at Autopackage fully recognized the pros and cons and vendoring/static linking. Autopackage recommends the following: dynamically depend on stuff that are common, but vendor/static link stuff that are uncommon. We believe that this is a good trade off between the pros and cons. Passenger follows this recommendation as well: we vendor the Boost C++ library. Few people have Boost installed, and when they have it installed it often isn’t the version that Passenger requires. Installing Boost is a huge, huge pain on MacOS X. In this case, the benefits that vendoring Boost gives us outweight the cons by far.
On the other hand, Apache is fairly common, and easy to install on most platforms. Rake, fastthread, etc. are also easy to install because of RubyGems. That’s why we dynamically depend on those things instead of vendoring them.
So it all boils down to making the right choices and correctly balancing the pros and cons of vendoring. There’s no silver bullet.
Luke’s article was quickly followed by a blog post by Debian developer Gunnar Wolf:
By using Ruby Gems, you dramatically increase entropy and harm your systems’ security.
To this, I say nonsense. It’s pretty well-known that Debian, and Linux distros in general, dislike foreign packaging systems, regardless of their merits. I see quite a lot of conversations on #rubyonrails @ irc.freenode.net that are somewhat like this:
Person A: hi, I’m using Debian/Ubuntu/some-other-Debian-derived-distro. I typed “gem install rails”. But when I type “rails foo”, it says “command not found”. what’s going on?
Person B: type “gem update –system && gem install rails”
Person A: wow, it worked! thank you!
It is painfully obvious that Debian did something to RubyGems. I was bitten by this very issue as well. Debian’s RubyGems package places binaries in /var/gems (or something like that, I don’t remember the exact location) instead of /usr/bin. Fine, I understand that they don’t want foreign packages to pollute /usr/bin, which is a managed directly. But the least they can do is adding /var/gems to $PATH by default, something which they didn’t. As a result, many people who installed Rails via Debian thought that Rails is broken, when it’s actually Debian who crippled RubyGems.
“Increase entropy”? I don’t even know what this is supposed to mean in the context of Ruby software distribution.
“Harm your systems’ security”? As I’ve already stated, vendoring has both pros and cons. If one dynamically depends on a library, then it means that the system administrator and library author are responsible for security updates, but it also means that a security update can potentially break the application. By vendoring stuff, the responsibility of security updates is mostly shifted to the application developer. This is a trade-off: there’s nothing wrong with it, and whether it’s the best thing to do depends on the situation. Gunnar however has an extremely purist view, along the lines of “if you don’t agree with us then you’re an idiot, regardless of the circumstances”.
It seems more like that Gunnar’s words are carefully picked, with the goal of creating knee-jerk reactions. Debian said pretty much the same thing about Autopackage in the past, and now they’re doing it again with RubyGems.
Luke, Gunnar, will you realize that all you’re doing is ranting about a problem, without offering any solutions? And no, creating native Debian packages is not a solution, the world isn’t comprised of just Debian.
Passenger 1.0.2 fixed MacOS X compatibility. Unfortunately, the OS X fix broke compatibility with 64-bit Linux. The 1.0.4 release was supposed to fix it for both platforms. Before we released 1.0.4, we had an OS X user test it, and the test result was positive. But soon after the release we got bug reports from people, telling us that 1.0.4 broke OS X compatibility.
The main problem lies in the file descriptor passing code. We’re struggling to get it right:
1. The original code was copied from Ruby. Apparently that didn’t work well on 64-bit MacOS X. This also means that file descriptor passing on 64-bit MacOS X Ruby is also broken, not just Passenger.
2. There is generally very little documentation about file descriptor passing. We found a bunch of code samples, but it turns out that all of the ones that we’ve tried have portability problems. The developer man pages don’t provide a lot of information on file descriptor passing.
We want to get this right, but we don’t have any 64-bit machines at all. So we’re looking for people who can help us with testing Passenger on 64-bit OS X and 64-bit Linux. We’d like to have at least 3 people from both categories. If you’re interested, please send a message to our discussion board, or join #passenger on irc.freenode.net.
Thank you.
Passenger version 1.0.2 has been released. We’ve finally finished our corporate blog now, so visit http://blog.phusion.nl/ for the release announcement, overview of changes, upgrade instructions, and more.
Now that we finally have an official corporate blog, people can stop complaining about my blog template. 
Note that our corporate blog’s template isn’t finished yet: it still needs some polish.
There are a few technical achievements in Passenger that we haven’t actively marketed. But we’d like the community to be aware of them, so we’ve written this blog post.
Not too long ago, Thin announced support for Unix domain sockets. This gave Thin an incredible speed boost. Switchpipe soon followed, with alpha support for Unix domain sockets.
It has always surprised me that the Ruby/Rails web containers didn’t support Unix domain sockets until early 2008. Unix domain sockets aren’t exactly rocket science or exotic: the X Window System has used them for client-server communication on localhost for as long as I can remember. Database systems such as MySQL prefer to use Unix domain sockets on localhost as well. It’s also well-known that Unix domain sockets are faster than TCP sockets.
There’s one down side to Unix domain sockets though: you have to remove them. Usually, server processes remove them during exit. But if the system crashes, or if those processes crash, then the socket files are never removed, and the system administrator will have to do that manually. Ouch, maintenance overhead.
Passenger uses Unix sockets extensively, and has done so since the first release. But there’s one difference compared to Thin and Switchpipe: Passenger uses Unix sockets in the abstract namespace (as opposed to on the filesystem) whenever possible. Abstract namespace Unix sockets have filenames, just like regular Unix sockets, but they do not appear as files on the filesystem. This means that after a reboot, or if Apache crashes, no stale files will be left on the filesystem. This way, the system administrator doesn’t have to manually remove stale files if something went wrong.
Passenger strives for a concept that we call “zero maintenance”. We believe that software should Just Work(tm), and system maintenance overhead should be as low as possible. The system administrator shouldn’t have to worry about stuff like stale files, if he doesn’t have to. Ideally, the system administrator should be able to forget that he ever installed Passenger at all. The usage of abstract namespace Unix sockets is one of the mechanisms that we use to achieve that goal.
Mongrel and Thin write so-called PID files to the filesystem. PID files contain the PIDs of background processes, and are used for shutting down those processes. But if the system crashes, or if those background processes crash, then the PID files will never be removed, and they become stale PID files. Early Mongrel versions refuse to start if there are stale PID files, forcing the system administrator to remove them manually. Passenger on the other hand doesn’t use PID files at all. If one shuts down Apache, then all Rails processes are shut down as well.
So what happens if Apache crashes? The Rails processes will exit as well. We use so-called “owner pipes” in Passenger, pipes that are shared between Apache and the Rails processes. Owner pipes are essentially a mechanism for reference counting processes. If Apache crashes, then the owner pipe is automatically closed. The Rails processes will detect this, and will shut down gracefully.
It has been a little more than one week since the first public release of Passenger. In this one week, lots and lots of people have blogged about us, including even O’Reilly. The feedback has been overwhelmingly positive!
A few nice quotes from the community:
Thanks for the praise, everybody.
One may wonder why our “Enterprise”/Donation page was taken down so soon. It’s because people have been very, very generous with regard to donating to us. In fact, so generous that Paypal decided to block our account under the guise of “anti-money laundering investigation”. But everything has been cleared up now, so the donations page is once again online.
If you’d like to help us survive the coming winters, or just want to support Passenger’s development, then please consider getting an “enterprise” license.
We’ve been interviewed by InfoQ. This interview elaborates some things about the name and the license. It also elaborates our upcoming memory optimizations for Passenger/Ruby, which allows them to use less memory. More information about this soon, honestly.
On a side note, we’re preparing the release of Passenger 1.0.2. It will soon be available.
Hi there,
Our apologies for the later than usual update, but things are getting really busy here at Phusion and this has been the first opportunity that we’ve been able to blog about the latest developments again! First off, we’d like to take the opportunity to ask all enterprise licensees to check their mail as they’re bound to find something nice there
Also, the next batch of enterprise licenses will become available very soon. We just wanted to let you guys know that we’re still alive and are working around the clock on some very interesting projects for our clients. Regardless of this, we want you guys to know that we’re still actively involved in the development of Phusion Passenger and Ruby Enterprise Edition and for those who want to know a bit more about Phusion Passenger and Ruby Enterprise Edition, we’ve recently done a mini interview with the fine people at infoq, and we encourage you to go check it out over here.
Even though we try to stay as actively involved as possible, we would also like to encourage the community to submit patches for the improvement of Phusion Passenger. We’re confident that we can make Passenger even better together (wow, that even rhymes so it must be true right? ;-)).
Also, it has come to our attention that some of you guys have expressed a concern about the memory use of Phusion Passenger and we have reason to believe that the memory wasn’t always measured in an accurate way. This is for the greater part the result of a well known problem with the innacurate display of memory usage by ps and/or top.
We’ll try to write an article on this as soon as possible where we will address this ‘issue’ with the appropriate measuring tools as well. Hopefully, this should set the record straight and take away any reluctance to try out Phusion Passenger as we’re pretty confident that Phusion Passenger uses a lot less memory than some people seemed to have inferred.
We’d like to leave it at this for now, and we’ll keep you guys posted and would love to stay in touch with you guys :-).
Cheers,
Hongli Lai
Ninh Bui
After weeks of teaching it hardcore computer science kung-fu, we’ve finally reached a stage in that we think our little baby is now all grown up. It is no longer solely dependent of our care, and from this moment on, Phusion Passenger is available for the masses.
Phusion Passenger has been the result of many many many (did I say many? I mean MANY) hours of hard work by both Hongli and I ( Ninh ), but it couldn’t have been made possible without the help of our beta testers. In particular, we’d like to thank Dallas of Dreamhost for being so patient and we’re looking forward to seeing Passenger being fully integrated in your hosting plans. Also, we’d like to thank Alex of Twitter for trying his best to test Passenger in time.
Also, we’d like to thank the Ruby on Rails core team for their insightful feedback and support, and in particular, we’d like to thank David Heinemeier Hansson, Jeremy Kemper and Pratik Naik for this.
Yukihiro Matsumoto (matz), thank you for the kind words on our project, it really kept us motivated.
Ryan Bates of Railscasts, thank you so much for doing the screencast on Phusion Passenger. We’re looking forward to meeting you in person at Railsconf and being able to thank you properly Depending on the paint we’re allowed to carry with us, shaking a green hand may be involved
Antonio Cangiano, thanks for the coverage. It was when we saw that a technical evangelist at IBM had written about us that it truly hit us that we were doing something important.
Russel Norris, you really should not underestimate the importance of your great sense of humor for an open source project :-D.
Since we’re from the Netherlands, we’d also like to thank our fellow dutchmen Stefan Fountain of Soocial and Thijs van der Vossen of Fingertips for their support on this project.
Also, thank you Weyert de Boer for being a great friend and helping us streamline the markup of our website. Even though we haven’t been able to complete it yet, we’re confident that we’ll be able to finish this with your help and we thank you for this.
Lastly, we’d like to thank Hans Scholten of the University of Twente for offering to help us on writing a scientific paper on our work with Ruby Enterprise Edition and Phusion Passenger.
To you the reader, we hope that you’ll enjoy using Passenger and we thank you for your patience. We’ll be seeing you guys at Railsconf USA 2008 (and probably Apachecon USA 2008 as well! )
Be sure to check us out there, since we’ll be doing a talk on Phusion Passenger (mod_rails) and our garbage collector optimization work on MRI. The talk material will be based on our scientific paper to be published on eprints, so the tech savvy among you guys will be in for a treat :-).
For science and all things pragmatic, this was Hongli Lai and Ninh Bui of Phusion, The Computer Science Company, wishing you a good weekend as we’ll be seeing you guys monday again. We’re going to take that well-deserved time off now ;-).
P.S. Even though we’re currently getting swamped with clients, that shouldn’t stop you from contacting us for business enquiries. As a matter of fact, we’re planning on expanding our team, so if you’re a computer scientist / developer located in the Netherlands with a lot of work experience and interested in working with us, please contact us as well
Ninh wrote this a while ago:
“Lastly, we’d like to take the opportunity to thank Ryan Bates of Railscasts for creating an excellent screencast on Passenger, and you’ll be able to see this for yourselves at the public release of Passenger this week. Needless to say, it totally kicks the *censored* out of our initial screencast :-D”
So you guys are probably still waiting for the release of Passenger, and I’m pleased to say that we’ve been able to include the test results from Dreamhost (thanks Dallas :D) in the first public release that you’ll see within this week. (Hold on just a little longer guys , with a bit of luck, Twitter’s results will come in as well prior to the release of Passenger this week).
Instead of writing an elaborate blog post, I’ll keep it concise for the time being as we’re currently getting swamped at Phusion.
Today, after planning to do this for quite a long time now, we went to go see our Operating Systems/Computer Architecture teacher, Ir. Hans Scholten to discuss the possibilities for receiving ECTS for our work on both Passenger and Ruby Enterprise Edition (our ruby branch containing our garbage collector optimization).
From this, it became clear that our work wasn’t something trivial and Hans elaborated on the possibilities for publishing this work scientifically. Needless to say, we’re more than thrilled about this. After all, we’re “Phusion, the computer science company” right?
All kidding aside though, we do intend on taking Hans up on his offer to write a scientific paper on our work, seeing as this is just too great an opportunity to ignore. With his help, we’re more than confident about the quality of its contents.
Lastly, we’d like to take the opportunity to thank Ryan Bates of Railscasts for creating an excellent screencast on Passenger, and you’ll be able to see this for yourself at the public release of Passenger this week. Needless to say, it totally kicks the *censored* out of our initial screencast 
.
To all the people that have submitted a voice sample, we’ll be crediting you on the screencast section of the Passenger as well .
Hang on in there for just a little longer people , we’re almost there!
Thanks,
We’ve written a robustness comparison between Phusion Passenger (mod_rails), Thin, Ebb and Mongrel. See Ninh’s blog for the article.
Since we’re almost nearing the weekend, we thought we’d give you guys a little heads-up on the things we’re currently doing.
As you may have recalled from several days ago, we’ve asked for your voice in the most literal sense of the word for our new (series of) screencasts that we’re planning on doing.
We’re pleased to say that a lot of you guys out there have been sending in your voice samples, and to our great surprise, it wasn’t just guys who were sending in their voices :D. More on that later though, and for now, we’d like to again take the opportunity to thank you all.
Among those submissions, we’ve also received one from Ryan Bates which you may already know from his excellent Railscasts website. Greatly impressed by the quality of his previous work, we’re pleased to announce that Ryan Bates has agreed on doing the first ‘official’ screencast on Passenger. This screencast will be based on the latest beta that we’ve provided him with: don’t even bother asking him to leak it though, as he has assured us that such attempts will be in vain ;-).
But what about all those submitted voice samples? Well, we’re currently planning on doing a demo reel on Phusion as a company as well, and needless to say, we’re going to need a voice over for this as well.
It is for this reason that we’ve decided to take the people who’ve submitted their voice samples into consideration for this, since you all sounded more than suitable for this. Needless to say, this is only possible if you’re willing to do this, and we’ll be sure to contact you guys on this soon. Due to our busy schedules however, we may not be able to contact each and every one of you, and we ask for your understanding on this.
Whichever the case may be, we will however be crediting each and everyone of you on the Passenger screencast site.
Regarding the demo reel, we’re currently in talks with several video production companies and individuals for helping us create this, and we’ll probably be able to share some more information on this with you guys soon.
Concerning Passenger, we’ve decided to increase our sample space by accepting some additional large Ruby on Rails hosts for beta testing, in the hopes of being able to release it to the public even sooner. If you’re attending Railsconf like we’re planning on doing, then you’ll probably be able to ask them about their findings on Passenger as well
Also, you may have read our article on the performance of Passenger compared to other popular solutions out there. However, we haven’t touched the subject of stability/robustness yet, and since Passenger was designed for this as well, we’ve also tested this on the other popular solutions. We’ve been able to infer some interesting results, and we’ll be posting a detailed article on this as soon as we’re done doing administrative work for Phusion.
Lastly, even though we’re getting swamped at Phusion, this shouldn’t stop you from contacting us in case of a question/fanmail ;). We’d be more than happy to receive/answer them
With kind regards we are, your friends at Phusion,
- Hongli Lai
- Ninh Bui
By the time of this writing, a lot of people have been claiming that Passenger will cause a revolution in the Ruby on Rails community. And even though we agree with them to a certain point ( ), we believe that one should always have a trump card in case all else fails. We’ve been working on our trump card for quite some time now in great secrecy.
Today, we’re pleased to announce this trump card to the world: Ruby Ain’t Hypertext Preprocessor (RAHP).
RAHP is a Ruby parser written in PHP, which will enable you to run all your Ruby applications within a PHP environment! This was possible due to the fact that Ruby and PHP are both Turing Complete.
Since PHP still enjoys more support among webhosts than Ruby on Rails, we’ve decided to combine the best of both worlds with RAHP. And as you may already have heard from a friend, PHP scales really well in Enterprise environments, unlike Rails applications.
But wait! It doesn’t end there! Using Zend optimizer we’ve been able to infer a tremendous performance boost, and we think we may need to retract our previous benchmark post on Ruby vs PHP. :$
Come to think of it, we’re considering to discontinue our work on Passenger in favor of RAHP. We’ll have to sleep on it though.
Stay tuned!
Recently, Yukihiro Matsumoto (a.k.a. Matz, the creator of Ruby) blogged about Phusion Passenger (mod_rails) on his weblog. Since our Japanese is a bit rusty, i.e. non-existant 
, Matz was kind enough to mail us a summary in English from which we quote:
It is often said that Rails is weak on deployment; PHP runs fairly fast just by uploading scripts. Rails is slow on development mode, and requires restarting on production mode (and bit complex to configure). modrails might be the answer for it.
Needless to say, coming from Matz, this made us feel real warm and fuzzy from inside
Since the initial release of the teaser screencast, a lot of people have asked about Passenger’s (a.k.a. mod_rails) performance compared to Mongrel and Thin. Here’s the benchmark that you’ve all been waiting for.
In this benchmark, we compare the following software:
System specification:
Test applications:
require 'forwardable' in app/models/cart_items.rb. I added this line myself.Typo and Eldorado both use SQLite 3. Petstore uses MySQL, because for some reason Petstore performs very badly when SQLite 3 is used.
People have also recommended Lovd By Less as test application, but I couldn’t get it working (database setup failed).
The applications, when served by Passenger, are setup at the following test domains:
Both the Mongrel cluster and the Thin cluster are setup at http://mongrel_cluster.test/. (Only either Mongrel or Thin is running during the benchmark, not both at the same time.)
And of course, we always use the production environment.
Backend configuration:
ProxyPass / balancer://mongrel_cluster/ ProxyPassReverse / balancer://mongrel_cluster/ <Proxy balancer://mongrel_cluster> BalancerMember http://127.0.0.1:3000 BalancerMember http://127.0.0.1:3001 BalancerMember http://127.0.0.1:3002 BalancerMember http://127.0.0.1:3003 BalancerMember http://127.0.0.1:3004 BalancerMember http://127.0.0.1:3005 BalancerMember http://127.0.0.1:3006 BalancerMember http://127.0.0.1:3007 BalancerMember http://127.0.0.1:3008 BalancerMember http://127.0.0.1:3009 </Proxy>
First we benchmark Typo served by Mongrel clusters. Because the actual application code is loaded by Mongrel during the first request, we do not want to count the first request’s time. So we run the following command to “warm up” the Mongrels, and discard its result:
ab -n 1000 -c 100 http://mongrel_cluster.test/
Next, we test 10,000 requests with 100 concurrent users:
ab -n 10000 -c 100 http://mongrel_cluster.test/
Result:
Concurrency Level: 100 Time taken for tests: 32.628480 seconds Complete requests: 10000 Failed requests: 0 Write errors: 0 Total transferred: 54890000 bytes HTML transferred: 51480000 bytes Requests per second: 306.48 [#/sec] (mean) Time per request: 326.285 [ms] (mean) Time per request: 3.263 [ms] (mean, across all concurrent requests) Transfer rate: 1642.83 [Kbytes/sec] received
We repeat these steps Petstore and Eldorado.
Petstore:
(for Petstore we benchmark the URI “/shop/viewCategory.shtml?category=DOGS” because it’s written in the INSTALL file)
Concurrency Level: 100 Time taken for tests: 109.786026 seconds Complete requests: 10000 Failed requests: 0 Write errors: 0 Total transferred: 40750000 bytes HTML transferred: 37680000 bytes Requests per second: 91.09 [#/sec] (mean) Time per request: 1097.860 [ms] (mean) Time per request: 10.979 [ms] (mean, across all concurrent requests) Transfer rate: 362.47 [Kbytes/sec] received
Eldorado:
Concurrency Level: 100 Time taken for tests: 70.889388 seconds Complete requests: 10000 Failed requests: 0 Write errors: 0 Total transferred: 41130000 bytes HTML transferred: 35530000 bytes Requests per second: 141.06 [#/sec] (mean) Time per request: 708.894 [ms] (mean) Time per request: 7.089 [ms] (mean, across all concurrent requests) Transfer rate: 566.60 [Kbytes/sec] received
The benchmarking steps for Thin are the same.
Typo:
Concurrency Level: 100 Time taken for tests: 25.795470 seconds Complete requests: 10000 Failed requests: 0 Write errors: 0 Total transferred: 54910000 bytes HTML transferred: 51480000 bytes Requests per second: 387.66 [#/sec] (mean) Time per request: 257.955 [ms] (mean) Time per request: 2.580 [ms] (mean, across all concurrent requests) Transfer rate: 2078.78 [Kbytes/sec] received
Petstore:
(for Petstore we benchmark the URI “/shop/viewCategory.shtml?category=DOGS” because it’s written in the INSTALL file)
Concurrency Level: 100 Time taken for tests: 96.874308 seconds Complete requests: 10000 Failed requests: 42 (Connect: 0, Length: 42, Exceptions: 0) Write errors: 0 Non-2xx responses: 42 Total transferred: 40630350 bytes HTML transferred: 37547028 bytes Requests per second: 103.23 [#/sec] (mean) Time per request: 968.743 [ms] (mean) Time per request: 9.687 [ms] (mean, across all concurrent requests) Transfer rate: 409.58 [Kbytes/sec] received
Eldorado:
Concurrency Level: 100 Time taken for tests: 71.311031 seconds Complete requests: 10000 Failed requests: 5 (Connect: 0, Length: 5, Exceptions: 0) Write errors: 0 Non-2xx responses: 5 Total transferred: 41137070 bytes HTML transferred: 35518568 bytes Requests per second: 140.23 [#/sec] (mean) Time per request: 713.110 [ms] (mean) Time per request: 7.131 [ms] (mean, across all concurrent requests) Transfer rate: 563.34 [Kbytes/sec] received
The benchmarking steps for Passenger are the same.
Typo:
Concurrency Level: 100 Time taken for tests: 25.131980 seconds Complete requests: 10000 Failed requests: 0 Write errors: 0 Total transferred: 55200000 bytes HTML transferred: 51260000 bytes Requests per second: 397.90 [#/sec] (mean) Time per request: 251.320 [ms] (mean) Time per request: 2.513 [ms] (mean, across all concurrent requests) Transfer rate: 2144.92 [Kbytes/sec] received
Petstore:
(for Petstore we benchmark the URI “/shop/viewCategory.shtml?category=DOGS” because it’s written in the INSTALL file)
Concurrency Level: 100 Time taken for tests: 104.428742 seconds Complete requests: 10000 Failed requests: 0 Write errors: 0 Total transferred: 40300000 bytes HTML transferred: 36700000 bytes Requests per second: 95.76 [#/sec] (mean) Time per request: 1044.287 [ms] (mean) Time per request: 10.443 [ms] (mean, across all concurrent requests) Transfer rate: 376.86 [Kbytes/sec] received
Eldorado:
Concurrency Level: 100 Time taken for tests: 72.587018 seconds Complete requests: 10000 Failed requests: 0 Write errors: 0 Total transferred: 41660000 bytes HTML transferred: 35530000 bytes Requests per second: 137.77 [#/sec] (mean) Time per request: 725.870 [ms] (mean) Time per request: 7.259 [ms] (mean, across all concurrent requests) Transfer rate: 560.47 [Kbytes/sec] received
Here’s a nice summary of the requests per second, in a table and in a graph:
| Typo | Petstore | Eldorado | |
|---|---|---|---|
| Mongrel | 306.48 | 91.09 | 141.06 |
| Thin | 387.66 | 103.23 | 140.23 |
| Passenger | 397.90 | 95.60 | 139.27 |
I was pleasently surprised by the results. During Passenger’s development, our performance baseline was Mongrel behind mod_proxy. According to earlier (simple and naive) tests, Passenger performed similar to Mongrel. I expected Thin to be a bit faster than Passenger. These new benchmarks however suggest Passenger is faster than Mongrel, and is on par with Thin.
But you know what they say: there are lies, damn lies, and statistics. Your hardware and software is different, so if you benchmark it yourself, you’re likely to get (slightly?) different results.
I have good news and bad news.
Let’s start with the good news. Passenger (mod_rails) is almost ready for release. The code is as good as done (actually, it has been as good as done for well over a week now). The website is almost done. We’re just awaiting final test results from remaining beta testers.
The bad news: lots of people have commented about our current screencast, in particular about the now-infamous “scary green hand”. Being the perfectionists that we are, we’d like to redo the screencast before we release Passenger. Neither Ninh nor I are native English speakers, so we’re looking for a native English speaker who can help us with the voice-over part of the screencast. Preferably, someone with enough ‘gravitas’ in their voice, or in the case of a female voice, the ‘kindness’ should come out dripping from out of our audio speakers.
Would you like to volunteer? If so, please provide us with an audio sample. You can do this by either replying in this post, or by emailing us at “hongli at plan99 dot net” or “ninh dot bui at gmail dot com”! Thanks in advance.
Does anyone know a good Ruby on Rails app that I can use for benchmarking Passenger (mod_rails)? I don’t have any apps that I’m allowed to use for this purpose. Mephisto and Typo are out because they both use page caching, which would make the benchmark pointless.
Traffic for our Passenger (mod_rails) preview site has increased three fold ever since RailsEnvy and weblog.rubyonrails.org (thanks David) linked to us. We’ve received quite some emails from interested people. One of the questions that we receive most is:
There is the following saying: jack of all trades, master of none. At the moment, our goal is to make Passenger a great deployment system for Ruby on Rails, one that is matched by none in terms of ease of use and low maintenance overhead.
But some people ask:
mod_ruby uses a shared interpreter per Apache process, which means that multiple Rails applications on the same Apache setup with mod_ruby would share the framework classes. This doesn’t work well with the Rails model of using class-level configuration attributes, so it’s considered unsafe to use mod_ruby and Rails with more than one application running per Apache setup, because different applications may start sharing the classes.
Granted, these are implementation problems and not conceptual problems, so they can be fixed. So some people ask:
First, we must agree on the premise that Rails is a more popular framework than other Ruby frameworks. Wait, hold your flame thrower, this is not the reason.
Next, try thinking from the point of view of a new Rails developer. He’s introduced to lala-land: Don’t-Repeat-Yourself, convention over configuration, stuff that’s supposed to Just Work(tm).
Then he deploys his Rails app and finds out that he has to setup a Mongrel cluster (1 command but usually with 3 or 4 options that he’ll have to look up), configure Apache to proxy to the cluster (about 5 or 6 lines that he’ll have to setup), configure init.d/daemon tools/monit/god/whatever so that his Mongrel cluster is started at system startup, etc.
Oh, and you need to repeat this for every single Rails app.
Ouch, is this the proper Rails experience? I don’t think so. This is Passenger’s philosophy: deployment should be in line with the Rails philosophy. In other words, it should be brain-dead simple. Just imagine the money that you’ll save on health care for your system administrator if he doesn’t have to worry about all that stuff anymore.
Of course, the Rails philosophy doesn’t prevent deployment of other Ruby framework apps from being easy. So what’s holding us back? Well, here’s a few things that Passenger does but mod_ruby doesn’t:
We’ve seen many people failing to deploy Rails apps because of errors during application startup. Lighttpd’s FastCGI implementation redirects those errors to /dev/null. Not good, this fact bit me 2 years ago — when I was deploying my first Rails app — and made me wonder why the thing wouldn’t work. Mongrel prints errors either to stderr or to its log file. But people who are new to Rails deployment don’t know that they should be looking in the log files (or that there are log files in the first place). Plus, looking into the log file is yet-another-step. Wouldn’t it be great if you can see startup errors right in the browser?
Passenger provides user-friendly error pages. If a Rails app failed to start, then Passenger will tell you what happened. A few possible error pages are:
Here are previews of typical error pages:
As you can see, a lot of these error pages are framework-dependent.
As you may have noticed by now, we’ve put a lot of work into making sure that deployment is dead-easy. This is only possible if Passenger has knowledge about the frameworks themselves. We have a tight schedule and we only have 2 developers working on Passenger. Rails is more popular so it has more priority. We’re working around the clock to get this thing released.
Or we can get rid of the autodetection/preloading logic and just provide generic configuration options. But would you really want that? By no means should we force the system administrator to configure things. By default, stuff should Just Work(tm) out-of-the-box.
Hope this clears things up.
Thought I’d give you guys a little update on where we currently stand (yeah, we’re still alive ;)). As you may have already know, we’ve announced Phusion Passenger (a.k.a. mod_rails) last week and the reaction has been overwhelmingly positive!
Our beta testers now not only include several of the Ruby on Rails core team, but also some of the largest Ruby on Rails webhosts you may know of. Needless to say, we’re very thrilled about this, and in particular, we’d like to thank Pratik Naik, David Heinemeier Hansson and Derek Perez for their insightful feedback and support. Also, we’d like to thank community members such as Russel Norris for keeping the conversations fun. With this kind of support, we’re convinced of the quality of Passenger, since a lot of love has gone into making this.
Again, we’d like to express our gratitude to all for supporting us on this. It’s important to note that for an open source project, it’s things like these that keep developers like us going on.
And while we’re on the subject of ‘going on’: we’ve been working around the clock the past few days to get things done as fast as we can. While Passenger is nearing completion and its website still needs a bit of work, it still remains uncertain if we’re able to obtain a preview/announcement slot at Railsconf . We’d like to let you guys know however, that we’re putting a lot of effort into making this happen, since we really believe that Passenger has the potential to set of a revolution in the Rubyverse by possibly allowing more people to enjoy Ruby on Rails.
For now we ask you for your patience and we’ll try to keep you guys posted on a regular basis regarding our activity.
With kind regards we are,
PStore is a library in Ruby that “implements a file based persistance mechanism based on a Hash”. Ruby on Rails up until version 1.2 uses PStore as the default mechanism for storing sessions. But it’s pretty well-known that PStore “sucks”: people say that it’s slow, causes file corruptions, etc. This is one of the reasons why Rails 2.0 uses the cookie session store by default: it’s faster and needs less maintenance at the cost of a few (easy-to-avoid) caveats.
If you look at the Mongrel FAQ, then you’ll see that 3 items are devoted to telling you that PStore is the work of satan and that any sane web developer should ritually burn it. But this particular sentence caught my eye:
“Other things that can cause big pauses are:
- …
- Locking files wrong. Multiple processes locking files is a delicate thing to do. “
Locking files wrong? How can that possibly be? File locking, just like mutex locking, isn’t really rocket science.
Note that PStore’s RDoc advertises itself as transactional:
“# The transactional behavior ensures that any changes succeed or fail together.
# This can be used to ensure that the data store is not left in a transitory
# state, where some values were upated but others were not.”
I decided to take a look at its source code. What possibly could have gone wrong?
Well, I’m not sure what is wrong with it, but the relevant code, PStore#transaction, is a bit messy. I had a hard figuring out what it is exactly doing and why, but I figured that it does these things:
It’s not clear whether PStore has locked everything correctly. Oh, and PStore isn’t thread-safe (though it is reentrant).
So I rewrote PStore. The code is now easier to read and is faster. And as far as I know, everything is locked correctly so there shouldn’t be any concurrent issues.
By default, it doesn’t try as hard to ensure file integrity because harddisk I/O errors are very rare, but if file integrity is really an issue, then you can set pstore.ultra_safe_transactions = true to enable it. This option only has effect on Unix though: as of now, I haven’t bothered writing complex recovery code for Windows.
This is the benchmark program that I used:
http://pastie.org/170997
The results are as follows:
As you can see, PStore has become 2 times of even 3 times faster, depending on whether is ultra_safe_transactions is enabled! But let’s see what kind of effect it has on a dummy Rails 2.0 app that uses the PStore session store:
A small performance increase. (Though the cookie session store is still faster: 221.74 requests/sec.)
You can download it here:
http://pastebin.com/f163702f2
If you want your Rails app to make use of it, simply save it as “lib/pstore.rb”. For maximum stability, Rails’s PStore handler should be modified to ignore any errors encountered during unmarshalling.
One of these days I’ll send a patch to ruby-core so that this can be merged back upstream. But for now Passenger (a.k.a. mod_rails) has priority.
A video says more than a thousand words, so just click on the video link below.
Yes people, this is real! We at Phusion have been developing this for a while now, and it’s almost done. Currently we’re in private beta testing phase. Our beta testers include several Ruby on Rails core developers, and so far the feedback from our testers is overwhelmingly positive.
Stay tuned for more news!
I benchmarked Rails 1.2 vs 2.0 in my last blog post, and Rails 2.0 turned out to be 30%-50% faster. Then Eric Allam said:
“Hmm, another benchmark that completely ignores memory use and memory usage growth.”
Okay, let’s give it a try.
When measuring memory, people usually measure the wrong thing. There are 3 things that one can measure:
Even if you only have 256 MB RAM, the virtual memory size reported by a process can still be 4 GB! This is because processes can map files, hardware and other resources into its memory address space, without really loading them into physical memory. This is complicated by the fact that processes can share memory with each other. The VM size is quite meaningless when measuring real memory usage.
fork() is never called), we’ll be effectively measuring the total heap size, including any free heap slots. Free heap slots are reused by Ruby, so they really shouldn’t be counted.We want to count the number of used heap slots. Usually, there’s no way to do this. But I maintain a private Ruby branch (which will be released soon) with statistics information. We’ll use that for memory measurement. (My GC patch also provides statistics information, though the version I’m working on provides more accurate statistics.)
People interested in details should read a good book about operating systems. I recommend “Operating System Concepts” by Silberschatz et al. My “Operating Systems” classes professor has Dutch slides on the subject.
After a cold start with Mongrel 1.1.3, the private dirty memories were as follows:
Nice! A 6 MB memory reduction after a cold boot!
Though, as I’ve said earlier, these numbers don’t really mean much. The application code (i.e. controller and model code) haven’t been loaded into memory yet. And Ruby doesn’t garbage collect until its heap is full. So let’s find out what happens after 3000 requests and a garbage collection run. I added these actions to PostsController:
def gc
headers["Content-Type"] = "text/plain"
GC.start
render :text => ObjectSpace.statistics
end
And this route:
map.connect '/gc', :controller => "posts", :action => 'gc'
Then I ran, for both apps:
ab -n 3000 http://localhost:3000/ links http://localhost:3000/gc
The memory usages were:
Nice! I didn’t expect this, but apparently Rails 2.0 uses a lot less memory!
[EDIT: Some people have contributed a chart. Thanks.]
There’s still a lot of misinformation about Rails on the Internet. People are still claiming that Rails doesn’t scale, is too slow, etc. Rails 2.0 has been released for a while now, and a year ago someone blished a benchmark, in which it is claimed that Rails edge (what is now 2.0) is slower than 1.2. I’ve decided to benchmark Rails 1.2 and 2.0.
I’ve written two dummy applications, one for 1.2 and one for 2.0. The reason why I’m not just running the same app under 2.0 is because Rails 2.0 introduced a new cookie session store, among other things. It is claimed that this session store is much faster than the old default PStore cookie store.
So first I create a Rails 2.0 application:
mkdir bench rails 2.0 cd 2.0 ./script/generate scaffold Post title:string content:text created_on:timestamp updated_on:timestamp rake db:migrate RAILS_ENV=production
Note that scaffolding in Rails 2.0 is no longer dynamic. The scaffold generator now generates a database schema and some static HTML and code. This is equivalent to Rails 1.2’s scaffold_resource.
I also modified routes.rb so that the root URL is mapped to the ‘posts’ controller. And of course, I delete ‘public/index.html’.
Next, I create a Rails 1.2 application:
cd .. rails _1.2.6_ 1.2 # This is a nice RubyGems trick. It allows me to specify which Rails version I want to use. cd 1.2 ./script/generate scaffold_resource Post title:string content:text created_on:timestamp updated_on:timestamp cp ../2.0/config/database.yml config/ # We want to use SQLite, just like Rails 2.0. rake db:migrate RAILS_ENV=production
I also modified routes.rb so that the root URL is mapped to the ‘posts’ controller, and deleted ‘public/index.html’.
I ran both applications in Mongrel (with the ‘production’ environment). The results of ‘ab -n 5000 http://localhost:3000/’ are as follows:
Wow, what a difference! Rails 2.0 is 50% faster in a dummy application!
In Rails 1.2, a lot of time is spent in the session store. Let’s see what happens if we specify “session :off” in both applications:
Wow! Even with sessions off, Rails 2.0 is still 30% faster! So the cookie session store isn’t the only thing responsible for the performance improvement!
Edge Rails has even more performance improvements. Things are definitely going the right direction.
I moved my blog to mikehearn.wordpress.com - I got tired of having to maintain it myself. If you notice any problems let me know.
I totally fell in love with RSpec. Although I’m not really into BDD, I’ve found that RSpec makes my tests easier to read and easier to organize.
But their website really is a problem. Its navigation is pretty confusing. At first, I didn’t know that there are 3 more levels of pages under “Documentation” - I thought that the first page I see under “Documentation” is the only documentation available. This is because the navigation bar changes in ways that I think people would typically not expect. And their RDoc documentation looks horrible because it uses the default RDoc template.
Because they’re experimenting with Git since today, I decided to check out their sources and see whether I can contribute. I’ve redesigned their menu bar, and although it’s not perfect, it should be better than it currently is. Compare the current navigation with my redesign.
I’ve also submitted a patch for generating the RDoc documentation with my “Horo” RDoc template (which is based on the Jamis RDoc template). A sample can be found at: http://izumi.plan99.net/rspec-horo/
Hopefully these patches will get accepted.
I think that the default RDoc template looks horrible. The colors are not good at all, and the top frames waste a lot of space. On the other hand, the Jamis RDoc template looks a lot better.
Today I ran into a problem: RubyGems can generate RDoc documentation for projects, but how do I force it to use the right template? The Jamis template installation instructions don’t work with the Rake RubyGems task. Eventually I found out that I had to do this:
Rake::RDocTask.new do |rd| ... rd.template = "./doc/template/jamis" ... end
And put ‘jamis.rb’ into the folder ./doc/template/
Last time, I talked about the basics of Singularity: SIPs, manifests, connections, and software isolation. Let’s have a look at how this applies to driver development.
Drivers are a Big Deal for us, because:
So getting drivers right would be a huge win, both for reliability, and for building a foundation on which we can get security right too.
drivers in singularity
Because Singularity is a type system based micro-kernel, a driver is simply a set of Sing# classes (Sing# is C# with extra bits), shipped as verifiable MSIL bytecode images. It runs as a SIP, like any other program does. Obviously, using C# instantly solves a large class of common bugs - pretty much any exploit with “overflow” in its name is solved by using these languages.
You can’t represent the specialized instructions needed to control hardware in safe MSIL, so we need a workaround. This comes in the form of an unsafe DLL which provides objects abstracting the hardware control instructions. This DLL (Singularity.DriverRuntime) is a part of what Microsoft Research call the “trusted computing base”, but that makes it sound like it’s to do with Palladium and DRM, which it isn’t. It’s easier to think of this DLL just as logically part of the kernel, even though it doesn’t run in the context of the main kernel program. These classes have names like IoPortRange, IoIrqRange and so on, and provide a simple OO interface to programming hardware.
Singularity won’t let you construct objects of these types yourself, you have to use a set of special static method calls. And you can’t invoke them directly either. It seems we’re stuck - how do we get a reference to such an object, if we can’t construct them ourselves? And what’s the point of being so awkward?
The answer is that Singularity requires you to annotate your code with metadata, describing what hardware your driver is for and what resources it needs. This is done in the usual .NET fashion, with attributes that decorate your classes and fields. At install time, this metadata is used to build an XML manifest, describing what resources your code needs, and is input to a compile time transform, which fills out an empty method you provide. This transform is itself a part of the kernel, and is applied when you install the driver - it is what adds the protected call to the hardware abstraction DLL.
In this way, Singularity ensures that it has accurate declarative metadata on what exactly the driver needs. Because you can’t get hold of the relevant objects and thus program the hardware unless you define accurate metadata (and this is enforced by the system), the metadata is guaranteed to be accurate. That, in turn, makes it easy to solve problems like avoiding driver conflicts and figuring out the order in which drivers should be loaded at bootup.
the hole in the plan
This would all be fine and dandy, if it weren’t for The Catch (why is there always a catch?). The catch is this - any driver that can access a DMA capable device can subvert the type system by overwriting arbitrary areas of physical memory.
Direct Memory Access is an optimisation that lets hardware devices directly write data into RAM, bypassing the CPU. It’s very useful and speeds things up a lot, especially when moving lots of data around like with graphics cards, hard disks, network cards and so on. Unfortunately, DMA not only bypasses the CPU but also the MMU, thus a driver that can control a DMA capable piece of hardware can control the contents of memory. In a regular operating system that doesn’t matter, because such drivers are running in kernel mode anyway and are fully trusted. But for Singularity it’s a problem.
Fortunately for us, DRM comes to the rescue! Yes, I’m serious. Newer CPU architectures being developed by Intel and AMD feature something called, appropriately enough, an IOMMU. This does for hardware devices what the regular MMU does for the CPU - regulates memory access by mapping DMA read/writes through a set of page tables. By configuring the IOMMU, you can stop hardware from fiddling with memory. This feature was originally intended to stop people breaking into kernel space by plugging in physical pieces of hardware, which apart from the obvious downsides for DRM manufacturers can also be used to do things like unlock screens. And of course it protects computers from buggy hardware, which is not such an uncommon thing.
It’s also mighty handy for us, because it closes the hole that would mean DMA-aware drivers could break out of their type-verified jail.
how it helps security
Before you can make something secure, you need a threat model. What exactly do we want to stop?
I’m going to skip over that here and mumble something about stopping malware and viruses. It’s vague and not that useful, but it’ll help illustrate these examples.
Back in 2001 the idea of Microsoft restricting what software you could put on your system seemed grotesque. They were just coming to the tail end of the the browser wars, broadband was just beginning its rollout and the general problem of botnets had not yet reared its ugly head. The potential for abuse seemed way too high, the benefits way too low. Despite that, drivers were known to be such huge reliability problems that a signing program was instigated …. Microsoft would provide you with a unit test suite, and if it passed, that suite would sign the driver for you. The test suite checked for many common problems and didn’t require anybody to hand over their code to Microsoft. Seemed like a good idea, right?
Well, of course it was controversial. Worse, it didn’t necessarily work. Rather than pass the tests, some manufacturers cheated. Rather than risk a schedule slip, or pay the $250 verification cost, some moved the mouse and clicked OK on the unsigned driver warning for you. D’oh.
If we’re willing to be a bit despotic and require signing that cannot be disabled for some things, can we do better? Yes, but it would suck to require every driver to be signed. We can avoid this in the most common cases.
For instance, thanks to the declarative metadata, we can now statically inspect a driver and say, well, gee, this driver will only be activated for devices of class “sound card”, and it doesn’t use DMA, and it doesn’t read or write any files (we know this because we can statically verify that it can’t get any connections to the filesystem server). What can this driver do, except mess up the sound card? Not a whole lot, actually. It could still hose the box by misprogramming the hardware, but such bugs are rare. But it can’t blue-screen us, because it’s not running in the actual kernel. If it crashes, the worst that happens is audio is interrupted until the driver restarts and programs re-establish their connections (assuming we write software to handle driver crashes of course - more on that later).
Generally, we can define some set of privileges (in Singularity this is equivalent to defining some set of connections a SIP can access) which might be dangerous, and only require signing for those drivers. What might trigger such a requirement? Accessing system files or configuration is an obvious case. Being a network driver is another case (how to DoS a target without falling foul of the firewall? Become a network driver). Fortunately, i
t’s rare that drivers need access to anything other than the hardware and their own configuration data, and network drivers are typically uninteresting enough that the default drivers suffice for most people. Thus the majority of drivers can be safely written, distributed and installed without Microsoft ever needing to be involved whilst preserving the security of the system.
Most experienced computer users will agree with me when I say that typing something is often quicker and easier than navigating a deep menu structure. I hate the standard windows start menu with a passion, which means I was quite happy when I found the nice open source tool called Launchy. Now just a quick <alt>+<space> away, I can search an indexed db of my launchers. That's on windows...
On Linux, I've been using a combination of terminal (requires you to know the binary name, plus tab completion only matches beginning of strings in bash), panel launchers and the deskbar applet.
Unfortunately, the deskbar applet lacks the sex appeal of Launchy, and it's always crowded with results I seldom want (yeah, I know I can configure it and add/remove plugins and what-not, but it still lacks sex appeal ;-) ).
Via planet gnome the other day, I found a new player in town - Gnome DO! It's a slick little app, very similar to Launchy on Windows, but slightly on stereoids. By default, you just type something and it'll run it for you when you hit enter. But for things which has multiple actions, there's also an "action" section which can be reached by hitting <tab>. So if I type "movie", it will display an icon showing my movie folder with default action to open it in nautilus. But right now, I don't want to do that so I hit <tab> and enter "terminal" giving me the option to open a gnome-terminal in that folder. Slick.
Now if someone could add SnagIt capabilities to the screenshot tool, I'd be an even happier Linux user. :-)
Last updated:
May 09, 2008 01:01 PM
All times are UTC.
Powered by:
