The Interesting Case of Diminishing Responsiveness

I’m working up a second run of PHP framework benchmarks on more powerful hardware than the last time. I’ll talk more about that in a future entry, but in the mean time, I wanted to share an experience that may help you out.

I was running the ab (Apache Benchmark) tool against Zend Framework, Solar, and Cake, using the minimalist “hello world” applications I described the previously. The ZF requests/minute were consistent between runs, always “Z” amount (plus or minus a couple percentage points).

But the Solar and Cake requests/minute declined considerably from run to run. For example, the first benchmark run would yield “X” requests/minute; the 2nd run would yield 0.93X, the 3rd 0.85X, then 0.75X, 0.64X, 0.52X, and so on. This was with the opcode caching turned on; without it, the decline was less severe, but it was still there. Very depressing.

Because I know Solar better than I know Cake, I started looking around the codebase for places to improve. Was it the way I was loading classes, thus causing the opcode cache to work inefficiently? Was it a memory leak in Solar, or PHP itself, or maybe in the web server somehow?

Nope: it was sessions.

Solar and Cake each call session_start() on new requests. Because the “ab” tool sends a new request each time, the frameworks start a new session each time. This means that 1000 requests will generate 1000 new session files. Because the benchmarking makes thousands of requests, the /tmp directory was overflowing with sess_* files (50,000 or more). Having a very large number of files in a single directory was causing the filesystem to drag, and so response time diminished exponentially over time.

Notice that Zend Framework did not have this problem, because it doesn’t have a session-management class; thus, no sessions were created, explaining why its performance stayed consistent from run to run.)

The solution (for benchmarking purposes) was to force the session ID value to ‘abc’ by calling session_id('abc') command at the top of the Solar and Cake bootstrap scripts. This makes the script think there’s already a session in place, and so it doesn’t create a new file. Once I did that, the diminishing responsiveness disappeared, and I got consistent results from run to run.

How Fast Is Your Framework?

UPDATE (2007-01-01): I have conducted a new series of benchmarks; see them here. The numeric results below should be considered as outdated, although the history, methodology, and approach are still valid.

Wherein I relate the results of benchmark testing on three (plus one)four frameworks: Cake, Solar, Symfony, and Zend Framework(plus Cake). Anger-inducing broad-brush overview: Solar is 4x faster than Zend, and almost 2x faster than Symfony. Read on for all the nuances and caveats.

UPDATE: Have updated the Cake results, now that I’ve figured out how to turn off database access.

UPDATE 2: Added note about “views” in Zend Framework, also added list of loaded files and classes for each framework.

Continue reading

Solar 0.24.0 Released

Last night, I released version 0.24.0 of Solar, the simple object library and application repository. Solar is a PHP 5 framework for rapid application development.

On an administrative note, Clay Loveless has set up a Trac installation for us, so now you can report bugs and request enhancements without having to join the Solar-Talk mailing list. You can keep up with the Trac issues via the Solar-Trac mailing list if you like.

You can read the change notes for a full run-down, but there are a few changes in particular that I want to highlight.

Continue reading

AJAX Is Interesting, But The Basics Are Better

I had the good luck to share a cab back to the airport with Wez Furlong after php|works. Wez had presented a new talk about how to work with email properly (jokingly subtitled “Not PDO” by everyone there), and was curious to know how it was received. I replied that I liked it, but struggled for a moment on “why” … and then it hit me: the reason I liked it was that it was about a fundamental operation that still seems to trip developers up regularly.

Although I told Cal Evans when he interviewed me that the next technology I’m really interested in is AJAX (particularly its implementation in Protaculous), I’m not as excited by it as everyone else seems to be. I think it’s neat, but good grief, if wide swaths of developers can’t do input filtering properly, what good can adding AJAX do? Get the basics right first, then you can do progressive enhancement as you go.

Teaser for php|works

I’ve noted before that I’ll be presenting at php|works in a couple of days; it’s in the 4-5pm slot on Wednesday (13 Sep). The synopsis is public, but I wanted to give a little teaser about the presentation, in hopes of drumming up greater attendance. The talk is titled “Organizing Your PHP Projects”, and the subtitle is “Project Planning in One Lesson.”

Continue reading

Solar and TypeKey Authentication

As astute observers will have realized, the most-recent release of Solar had a new authentication adapter in it: Solar_Auth_Adapter_Typekey.

This particular class comes from work I’m doing at my current employer, Mashery. The Mashery folks are very open-source friendly, and approve of contributing back to useful projects, so I want to go out of my way and thank Mashery explicitly for this. (Thanks Mashery!)

Likewise, the core code in Solar_Auth_Adapter_Typekey comes directly from the PEAR Auth_TypeKey work by Daiji Hirata (which strangely has not been accepted yet at PEAR; their loss is Solar’s gain). Thanks, Daiji, for contributing this code to Solar!

In this article, I’m going to talk a bit about Solar authentication in general, and then TypeKey authentication in specific.

Continue reading