Publish Your Failures; or, The Way Of All Frameworks

The blog post Is ORM Abstraction A Pipe Dream? is one of the best technical articles I’ve read in maybe a year. Not because it describes a successful technical solution, but because it describes an attempt and a failure to achieve a goal. It reminds me of this quote from Feynman in Cargo Cult Science:

If you’ve made up your mind to test a theory, or you want to explain some
idea, you should always decide to publish it whichever way it comes out.
If we only publish results of a certain kind, we can make the argument
look good. We must publish both kinds of results.

(Cargo Cult Science is one of those essays that everyone should read, especially programmers who think of themselves as scientists or science-minded.)

What Feynman is saying here is that, when it comes to expanding a body of knowledge, the failures are just as important as the successes, perhaps more so in some cases. (Be careful here; they have to be “honest” failures, where you had some reason to believe in advance that it had a good chance of working.)

So what is it about the “Pipe Dream” article that impressed me? It is that the the author first signals his tribe membership by mentioning his “framework of choice”, then proceeds to try to do some work outside of that tribe. In doing so, he is expanding his knowledge and skills so that he can be useful outside the norms of that particular tribe. He realizes that there are other ways to do things than the way his tribe does them, he does some work with those things, he tries to unify them, and he fails at the unification. In the end, he has no technical artifact to show for his exploratory work, but he has now expanded his skills as a programmer beyond his “framework of choice.”

Your Framework Will Fail You

And being able to operate outside a framework is really important, because the framework you choose today, even if it’s the self-described “greatest PHP framework of all time”, is going to fail you in some non-trivial way in 3 years or so, even if it has a “long-term support” version. (After all, how long is long-term in the web world?)

Take a look at some points from this article by Yannick Mahe titled The consequences of living with a legacy PHP framework (note that “legacy” in this case is only 4-5 years):

You can no longer rely on the community. …

Documentation can be hard to find now. …

I can effectively no longer use plugins. …

[This framework] used Prototype (scriptaculous) as its Javascript framework … UX advances which could require just a simple jQuery plugin have to be written from scratch. …

This framework is not compatible with newer versions of PHP. It was written for PHP 5.2, and is compatible with PHP 5.3 but no longer works on PHP 5.4. …

Basically I’m one third less productive when programming than I could be.

Is that a slam against the framework or its defenders? No more than saying “the sky is blue” is a slam against the physics of light diffusion in atmosphere. “The framework is going to fail you” is a truism regardless of the framework.

And everybody uses a framework, whether they recognize it or not. I have said at other times that a “framework” can be defined as “that collection of tools and techniques you reuse on a regular basis from project to project”. So everyone uses a framework one way or another, whether formal or informal, well-architected or not, designed with intent or evolved through use.

How To Handle Failure

So if all frameworks are going to fail you, and you can’t avoid using one, what are your options?

All too often, software is like a complex system in that requires a constant input of energy, and complex systems fail. So the key point here is not “the framework is going to fail you.” The key point is “how easy will it be to handle that failure when it happens?”

A developer twin of Nassim Taleb might argue that whatever framework you have needs to be antifragile. That is, when one subsystem fails or becomes obsolete, it should not render the rest of the system unusable. Indeed, the failure of a subsystem should lead to a replacement that makes the system stronger in the future.

Frameworks in general are especially susceptible to their subsystem failures: they are usually developed as a thing of whole cloth (yes, even the ones that advertise they are “component based” — the components might be OK, but the framework often is not). The different parts depend on each other, and if one fails, you need to dive into the guts of the framework proper to fix it in place, which frequently leaves you worse off than before when it comes to maintainability.

Instead, for a robust or antifragile system, you should be able to swap out the different subsystems when they no longer meet your needs. As each piece fails (and each one will eventually fail to meet your needs), you’ll swap out each of the pieces at different times. Eventually the system will be composed entirely of replacements, but the whole thing will have kept running the whole time.

And that’s where we come back to the “Pipe Dream” article. By taking some steps outside his “framework of choice” the author is beginning to get an idea of how to integrate independent and disparate systems into a whole. In the long run, that ability is going to serve him better as a programmer than his knowledge of any particular framework.

Afterword: Aura for PHP

Of course, I’m going to mention Aura here. The ideals illuminated by article are the ideals that drive the development of the Aura, even to the point of prejudicially declining certain feature requests.

The Aura project is a collection of truly independent and fully decoupled libraries. None of the libraries has any external dependencies. The project is “library-first” and not “framework-first”, even though we do offer a framework built of the combined packages. The libraries evolve independently of the framework and can be incorporated individually or in any combination you like into any project.

If you liked the ideas behind this article, then the Aura project is for you. Download a single package and start using it in your project today, with no added dependencies.


UPDATE (30 Oct 2013): The title on the final section has been prefixed with the indicator “Afterword” since at least one person mistook it for a summary.

A Peek At Aura v2: Aura.Sql and ExtendedPdo

… it turns out that merely extracting the SQL parts was not good enough for some of our audience. They asked, “Can we download just the SQL connection classes, without the query objects or the mapper?” As with Solar, we had to answer, “No; although the Aura.Sql package is independent from every other Aura package, the various SQL tools all have be taken together as a whole.”

As a result, we have done even more decoupling with the Aura.Sql-v2 packages. We have managed to make these packages independent from each other:

Aura.Sql-v2 is composed primarily of an extended PDO implementation, along with a connection manager and a bare-bones profiler. Its only dependency is the native PDO; there are no userland dependencies. As an added bonus, we have taken pains to make it PHP 5.3 compatible.

Aura.Sql_Query is a package of SELECT, INSERT, UPDATE, and DELETE query objects for MySQL, PostgreSQL, SQLite, and Microsoft SQL Server. It has no dependencies at all, not even on PDO. You build your query with the object, then pass it along to the database abstraction layer of your choice. Because it uses traits, it is for PHP 5.4 and up.

Aura.Sql_Schema comprises “list tables” and “list columns” functionality for MySQL, PostgreSQL, SQLite, and Microsoft. As with Aura.Sql-v2, its only dependency is the native PDO (note, not the Aura.Sql-v2 ExtendedPdo). Also as with Aura.Sql-v2, it is PHP 5.3 compatible.

With this dedicated effort at decoupling and true indpedendence, you can now use just the extended PDO connection object, or the query objects, or the schema discovery tools, without having to download the others. Of course, you can use them all in concert if you like.

via Aura for PHP.

Quality, Features, and Schedule

From the book Lost Moon, retitled Apollo 13 after the movie was made:

Apollo was downright dangerous. Earlier in the development and testing of the craft, the nozzle of the ship’s giant engine…shattered like a teacup when engineers tried to fire it. During a splashdown test, the heat shield of the craft had split open, causing the command module to sink like a $35 million anvil to the bottom of a factory test pool. The environmental control system had already logged 200 individual failures the spacecraft as a whole had accumulated roughly 20,000.

In January 1967, one of the first Apollo spacecraft caught fire during an on-the-ground test, killing astronauts Gus Grissom, Ed White, and Roger Chaffee. At that point, NASA decided that quality was more important than schedule and they overhauled the Apollo project (although they still managed a moon landing 2-1/2 years later).

Far too many software projects are like that. It usually takes a serious failure, one where blame cannot be shifted, for managers to realize that their schedule is unrealistic. Via Quality, Features, and Schedule | askblog.

If you inject a container into your class, you are using Service Locator, not Dependency Injection

A while ago I tweeted

ContainerAware is the new Singleton.

While many people agreed by retweeting and faving. I feel the need to elaborate some more on this statement and safe the explaination for the future.

TL;DR: No class of your application (except for factories) should know about the Dependency Injection Container (DIC).

The ContainerAware interface (actually ContainerAwareInterface, ContainerAware is a basic implementation of it) is part of the Symfony2 API, but a similar concept is known from many other frameworks and many applications rely on it. It defines only the one method setContainer(), which allows to inject the DIC into into an object so that it can directly retrieve services from it.

I wouldn’t call it “the new Singleton”, I’d call it “Service Location.” If you use a dependency injection container inside your class to bring dependencies into the object, you are using Service Locator, not Dependency Injection. Solar used SL, but when we started Aura (a collection of truly decoupled library packages, each with no dependencies) we switched over to DI proper. Real DI has been a huge win.

Via ContainerAware Considered Harmful – Qafoo GmbH – passion for software quality.