A Response To “On php-fig and Shared Interfaces”

(This is a response to Matthew Weier O’Phinney. Full disclosure: MWOP is both my friend and respected peer. Also, like MWOP, I am a voting member of PHP-FIG, and was directly involved in the recent LoggerInterface discussion among that group.)


Matthew wrote:

My primary problem with the idea of shared interfaces is that I feel there is always room for new thinking and ideas in any given problem space, and that this thinking should not be restricted by what already exists.

By definition, truly new thinking is not generally restricted by what already exists. 😉 If lots of developers adopt new thinking on an old problem, I think it is completely within reason to deprecate an older PSR and write a new one that codifies the new approach.

Even so, I always thought a major point of PHP-FIG was to recognize which things we already do much the same way and codify PSRs around them. (I will admit I continue to be frustrated by the unwillingness of other members to do basic research and discovery of this kind.)

Thus, when we consider functionality …

  • that does already exist,
  • that has been implemented by multiple different projects,
  • in ways that are similar despite their varying origins,

… I think it can be very useful to reduce the differences and combine the commonalities into a shared interface.

In the specific case of logging, as Matthew gives, I reviewed each member project’s logging interface or base implementation:

https://groups.google.com/group/php-fig/msg/6e4285dc3116331c?dmode=source&output=gplain&noredirect&pli=1

True, I was examining them in reference to the string-interpolation portion of the proposed LoggerInterface. But I was struck overall by how similar they really were in terms of the basic notification methods. Differences were minor at best. For something like that, a shared interface sounds to me like an unqualified good.


Matthew also wrote:

Secondarily, I feel that it’s okay for a given project to be selective about what capabilities it requires for its internal consumption and consistency, and should not limit itself to a standardized interface.

I completely agree. There’s nothing that says a development effort is required to adopt anything from PHP-FIG. If someone thinks their way is better, then by all means ignore any and/or all PSRs entirely.


To sum up (and MWOP, please correct me if I’m not getting what you’re saying):

One is able to imagine reasons why having shared interfaces of the kind described above is in opposition to, or at best orthogonal to, better development practices and greater innovation across PHP land.

Even so, I assert that shared interfaces as described, while maybe preventing an imaginable ideal in theory, instead promote an actual good in practice.

The actual good can never be as desirable as the imaginable future ideal, but it does have the benefit of being real and present.


UPDATE: MWOP informs me by private communication that I have missed his point. I await his next missive. Here’s his response: http://mwop.net/blog/2012-12-20-on-shared-interfaces.html#comment-744905017

On Decoupling and Dependencies: Answering The Critics of Aura’s Approach

“There are no solutions … only trade-offs.” — Thomas Sowell

tl;dr: “As a library author it’s tempting to bring in dependencies for various reasons, but as a library user it’s frequently convenient to not have cascading dependency sets. … If a user already has something they like, make it as easy as possible for them to keep using it without making them install things they don’t actually need. That is the kind of duplication we really want to avoid. This applies not only to HTTP response delivery, but to caching, logging, translation, templating, and a host of other functionality.”


Recent articles from me and others regarding the Aura project for PHP 5.4+, dependencies, and decoupling have generated a lot of commentary, some of it negative, and some of it misdirected. For reference, the posts are:

In this post, I’ll respond to some of that commentary. I will sum up similar comments and address those summaries.

Regarding “Dependencies Are Bad”

Bernhard “Webmozart” Schussek writes an otherwise good post that, unfortunately, begins by misconstruing my statements:

Aura is creating some buzz that components of PHP frameworks should not have any dependencies.

The general attitude of people arguing against dependencies is that dependencies are a bad thing.

Matt Robinson makes a related comment:

You’ve made a judgement call that dependencies should be avoided at all costs – I think that’s going a bit too far.

I can understand how some readers might infer the “dependencies are bad” sentiment from the interviews, especially if they did not read with thoughtful consideration. Their assessment is incorrect.

For the record: dependencies are not a bad thing in and of themselves, nor are they a good thing in and of themselves. Choosing to have one or more dependencies is a trade-off in software development, where the gains and losses from having the dependency must be judged according to the context of the development goals.

If dependencies are neither good nor bad in themselves, why does Aura avoid them so assiduously?

To learn the answer, please recall the origin of the Aura project (which is essentially the second major version of the Solar framework):

One of the repeated questions we got regarding Solar went like this: “I want to use just one part of Solar; can I do that without having to download and configure the whole framework?” Of course the answer to that was “not really.” It was a monolithic framework, where all the pieces were designed to work with each other in a relatively dependent fashion.

So with Aura, we started out from the other direction. We wanted the pieces to be usable on their own, without any other dependencies. Only after that would we build a framework out of the pieces.

As you can see, it is expressly our goal to serve a particular set of users as our primary focus: those who want independent libraries without added dependencies.

As a library author it’s tempting to bring in dependencies for various reasons, but as a library user it’s frequently convenient to not have cascading dependency sets. Call it a case of taking our audience seriously, and going as far as we can with it.

Regarding “Duplication”

Matt Robinson asks:

[W]hat’s the real difference between AuraHttpMessageResponse and AuraWebResponse? They look remarkably similar and have significant overlap in interface and purpose. I might be wrong, but it looks like you’ve got this big chunk of duplicated effort for the sake of avoiding one line in a composer.json file. 🙂

They are remarkably similar, since they serve a similar purpose; that is, to describe an HTTP response. I think “big chunk” is an inaccurate characterization, though; it’s 2 classes out of 56 between both packages.

As Matt allows, he is in fact wrong. 😉 The duplication is not for the sake of avoiding one line in a Composer file. It is to make sure that people who want to use Aura.Web package are not required to additionally install the entire Aura.Http package to get access to a single class.

Many library users out there already have HTTP delivery mechanisms of their own choosing, and have neither the desire nor the inclination to drag another one into their codebase. All these users need is a way to describe an HTTP response. It’s easy to do that in a single file.

Because of this single class of duplication that exists in Aura.Web, library users can build a relatively small amount of glue code to send the AuraWebResponse using their existing installation of Guzzle, Buzz, the HTTP extension, plain old PHP, or even the Aura.Http package. This is the tradeoff: a single duplicated file, versus many dozens of files of duplicated, unused, and unneeded functionality.

That is the central point for the Aura project: if a user already has something they like, make it as easy as possible for them to keep using it without making them install things they don’t actually need. That is the kind of duplication we really want to avoid. This applies not only to HTTP response delivery, but to caching, logging, translation, templating, and a host of other functionality.

Stan, who admittedly is part of the Aura target audience, already knows this:

[T]here are some of us out there that have a hybrid of components tooled together. The more dependencies that are forced on us (guilty by assoc) the harder that is to do. So maybe my app that’s routed by Aura, templated by Twig, works with the DB using Doctrine ODM, utilizes Symfony for the Finder and Yaml components and Zend for the console could find itself in a pickle when more baggage is added to the mix through dependencies that might not be necessary.

Amy Stephen also figured out the point pretty quickly …

Disagree with Paul’s assertion in the comments that such a strategy does not lead to some duplication of code or function or that the methodology means some functionality be sacrificed. Of course, it does. That’s the ying and the yang of it – get one thing clean and another dirty – the cost, time, quality triangle – It’s math. It’s always a balancing act building software.

… although to be fair, I didn’t say “does not lead to some duplication”; I said duplicated code was “almost nonexistent.” 😉 Perhaps that too was a mischaracterization: a few classes here and there, among hundreds of classes and a dozen packages, seems rather small to me. Others may disagree.

Regarding “Not Invented Here”

Some commenters claim that this approach promotes “NIH” syndrome. Drak said:

Seems to me like you are promoting NIH. The very concept that a decoupled component cant have dependencies means you have to invent everything yourself.

Webmozart similarly opined:

I think this needs some urgent clarification before this way of thinking becomes mainstream and PHP creeps back into its NIH-hole.

I would argue that the offering decoupled packages in the way the Aura does leads away from not-invented-here syndrome, not toward it.

Recall that one of the driving motivations behind Aura was to extract the parts of Solar that users wanted as separate packages. Those users wanted very badly not to engage in NIH by using Solar, but could not, because it was delivered as a monolithic whole. Those users can now download independent, decoupled Aura packages and avoid rewriting the functionality contained therein.

Bertrand lobbed a shot over the net at, I guess, the Symfony crowd, when he said:

Ha, ha, who’s got the NIH syndrom ?

Event_Dispatcher, written in 2005, 3 years before the one in your example…

http://pear.php.net/package/event_dispatcher

This is a comment I sympathize with. It seems to me that many accusations of NIH-ism are more complaints and frustrations that the accused is not using the libraries and packages the accuser prefers.

Regarding “Tests As A Way Of Discovering Dependencies”

Some commenters were dissatsifed with my use of unit testing requirements to discover what a package really depends on, as opposed to what its composer.json file states. Webmozart said:

There’s a distinction between required dependencies and optional, supported libraries. For example, the Symfony2 Validator optionally supports annotations, in which case you need DoctrineCommon (why should it reinvent annotation parsing?). You can use the Validator without any of these supported libraries.

Obviously, the support for these libraries is tested, so for running the tests you also need the supported libraries (doh). I fail to see how this is bad.

Lukas covered this as well:

[T]he issue is that the way [Paul] was comparing. Paul was making a very incorrect conclusion that anything that is needed to run the tests illustrates “cross-package dependencies”. Yet if you look at the composer.json (which I would have expected Paul to be familiar with) then you would notice that infact the Validator component has ZERO dependencies beyond PHP itself:

https://github.com/symfony/Validator/blob/master/composer.json

This was a topic I had hoped I would not have to address. All my other statements regarding dependencies have been as clinical as I could make them, without reference to words like “good” or “bad” in describing the various practices. For this topic, though, I must make some statements that others will find negative or perjorative. Because of that, I am going to do it in an entirely separate post, so as not to pollute this commentary thread too harshly. Look for it in a few days.

Conclusion

Thanks to everyone for their comments and critique, and please be sure to check out the Aura project!

New Aura System Release, Including Aura.Framework and Aura.Demo

Late last night we released version 1.0.0-beta4 of the full-stack system, which composes all the independent Aura packages into a coherent framework using the Aura.Framework package in a project skeleton. We have extracted the “hello world” portions into their own Aura.Demo package so that it can be removed or disabled when you begin your project. Download it and take it for a spin!

via New System Release, Including Aura.Framework and Aura.Demo.

Aura.Micro: Experimental Replacement for Silex

Stan Lemon writes on the Aura Project for PHP 5.4+ blog about an experiment he’s trying:

I was recently working on a small project that used Silex. As I browsed my vendor folder, I realized how much extra “stuff” I had inherited with Silex. There were a bunch of other components required when all I wanted was some quick and easy routing, micro-framework style.

When I think about going lean I always find myself coming back to Aura. Micro-frameworks are not a new to idea to Aura, so I wondered if I could take the elegance and ease of Silex by wrapping up Aura.Router and exposing it through a similar API. The result is Aura.Micro, a light wrapper for Aura.Router to get a Silex-style API.

Read the whole thing here: Aura.Micro — Experimental Replacement for Silex.

Aura: More Decoupled Than Symfony 2 and Zend Framework 2

The guys at PHPMagazin have posted a followup question about the Aura Project for PHP. Here is our original English conversation.


I’d like to follow up on the last point where we talked about other frameworks. I think that the second generations of symfony and Zend Framework have gone through a huge decoupling process. This becomes evident when you look at symfony2 components being used in Drupal 8 or web tutorials which show you mixed use of zend- and symfony components.

So I think we could elaborate further on that last bullet point we discussed. Does the above change your angle in this regard?

Not in this case. Being able to use Symfony2 components or ZF2 modules is not quite the distinction I am making. The distinction is that all Aura packages (with the exception of the Framework package) are completely independent, and have no cross-package dependencies, whereas at least some of the components from Symfony2 and ZF2 have dependency requirements.

By way of comparison, let’s examine something that ought to be relatively straightforward: input validation and filtering. We’ll start with ZF2, then move on to Symfony2, and end with Aura. In each case, we will try to download the package and run its tests; this should be a good indicator of whether or not the package can be used independently. We will discover that you can’t do it at all with ZF2, you can kind of do it with Symfony2, and that it’s trivially easy with Aura.

Zend Framework 2

Let’s start by downloading the ZF2 InputFilter package. It doesn’t appear to be available for download by itself — there is the main ZF2 repository at Github https://github.com/zendframework/zf2 but that’s the whole framework, not just the InputFilter.

OK then, we can at least use Composer to incorporate the InputFilter into a project. We will set up a Composer file per the ZF2 instructions and install the InputFilter package. Our composer.json file looks like this:

{
    "repositories": [
        {
            "type": "composer",
            "url": "https://packages.zendframework.com/"
        }
    ],
    "require" : {
        "zendframework/zend-inputfilter" : "2.0.*"
    }
}

After downloading Composer into the same directory, we run the installer, and this is what we see:

$ ./composer.phar install
Loading composer repositories with package information
Installing dependencies
  - Installing zendframework/zend-stdlib (2.0.5)
    Downloading: 100%

  - Installing zendframework/zend-servicemanager (2.0.5)
    Downloading: 100%

  - Installing zendframework/zend-filter (2.0.5)
    Downloading: 100%

  - Installing zendframework/zend-i18n (2.0.5)
    Downloading: 100%

  - Installing zendframework/zend-validator (2.0.5)
    Downloading: 100%

  - Installing zendframework/zend-inputfilter (2.0.5)
    Downloading: 100%

zendframework/zend-stdlib suggests installing pecl-weakref (Implementation of weak references for StdlibCallbackHandler)
zendframework/zend-servicemanager suggests installing zendframework/zend-di (ZendDi component)
zendframework/zend-filter suggests installing zendframework/zend-crypt (ZendCrypt component)
zendframework/zend-validator suggests installing zendframework/zend-db (ZendDb component)
zendframework/zend-validator suggests installing zendframework/zend-math (ZendMath component)
Writing lock file
Generating autoload files
$

In order to use input filtering from ZF2, six other packages are required, and a few others are suggested.

Now that we’ve installed it, where are the tests? They’re not provided with the Composer package, although I suppose they are available with the framework as a whole. It appears the Zend offering is not entirely self-contained.

(If we look closely, we see that InputFilter is composed of at least two other packages that might suit our needs, zend-filter and zend-validator. However, when you examine them, they’re not comparable to Aura.Filter and Symfony2 Validator.)

Symfony 2

Let’s do the same thing with Symfony. This time it is downloadable through Github: https://github.com/symfony/Validator

Let’s clone the package and run the tests.

$ git clone https://github.com/symfony/Validator.git
Cloning into Validator...
remote: Counting objects: 3459, done.
remote: Compressing objects: 100% (672/672), done.
remote: Total 3459 (delta 2708), reused 3451 (delta 2700)
Receiving objects: 100% (3459/3459), 621.73 KiB | 730 KiB/s, done.
Resolving deltas: 100% (2708/2708), done.
$ cd Validator/Tests/
$ phpunit
[phpunit fails]

It appears we can’t just download the package and run the tests. The README states we need to use Composer and install all the --dev dependencies, so we’ll do that. First we download Composer into the cloned repo, then:

$ ./composer.phar install --dev
Loading composer repositories with package information
Installing dependencies
Nothing to install or update
Loading composer repositories with package information
Installing dev dependencies
  - Installing symfony/yaml (dev-master bed4fdd)
    Cloning bed4fddc24392513e01b32a78d600b1272ed9a6c

  - Installing symfony/locale (dev-master 2dceded)
    Cloning 2dcededb060dfb6289ad8bb3f2a7a4e00929c4dc

  - Installing symfony/http-foundation (dev-master 067c310)
    Cloning 067c310fe4d0691a24adc97f39500233a58e42cb

Writing lock file
Generating autoload files

Interesting: in order for the tests to run we need three other packages. It looks to me like there are cross-package dependencies. Examining the codebase reveals this to be true:

  • If you want to use the YAML loader included with the Validator, you need that YAML package after all.

  • If you want to validate against anything related to locales or languages, you need the Locale package. For example, the LangaugeValidator.php file makes a static call to SymfonyComponentLocaleLocale::getLanguages().

  • If you want to use Annotations with the Validator, it looks like you need Doctrine, which isn’t a part of the Symfony vendor hierarchy at all. Goodness knows what that will require.

Anyway, now we can run the tests; I have omitted the test progress output.

$ phpunit
PHPUnit 3.7.9 by Sebastian Bergmann.
...
Time: 8 seconds, Memory: 17.75Mb

OK, but incomplete or skipped tests!
Tests: 987, Assertions: 1091, Skipped: 12.
$

12 tests have to be skipped because of missing dependencies. When we run phpunit --verbose we discover that:

  • 3 tests are skipped are because APC is not loaded for the command line, which is not such a big deal;

  • 6 are skipped because “The Doctrine Common library is not available”;

  • 3 are skipped because “Annotations is required for this test”.

As with the ZF2 InputFilter component, it looks like the Symfony2 Validator component is not entirely self-contained. It has external dependencies that must be fulfilled in order for it to be fully useful.

Aura.Filter

Finally, we have the Aura.Filter package. It is downloadable through Github at https://github.com/auraphp/Aura.Filter. Let’s try to clone it and run the tests; I have again omitted the test progress output.

$ git clone [email protected]:auraphp/Aura.Filter.git
Cloning into Aura.Filter...
remote: Counting objects: 1105, done.
remote: Compressing objects: 100% (373/373), done.
remote: Total 1105 (delta 631), reused 1076 (delta 602)
Receiving objects: 100% (1105/1105), 725.94 KiB | 606 KiB/s, done.
Resolving deltas: 100% (631/631), done.
$ cd Aura.Filter/tests/
$ phpunit
PHPUnit 3.7.9 by Sebastian Bergmann.

Configuration read from /Users/pmjones/Aura.Filter/tests/phpunit.xml
...
Time: 4 seconds, Memory: 9.75Mb

OK (1009 tests, 1443 assertions)
$

No external dependencies, and nothing extra is needed for the tests to run. The package is completely self-contained, independent, and decoupled. Now compare the memory use and time taken to that of Symfony2: about half the memory used, and about half the time taken, to complete about 30% more assertions in the tests. (Incidentally, we have 100% test coverage of the source classes, but I don’t know how that compares to Symfony and Zend.)

Conclusion

None of the above is to meant to say that Zend Framework or Symfony2 are poorly architected, not useful, or any other negative thing. They are good projects, and the components appear to be good too.

The only thing I am saying is that their offerings of separate components are not always very well decoupled. This is because they started with a framework and tried to extract pieces from it. (To be fair, some components from both of those projects really are dependency-free, but not all of them.)

But in Aura, with its “libraries first” approach, every package is truly decoupled, independent, and self-contained, with zero cross-package dependencies. That is the major difference I want to emphasize.

How Is Aura Better Than (er, Different From ;-) Than Symfony and Zend?

I did an email interview with the folks at PHP Magazin; their German version is here. What follows is our original email exchange in English.

First of all, congratulations for releasing Aura 1.0!

Thanks! Most of the packages are at 1.0, but there are still three that are in beta; I expect them to go “stable” soon as well.

Why did you do it?

Aura is essentially a second major version of the Solar Framework. (Solar was the first E_STRICT framework for PHP 5; its development pre-dates that of the Zend Framework.) One of the repeated questions we got regarding Solar went like this: “I want to use just one part of Solar; can I do that without having to download and configure the whole framework?” Of course the answer to that was “not really.” It was a monolithic framework, where all the pieces were designed to work with each other in a relatively dependent fashion.

So with Aura, we started out from the other direction. We wanted the pieces to be usable on their own, without any other dependencies. Only after that would we build a framework out of the pieces. We called this our “libraries first, framework second” principle. This means you can use just one Aura package if you want, and you won’t get a lot of of other packages as dependencies; each one is completely self contained, including its tests. Each one uses separated interfaces and data transfer objects as necessary to move information across package boundaries.

In addition to that, we wanted to take all the lessons we learned from Solar and break backwards compatibility to start over again. The single biggest BC break has been moving away from a Service Locator implementation and the universal constructor, toward a more formal Dependency Injection oriented system. That one change has made for gigantic improvements in decoupling, testability, and package independence. (I have to thank Jeff Moore here for being patient with me and slowly getting me on the dependency injection track.) We don’t even use superglobals within the packages; everything from the environment has to be copied into the objects, which makes things really easy to test.

Why did you decide it has to be PHP 5.4? What’s the advantage?

When we started the Aura project in 2010, we targeted PHP 5.3, since it was the most recent PHP version at the time. Closures and traits especially have a lot of powerful uses if you approach them wisely. Then PHP 5.4 came out in January 2012. Almost all of the Aura packages were still in development at the time, so we figured we might as well target PHP 5.4, with its short-array [] syntax and “callable” type hint. Those things seem small, but once you start using them, they are *so* convenient (and frankly they make the code look prettier :-).

You seem to love small packages. What do you think of the microframework approach Ed Finkler published in the beginning of the year?

I think Ed has a strong point, although to be clear I don’t think he’s so much about “microframework” as he is about “micro-PHP” in general.

It used to be, back in the PHP 3, 4, and early 5.x days, that the word “framework” was a dirty word in PHP land. (The word “CMS” was OK though.) Then, right after Ruby on Rails came out, suddenly a “framework” was a good thing. Lots of developers got on board with that, and we did the same with Solar.

So in a way, I think moving back to a library-oriented approach is a natural tendency for the PHP world. Frameworks still have value, especially for early-to-mid-career developers, or for teams where you need a standardized development process but don’t have a strong senior-level architect on staff. But a lot of senior developers want to be able to pick and choose between libraries, and they want to be sure they understand what the library is doing (and why, and how). And they want to be able to replace the pieces they end up not liking. That’s a lot easier when you have independent libraries than when you have a monolithic framework.

What can one do better with Aura than they can with Zend or Symfony?

A lot of PHP developers are stuck with codebases they didn’t build themselves, or that they need to improve carefully over a long period of time because the business is dependent on it for revenue. For those PHP developers, switching the project to a framework isn’t an option. The Aura project, being composed of independent packages, lets these developers use just the individual independent parts they need for their existing projects, and slowly improve the quality of their codebase. It’s easier to refactor your project one part at a time using Aura than it is to start all over again with a monolithic framework.

If you’re lucky enough to be able to start a brand-new project, Aura also provides a framework system that glues all the other packages into a cohesive whole. If you’re the kind of developer who wants to use a full-stack framework, but you also want to be able to pull out parts of the framework and replace them with your own implementations, Aura (because it was built with a “libraries first” approach) makes that a lot easier for you than Zend or Symfony does. There are no cross-package dependencies like there are with Zend and Symfony, and we use separated interfaces for things that should have replaceable implementations. (The framework package is still beta, but it appears to be working just fine.)

Thank you very much and keep up the good work!

Thank you for your interest and attention!

Aura: First 1.0.0 Stable Library Releases!

The Aura Project for PHP 5.4 is happy to announce its first release of 1.0.0 stable library packages. The packages include:

In addition, we have initial beta releases of three new libarary packages:

Each library package is independent of all the others, so you can pick and choose which ones you want to use. With few exceptions, each class in each package has 100% PHPUnit test coverage. Each package has a full README for getting started. They all have a composer.json file and are registered at Packagist.

While the above packages are completely independent of each other, Aura also has a Framework package that binds them all together. Releasing the next version of that Framework and system skeleton is the next thing for us to do.

You can learn more about the Aura Project at http://auraphp.github.com, and please be sure to join our mailing list at http://groups.google.com/group/auraphp.

Update: I originally reported that Aura.Uri was a beta package; it is in fact stable.

Regarding “Simply” and “Just” In Documentation

When reading documentation, manuals, narratives, instructions, tutorials, and the like, I frequently see phrases like: “If you want to do X, simply …” and then several steps of instructions. Or, “In order to do Z, just do this, that and the other.”

You don’t need the words “simply” and “just” in your writing. If the instructions are simple, they will *look* simple. If they are not, using “simply” and “just” indicates that the writer *wants you to think* they are easy. They are marketing words, reflecting the way the author wants you to think about the work, and are not descriptive of the work itself. Eliminate these words (and their various variations) from your writing, and your meaning will be much clearer.