Atlas ORM 1.2.0 Released

The 1.2.0 release adds the ability to define WHERE conditions on relationships. (The 1.1.0 release added functionality to ignore foreign key string case when wiring up objects in memory, and 1.0.0 was released pretty quietly a couple of weeks ago.)

Try it out today, because you like keeping your persistence layer separate from your domain layer.

Now, read on for some history, if you care about that kind of thing.


Many years ago, we on the Solar project developed Solar_Sql_Model, an Active Record type of ORM. Overall I liked it well enough, though (as with anything) it had its strengths and weaknesses.

Since then, after extracting the Solar components to Aura libraries, I’ve mostly lived without ORMs. The majority of my legacy consulting work has not made use of them; where a legacy project did have an ORM of some sort, it was a custom in-house piece of work.

However, about three years ago, I hired on with a startup to build out their backend from scratch. At the time, I wanted to do “real” Domain-Driven Design, with entities and aggregates and value objects and everything else. That meant keeping the domain system separate from the persistence system, and that in turn meant Active Record was not an option. Doctrine, a domain model data mapper, was the next logical choice, but on review it was not to my liking. (The annotations, among other things about Doctrine, just rubbed me the wrong way.)

So instead of an Active Record implementation, or Doctrine, I figured that it would be enough to use a Table Data Gateway on top of Aura.Sql and Aura.SqlQuery to retrieve rows, then map the rows over to domain objects. This was fine, for as far as it went, but the problem was “relationships.” Oh dear Codd, the relationships.

Selecting one or many objects from a single table was no big deal, but doing multiple selections from multiple tables and building up the selection statements for those relationships was a tedious, tiresome, and time-consuming burden. (Wiring them up in memory once they’d been selected was not too bad, given Aura.Marshal, but it was still more effort than I’d’ve rather been expending.)

So it all “worked,” but it was just not satisfying at all. The DDD portions, for their part, worked out great, as did the separation of the domain layer from the persistence layer, so I was pretty sure I was on something like the right track.

Then I read this article from Mehdi Khalili. It’s fantastic and you should read the whole thing. In summary, Khalili points out that it is perfectly reasonable to use a persistence model in addition to a domain model. That is, you do not necessarily have to map directly from the database to domain objects; you can map from the database to persistence objects for the data, and then later compose or map the persistence model into the domain model for behaviors.

This was a revelation to me, something that I’d never considered, or even heard of before. It alleviates wide swaths of DDD-related burdens.

As a result of my relationship-related burdens at the startup, and after reading the Khalili article, I put together Atlas, a mapper for your persistence model. Like everything else I’ve been doing for the past several years, Atlas is built in discrete layers:

  • PDO at the very bottom;
  • Aura.Sql around that, to provide convenience methods for binding and fetching;
  • Aura.SqlQuery in parallel, to support query building;
  • all of those composed into a table data gateway system to emit Row objects from tables;
  • and finally a mapper system on top of that to emit Record objects composed of table Rows and their relationships

As such, each Record object is composed of a Row (from the “main” table) and its Related objects (themselves Records, each of which is composed of a Row and Relateds, and so on).

Atlas uses the term “Record” to indicate that the object is not a domain entity or aggregate. You can use Records directly for straightforward CRUD/BREAD operations, or you can map them over to your domain objects.

Fethcing deep relationship structures is no big deal; see this article from Andrew Shell using 25 tables in different complex relationships. (Andrew’s project also shows how to keep the persistence and domain layers separate, and incorporates a debug bar implementation for Atlas.)

So, if you want a data mapper implementation that models your persistence layer, and the relationships therein, Atlas is just the thing for you. Try it out today!

Symfony 4: Directory Structure, and Common Practices

Fabien has published his plans for the new Symfony 4 directory structure. Not that it matters much to anyone in Symfony-land, where I have no status that I’m aware of, but I am happy to see the changes described.

Having said that, and recognizing that Fabien obviously has final say over his own projects …

The new etc/ directory is the equivalent of the current app/config/ directory.

Web files under web/

… it might be nice if Symfony 4 adopted more existing common practices, used by roughly 70% of Packagist packages. That is, to use config/ for the top-level config directory, and public/ for the top-level document-root directory.

More specifically:

  • config/ is used ~10x more than etc/ (ref)

  • public/ is used ~2x more than web/ (ref)

As a side note, that research resulted in the pds/skeleton publication.

(This blog post originated as a comment on Reddit.)

Command-Line Output: Consider Logging Over Streams

When writing command-line applications for PHP, consider using a logger for screen output. This is something I’ve done with great success in several projects. Producer, for example, uses a very light standard I/O logger class that writes output to STDOUT and STDERR resource/stream handles. Every command that generates output uses that standard I/O logger (cf. the issues command).

This has a couple of advantages:

  • Your command-line code gets stdout/stderr output separation practically for free, using a common PSR-3 interface.

  • If you incorporate that command-line tool into another class, you can easily inject a different PSR-3 logger so that output is captured elsewhere, instead of writing to stdout/stderr. Among other things, that makes it relatively easy to test the output of your command-line code without having to use output buffering.

I think this approach works best for non-interactive commands. If you have to read keyboard input from the user as part of the command, using a logger for output might not make a lot of sense. But if all of the command inputs are handled as options or flags, using a logger for output can be great.

Regarding A Recent Event

KING HENRY V: We judge no less. Uncle of Exeter,
    Enlarge the man committed yesterday,
    That rail'd against our person: we consider
    it was excess of wine that set him on;
    And on his more advice we pardon him.

SCROOP: That's mercy, but too much security:
    Let him be punish'd, sovereign, lest example
    Breed, by his sufferance, more of such a kind.

KING HENRY V: O, let us yet be merciful.

CAMBRIDGE: So may your highness, and yet punish too.

GREY: Sir,
    You show great mercy, if you give him life,
    After the taste of much correction.

KING HENRY V: Alas, your too much love and care of me
    Are heavy orisons 'gainst this poor wretch!
    If little faults, proceeding on distemper,
    Shall not be wink'd at, how shall we stretch our eye
    When capital crimes, chew'd, swallow'd and digested,
    Appear before us? We'll yet enlarge that man,
    Though Cambridge, Scroop and Grey, in their dear care
    And tender preservation of our person,
    Would have him punished. And now to our French causes:
    Who are the late commissioners?

CAMBRIDGE: I one, my lord:
    Your highness bade me ask for it to-day.

SCROOP: So did you me, my liege.

GREY: And I, my royal sovereign.

KING HENRY V: Then, Richard Earl of Cambridge, there is yours;
    There yours, Lord Scroop of Masham; and, sir knight,
    Grey of Northumberland, this same is yours:
    Read them; and know, I know your worthiness.
    My Lord of Westmoreland, and uncle Exeter,
    We will aboard to night. Why, how now, gentlemen!
    What see you in those papers that you lose
    So much complexion? Look ye, how they change!
    Their cheeks are paper. Why, what read you there
    That hath so cowarded and chased your blood
    Out of appearance?

CAMBRIDGE: I do confess my fault;
    And do submit me to your highness' mercy.

GREY    |
        | To which we all appeal.
SCROOP  |

KING HENRY V: The mercy that was quick in us but late,
    By your own counsel is suppress'd and kill'd:
    You must not dare, for shame, to talk of mercy;
    For your own reasons turn into your bosoms,
    As dogs upon their masters, worrying you.

I hope this is the last thing I will have to say on the matter.

WikiMedia, Clean Architecture, and ADR

tl;dr: Action-Domain-Responder is a natural fit for the HTTP user-interface portions of Clean Architecture (or Hexagonal), especially with Domain Driven Design. Just be sure to remember to separate the HTTP response presentation from the action code.


I.

Jeroen de Dauw has a fantastic post on Implementing the Clean Architecture in PHP, with Domain Driven Design elements. You should read the whole thing, and examine the implementation codebase, for a number of useful insights. Though I might quibble over some elements of the implementation, I think it is a good offering, and serves as a solid reference point.

In his article, Jeroen notes they are using Silex for their HTTP user-interface system, and describes the logic of each route action:

Inside this [Silex action] we construct our framework agnostic request model and invoke the Use case with it. Then we hand over the response model to a presenter to create the appropriate HTML or other such format.

That is a very near paraphrase of Action-Domain-Responder:

  • The Action marshals input from the HTTP request
  • The Action invokes a Domain element with that input and gets back a result
  • The Action passes that result to a Responder to build the HTTP response

In Jeroen’s implementation, each Action is a closure defined in the routes.php file. The Action marshals input from the HTTP request using a “request model” (an input object tailored to the domain) and passes it to a “use case.” Each “use case” is an entry point into the Domain, and returns a “response model” (the domain result).

The only place where Jeroen’s implementation deviates from ADR is that the Action code builds the presentation itself, instead of handing off to a Responder. (This may be a result of adhering to the idioms and expectations specific to Silex.)

Because the rest of the implementation is so well done, refactoring to a separated presentation in the form of a Responder is a straightforward exercise. Let’s see what that might look like.

II.

First, as an example, review the code in the check-iban action. The following reorganization of that action code makes the ADR pattern more obvious:

<?php
$app->get(
    'check-iban',
    function( Request $request ) use ( $app, $ffFactory ) {

        // marshal input
        $input = new Iban( $request->query->get( 'iban', '' ) );

        // invoke domain and get back result
        $result = $ffFactory->newCheckIbanUseCase()->checkIban($input);

        // presentation
        return $app->json(
            $ffFactory->newIbanPresenter()->present( $result )
        );
    }
);
?>

Very clear and straightforward. However, the presentation work is embedded in the action with the $app->json(...) call. (My guess is that’s probably a result of working with existing Silex idioms.)

Another good example is the list-comments.html action. Reorganizing the logic to make the ADR pattern more obvious gives us the following:

<?php
$app->get(
    'list-comments.html',
    function( Request $request ) use ( $app, $ffFactory ) {

        // marshal input
        $input = new CommentListingRequest(
            10,
            (int)$request->query->get( 'page', '1' )
        );

        // invoke domain and get back result
        $result = $ffFactory
            ->newListCommentsUseCase()
            ->listComments( $input );

        // presentation
        return new Response(
            $ffFactory->newCommentListHtmlPresenter()->present(
                $result,
                (int)$request->query->get( 'page', '1' )
            )
        );
    }
);
?>

Again, the presentation work is embedded in the action code.

In general, it is better to completely separate the presentation work from the action code. Remember that in an HTTP context, the presentation is not just the body of the HTTP response. Instead, the presentation is the entire HTTP response, including headers and status. (For more on this, see The Template Is Not The View.)

With the above examples, because they are already so well structured, it would be easy to extract the presentation to a Responder class. For example, the list-comments action could have the presentation work completely removed like so:

<?php
// hypothetical class with the extracted logic
class ListCommentsHtmlResponder
{
    public function buildResponse($request, $result, $ffFactory)
    {
        return new Response(
            $ffFactory->newCommentListHtmlPresenter()->present(
                $result,
                (int)$request->query->get( 'page', '1' )
            )
        );
    }
}

// the refactored action code
$app->get(
    'list-comments.html',
    function( Request $request ) use ( $app, $ffFactory ) {

        // marshal input
        $input = new CommentListingRequest(
            10,
            (int)$request->query->get( 'page', '1' )
        );

        // invoke domain and get back result
        $result = $ffFactory->newListCommentsUseCase()->listComments($input);

        // hand result to responder
        return $ffFactory->newListCommentsHtmlResponder()->buildResponse(
            $request,
            $result,
            $ffFactory
        );
    }
);
?>

Now the presentation work of building an HTTP response is cleanly separated from the rest of the action code.

III.

When separating concerns along these lines, you begin to see the similarities in the presentation work, and can start to reduce repetition across the codebase. For example, any Action that delivers a JSON response might use the same base JSON Responder.

Eventually, you may realize that the logic of each action is effectively identical. That is, you always collect input, pass that input through the domain to get back a result, and pass that result to a response builder.

When that realization occurs, you can build a single action handler that coordinates between injected input marshals, domain entry points, and response builders. That’s exactly what the Arbiter ActionHandler does, and Radar uses that in turn to specify the input + domain + responder callables for each route.

At that point, you are out of the business of writing action methods entirely. Then the user-interface code can focus on marshaling inputs going to the domain, and on presenting the results coming out of the domain – which is exactly how things should be.

P.S.

Jeroen’s writeup also reveals that at least some of the elements in his implementation are returning something like Domain Payload objects. Cf. the ValidationResult class, used in the validate-payment-data action among other places.

I’m a big fan of the Domain Payload pattern in ADR, and using a Domain Payload for all returns received by the action code. Doing so simplifies the response-building logic even further; for example, by collecting common “success” and “failure” presentation work across different JSON responders.

Then there’s this bit about containers:

We decided to go with our own top level factory, rather than using the dependency injection mechanism provided by Silex: Pimple. Our factory internally actually uses Pimple, though this is not visible from the outside. With this approach we gain a nicer access to service construction, since we can have a getLogger() method with LoggerInterface return type hint, rather than accessing $app[‘logger’] or some such, which forces us to bind to a string and leaves us without type hint.

This resonates with some other ideas I’ve been toying with, namely that the user-interface container might better be separated from the domain container. They can be wired up separately from each other, making it easier to package the Domain portions independently from the user-interface portions, and enforcing a “physical” boundary between the two.

Overall, congratulations to Jeroen on putting together such a good writeup.

Radar Project Skeleton Now Stable

I am happy to announce that the project skeleton for Radar, an Action-Domain-Responder system for PHP, is now stable and available for use.

One significant difference between this release and the last alpha is that it now uses the standard pds/skeleton names for directories. Existing Radar projects will not need to change directory names, but be aware that starting a new project with the 1.0.0 version will use “public/” instead of “web/”.

Many thanks to everyone who contributed to this release, especially Jake Johns, who put together a post-create-project command to “clean up” the initial installation.

The “pds/skeleton” Standard Is Now Stable!

I am proud to announce that the first PHP Package Development Standards publication, pds/skeleton, has been released as stable at version 1.0.0.

This publication has been a great working effort. Everything from researching first a subset (and then the entirety) of Packagist, to putting together the first drafts, to working with reviewers and refining the publication, has been a wonderful experience. From the first uncommitted work in early Nov 2016, to the stable release a few days ago, the whole process has taken just about 12 weeks of evening and weekend time.

Many thanks to the early reviewers (you know who you are!) for your input, criticism, and suggestions. Thanks also to the issue submitters and commenters, and especially to to everyone who submitted a pull request. These people contributed serious effort and attention to the publication, which helps to show that the publication really is a community-based work.

Roughly 78,000 packages already comply with the pds/skeleton standard, although they may not know it. To formally show that your package has adopted the standard, “require-dev” it via Composer, or display a badge on your README.

Although I have a few ideas in mind, what do you think the next PDS publication should focus on? Let me know if you have a particular area of interest.

How Many PSR-7 Implementations Exist?

More specifically, how many implementations of PSR-7 ServerRequestInterface exist?

Initially, it appears the answer might be as many as three dozen.

But increasingly it looks like the real answer, to a first approximation, is “zero.”

To implement ServerRequestInterface, the methods withAttribute() and withParsedBody() MUST maintain the immutability of the message object. However, none of the existing implementations do that.

To see for yourself, clone my PSR-7 Immutability Testing repository and run the tests, which cover the three most popular PSR-7 implementations.

Immutability is very difficult to implement – at least, not without some serious restrictions, none of which are specified by PSR-7 ServerRequestInterface.The more I work with it, the more I begin to think that it is not fully implementable-as-specified in the first place.

UPDATE: Apparently I was on to something when I suggested that PSR-7 cannot be implemented as written. Sara Golemon (who has forgotten more about PHP internals than I will ever know) points out, “It is not technically implementable. And thus you’re technically correct, the best kind of correct.” (Archived for posterity at https://archive.is/BnkGb .)

UPDATE 2: Reddit commenters point out that the ServerRequestInterface::getAttributes() method states “Attributes will be application and request specific, and CAN be mutable.” This seems at odds with the withAttribute() method, which requires that the object remain immutable. At the best, this seems like an oversight in the spec, and at the worst it is an internal contradiction.

Beta2 of pds/skeleton now available!

I am excited to announce that pds/skeleton 1.0.0beta2 has been released. (The pds/skeleton publication describes a standard PHP package skeleton, as backed by research into the PHP package ecosystem.)

Among other things, this release incorporates some command-line tooling to validate, and generate, your PHP package skeleton.

Barring unforeseen events, I expect the next release to be stable.

Thanks to everyone who made this release possible, both direct contributors, issue reporters, and everyone who commented on the research!

PSR-7 vs. the Server(Request|Response) RFC

tl;dr: PSR-7 aims to model HTTP messages, whereas the RFC aims to make some non-OO PHP functionality more OO-ish.


I had thought the distinction between the purpose of PSR-7, and the purpose of the server-side request/response object RFC, was obvious from their descriptions. But that is apparently not the case.

I would prefer to discuss the RFC entirely on its own merits, and not dwell on the various strengths and weaknesses of PSR-7. However, to reduce confusion on this topic, I am happy to take some time to expound on the differences.

To understand those differences more clearly, we need to start with a history of PSR-7.

Purpose of PSR-7

PSR-7 was born to answer the question, “How can we model HTTP messages in PHP for sending a request, and getting back a response?” That is, how can we standardize the model of an HTTP request message for sending, and the model of the returned the HTTP response, when using PHP as an HTTP client?

The entrance vote passed in Jan 2014 after about a year of pre-work, with Michael “Guzzle” Dowling as lead: https://groups.google.com/d/topic/php-fig/H1Lr7FYxj94/discussion. You can see the original draft at https://github.com/php-fig/fig-standards/pull/244/files.

What you’ll find in the draft is one pair of request/response interfaces, descended from a message interface, with stream as message body, and no URI specification. These were designed primarily as client interfaces; all the referenced projects in the draft were client-side. (As a side note, they were mutable. Dowling said, “Having mutable and immutable messages would add a significant amount of complexity to a HTTP message PSR and would not reflect what is currently being used by a majority of PHP projects.”)

After 8 months, Dowling stepped down in August 2014, citing a lack of time and motivation. He also said: “I don’t think there’s one way to represent HTTP messages, clients, or servers in PHP.” https://groups.google.com/forum/#!topic/php-fig/XwFcqSmqzGk

Shortly thereafter, in September 2014, with encouragement from many (including myself), MWOP of Zend Framework takes over PSR-7. We learn that he has “Sencha Connect” and middleware on the brain:

The reason I wanted to port Connect is this: an application consists of middleware. Each middleware is a callback that accepts a request, response, and a callback called “next” (which is optional, actually):

function (request, response, next)

I know from Michael Dowling that the original intent for PSR-7 was to define HTTP messages that could then be used in HTTP clients. I am here to argue that they are even more important when considering server-side applications.

At this point, we see that PSR-7 has been expanded to answer a second question: “How can we model HTTP messages for receiving a request, and sending back a response?” This is in addition to the original goal, but idea is the same: building a standard model of HTTP messages.

(For full disclosure, note that I became a sponsor on PSR-7 in December 2014, along with Beau Simensen as the coordinator.)

It is during MWOP’s tenure, before the successful acceptance vote in May 2015, that we see the PSR-7 interfaces expand in number, and become “immutable” (with one intentional exception, and other unintentional exceptions).

So we can see that the purpose of PSR-7 is to model 2 sets of HTTP messages using 7 interfaces: one set for when PHP sends a request and receives a response, and an addition set for when PHP receives a request and sends a response.

The Purpose of the Server(Request|Response) RFC

The proposed RFC starts out by asking a different question. It is not concerned with modeling HTTP messages, whether when sending or receiving them. Instead, it asks: “How can we take the request-related superglobals in PHP, and the various response-related global functions in PHP, and encapsulate them in objects, to make them at least a little more object-oriented?” Becuase the RFC begins with a different question, it leads to a different answer.

You end up with a ServerRequest object that exposes almost only properties, mimicking PHP’s superglobals. The properties are read-only, since they represent user input that should be copied out, not changed-in-place. As a convenience, a lot of common $_SERVER['HTTP_*'] values are parsed into more usable representations. Conceding the needs of application development, there are properties and methods for truly immutable values relating to application-specific parameters, parsed content input, and so on.

You also end up wih a ServerResponse object that exposes only methods, mimicking some of PHP’s global functions. Instead of emitting headers and cookies on each call to the related methods, it buffers and retains the header and cookie values until you decide to send them. As a collection point for those values, and for content, you can inspect the state of the object prior to sending, and modify it as needed. It has some convenience methods, not least of which includes sending content as a download, or as JSON, with the appropriate headers.

Conclusion

I hope this helps to clear up any confusion as to the purpose of the RFC, vs. the purpose of PSR-7. They start with different questions, and have different goals. I think it would be better to see them as orthogonal to each other at worst, and complementary at best.