The “pds/skeleton” Standard Is Now Stable!

I am proud to announce that the first PHP Package Development Standards publication, pds/skeleton, has been released as stable at version 1.0.0.

This publication has been a great working effort. Everything from researching first a subset (and then the entirety) of Packagist, to putting together the first drafts, to working with reviewers and refining the publication, has been a wonderful experience. From the first uncommitted work in early Nov 2016, to the stable release a few days ago, the whole process has taken just about 12 weeks of evening and weekend time.

Many thanks to the early reviewers (you know who you are!) for your input, criticism, and suggestions. Thanks also to the issue submitters and commenters, and especially to to everyone who submitted a pull request. These people contributed serious effort and attention to the publication, which helps to show that the publication really is a community-based work.

Roughly 78,000 packages already comply with the pds/skeleton standard, although they may not know it. To formally show that your package has adopted the standard, “require-dev” it via Composer, or display a badge on your README.

Although I have a few ideas in mind, what do you think the next PDS publication should focus on? Let me know if you have a particular area of interest.

How Many PSR-7 Implementations Exist?

More specifically, how many implementations of PSR-7 ServerRequestInterface exist?

Initially, it appears the answer might be as many as three dozen.

But increasingly it looks like the real answer, to a first approximation, is “zero.”

To implement ServerRequestInterface, the methods withAttribute() and withParsedBody() MUST maintain the immutability of the message object. However, none of the existing implementations do that.

To see for yourself, clone my PSR-7 Immutability Testing repository and run the tests, which cover the three most popular PSR-7 implementations.

Immutability is very difficult to implement – at least, not without some serious restrictions, none of which are specified by PSR-7 ServerRequestInterface.The more I work with it, the more I begin to think that it is not fully implementable-as-specified in the first place.

UPDATE: Apparently I was on to something when I suggested that PSR-7 cannot be implemented as written. Sara Golemon (who has forgotten more about PHP internals than I will ever know) points out, “It is not technically implementable. And thus you’re technically correct, the best kind of correct.” (Archived for posterity at https://archive.is/BnkGb .)

UPDATE 2: Reddit commenters point out that the ServerRequestInterface::getAttributes() method states “Attributes will be application and request specific, and CAN be mutable.” This seems at odds with the withAttribute() method, which requires that the object remain immutable. At the best, this seems like an oversight in the spec, and at the worst it is an internal contradiction.

Beta2 of pds/skeleton now available!

I am excited to announce that pds/skeleton 1.0.0beta2 has been released. (The pds/skeleton publication describes a standard PHP package skeleton, as backed by research into the PHP package ecosystem.)

Among other things, this release incorporates some command-line tooling to validate, and generate, your PHP package skeleton.

Barring unforeseen events, I expect the next release to be stable.

Thanks to everyone who made this release possible, both direct contributors, issue reporters, and everyone who commented on the research!

PSR-7 vs. the Server(Request|Response) RFC

tl;dr: PSR-7 aims to model HTTP messages, whereas the RFC aims to make some non-OO PHP functionality more OO-ish.


I had thought the distinction between the purpose of PSR-7, and the purpose of the server-side request/response object RFC, was obvious from their descriptions. But that is apparently not the case.

I would prefer to discuss the RFC entirely on its own merits, and not dwell on the various strengths and weaknesses of PSR-7. However, to reduce confusion on this topic, I am happy to take some time to expound on the differences.

To understand those differences more clearly, we need to start with a history of PSR-7.

Purpose of PSR-7

PSR-7 was born to answer the question, “How can we model HTTP messages in PHP for sending a request, and getting back a response?” That is, how can we standardize the model of an HTTP request message for sending, and the model of the returned the HTTP response, when using PHP as an HTTP client?

The entrance vote passed in Jan 2014 after about a year of pre-work, with Michael “Guzzle” Dowling as lead: https://groups.google.com/d/topic/php-fig/H1Lr7FYxj94/discussion. You can see the original draft at https://github.com/php-fig/fig-standards/pull/244/files.

What you’ll find in the draft is one pair of request/response interfaces, descended from a message interface, with stream as message body, and no URI specification. These were designed primarily as client interfaces; all the referenced projects in the draft were client-side. (As a side note, they were mutable. Dowling said, “Having mutable and immutable messages would add a significant amount of complexity to a HTTP message PSR and would not reflect what is currently being used by a majority of PHP projects.”)

After 8 months, Dowling stepped down in August 2014, citing a lack of time and motivation. He also said: “I don’t think there’s one way to represent HTTP messages, clients, or servers in PHP.” https://groups.google.com/forum/#!topic/php-fig/XwFcqSmqzGk

Shortly thereafter, in September 2014, with encouragement from many (including myself), MWOP of Zend Framework takes over PSR-7. We learn that he has “Sencha Connect” and middleware on the brain:

The reason I wanted to port Connect is this: an application consists of middleware. Each middleware is a callback that accepts a request, response, and a callback called “next” (which is optional, actually):

function (request, response, next)

I know from Michael Dowling that the original intent for PSR-7 was to define HTTP messages that could then be used in HTTP clients. I am here to argue that they are even more important when considering server-side applications.

At this point, we see that PSR-7 has been expanded to answer a second question: “How can we model HTTP messages for receiving a request, and sending back a response?” This is in addition to the original goal, but idea is the same: building a standard model of HTTP messages.

(For full disclosure, note that I became a sponsor on PSR-7 in December 2014, along with Beau Simensen as the coordinator.)

It is during MWOP’s tenure, before the successful acceptance vote in May 2015, that we see the PSR-7 interfaces expand in number, and become “immutable” (with one intentional exception, and other unintentional exceptions).

So we can see that the purpose of PSR-7 is to model 2 sets of HTTP messages using 7 interfaces: one set for when PHP sends a request and receives a response, and an addition set for when PHP receives a request and sends a response.

The Purpose of the Server(Request|Response) RFC

The proposed RFC starts out by asking a different question. It is not concerned with modeling HTTP messages, whether when sending or receiving them. Instead, it asks: “How can we take the request-related superglobals in PHP, and the various response-related global functions in PHP, and encapsulate them in objects, to make them at least a little more object-oriented?” Becuase the RFC begins with a different question, it leads to a different answer.

You end up with a ServerRequest object that exposes almost only properties, mimicking PHP’s superglobals. The properties are read-only, since they represent user input that should be copied out, not changed-in-place. As a convenience, a lot of common $_SERVER['HTTP_*'] values are parsed into more usable representations. Conceding the needs of application development, there are properties and methods for truly immutable values relating to application-specific parameters, parsed content input, and so on.

You also end up wih a ServerResponse object that exposes only methods, mimicking some of PHP’s global functions. Instead of emitting headers and cookies on each call to the related methods, it buffers and retains the header and cookie values until you decide to send them. As a collection point for those values, and for content, you can inspect the state of the object prior to sending, and modify it as needed. It has some convenience methods, not least of which includes sending content as a download, or as JSON, with the appropriate headers.

Conclusion

I hope this helps to clear up any confusion as to the purpose of the RFC, vs. the purpose of PSR-7. They start with different questions, and have different goals. I think it would be better to see them as orthogonal to each other at worst, and complementary at best.

RFC: ServerRequest and ServerResponse

The RFC is at https://wiki.php.net/rfc/request_response.

The message opening discussion on Internals is at http://news.php.net/php.internals/97461.

The extension itself is available at https://pecl.php.net/package/request, with documentation at https://gitlab.com/pmjones/ext-request.

(Many thanks to John Boehr for doing the actual heavy lifting of writing the C code.)


Nearly every PHP framework and library-collection since 2000 has had classes to encapsulate the “request” and “response” elements of a PHP application. A handful of examples include:

There are many others. They all do essentially the same things:

  • Copy the $_GET, $_POST, $_SERVER, etc. superglobals into a “request” object. Some make them available through a method that standardizes the logic to get a default value when a key is not present, a la return (isset($_GET[$key]) ? $_GET[$key] : $defaultValue).

  • Add convenience methods to the “request” object so that you can determine the HTTP method, the values of various headers, and so on.

  • The “response” object is a place to hold headers, cookies, status, and content, so they can all be inspected and modified before sending, and to make testing easier. (This is because the header(), setcookie(), etc. functions in PHP are not especially amenable to inspection, modification, and testing – at least, not without being wrapped somehow.)

  • The “response” object often has some convenience methods to send JSON content, send files for download, and so on.

Why do framework and library-collection authors write these request and response objects? Because PHP, even though it is a web-centric programming language, and even though it provides all sorts of classes for all sorts of functionality, it has never had classes for server-side requests and responses. This RFC helps to improve this situation in PHP 7 and later.

PDS “skeleton” Standard Now In Beta

I am proud to announce the immediate availability of pds/skeleton 1.0.0-beta1 for inclusion in your PHP package.

The pds/skeleton standard defines specific names for specific kinds of root-level directories in PHP packages. If you use all the directory and file names in the standard, your package will look something like this:

bin/            # executable files
config/         # configuration files
docs/           # documentation
public/         # web server files
resources/      # other resource files
src/            # PHP source code
tests/          # tests
CHANGELOG       # change notes
CONTRIBUTING    # contribution guidelines
LICENSE         # licensing and copyright
README          # read-me-first file

Believe it or not, roughly 78 thousand packages on Packagist, from tens of thousands of vendors, already appear compliant with pds/skeleton.

To formally indicate that your package attempts to comply to pds/skeleton, add it to your package manifest as a development requirement. For example, if you use Composer, you can do something like the following:

"require-dev": {
    "pds/skeleton": "~1.0"
}

(Alternatively, mention in your README that the package attempts to comply with pds/skeleton.)

This standard will remain in beta for at least two weeks, to give ample time for additional public review. If you find errors or omissions, please be sure to open an issue on it.


As a side note, Producer will end up supporting pds/skeleton and validating against it by default. This will mean a BC break for Producer, so look for a 3.x version soon!

ADR with HTTP Caching

Great conversation with “MatTheCat” on Github today, regarding HTTP caching headers and the ADR pattern.

A quick summary:

  • You want to return a 304 “Not Modified” response if the proper preconditions are met.
  • You can only tell if the resource is unmodified if you touch some form of storage.
  • In ADR, one rule of thumb is “if it touches storage, it probably goes in the Domain.”
  • So you need to do all the precondition-checking in the Domain, not in the Action.
  • The Domain can return a payload indicating “not modified” as appropriate, and the Responder can read that payload status to send back a 304.

You can read the whole thing here. Thanks for the great question Mathieu!

Efficient use of mysqli_result::$num_rows

I frequently see this pattern in legacy applications using mysqli:

$result = $mysqli->query("
    SELECT *
    FROM table_name
    WHERE conditions = 'whatever'
");
if ($result && $result->num_rows > 0) {
    return;
}

The developer’s intent here is to see if there are any rows at all in the database that match a certain condition. He does so by issuing a query, then asking the result object how many rows it has. The developer doesn’t actually want any data from the result, and doesn’t care about the actual row-count itself; this is just a check to see if at least one row exists in the database.

This is a poor conservation of resources. The database does the work needed to select all the columns for all the rows matching the conditions, allocates memory for them, and returns them. But the developer discards all that immediately.

To accomplish the same end, it is less resource-intensive and just as effective to query for a single column and limit the results to a single row:

$result = $mysqli->query("
    SELECT col_name
    FROM table_name
    WHERE conditions = 'whatever'
    LIMIT 1
");
if ($result && $result->num_rows > 0) {
    return;
}

Now the database only does the work needed for a single column and a single row.

(As a side note, I find it interesting that I have not seen this pattern at all in projects using PDO. I’m not sure why this would be. Perhaps there is some originating example code for mysqli somewhere that has gained a life of its own through copying and reuse.)

UPDATE: Perhaps a better way to conserve resources, courtesy of Reddit user marcjschmidt, is to use a COUNT() in the query, then fetch the count of rows, something more like this …

$result = $mysqli->query("
    SELECT COUNT(*)
    FROM table_name
    WHERE conditions = 'whatever'
");
if ($result && $result->fetch_array()[0] > 0) {
    return;
}

… thereby avoiding the use of mysqli_result::$num_rows completely.

UPDATE 2: Lots of commentary in the above Reddit thread. To summarize this blog post: selecting all columns of all rows, then examining $num_rows, and then discarding the result set, is a terrible way of determining if there are any matching rows at all in the database. It is trivially easy to something much better, whether by using a LIMIT 1 and $num_rows, or some form of COUNT(), or perhaps some other approach.

PHP-PDS: Interview on Voices of the ElePHPant

My good friend Cal Evans interviewed me about the PHP-PDS initiative and its first offering, pds/skeleton last week; here it is for you to enjoy.

(Via https://voicesoftheelephpant.com/2016/12/20/interview-paul-m-jones/.)

While we’re here, you might want to check out his virtual user group Nomad PHP, and the series of one-day virtual training conferences at DayCamp4Developers, because you’ll probably learn something useful there.

PECL Request Extension: Beta 1 Released!

I am happy to announce that the PECL extension for server-side request and response objects has reached beta status! (Documentation here.)

This release adds four new read-only properties to the ServerRequest object:

  • $forwarded is an array representation of $_SERVER['HTTP_FORWARDED'] (cf. the Forwarded HTTP Extension),

  • $forwardedFor is an array computed from treating $_SERVER['HTTP_X_FORWARDED_FOR'] as comma-separated values.

  • $forwardedHost is the $_SERVER['HTTP_X_FORWARDED_HOST'] value, and

  • $forwardedProto is $_SERVER['HTTP_X_FORWARDED_PROTO'] value.

If you use trusted proxies a lot in your work, you can use those values to compute the “real” client IP address in your application.

This completes the intended initial functionality of the extension. You should install it and try it out, because it might be make your work easier.