PHP ssh2.sftp opendir/readdir fix

This bug https://bugs.php.net/bug.php?id=73597 related to the PECL ssh2 extension bit us yesterday, so this post is a public service announcement that will (hopefully) save you from writing your own workaround like I almost did.

Problem: PHP 5.6.28 (and apparently 7.0.13) introduced a security fix to URL parsing, that caused the string interpolation of the $sftp resource handle to no-longer be recognized as a valid URL. In turn, that causes opendir(), readdir(), etc. to fail when you use an $sftp resource in the path string, after an upgrade to one of those PHP versions.

Solution: Instead of using "ssh2.sftp://$sftp" as a stream path, convert $sftp to an integer like so: "ssh2.sftp://" . intval($sftp) . "/". Then it will work just fine.

Thanks to the people who fixed the URL parsing security flaw, thanks to the people who wrote the PECL ssh2 extension, and thanks to the people who provided the fix.

Conserving On The Wrong Resource

(This is a blog post I’ve had in a “write-me!” folder for years; I am now publishing it in its unfinished form to get it off my mind.)

Programmers are acutely aware of their limited resources: CPU, memory, storage, requests-per-second, screen space, line length, and so on. If programmers are aware of a resource constraint, they conserve on it; sometimes they call this “optimizing.”

Are you aware of a loop? Then you try to “optimize” it, or conserve on it, e.g. by not using a count() in the loop declaration.

Are you aware of the number of lines? You might try to conserve on the number of lines, perhaps by removing blank lines (especially in docblocks).

Are you aware of aware of the size of an array, or the number of objects in the system, and so on? You try to conserve on those, to reduce their number.

But this is sometimes a trap, because you conserve or optimize on a low value
resource you are aware of, at the cost of not conserving on a high-value resource you are not aware of. For example, optimizing a for loop by not using count() in its declaration will have little impact if the loop is executing queries against an SQL database.

Lesson: conserve on high-value resources that you may not be aware of, not low-value ones that are obvious but of little consequence.

The PHP 7 “Request” Extension

tl;dr: The new request extension provides server-side request and response objects for PHP 7. Use it as a replacement object for request superglobals and response functions. An equivalent PHP 5 version is available in userland as pmjones/request.


You’re tired of dealing with the $_GET, $_POST, etc. superglobals in your PHP 7 application. You wish $_FILES was easer to deal with. You’d prefer to wrap them all in an object to pass around to your class methods, so they’d be easier to test. And as long as they’re all in an object, it might be nice to have parsed representations of the various incoming headers. Preferably, you’d like for it to be read-only, so that the various libraries you use can’t modify superglobal state out from under you.

Likewise, seeing the “Cannot modify header information – headers already sent” warning is getting on your nerves. You’d like to get away from using header(), setcookie(), and the rest. You know they’re hard to inspect, and they’re hard to test. What would be great is to have all that response work wrapped in another object that you can pass around, and inspect or modify it before sending the response back to the client.

You could maybe adopt a framework, but why do that for your custom project? Just a pair of server-side request and response objects would make your life so much easer. Why can’t there be set of internal PHP classes for that?

Well, now there is. You can install the request extension from John Boehr and myself to get ServerRequest and ServerReponse objects as if PHP itself provided them.

ServerRequest

After you install the extension, you can issue $request = new ServerRequest(), and then:

Instead of ...                          ... use:
--------------------------------------- ---------------------------------------
$_COOKIE                                $request->cookie
$_ENV                                   $request->env
$_GET                                   $request->get
$_FILES                                 $request->files
$_POST                                  $request->post
$_SERVER                                $request->server
$_SERVER['REQUEST_METHOD']              $request->method
$_SERVER['HTTP_HEADER_NAME']            $request->headers['header-name']
file_get_contents('php://input')        $request->content
$_SERVER['HTTP_CONTENT_LENGTH']         $request->contentLength
$_SERVER['HTTP_CONTENT_MD5']            $request->contentMd5
$_SERVER['PHP_AUTH_PW']                 $request->authPw
$_SERVER['PHP_AUTH_TYPE']               $request->authType
$_SERVER['PHP_AUTH_USER']               $request->authUser

Likewise:

Instead of parsing ...                  ... use:
--------------------------------------- ---------------------------------------
isset($_SERVER['key'])                  $request->server['key'] ?? 'default'
  ? $_SERVER['key']                     (good for all superglobals)
  : 'default';
$_FILES to look more like $_POST        $request->uploads
$_SERVER['HTTP_CONTENT_TYPE']           $request->contentType and
                                        $request->contentCharset
$_SERVER['HTTP_ACCEPT']                 $request->accept
$_SERVER['HTTP_ACCEPT_CHARSET']         $request->acceptCharset
$_SERVER['HTTP_ACCEPT_ENCODING']        $request->acceptEncoding
$_SERVER['HTTP_ACCEPT_LANGUAGE']        $request->acceptLanguage
$_SERVER['HTTP_X_HTTP_METHOD_OVERRIDE'] $request->method
$_SERVER['PHP_AUTH_DIGEST']             $request->authDigest
$_SERVER['HTTP_X_REQUESTED_WITH']       $request->xhr
  == 'XmlHttpRequest'

Those properties are all read-only, so there’s no chance of them being changed without you knowing. (There are some true immutables for application-related values as well; see the documentation.)

ServerResponse

For responses, you can issue $response = new ServerResponse(), and then:

Instead of ...                          ... use:
--------------------------------------- ---------------------------------------
header('Foo: bar', true);               $response->setHeader('Foo', 'bar');
header('Foo: bar', false);              $response->addHeader('Foo', 'bar');
setcookie('foo', 'bar');                $response->setCookie('foo', 'bar');
setrawcookie('foo', 'bar');             $response->setRawCookie('foo', 'bar');
echo $content;                          $response->setContent($content);

You can inspect the $response object, and then call $response->send() to
send it to the client.

Working with JSON?

// instead of ...
header('Content-Type: application/json')
echo json_encode($value);

// .. use:
$response->setContentJson($value);

Sending a $file for download?

// instead of ...
header('Content-Type: application/octet-stream');
header('Content-Transfer-Encoding: binary');
header(
    'Content-Disposition: attachment;filename="'
    . rawurlencode(basename($file)) . '"'
);
$fh = fopen($file, 'rb+');
fpasthru($fh);

// use:
$fh = fopen($file, 'rb+');
$response->setContentDownload($fh, basename($file));

Building a complex header? Pass an array instead of a string:

$response->setHeader('Cache-Control', [
    'public',
    'max-age' => '123',
    's-maxage' => '456',
    'no-cache',
]); // Cache-Control: public, max-age=123, s-maxage=456, no-cache

$response->setHeader('X-Whatever', [
    'foo',
    'bar' => [
        'baz' => 'dib',
        'zim',
        'gir' => 'irk',
    ],
    'qux' => 'quux',
]); // X-Whatever: foo, bar;baz=dib;zim;gir=irk, qux=quux

Find Out More

You can read the rest of the documentation at https://gitlab.com/pmjones/ext-request to discover more convenient functionality. And if you’re stuck on PHP 5.x for now, the extension has a userland version installable via Composer as pmjones/request.

Try out the request extension today, because a pair of server-side request and response objects will make your life a lot easier.

Definition of “Done” ┬áin a web project

– Source code meets our coding standards.

– High enough level of unit test coverage for routes, action methods and controllers.

– High enough level of unit test coverage for business logic and repositories.

– High enough level of automated UI and integration test coverage.

– Code has been peer reviewed.

– Code must be completely checked in to the source control system and the build and all the automated tests should be green.

– UI looks nice and works on different resolutions on major browsers and browser editions.

– UI fulfills the accessibility requirements.

– UI works with and without javascript enabled.

– End user help/documentation/tooltips are done.

– Any auditing/tracing code is added and the output is useful and readable.

– Security permission checks have been implemented and validated via automated tests.

– Automated database migration scripts are provided and testedSample data needed to test the feature is scripted, if required.

– Users have tested the feature and are happy with it.

I’ll nitpick and note that “MVC” is not an application architecture, but whatever. Source: Definition of Done in an MVC project