Yawp 1.0.5 Released

Yawp is a single PEAR-compliant class that encapsulates a number of other PEAR classes, and ties them all together with a simple configuration file. This release provides a minor functionality improvement.

Previously, when a hook script was called, it would be included directly in the calling code (e.g., a ‘start’ hook would be include()-ed in the middle of the Yawp::start() method; same for login, logout, and authErr hooks). This means it would be possible for the included script to use (and possibly overwrite) variables in the calling Yawp method; while not a security violation, that kind of thing could lead to unexpected behavior, and requires a much closer knowledge of the Yawp internals than should be necessary.

To solve this problem, I have added a method called run() — all it does is include a file. It doesn’t even need a parameter (using one would create a new variable in the scope of the function, and we want *everything* isolated for the included script).

function run() { include func_get_arg(0); }

This has the benefit of executing a script in its own scope, so that the hook script cannot accidentally overwrite variables in the calling Yawp method. All hooks now use the Yawp::run() method instead of include().

Restrictive (“Secure”) Compiler for Savant

This page is to track issues related to the new “secure” compiler for Savant. A number of people (notably and most recently RevJim) have opined that Savant needs a decent secure compiler so that untrusted users can be allowed to edit templates. I agree.

Note: The compiler is not a required element. The normal use of Savant remains; that is, PHP itself is the template markup. The only time you really need to compile a template is when you have anonymous or otherwise possibly mailicious users, and this compiler exists to support that relatively rare case.

So the idea now is to replace the existing “basic” compiler with a more-secure version; you can view the source code here. Note that it depends on Josh Eichorn’s excellent PHPCodeAnalyzer script, which itself depends on the tokenizer function in PHP 4.3.0 and later.

I don’t know if I like calling the new compiler “secure” or not, but it sure is restrictive. Here are the built-in restrictions and features:

  • The markup language is PHP in most cases, just surrounded in non-PHP tags
  • Prefix and suffix tags default to “{” and “}”, but are user-definable.
  • No <?php … ?> or <? … ?> tags allowed
  • Simple variable echoing via {$var}
  • Comments via {* … *}
  • Control structures are regular PHP ( {if (…):}, {else}, {foreach (…):}, etc)
  • No support for switch/case, but break and continue are allowed
  • Plugins supported via {[‘pluginName’, ‘arg1’, $arg2, … ]}
  • Certain language constructs are disallowed: eval, global, include[_once], require[_once], parent, self
  • The only way to include other templates is via the {tpl} tag
  • If the Savant $_restrict flag is on, template requests are restricted to specific allowed paths
  • Access to superglobals ($GLOBALS, $_GET, etc) is disallowed
  • Access to private $this properties is disallowed
  • Variable-variables and variable-functions are disallowed
  • Only whitelisted functions are allowed (the whitelist is user-definable)
  • Only whitelisted static method calls are allowed (the whitelist is user-definable)
  • Use of $this by itself is disallowed; it must be followed by -> (e.g., “$this” generates an error, but “$this_thing” and “$this->property” are allowed)

I think that’s it. If the compile generates errors, the compiled script is not saved and the compiler returns a list of restriction violations with line numbers (the number correspond to both the source template and the compiled template becuase the markup language is very close to native PHP).

Comments? Questions? Have I forgotten to take something into account? It’s only been a week, so I am sure to have missed some form of sneakiness.

Savant Article in PHP-Mag

My article on Savant has been published in the 01.2005 edition of International PHP Magazine. I subtitled it “How I learned to stop using Smarty and love PHP for template markup” (so what if I love old movies?).

In the Development track, Paul Jones elaborates on how he learned to stop using Smarty and love PHP for template markup. He holds forth on Savant, a lightweight, object-oriented, fast and powerful solution that leverages the power of PHP markup with the convenience of plugins and path management Γ’β‚¬β€œ all without compiling.

Toward a Secure Compiler for Savant

Savant uses PHP for its template markup, but also supports extneral compilers. Unfortunately, the example compiler in the distro is not that secure.

But! Joshua Eichorn has put together a spiffy PHP code analysis tool called PHPCodeAnalyzer. It takes PHP code, runs the PHP tokenizer on it, and reports back what functions, methods, etc. are present in the code. This could serve as a basis for a secure compiler for Savant, which in turn might be the tipping point for RevJim.

(Reverend, if you’re reading, I’d be very happy to hear your comments on this).

Code Tests As Code Tutorials

Early in the development of Savant (back when it was HTML_Template_Dummy) I broke the assign() method without knowing it, then distributed the source to early adopter testers. Of course, they discovered the break right away. Embarrassed, I wrote up a quick series of “visual” test scripts to run on each release. They are not automated; basically, they instantiate Savant and print out the results of various method calls, which I then eyeball to look for problems. While not optimal, and certianly not “best practice,” it’s good enough most of the time.

However, such “eyeball” tests seem to have an unexpected benefit. I just got a comment from Alex at Sourcelibre saying:

In version 2.3.2, the directory … Savant2/tests are really usefull. I almost always prefer examples to explanations, and these examples are just perfect.

Well look at that. I wrote up code examples and I didn’t even know it. While it’s not documentation per se, it appears to add a lot of value to the package.

So now there’s at least one more reason to write non-automated tests for your libraries: if the tests are designed to be human readable, not machine readable, then they can serve the purpose of testing **and** tutorial.

Savant 2.3.2 released

Savant sees a new release today, version 2.3.2. It’s a bugfix release; the change notes are:

* can now use assign() to assign a null value by name (thanks, Michael Caplan)

* the dateformat plugin now correctly refers to named custom formats (thanks, maxim at inform-link dot co dot uk)

* added tests for named custom dateformat strings

In related news, it seems that Monte Ohrt (or someone using his name) of Smarty fame wanted to take issue with some of my comments about caching over at the house of Reverend Jim. Give Jim some traffic and see for yourself which point of view you agree with; when you’re done, be sure to check out his photography work.

Finally, the Savant site has seen quadruple its normal traffic in the past 24 hours. Normally it gets about 1000 hits/day, but right now it’s at 4000. Could it be that my new article for PHP Magazine on Savant has been released?

Savant is a template system for PHP that uses PHP itself as the template markup language. Savant has plugins, output filters, customized error handling, and allows you to hook in a compiler object to deal with customized non-PHP template markup. I call it the simple, elegant, and powerful alternative to Smarty.

Update: Looks like SitePoint Tech Times has a link to Savant, which explains the traffic increase. Thanks to Sean from #pear on IRC for the information, and thanks to the SitePoint folks for mentioning the project.

Update 2: At midnight: 6763 hits in 24 hours. Sweet. πŸ™‚

DB_Table: Pre-Defined Per-Query Fetchmodes

Ian Eure just provided a patch to DB_Table that is genius in its simplicity. While I have not tested it, the patch looks like it will cause no trouble. In short, in your predefined SQL queries, you can specify that the query should return in DB_FETCHMODE_ASSOC, DB_FETCHMODE_ORDERED, or any other fetchmode. For example:

// predefined query for a list of all rows
$this->sql['list'] = array(
    'select'    => '*',
    'from'      => $this->table,
    'order'     => 'some_column DESC',
    'get'       => 'all',
    'fetchmode' => DB_FETCHMODE_ASSOC

// predefined query for a single row
$this->sql['item'] = array(
    'select'    => '*',
    'from'      => $this->table,
    'get'       => 'row',
    'fetchmode' => DB_FETCHMODE_OBJECT,
    'fetchmode_object_class' => 'myDataObjectClass'

With that, you can call $dbTable->select('list') and get an array of rows, or call $dbTable->select('item', "id = '9'") and get an object of type myDataObjectClass.

This is only in CVS right now, as I’m trying to add various Oracle restrictions to DB_Table, but it was just too neat to not add in right away. Thanks, Ian Eure.

DB_Table is a PHP class to automatically create RDBMS tables and XHTML forms. It includes a “poor man’s” data type abstraction wherein the database is forced to store date and time data in ISO formats, which means you do not need to do date/time magic on your queries before executing them (and you don’t need to do magic on the return results, either).

YaWiki 0.18 alpha released

YaWiki 0.18 alpha is ready for download. The change notes are:

* updated to take advantage of Text_Wiki 0.22 and later

* configuration change: Text_Wiki parsing and rendering is much more configurable via [Text_Wiki_*] groups in Yawp.conf.php (instead of via the [yawiki] group)

* now supports RSS feeds for all pages in all areas, all pages in one area, or one page in one area

* yawiki::getPage() now takes optional $area, $page, and $dt arguments

* clicking on the “Revert” button when no date-time is selected now reverts to the most-current saved version

* cannot delete Main (or other default) area in area_list.php

* other areas in the wiki are automatically set up as interwiki sites

* acl.php area name list now shows a ‘*’ wildcard

* there is now a “settings” link on area_list.php (clicking on the area name will take you to the page list)

One big deal is that you can configure Text_Wiki directly now, with all the new CSS options that Text_Wiki 0.23.1 has. Another is that public RSS feeds are available for the entire wiki, one area (all pages), or one page in an area. No RSS feeds for comments yet, because that’s part of a separate project (Yet Another Comment System, or Yacs, which has until now been embedded in YaWiki in pre-alpha form).

With this release, I am willing to call YaWiki feature-complete for a 1.0 release. Keep the bug reports coming (there have been remarkably few, hope it stays that way) and I’ll start commenting and documenting in earnest. With luck it can go beta over the next couple of months, then a stable release early next year.

YaWiki is a wiki-like CMS (or a CMS-like wiki, if you prefer) designed for collaborative documentation efforts. It is built from PEAR and PEAR-compliant components such as Savant, DB_Table, Text_Wiki, and uses the Yawp foundation for rapid application development.

Savant and The Right Reverend Jim

RevJim has a great critique on one of my many posts about Savant vs. Smarty. His analysis is accurate, fair, and open. You really should read the whole thing. In the mean time, here are some highlights of what I want to respond to:

As of a few verisons ago, PHPSavant has a compiled templates option. In the event that your security needs would keep you from allowing template authors direct access to PHP, you can author a PHPSavant template compilation class that will, when activated, convert template markup of your own design into proper PHPSavant template code.

Smarty has the advantage over PHPSavant because the template compiler and, therefore, the language of the template is already written. This means, first and foremost, that I donÒ€ℒt have to write a full-featured, flexible template compiler like I would with PHPSavant. This also means, though this is less important, that users of differnet Smarty based applications can join forces to teach one another and lead by example when it comes to what Smarty is capable of. With PHPSavant, those authoring templates in PHP would be able to help one another. But the users of applications that employed PHPSavant with a template compiler would find themselves in less company.

He concludes, “if PHPSavant were to implement a template compiler that was as featureful as the Smarty template language and provide a means in which to request template compilation only if a template had been changed, Smarty would, unless there are any major speed differences, have no reason to exist whatsoever as PHPSavant would cover the needs of both the ‘pure PHP template junkies’ and the ‘template tag sugar addicts’.”

Jim, please tell me you didn’t try to set me up for this one. πŸ™‚

Such a compiler already exists in basic form as part of the standard Savant2 distribution. It converts a limited markup, and only compiles when the template source has been modified. While not 100% locked down, part of the plan is to add a feature suggested by Joshua Eichorn to have the PHP tokenizer extension pass through the script to make sure only “whitelisted” PHP has been generated.

The markup is as follows:

Markup Action
{$this->var} Prints an assigned variable
{$var} Prints a local variable
{if (…): } Starts an if block
{elseif (…): } Starts an elseif block
{else (…): } Starts an else block
{endif} Ends an if/else/elseif block
{for (…): } Starts a for loop
{endfor} Ends a for loop
{foreach (…): } Starts a foreach loop
{endforeach} Ends a foreach loop
{while (…): } Starts a while loop
{endwhile} Ends a while loop
{[‘pluginName’, $this->arg0, $arg1, ‘arg2’, …]} Activates the plugin named ‘pluginName’ with an argument list
{* … *} comments
{tpl ‘template.tpl.php’} Includes a template

The markup is converted directly to PHP and matches very closely the PHP an author himself would write; the compiled PHP code is very clean and easy to debug.

While this basic example compiler is not as full-featured as the Smarty compiler, any basic functions that are missing can easily be added, if for no other reason than the codebase is much prettier (it adheres to the PEAR standard) and nicely object-oriented. Savant is also a lot lighter than Smarty, so there’s less code to slog through.

Does this mean there’s no reason for Smarty to exist? Certainly not; while I have no use for Smarty, that does not mean there is no use for it period. But the existence of this basic compiler, I think, goes a long way toward satisyfing some of RevJim’s desires regarding a non-Smarty template system.

On a related note, Jim is dead-on about collaborative effort between users of different markups; because Savant does not dictate what compiler you must use, anybody can come up with any kind of markup they want. It would be easy for this to lead to chaos on the compiler front.

However, I see this as a strength, not a weakness. Because the compiler is separated from the core templating system, the “best” or “most useful” compilers can evolve separately from the core template functions. Different communities can write compilers to suit their particular needs, all while using the same core functions and plugin sets. Thus, you get the best of both worlds; a relatively static core package (top-down) that provides an easy, dynamic, and well-defined way to extend and enhance the functions of that package (bottom-up).

UPDATE: See the Savant2_Compiler_basic documentation for more information.

UPDATE: Jim responds! It’s bloggy ping-pong! πŸ™‚

New Text_Wiki Mailing List

If you use the Text_Wiki package you can now subscribe to its dedicated mailing list at [email protected].

One of the inconveniences of PEAR is that is has no package-specific mailing lists. Thus, if you want to talk about a specific package and no other, you have to get all the traffic whether or not it is related to the package in which you are truly interested. As I said, it’s an inconvenience, but having it would be a nice touch. Thank goodness for Tigris. πŸ™‚

Text_Wiki is an object-oriented PHP library to parse wiki text and then render it into any of a number of formats. Each parsing and rendering rule is its own class, so you can add/change/delete parsing rules quite easily. Aaron Wormus gives it a favorable review (thanks Aaron! πŸ™‚ and provides some example code on how to write your own rules at the end of PHP Barnstormer #15.