Code Tests As Code Tutorials

By | December 9, 2004

Early in the development of Savant (back when it was HTML_Template_Dummy) I broke the assign() method without knowing it, then distributed the source to early adopter testers. Of course, they discovered the break right away. Embarrassed, I wrote up a quick series of “visual” test scripts to run on each release. They are not automated; basically, they instantiate Savant and print out the results of various method calls, which I then eyeball to look for problems. While not optimal, and certianly not “best practice,” it’s good enough most of the time.

However, such “eyeball” tests seem to have an unexpected benefit. I just got a comment from Alex at Sourcelibre saying:

In version 2.3.2, the directory … Savant2/tests are really usefull. I almost always prefer examples to explanations, and these examples are just perfect.

Well look at that. I wrote up code examples and I didn’t even know it. While it’s not documentation per se, it appears to add a lot of value to the package.

So now there’s at least one more reason to write non-automated tests for your libraries: if the tests are designed to be human readable, not machine readable, then they can serve the purpose of testing **and** tutorial.

7 thoughts on “Code Tests As Code Tutorials

  1. Travis Swicegood

    I’d have to say I sort of disagree with you here – only “sort of” because I think a properly commented automated test could serve the same purpose. I try to make it a habit to include a final test case that duplicates all of the individual tests but in a “real world” scenario. For Savant, that might be a full initialization of Savant, loading a template, checking for a few of the correct patterns followed by the same for plug-ins, filters, and finally both together. Basically, it would be what you have right now with the exception that you’re relying on the computer to “eyeball” the results.

    Of course, its usefulness might be mitigated by the fact that it would be buried in the framework of the automated tests, but if you have the proper pointers to it folks could find it.

    Reply
  2. Paul M. Jones

    Hi, Travis –

    I can’t say I know much about automated testing to begin with, so my opinion here is likely uninformed. However, I have yet to see a testing suite that lends itself to easy understanding of the underlying code; that is, the automated test itself is obtuse and unintuitive. As you point out, framework of the test itself hampers the new user’s analysis of (a) what the test is doing, (b) why, and (c) how that applies to the use of the package.

    Not saying that automated testing is bad (far from it! it is good!) only that the automated tests I have seen would not be good code examples or code tutorials for the package they test.

    Reply
  3. Sean

    PHP’s .phpt (php tests) are extremely simple, and don’t suffer from being “obtuse and unintuitive”. I don’t know if they should considered a “test suite”, though.

    S

    Reply
  4. Harry Fuecks

    I’m deep in the “write automated tests” camp simply out of laziness – they give me an absolute result immediately that I can re-run again and again, which I find extremely valuable for development. Granted they aren’t friendly if you’re not familiar with unit testing and, generally, don’t serve as practical examples. Prefer to write examples seperately.

    But can see where you’re coming from and the value of it. Perhaps the phpt tests make a good compromise.

    Reply
  5. Travis Swicegood

    Here’s a sample of one of the tests I have running on a class called Filter. Filter takes a key, value, and a valid Comparison object and creates a Filter for filtering results. Anyhow, here’s one of the tests:

    /**
    * Test to insure that filter is providing information back properly.
    */
    public function test_GoodReturn()
    {
    $test = new Filter('keyName', 'valueName', new Comparison('='));
    $this->assertTrue($test->isValid());
    $this->assertEqual($test->__toString(), 'keyName = valueName');

    $test->setValueDelimiter('"');
    $this->assertEqual($test->__toString(), 'keyName = "valueName"');

    $test->setKeyDelimiter('`');
    $test->setValueDelimiter('');
    $this->assertEqual($test->__toString(), '`keyName` = valueName');

    $test->setKeyDelimiter('');
    $test->setKey('newKey');
    $this->assertEqual($test->__toString(), 'newKey = valueName');

    $test->setValue('newValue');
    $this->assertEqual($test->__toString(), 'newKey = newValue');
    }

    As the code progresses, I think it’s fairly straight forward as to what is happening. All of the “$this” references point to the Filter_Test which extends UnitTestCaseExt – my personal flavor of Simple Test – which contains all of the assert* methods. If you know what assert() is, though, it’s easy to assume that assertEqual() is checking to insure two values are equal then the rest of it is just a matter of reading the code to see how it’s working.

    I would check out Marcus Baker‘s article on Test Drive Development.

    Reply
  6. Sebastian

    I have worked in a project where it was the case that we needed to document a interface to stored procedures in a very complex database for a 3rd party acessing here.
    So the tests where a) documentation and b) used as integration tests when moving from a dev. server to the live system.
    As the real world bites hard, things tend not to work when you need it and my own experience was here that one documentation and test suite spared a lot of discussions and research on what does not work in which subsystem.
    Systems which are testable tend to follow rules of low coupling and cohesion as simpler they are to test … those things are connected.
    Whats automatable is simple …..

    Reply

Leave a Reply

Your email address will not be published. Required fields are marked *