TDD, Test-First, and Ravioli Code

By | July 14, 2007

I know that test-first and test-driven development (TDD) are popular methodologies these days, but something about those processes has always met with a level of mental resistance from me. Even though it sounds good in theory, I have been intuitively wary of the “test-first” mentality for some reason. My own approach is closer “remember to code so you can test it later” and then “test-last” after the API is mostly stable. This comment from Slashdot is a good summary of my feelings on the matter.

Thinking about testing early — good. Writing unit tests — good. The test driven development mentality (write tests instead of design, write unit tests before coding) — bad. … Thinking about testing early is useful, it may cause you to think about corner cases. But writing them first causes 2 problems — you end up writing the code to solve the tests (rather than solving the problem) and/or you end up throwing away half the test suite in the middle when you refactor the design.

AuMatar

Just that you have tests does not guarantee that you’re solving the right problem, or solving it well, or that your solution is comprehensible to others.

In the same conversation, someone brings up the term “ravioli code”, which I had never heard before. The idea is that instead of long strings of procedural “spaghetti” that are difficult to untangle, there are lumps of methods jumbled together:

The problem is that it [Ravioli Code] tends to lead to functions (methods, etc.) without true coherence, and it often leaves the code to implement even something fairly simple scattered over a very large number of functions. Anyone having to maintain the code has to understand how all the calls between all the bits work, recreating almost all the badness of Spaghetti Code except with function calls instead of GOTO. It is far better to ensure that each function has a strong consistent description … rather than splitting it up into smaller pieces (“stage 1 of preparing to frobnicate the foo part of the foobar”, etc.) with less coherence. The principal reason why this is better is precisely that it makes the code easier overall to understand.

People who deliberately split code up into ravioli, and especially those who advocate that others do so, are “dangerous idiots” precisely because they’ve lost sight of the fundamental need for comprehensibility …

dkf

I think that a lot of TDD and test-first idealists and evangelists* end up with ravioli code that is well-tested but still difficult to comprehend.

Some people will read this and think that I am against unit testing; that would be an incorrect interpretation. TDD and test-first dogmatism are what bother me, not testing per se.

(* Note that I said “idealists and evangelists” — not all of the test-firsties are like this.)

30 thoughts on “TDD, Test-First, and Ravioli Code

  1. Kevin Dangoor

    I’m a test first person, but I’m certainly not dogmatic (there are tradeoffs to everything… once you realize that, it’s hard to be dogmatic).

    That said, I think that writing the tests before writing the code that makes the test pass is actually good practice. Doing so helps to *verify the test*. Every now and then, you’ll write a test that magically passes before you’ve written the code that should make it pass. That either means that the test is faulty (the likely case) or that you had actually already met the requirement without realizing it.

    Doing TDD also helps to ensure that your code is testable. You’ll generally have looser coupling. Loose coupling is generally a good thing. But, I agree that you have to be careful about not getting into the “ravioli” mess that you describe above. It seems easy to avoid, though. If you find yourself writing the same 3 lines of code over and over again, that’s a good sign that you need some more conveniences in your API.

    Regardless of your methodology, it’s always possible to write crappy code.

    Reply
  2. Noel Darlow

    I have to ask who are the dogmatists: people who try to encourage others to learn an important technique or those who snipe at things they don’t know? I really don’t understand that sort of attitude. Accusations of evangelism or dogmatism against un-named individuals don’t really qualify as technical argument. Neither does the complaint about short methods – that’s just basic literacy. Do test suites and code get torn up? Well, yes, but that’s the whole point. TDD is a design tool where design emerges gradually, in little steps and only in response to real requirements. It’s expected that your first ideas will change as you learn about the domain and it’s a very practical and focussed way to work.

    Reply
  3. Amr Elssamadisy

    You have grossly misrepresented test driven development – but that’s ok.

    So here are some of the fallacies:

    1) “write tests instead of design” is patently false. Those who properly test-first design all the time – with every single test written they are designing. The well known development loop is “Red-Green-Refactor” which translates to “Write a failing test” (red), “make it pass” (green), “refactor”(improve your design).

    2) “you end up with code to solve the tests instead of the problem” is also false. Think with me for a moment – where do these tests come from if you haven’t yet written the code to solve the problem? Hmm…. Well – they don’t come from the design because you have yet to write the code. The only thing you really have to base your tests on are the REQUIREMENTS. Tests written first are a form of executable requirements. Therefore you tests help you focus on the requirements. They also force you design – with every tests you are having to make coupling and cohesion decisions – “where does this method belong?” “what does the method look like?” “where will it get its data”. You find yourself going through all the GRASP principles (Larman – OOA&D).

    There is much much more to this – I could probably write a book… But several people already have :)

    Finally – I know it gives you a bad feeling. It gave me one too before I got hooked. I had seen enough very smart people endorse it and use it (Ward Cunningham, Martin Fowler, and many others) and decided to “suspend my disbelief” and try it for 3 months until I grocked it and then make my personal decision. I suggest you do the same.

    Reply
  4. Nick Gerner

    I’m into the idea of testing early, and the promise that TDD holds. The notion of getting someone to make requirements concrete up front, before you’ve invested dev time seems like such a good idea, I can’t ignore it.

    I suspect that this works better for bigger projects where big teams create the requirements, big teams do the development, and big teams do the testing. In fact, I know that making requirements concrete is something sorely lacking in big projects. I work on ‘em ;) And I wish we’d spend more time to make our requirements precise by building tests early.

    That said, I agree about wanting to avoid dogma, and getting bogged down into tests just for the sake of tests.

    –you end up writing the code to solve the tests (rather than solving the problem) and/or you end up throwing away half the test suite in the middle when you refactor the design.–

    I’m with dfk on the first one (terrible!), and mostly on the second. But if you’re refactoring the design, what do you expect? That there’s nothing which will have to be…refactored?

    Reply
  5. Sebastian Bergmann

    From the Testivus Manifestivus:

    “Test first, during, or after – whatever works best for you.”

    and

    “The earlier you test, the better it is, but even tests written after you think the code is done are better than no tests at all. Do what works for you.”

    That said, well-practiced TDD does not lead to “ravioli code” in my experience.

    Reply
  6. pmjones Post author

    Hi Noel,

    You said, “I have to ask who are the dogmatists …”.

    [snark]I’d say one way to identify a “dogmatist” is: a person who perceives an attack on a beloved process or technique where there is no such attack, and responds reflexively and aggressively to it.[/snark]

    Sorry, could not resist. :-)

    Reply
  7. Noel Darlow

    I see. You believe that calling people evangelists and dogmatists is a form of praise? My mistake then.

    Reply
  8. pmjones Post author

    Hi Noel — if *you* want to believe that it’s praise, then I am happy to let you believe so. I’m sure you’ve written a test for the assertion, and that it passed just fine. Whether or not that test accurately gauges reality, well, perhaps that’s another issue. ;-)

    Reply
  9. Hans Lellelid

    What are some examples of ravioli code as a result of test-first practices?

    I’m not very strict about writing tests first, though I think this is a good practice because this practice has been shown (see, for example, Code Complete) to minimize the amount of time between writing mistakes and catching them. I don’t use unit tests to determine my software architecture and I don’t write code exclusively to make unit tests pass; however, when I am building a complex piece of code, I find it extremely useful to sketch out all the expected behavior and write the tests that ensure that behavior before I start writing out the business logic. It’s not only a useful way of thinking through the various execution paths/branches, but it also provides tests that will assure that the code continues to function properly as it is tweaked or refactored down the road.

    This sort of argument reminds me of good ol’ days arguing about the merits of exceptions on the PEAR dev list. People who have no experience with a development approach or technology start looking around for arguments on why said technology is bad. I am really disappointed to see how widespread this reactionary stance to software engineering innovation is in the PHP world.

    Reply
  10. Pingback: Paul M. Jones » Blog Archive » Testivus (for the Rest of Us) and the Testing Gene

  11. pmjones Post author

    Hi Kevin –

    You say “there are tradeoffs to everything” and I completely agree. However, I’m not so concerned with “verifying the test” as I am making sure the solution is comprehensible to others.

    As far as being able to write crappy code under any methodology, you are exactly correct. For code to be testable is a good thing, but only if the code itself is good — if someone writes junk that passes all its tests, the testability and test-pass-rate is meaningless.

    I’ve intended no slam against TDD or test-first, only a personal intuition that good coders who don’t use TDD will still produce better solutions than bad coders who adhere slavishly to TDD.

    Reply
  12. pmjones Post author

    Hi Nick,

    You said, “The notion of getting someone to make requirements concrete up front, before you’ve invested dev time seems like such a good idea, I can’t ignore it.”

    I agree, it is a compelling idea. My experience has been that no matter how well the requirements are specified in advance, whether by the customer **or even the developer**, you simply will not know enough about the problem until *after* you have worked with it for a while. There are always unstated or unrecognized complications that you discover only after building for a while. (This is one reason for the “build one to throw away” advice from Fred Brooks’ “Mythical Man-Month”.)

    In my opinion, test-first ignores this, hoping that writing tests will obviate the need to work with the problem for a while before settling on a design; I think this hope is misguided. No amount of planning and design can replace actually working through the problem; a soldier knows that “the map is not the territory”.

    Having said that, “planning is everything, but plans are nothing”. One is wise to do some amount of planning, and test-first may be a useful technique for some people there.

    Test will help to assure that the *code itself* is working as intended, but it is no guarantee that you have solved the problem in a useful and comprehensible way. To me, test-later or test-during makes more sense, because you more-fully understand the problem, after you’ve thrown away your first try (or your first few tries ;-).

    Reply
  13. pmjones Post author

    Hi Hans –

    I don’t mean to be a tease, but I don’t have publicly available examples. The best I can tell you is that my only experiences with “ravioli code” have come only from test-first and TDD devotees (admittedly a small number ;-). This is not to say that all the test-first guys I know produce ravioli; Jason Sweat comes to mind as an example.

    You talk about test-driven design as a planning tool, and I think I make reference to that kind of usage in my reply to Nick above. That does sound like a good use use TDD to me.

    As far as a “reactionary stance” to testing, I think I have pointed out repeatedly that I like tests and want more, not less. My reaction is to “test-first is the One True Path and you are Wrong to think otherwise”. I don’t think it’s bad at all, any more than I think object-oriented programming is bad. Testing has its place in the developer’s set of practices like anything else.

    Having said that, test-first and TDD are not replacements for considered and thoughtful approach to problem-solving. They may augment problem-solving in some cases, but religious devotion to them (or any particular technique) is not necessarily wise.

    Reply
  14. Hamish McTavish

    “a person who perceives an attack on a beloved process or technique where there is no such attack, and responds reflexively and aggressively to it.”

    I would have to agree with that statement. Only McGuff (or is it McGruff?) would take an aggressive attitude against those who would dare question the benefits, or not as the case may be, for something that they are favour of.

    People like McGuff (or is it McGruff?) just don’t like it when people argue the other way when it comes to something they are well into it, can they? They just cannot stand it, though people out there who like to think with their own mind, and take their own route, rather than just jump of the cliff like the other lemmings.

    Reply
  15. Mark Twain

    Hi Paul,

    I think you might have missunderstood some parts of how the TDD process works – at least of how I think about it.

    Writing tests first allow for using a top-down approach to design – that is, start with the client code and work your way down. The test cases are a way to capture how you want the design before you implement it.

    Refactoring is also an important part of TDD. The normal workflow is red-green-refactor. Having unit tests early in a project also minimizes the risk of breaking anything. This simplifies changing the design as you learn more about problem domain and requirements throughout the project.

    Reply
  16. Noel Darlow

    pmj:

    You still haven’t explained what triggered your complaint about TDD dogmatists. To whom were you referring? A book? A magazine article? A web site? Perhaps you have a link you can share so that we can find out what you are talking about?

    I think this is important. Either you can back up what you say and I owe you an apology or these people are figments of your imagination and you owe us one.

    Reply
  17. John DeHope

    I noticed you provide snippets of slashdot posts, but none of the ravioli code you mention. That makes it really hard to gauge the weight of your argument. How about an example?

    Reply
  18. Pingback: PHPDeveloper.org

  19. Hamish McTavish

    Noel – Go and McGuff somewhere else. This was an enjoyable blog until you came along :(

    Reply
  20. Pingback: developercast.com » Paul Jones’ Blog: TDD, Test-First, and Ravioli Code

  21. E. DeFazio

    I heartily agree with your arguments about TDD… posted about this before on artima:
    http://www.artima.com/forums/flat.jsp?forum=106&thread=137207&start=15&msRange=15
    BTW,
    Noel, if you want to find some dogmatists, maybe you should look at those who responded to this post.

    I also find that those beating the loudest on the TDD drum state:
    TDD is fine, and anyone who disagress with TDD simply does not understand what TDD is…

    This is what I disagree with about TDD… (verbatum from Scott Ambler http://www.agiledata.org/essays/tdd.html)

    “A programmer taking a TDD approach refuses to write a new function until there is first a test that fails because that function isn’t present. In fact, they refuse to add even a single line of code until a test exists for it. Once the test is in place they then do the work required to ensure that the test suite now passes (your new code may break several existing tests as well as the new one). “

    Reply
  22. Noel Darlow

    E. DeFazio:

    “Noel, if you want to find some dogmatists, maybe you should look at those who responded to this post.”

    Whom do you mean? I see several people who have pointed out – perfectly politely and quite correctly – that Paul seems to misunderstand some of the ideas behind TDD.

    That’s OK. Nobody should ever be attacked for not knowing something. Everybody should be encouraged to ask questions. At the same time you ought not to snipe at things which you do not understand simply because you don’t understand them. There’s too much of this in php (usually in OOP v procedural debates). What really bothers me is that this is harmful for those who are trying to learn and don’t yet know enough to see through it. I wouldn’t like anyone to go away from this thinking that TDD is an over-hyped dogma.

    It’s all a bit Goebbells isn’t it? Making up non-existent threats, lashing out at people who challenge the.. er.. dogma about the dogma. I guess we can call that the metadogma.

    It sounds like a reasonable thing to say. “I hate dogma!” “Yeah me too!” “Let’s get them!” Nobody is going to stand up and say well actually I quite like dogma and could I have some unfounded accusations to go with that please? In fact it’s completely meaningless. Lips are moving, words are coming out, but no information is actually being processed. There never were any dogmatists to begin with and it was simply dishonest to claim there were.

    In the world of php I can think of only a tiny handful of people who give talks, write articles or answer forum posts about TDD. None of them are what I would call dogmatists. They are all people I admire because of the effort they put into helping others learn.

    When I first saw the blog entry I thought: great! Something about testing; this should be interesting. I wish I’d been able to talk about that instead of the metadogma but, as Hamish McStalker has correctly pointed out you can sometimes find me dogmatically answering testing questions on sitepoint.com under the name of McGruff – when I’m not being rude about the good Dr Hamish.

    Reply
  23. pmjones Post author

    Noel –

    Your use of Goebbels, even in passing, invokes Godwin’s Law. Any valid points that you may have made in your post are completely overshadowed by this.

    I believe you have successfully demonstrated your inability to respectfully disagree and be civil to others in a public forum. If you wish to continue this behavior, I ask that do you do so on your own blog, and use its trackback/pingback system, instead of abusing the commenting privilege I have extended to you and the public-at-large.

    Amr Elssamadisy, for example, shows the ability to disagree in a civil manner, and even goes to far as to make a blog entry of his own in relation to the disagreement:

    http://www.infoq.com/news/2007/07/AgileBadForDesign

    I hope that you will have the common courtesy and self-restraint necessary to act as a good guest on this blog; if not, I will attempt, at my discretion, various measures to prevent and/or remove any further inappropriate comments from you.

    Reply
  24. Noel Darlow

    I’ll be glad to apologise if you can show us some real examples of TDD dogmatism in php. If not, my comments stand.

    Reply
  25. pmjones Post author

    Hi Noel — I’m not asking you to apologize. I’m asking you to behave yourself. Your comments do indeed stand.

    Reply
  26. Noel Darlow

    I’m sorry I don’t know how to “behave”. I only know how to be fair.

    Reply
  27. Chris Balavessov

    Hi Paul,

    Thanks for a very enlightened and may i say courageous post! I work for a small Java shop doing XP development and have personally witnessed many of the problems that your post mentions in our code base.

    TDD is a major tenet in our process and it has served us very well over the years. However it seems that over time the drive to split the code upfront into easily testable chunks has produced areas with particularly low coherence, with things only getting worse over time.

    This has actually become a problem with a large visibility in the company to the point that even the business people in the company now understand what “low coherence” means!

    Personally I find this code with low coherence can sometimes be much harder to refactor in a meaningful way, especially if the refactoring is not of some relatively trivial kind (e.g. I am doing an architectural enhancement, even a very minor one.). In these situations I sometimes (but not always) find myself having little option but to turf all the old code together with all the unit tests because of how badly and counter-intuitively things are split-up.

    Thanks again for the post, I am planning on using it to further shed some light on our cohesion problem in front of the business people in my company.

    With that said, the benefits of TDD far outweigh the problems it can introduce, at least for our company. It would be great if the discussion this post has started crystallizes some way of evolving TDD to avoid the low cohesion/ravioli code pitfall. The way it is currently used out there is just too easy to get wrong IMHO.

    Reply
  28. Pingback: Spaghetti? Ravioli? Farfalle? – PHP in Action

  29. Pingback: Thoughts On TDD (A Case Study With Ruby And RSpec)

  30. Pingback: Tweets that mention Paul M. Jones » Blog Archive » TDD, Test-First, and Ravioli Code -- Topsy.com

Leave a Reply

Your email address will not be published. Required fields are marked *