Faith in DRY; no hope for software

DRY means Don't Repeat Yourself. It is, or at least should be, a fundamental principle of software engineering. I believe in DRY. Strongly. Perhaps more strongly than I believe in God. But I don't advocate for it, because advocating DRY implies the following beliefs:

1. lots of people don't believe in DRY, and
2. of those people, lots can be convinced of DRY.

I believe (1) but not (2), and that combination deprives me of hope for software. As in, all hope is lost. Seriously. Not that all the computers in the world will soon stop working, as was feared for Y2K, or something dramatic like that. Just that we are doomed to never make any progress beyond where we are today. Today we produce software through a laborious, undisciplined process that combines the low quality work of many with the heroic high quality work of a few. What would progress look like? Well, an efficient, disciplined process that raises the minimum quality of the work and requires no heroism. But it ain't gonna happen. Not enough people believe in DRY and other good practices like it.

A person should come to believe in DRY automatically after their first experiences writing software. If this has not happened, you should not try to convince them of DRY; rather, you should try to convince them to stop writing software. This is not to say that such a person is bad or unintelligent; they are just bound to write software of low quality.

There are people who "get" DRY and people who don't. Trying to convince someone of DRY is asking them to change something too fundamental. It is somewhere between a way of thinking and a core belief. It is not something whose value you can prove to someone. The trade-offs are too complex to measure objectively.

Now, you may say that I am being too extreme in that there must be some people who are on the fence and can be convinced of DRY. The only wiggle room for hope I see here is that some "closet" believers can be encouraged to come out when other believers set a good example by practicing their faith openly and vigorously. Also, you can of course share techniques for DRY among believers. But that's not what we're talking about here.

Now, you may also say that I am being too extreme in that even a non-believer must be able to play a productive role in writing software if that role is appropriately bounded. Perhaps this can help a little, but I see two reasons why it can only help a little.

The first reason bounding only helps a little is that DRY is still important at small scales. At small scale it is easiest to achieve, and therefore perhaps least interesting to discuss among believers, since the techniques are more obvious, e.g. the use of subroutines. But please remember that there are people who basically do not believe in subroutines. I have worked with people who think nothing of a subroutine that is 500 lines long. In fact it makes more sense to them that way. They say they like that they can read it straight through like English text without being distracted by "jumps" to subroutines. The fact that DRY is not practiced even when it is easy to achieve shows how deep non-belief can run.

The second reason bounding only helps a little is that bounding someone's work can mean forcing them to repeat what others have done. DRY really includes the less-catchy abbreviation DRSE, Don't Repeat Someone Else. Another way of looking at this is that DRY in some sense works against modularity. Or, to be more precise, DRY frequently demands re-drawing the boundaries of modules when commonalities are discovered as those modules develop. So restricting someone's work to a pre-determined module can mean inviting a large-scale DRY violation. Now, non-believers would usually not see such commonalities, and even if they saw them, they would never exploit them. The point is that bounding their work does not bound the impact of their non-belief. You still end up with software that repeats itself.

Even though we're never going to convert the non-believers, it would be nice if our tools at least allowed us to practice our faith. Instead, particularly at large scales, they require us to go to great lengths to practice DRY. The trade-offs, already hard to justify objectively, become even harder to justify at large scale.

For an example from my experience, build systems do a terrible job of supporting DRY. In some sense these are small-scale problems but I consider them large-scale in the sense of considering stuff like build systems and test code that isn't part of the final product.

Build systems suffer from the same problem of any system that encodes information without abstraction mechanisms to avoid repetition: they make the fatal mistake of distinguishing data from code. Have the 60-year-old discoveries of LISP taught us nothing? Concretely, why can't I specify my build using a real language, with subroutines? Even better if this real language is an existing rather than novel one.

To be fair, some build systems run counter to this trend; for example Buildbot builds are specified in Python. An alternative is of course to continue to distinguish data from code, but make it easy for the user to generate the data, thus abolishing the distinction. This is usually what practicing DRYers end up doing, but systems often make it very hard to do.

I've just begun to dip my toes into the big water that is database programming, but it already seems to me that, despite mighty efforts to the contrary, the state of the art is still generating SQL queries in a way that requires repetition and therefore invites inconsistencies between the queries, the database, and the surrounding code. Perhaps C#'s LINQ makes some progress on this front; I don't know. And even if they did not widely succeed, perhaps the fact that mighty efforts that have been made (e.g. object-relational mapping) is encouraging.

I will end on that encouraging note.