The fallacy of software development principles
Why SOLID, TDD, or Clean Code won’t make us good programmers
What is good code? To research this question, imagine we put 100 software developers into one room, give them 10 code bases to evaluate, and ask them to create a ranking based on code quality. In reality, this will probably take a good while, and also have significant risk of domestic violence. But since this is just a thought experiment, we can assume that it all goes smooth and civilised. While the eventual survey result would certainly not be unanimous, it’s still likely that there will be an overall trend that the majority of participants can agree on.
So we go on and ask the developers how they came to their judgement of what “good code” is, to which they say things like: good code has a clear and intuitive structure. Good code works reliably. Good code is expressive, meaningful and self-explanatory. Good code is up-to-date. Good code feels easy to work with.
That all sounds great, but these statements are still a bit too vague for our purposes. We dig deeper: what does it actually take to achieve high code quality? What skills and practices do we have to apply to get there? And how can we teach these to other people? Murmurs go through the room, and our questions spark numerous discussions among the attendees. We seem to be onto something here, so we ask the developers to compare the best code bases of their ranking with the low-rated ones, and to identify common traits for either end. We get answers such as:
- “The worst code bases have many instances of code duplication à la ‘copy pasta’, whereas the best code bases aim to reuse common functionality.“
- “While the poor code bases have lots of functions that are long and convoluted, the great ones split up the procedures into smaller, dedicated methods.”
- “In the low-quality code bases, there are only few and course-grained tests. The good ones, on the other hand, have comprehensive coverage and verify each behavioural aspect in a separate test case.”
Such statements go on and on, and after a while we have gathered a bunch of insights, which we are able to condense into handy, actionable principles: “Don’t repeat yourself!”, “Separate concerns!”, “Test first!”.
That, more or less, is how software development principles come to be. First, we make observations about what works well and what doesn’t; then, we try to discern common patterns that are characteristic for the good and for the bad; and eventually, we come up with a principle that makes sure to avoid all this hassle in the first place. It’s as easy as that, so the only thing that’s left for us to do is to invent a catchy slogan and spread the word, to solve the problem once and for all.
Unfortunately, we may start to notice that our principles have some flip sides in practice. It’s true that good code bases tend to avoid needless duplication – but by merely telling someone “don’t repeat yourself!”, they can still legitimately produce an entangled mess of interdependent code, riddled with obscure abstractions whose sole purpose it is to avoid any repetition no matter what.
There is no doubt that large and complex methods are characteristic for “spaghetti code” – but a strict rule like “functions shouldn’t be longer than x lines of code!” can just as well have the effect that we now have to jump between 13 separate methods across 4 different files, only to understand something that’s actually a single, consecutive control flow.
In the name of TDD, I have actually witnessed someone suggesting in all seriousness that we should test for the presence of methods in an interface by asserting whether the reflection API would return the declared method name. I mean, if you take TDD at face value, it’s hard to argue with that – when you believe that every line of code must be preceded by a failing test case, then this is pretty much the only conclusive way to build up an interface. When applying common sense, however, it really is a pointless exercise that doesn’t lead anywhere useful.
Software development principles are leaky abstractions
These problems are home-made. Or, to say it in our own language: software development principles are leaky abstractions. The reality is that creating software is a brain-racking and highly individual process, which requires careful consideration to balance the trade-offs on a case-by-case basis. Producing high-quality code is largely a matter of craft and experience, and while there certainly are some underlying virtues to that, it’s a human process that by nature is fuzzy, organic, and somewhat arbitrary.
Our principles, on the other hand, try to paint a different picture: they tar all situations with the same brush, optimise for isolated and very specific aspects, and promote the illusion that complexity will automatically be tamed by simply following a set of prefabricated rules, one at a time. That doesn’t quite fulfil itself in real life, however, and the counter-examples above may illustrate how this approach can fail to deliver.
When it comes to the educational aspect, software development principles suffer from the misconception that cognitive processes were bidirectional. It’s true that we can infer common principles by analysing patterns and correlations in practice, but that is a one-way street. The generalisations, which led us to these principles in the first place, are not of much service to those who try to learn a practice from the ground up. Due to their inherent lack of richness and detail, our principles don’t yield the right outcomes in a reliable way, let alone manage to reproduce the original level of proficiency and quality that they were based on.
For an expert, who sees things from above, it’s all too easy to boil down their wisdom into smart-sounding proverbs. They have been there many times, they know all the nuance, and they can count on their gut to tell what matters and what doesn’t. Therefore, they don’t have to bother how well their “rules of thumb” hold up on close inspection – as virtuosos, they can always play it by ear anyways.
For a novice, however, that’s not how it works. They don’t have the intuition that would let them anticipate potential ramifications in advance, or to directly spot the key aspects that are critical for making good decisions. Without having a complete picture of how things play together, simplistic guidelines don’t convey enough meaning for a beginner to navigate the complexity. On the contrary: what they really need is exactly all the ifs and buts that our software principles are lacking to begin with.
Individuals and interactions over processes and tools
DRY, KISS, SOLID, YAGNI, Clean Code, TDD, DDD, BDD, Agile, Scrum, XP – you name it. No matter where you look, the world of software development is saturated with principles, paradigms, and methodologies. They all make well-meant and alluring promises, but unfortunately, they are often focused on technicalities to an extent that makes people struggle to see the forest for the trees. That’s hardly surprising. Some of these cases are particularly tragic, as they have effectively managed to cultivate practices which are diametrically opposed to what they initially set out to achieve.
How did that happen? My theory is that our desire for rules and formalisation stems from our own cognitive bias. As software developers, we think in terms of structures, flows, and systems. We are trained to recognise patterns of how things work under the hood, we define models that reflect the most essential rules and mechanics, and we solve problems by creating universal and reproducible frameworks. This is a great approach when instructing a computer what to do, so it may seem all too logical to try to apply this to human processes as well.
Except that humans aren’t computers. Creating and maintaining code bases is a creative and collaborative effort to a large degree. Working together and organizing ourselves within a team is inherently social, where the involved people have widely different backgrounds, characters, and preferences. The more rigorous the processes and principles are, the less they match and reflect our human reality. It’s beyond question that we need some basic rules and agreements. But with increasing degree of formalisation, we continuously foster a mindless and anemic culture. Instead of changing anything for the better, this might then lead to pain and frustration on various ends.
It’s the same old trap that always snaps shut when rules and principles grow out of hand. The focus shifts from competence to compliance: it becomes harder and harder for people to comprehend where things are really coming from, and they forget (or don’t even learn at all) how to make good decisions based on their own reasoning.
In order to round off this blog post, I’d like to draw the only logical conclusion from all these realisations, which is to coin a principle for them: “the fallacy of software development principles”. The name is a bit simplified, admittedly, since it really is supposed to cover all kinds of paradigms, processes, and methodologies likewise. But remember how we said that principles are leaky abstractions? The most important thing is that the name is sufficiently generic to resonate with experts, while also sounding clever enough to impress the novice.
I’m well aware that there is some irony in proclaiming a principle that’s basically self-invalidating, but inventing my own software development principle is a long-standing desire of mine, so please don’t take that away from me.