THINGS THAT CAN KILL A SOFTWARE PRODUCT – TECHNICAL DEBT
You’re coming home after a hard day at work. You throw your bag on the couch and grab the two slices of cold pizza left over since last night (you hope), barely feeling the taste of it. With the last bit of energy you throw the box on top of the pile of other boxes from food delivery you have lying around in your room, next to the pile of dirty clothes. It used to be a nice room and you were proud of it, decorated it yourself, and you had friends over for a beer, but now, you’d not invite their dogs over. You promise yourself you’ll clean all that in the morning, and fall asleep.
You wake up scared by your phone ringing, and you barely get the cochance to get a quick shower before you storm out towards the office. Another crisis needs to be taken care of. Your room can wait. Maybe next week-end. If all goes well.
Sounds funny? It is, unless you’ve actually been in that spot, in which case it’s actually sad. The good news, though, is that the solution is already there: take time to clean the mess.
Over the course of the past 25 years, I’ve seen quite a few software products / projects that resembled the room described earlier. They were masterpieces of engineering when the development started, and ended up like a pile of dirt. As a fun fact, the Software Engineering community came up with a nice name for that. It’s called Technical Debt. Now, I’m not sure about you, but I’d definitely not leave a Legacy like that to my kids. They would put me in the worst retirement house when the time comes.
Sometimes, the Technical Debt affects even pretty recent projects, say 2-3 years old, top. I’ve seen Technical Debt being present even in projects built over the course of just a few months. How did that happen? Why did that happen?
What exactly is this Technical Debt?
Every time you skip a good practice (sometimes as simple as taking a food box to the trash after you’re done eating), you end up with a mess. And then, the mess will pile up, till the amount of mess you deal with is unmanageable. And it’s called a debt because you owe it to yourself, to clean it. Mainly to the future you. Who will otherwise hate you.
When it comes to Technical Debt, this is accumulated when the team skips the good quality practices specific to the development process: using the right architecture, writing the code with the right patterns & principles in mind (ex. SOLID Principles) and following good practices such as Test Driven Development, Peer Programming, Code Reviews, Continuous Integration, etc.
Why does Technical Debt happen?
There are many many reasons why a software product or project degrades over time and becomes really hard to maintain. You can consider a product as being a pyramid. The tip is the actual product and the base has 3 sides: people, technology and processes (practices). If you maintain a good balance on all 3 sides, the product can grow nicely and maintain that balance. If you ignore one, or push too much on another, the product will eventually fall. And fail.
When it comes to people, it’s clear that the lack of skills or motivation is quickly going to lead to technical debt.
Lack of proper processes and practices, specific to all areas of the product (DevOps, development, quality assurance, management, etc), or too much process, can also put the product on its head.
With technology, things are no different. Not enough investment and focus on technology (choosing and using the right tools and technology, not understanding it properly due to lack of training, etc) or too much emphasis on it and usage without a proper risk analysis and due diligence can lead to a catastrophic end of the product, making it unmanageable.
How can we know if we’re affected by Technical Debt or not? What is the impact of the Technical Debt?
Technical Debt manifests in different ways, depending on the product type and stage of development. Usually, the following signs suggest that Technical Debt is piling up and needs attention:
- Increasing number of defects
- Performance degradation
- Security problems
- Reduced team velocity / performance
- Low team morale, satisfaction and overall productivity
- Inability to adopt new technologies or upgrade existing ones
- Inaccurate estimations and delayed delivery
- Long turn-over time for new features and even for bug fixing
- Lack of tests, outdated tests or tests failing consistently
- Frequent updates of the same classes / modules in your source code repository
How can we avoid letting our products to head the wrong way and accumulate Technical Debt?
The answer is simple: prevent the mess as much as you can, but when you can’t (for a while), make sure you address it as early as possible.
Ideally, you and your team should have the practice of continuously preventing and addressing technical debt. Agile/ Scrum teams use the concept of Definition Of Done which is extremely valuable in ensuring that all quality criteria are met before a unit of work is considered completed, and ensuring that they are well understood by all team members.
Some of the standard Definition Of Done criteria you should be focusing on are:
- Unit tests have been built or adjusted accordingly (and you might want to also target a specific Unit Test coverage threshold). And of course, they need to pass.
- Integration tests have been built or adjusted accordingly (depending on the type of the product / project, you should also consider security tests and performance tests to be part of the Definition Of Done). Again, having them run automatically and also passing, is a must.
- Code has been reviewed both manually, using Code Review practices, and automatically (using tools for code style checking, secure coding validation, static code analysis, etc.). Ideally, you would have all these automated code reviews performed on every check in or pull request into your source code repository (ex. git).
- Documentation has been updated (both source code documentation / comments, as well as any other types of documentation required: ex. deployment documentation, architecture documentation, etc).
Even the most Agile teams have times when a bit of Technical Debt sneaks in. AND THAT’S OK. There are times when a deadline, an emergency or a critical opportunity requires some compromise. BUT THAT IS FINE. However, as soon as the fires are out, go back and clean the mess. Don’t let it pile up.
Over time, with most of the teams I worked on, immediately after each release we had a Sprint Break. This sounds like a Spring Break, but I can promise it’s not that fun. But it still has a level of entertainment and satisfaction to it, because, at the end, you might get a Technical Debt free product.
Back to the Sprint Break, depending on the size of the release and the amount of Technical Debt we had, we just used a couple of days for refactoring & cleaning up. It’s just like cleaning your house after your friends have left at the end of the party. It’s a pain to have to do that, you’re tired and you want to sleep, but you’ll love it in the morning when you see your house is clean & tidy.
How can we identify the amount of Technical Debt we have?
Well, let’s start with saying that most of the time this is not easy to identify.
If a product is built using the practices mentioned earlier and has Continuous Integration with Tests & Static analysis in place, the numbers that would give us enough information to evaluate the Technical Debt are already there: code Quality, Unit Tests coverage, etc. We just need to look at them.
If we don’t have anything in place, we need to start somewhere: a one time static code analysis, using the tools available for the given technology stack, a review performed by the entire team or a review performed by an independent 3rd party.
Ideally, all the tools being used and activities performed by the team are leaving a trace which can help in the analysis (change logs, release notes, commits, issue tracking comments and status changes, etc.), so that the amount of debt is identified and also the most affected areas can be easily evaluated.
Our product is full of Technical Debt. What’s the way out?
This is the hardest question that demands a pretty complicated answer: it depends.
And that’s why we’re going to dedicate a whole article to it.
Closing Notes
Software Products are like living organisms. And it’s our responsibility, as Software Industry professionals, to take care of those organisms. Technical Debt is a chronic disease that can affect any software development team, project and product. With the right treatment, lifestyle and hygiene, this can be addressed and its effects removed. Ignored, it can lead to catastrophes. Ignored, it will always generate other diseases. Ignored, it can kill a product and even a business.
Article by
