Analyzing Technical Debt
Podcast: Play in new window | Download (49.5MB) | Embed
Subscribe: Apple Podcasts | Spotify | Email | RSS | More
Technical debt is everywhere. If you find an organization without it, they either don’t have any code, they are lying, or they are about to go bankrupt. Some degree of technical debt is inevitable in any organization. Furthermore, if you are starting a new position, or trying to get control of organizational technical debt, you need to know something more than “there is a lot of technical debt here”. Like actual financial debt, sometimes technical debt is undertaken for a very good reason, and sometimes (like your Uncle Larry taking out credit card debt to buy lottery tickets), it is taken on for more dubious purposes.
Regardless, to make good decisions at any level from junior developer to CIO, you need to both be aware of technical debt, and have a realistic ability to get rid of the bad technical debt before it becomes a problem. Like financial debt, there are a lot of reasons it occurs, problems it creates, and strategies for getting rid of it. Depending on the type of debt, a variety of different strategies may be applicable. Let’s talk about some types of technical debt and how best to get rid of them.
When we talk about technical debt, we do ourselves a disservice by simply dismissing it as “debt”, especially when we are talking to business people. While we often pat ourselves on the back because we are “speaking their language”, it doesn’t help to speak anyone’s language if you are using the wrong words. When trying to express “technical debt”, we really need to characterize it more carefully so that we express more than just the fact that there is debt. As an industry, we need to more clearly express what the costs the debt imposes, what risks it creates, what value is created by it (if any), and the difficulty of getting rid of it.
Episode Breakdown
Classification
Reason for the debt.
At one end is debt that is used to purchase a revenue-generating resource (taking out a loan for a lathe). At the other end is debt that is used to purchase an expense generating resource (taking out a loan for an expensive yacht).
Interest on the debt.
You always pay the price. The question is how. At the low end, the “price is low” and can be handled indefinitely. At the high end, the price is ruinous, and stops spending on everything else.
Ability to discharge debt.
How easily can you get out from under the debt? At the low end, this is the work of less than an hour. At the high end, this requires a total system rewrite.
Consequences of not paying the debt.
What happens if you just ignore the problem. At the low end, ignoring the issue has no discernable impact. At the high end, the company ceases to exist.
Lost opportunity.
At the low end the debt isn’t damaging your future prospects. At the high end, it’s sabotaging your market opportunities and costing you clients.
Metaphors
Buying a metal lathe at a discount at a bankruptcy sale from a rich relative, with a cheap payment plan.
If you took on the technical debt to take advantage of an opportunity, it isn’t costing you much, you could get out of it easily, and it creates opportunities, it’s probably one of the last ones you should discharge. This is the kind of debt you should ideally be choosing if you are taking on technical debt. Almost no one is talking about this when they use the term “technical debt”, because they don’t notice it if it is like this. It is easily justifiable to pay this debt, since it makes you money. An example of this in code might be using a third party component suite that has some wrinkles, just so that you can get a prototype out. The thing will still work, but may not be what you would choose if you had unlimited options.
Inheriting a functioning business that needs a small loan for expansion.
If you took on technical debt for a brief period to get past a problem so that you can capitalize on an opportunity and can quickly pay it back, this is more like what a businessperson hears when developers say “technical debt”. This is also an acceptable kind of debt, which is why they don’t understand your concern. Few are talking about this kind of technical debt, but this is how many interpret it. Because of the short term nature of this debt, it’s actually a reasonable way to move forward if you have to do it. An example of this in code is hard-coding something like a password reset process so that you can show a proof of concept application to investors.
Buying a piece of real estate that you can rent out with a low interest rate.
Similar to the previous one, buying an asset that makes you money is not a bad idea. Again, this sort of debt is often acceptable, provided that you can get a return on it. While the debt is large, this sort of debt also comes with a large asset that you can sell off to get out of the problem. This is a larger debt than the metal lathe example, and might be something more along the lines of using a bulky framework for the first pass at building an application. If the app doesn’t provide the results you want, you might quickly dump it, or pivot to something else.
Buying clothes so that you can interview for a better job.
Now we’re getting into what many programmers mean when they say “technical debt”. This is a small piece of debt taken on to facilitate a larger gain. However, if it fails, you are still stuck with the debt and/or an asset that is greatly reduced in value. An example of this might be something like writing a hacky wrapper interface for a third party API so that you can meet a deadline. If you build on this debt (accrue interest), its cost can easily outstrip that of buying it outright (doing it right the first time), but might still be small enough to tolerate for a while.
Buying a house so that you have a place to live.
This is a big chunk of debt on an asset that “usually” appreciates. You may have some annoying terms taking this on. This sort of debt also comes with a lot of other ongoing expenses, but you are toast if you don’t have it. If this debt fails to produce, it may do so in a way that means you lose big when you are getting rid of it. An example of this is tight integration with a large, third-party application. While this may help you get a product to market, it will be a royal pain to decouple later.
Taking out a loan due to a medical emergency.
In this case, you are taking out a loan as an alternative to serious injury or death. The asset being protected by this kind of technical debt is the ability to survive at all, even if the cost is ruinous. In addition to the cost of the debt, the situation that caused the debt may still be around for a while, making it harder to pay. An example of this would be having a regulatory requirement that you missed in an enterprise environment and having to hack together a solution using MS Access and a pile of scripts. When you try to get rid of this, it may take a while.
Spending money on a student loan so that you might be able to get a job that is capable of servicing the debt.
This is a potentially massive chunk of debt that isn’t on an asset (at least not a transferrable one). It’s also the first of the really unacceptable ones, in terms of technical debt. In addition, you can’t escape this debt. Once you’re in, it’s going to be unbelievably difficult to get out of it. And the debt will continue to mount. Your only hope on this is that it creates opportunities that allow you to discharge the debt. An example of this might be building a large project with no automated testing, in the hopes that you can get a company like Google to buy it and then use their resources for testing. This gets speculative very quickly, as do many student loans in the US.
Taking out a loan to gamble/speculate.
This is a kind of technical debt that developers really like. The cost is constantly escalating, and subject to sunk cost fallacy. In addition, escaping this debt means paying it back, just to get back to where you started. Your only hope on this is that you are right and that a huge payoff happens. An example of this is resume-driven development on a large, critical system, using either a beta Microsoft framework, a new hipster javascript framework, or a Google product that has been around for more than 4 years.
Not changing your oil because you are always in a hurry and having to buy a new car prematurely.
This is the kind of technical debt that your “idea guy” CEO probably really likes. It’s not speculative so much as it is negligent due to other concerns. Escaping this debt means not only fixing the debt itself, but usually significant additional opportunity cost, demoralized staff, etc. There is no hope with this kind of technical debt. You can only hope to not be in the car when the brakes go out. Developers are usually talking about this because they don’t want to be steering when the whole thing hits the wall. An example of this is continuing to ship new code with ancient frameworks that have known security/stability/support problems, simply because you are chasing the next sale. Eventually a reckoning happens.
Inheriting a mansion in a state of disrepair and trying to fix it up so you can live in a mansion.
This is the kind of technical debt that “yesterday’s idea guy” gets into today. This is taking on technical debt simply because “that’s how we used to do it and it was good enough then”. This debt is extremely painful to get out of, both because of cost and emotional attachment to a bad idea. This is much worse when a particular approach was proved successful in the past. There’s not really any hope with this kind of technical debt. In addition to wasting the debt, it will also run off most anyone who is capable of actually fixing it. An example of this might be building enterprise software in classic visual basic today. While back in the day, it was great, it is far past its sell-by date, even if you personally can code in it.
Borrowing from the mob so you can maintain your crack habit and pay off your payday loans.
Not only is this debt taken out for drastically stupid reasons, but it is doubling down on previous stupidity. This debt is extremely painful (probably fatal) to get out of, has to be handled quickly, and puts you into a state where you can’t even make reasonable decisions to get out of it. Not only is this kind of technical debt hopeless, but the consequences may land on anyone unlucky enough to be nearby. An example of this is lacking a private key infrastructure and sending sensitive data around (in contravention of regulatory requirements) with a trivially guessable key, hoping you don’t get caught.
National debt
Exorbitant debt taken out without any thought of repayment. This debt assumes that consuming a free lunch is a productive economic activity. Not only is this debt hopeless, but it’s so common that no one even worries about it, because it hasn’t resulted in fiery ruin…yet. Examples of this are things like the Y2K problem, or the assumption that Moore’s Law will always be with us. We lived with the assumption of normalcy until we couldn’t.
Book Club
The Elements of Computing Systems
The Elements of Computing Systems starts with a chapter on boolean logic. This first half of the book is about building the hardware for a computer. Since all computing boils down to switches between on and off the first two chapters look at boolean logic and arithmetic. Building upon that the next chapters talk about how that is organized into sequential logic. In these chapters you will be building a virtual computer chip. Chapter four talks about the way that low level chip is programmed using machine language. The final chapter in the hardware section puts it all together to build a general purpose computer.
Tricks of the Trade
Will didn’t write anything.
Tricks of the Trade:
When talking to management about technical debt, be very careful about how management understands the words you use. The manager who hears what you are saying, may not interpret the message the way you intended it to be interpreted.