When technical debt strikes the security stack

Most veteran CISOs implicitly understand the concept of technical debt and how it increases the risk across IT assets and applications. The idea is simple in theory, if difficult in practice to address. Technical debt is the accumulation of all of those technical improvements slated for some other time—deferred work that’s put off because there’s not enough time, budget or staff to handle now. Just like financial debt, rack up enough of this kind of debt and you’ll be paying interest. It typically comes in the form of extra work, diminished reliability of systems and a big serving of risk because debt ridden systems are the ones most likely to be exposed to security flaws.

Typically, CISOs fixate on the risks that arise when technical debt accumulates across the IT stack. But the truth is that their own departments are often quietly racking up their own debts along the way, too.

“Security professionals are not immune from acquiring their own technical debt. It comes through a lack of attention to periodic reviews and maintenance of security controls,” says Howard Taylor, CISO of Radware. “The basic rule is that security rapidly decreases if it is not continuously improved. The time will come when a security incident or audit will require an emergency collection of the debt.”

In fact, not only is the cybersecurity department not immune — it may well be one of the most susceptible domains for accumulating the riskiest types of technical debt. “We’re often some of the worst about quickly throwing solutions together, often with zero consideration about how and if they’ll need maintaining,” says Chris Clymer, CISO at Inversion6. “The system thrown up during an incident quickly becomes something you are afraid to patch five years later.”

The fast evolution of threats and the swiftness of change across the security vendor landscape — one which goes through a constant cycle of point-product mania to rampant platform consolidation — makes it extremely costly and difficult to prioritize new investments and integrate them seamlessly into the existing security infrastructure. And the risk stakes are higher for security debt. The technical debt that lingers in security tooling doesn’t just tangentially increases risk — it directly contributes to an inability to prevent, detect, or respond quickly to material cybersecurity incidents.

We caught up with security experts to discuss where and how security programs are most likely to see technical debt accumulate and what they need to do to effectively pay these sources of debt down, as well as broader strategies for minimizing technical debt in the future.

Most common types of security debt

The the root cause of the most common types of security technical debt come down to the business needing to use its assets as long as possible and the inevitable consequences that arise from holding on too long to a hodge-podge collection of legacy systems.

“Common forms of security technical debt today include an overreliance on outdated security tools, inadequate security by design, and poor software development practices,” says Srikumar Ramanathan, chief solutions officer at Mphasis. “Legacy security tools may no longer provide adequate protection against modern threats and can be difficult to integrate with newer technologies. For instance, older firewall systems might not support advanced threat detection capabilities needed to defend against sophisticated attacks.”

Here are some of the most common ways these security debts mount, along with suggested strategies for addressing each.

Solution or functional debt

The most obvious form of security debt that arises especially from an overreliance on legacy security systems is solution or functional debt. Basically, the security stack in this case does not have the controls or functional capabilities to keep up with managing risk in a modern IT environment and detecting the newest attacker behaviors.

“Solution debt is extremely common. An organization bought a new product that at some point was cutting edge. That’s great, but over the years that solution is less and less cutting edge and really becomes a commodity,” says Maxime Lamothe-Brassard, CEO of LimaCharlie, explaining that security teams let products with gaps linger because of the challenge of moving products or switching vendor.

Often the security vendors themselves allow technical debt to accumulate within their own application and infrastructure, according to Omri Weinberg, co-founder and CRO at DoControl, who says that security users will not only suffer from outdated or inconsistent functional capabilities, but also poor integration across security and IT architectures and slow patch releases.

“These issues can hinder the ability to respond to threats effectively and maintain a strong security posture,” says Weinberg, who urges security organizations to do a better job holding their vendors’ feet to the fire and doing due diligence in the renewal and buying process. “Organizations should carefully evaluate their vendors’ track record in maintaining up-to-date and effective security solutions.” Weinberg says. “This is one reason why companies often switch vendors: to ensure they are using the most current and robust security technologies available.”

Tool sprawl and integration debt

The paradox of security technical debt is that many departments concurrently suffer from both solution debt that causes gaps in coverage or capabilities, as well as rampant tool sprawl that eats up budget and makes it difficult to effectively use tools.

“We recently worked with a client’s incident response team that on a day-to-day basis used seven tools that the security operations center (SOC) and engineering team had to configure, operate, and maintain, often leading to workarounds and an accumulation of technical debt in business rules and tool documentation,” says Andrew Kim, managing director and cyber strategy lead for Accenture Federal Services. “New joiners to the team spent weeks reviewing hundreds of pages of policies and procedures documentation before they were fully operational.”

Kim says that not only should CISOs be working to consistently eliminate security tool sprawl, but this is a consideration that should be elevated all the way up to the CIO. The first step in this process is to do a search inventory of tools and really analyze how they’re being used.

“Align those tools to your security operating model and rationalize the portfolio of tools. By conducting analysis of the tools, you’ll get a sense of where to focus technical debt reduction efforts,” he says, explaining that in his example the security team got executive buy-in and budgetary support by looping the CIO into the process.

Underutilization and non-configured deployed systems

Intertwined with the issue of solution debt and tool sprawl is the problem of underutilization and non-configured or poorly tuned security systems. “A lot of security debt is not because you aren’t investing in new tools but that you’re not leveraging the tools you’ve got,” says Frank Duff, security industry veteran and the chief innovation officer for Tidal Cyber.

Security teams often have tools out there that are either not being used much at all or are deploying them in a way that makes them not much use to security operations. This often happens when security teams focus on the wrong KPIs — maybe focusing on coverage percentage rather than security outcomes, according to Michalis Kamprianis, director of cybersecurity for Hexagon Manufacturing Intelligence.

“What is missing is a proper governance structure that will evaluate the security programs’ outcome based on the pre-defined criteria of risk reduction and security improvements, rather than pure numerical measurements of things that have no value,” he explains. “As an example, most projects start with a plan to cover a percentage of the environment, such as ‘We need to deploy EDR to 99% of the endpoints.’ This target can be explained, measured, and communicated to the business in an indisputable manner. Nevertheless, from the security perspective this doesn’t say anything.”

EDR is a great example, agrees Duff, who says that many security departments linger in a state of underutilization by sticking in ‘detect only mode.’ “Almost every EDR vendor comes in detect only mode because they don’t want their users to deploy a solution and immediately run into a bad user experience being locked out. So then what happens is they get left in detect mode and they’re not actually protecting you. We can’t be having that because now you’re buying the tool for one thing and it’s doing something else.”

Security teams have to be better about asking the right questions and making the right measurements about the efficacy of deployed systems and their current configuration and tuning, Kamprianis says.

Some questions to ask are ‘Does it block what it needs to block? Does it provide enough information to the incident response team to react? Does it generate an acceptable number of false positive alerts or is it too eager?’” Kamprianis says. “All these are coming from the invisible effort of configuration and operationalization of the solution.”

Additionally, Duff says that if an organization starts to identify tools that should be spurring better security outcomes but are not, the next question to ask is ‘why?’ Sometimes it could just be a matter of insufficient training or resources provided to operators so they can effectively use a tool. Other times it could be an issue of usability in the tool itself. “You should be investing in training to push your people up there so that they can actually leverage the tool,” he says. “Or maybe it’s a user interface issue at that point, you should be just getting rid of it and replacing it with an alternative. There are dimensions that you should be looking at in terms of the usability in addition to price.”

Poor detection engineering

One of the unique sources of technical debt in care and feeding of security products occurs in the niche field of detection engineering. Often the efficacy of a solution comes down to the security team’s ability to keep up on fresh tuning and intel that feeds their automated detection engines, according to Lamothe-Brassard.

“Detection engineering is often a large source of technical debt: over the years, a great detection engineering team can produce many great detections, but the reliability of those detections can start to fade as the rest of the infrastructure changes,” he says. “Great detections become less reliable over time, the authors leave the company, and the detection starts to be ignored. This leads to waste of energy and very often cost.”

He explains that there’s no easy way to combat this debt. It comes down to consistent and methodical work put into refining and keeping track of detection rules. “An emphasis on continuous testing and documentation is so critical to any detection engineering organization.”

Controls and rules debt in architecture

Finally, another less visible but very costly (and risky) source of security debt occurs in the interface between security tooling and IT infrastructure. This is the accumulation of debt around the rules that govern traffic flows and identity and access management. Poorly maintained or tracked firewall rules is a classic case of this kind of debt.

“Technical debt in firewall rules and cloud NSGs impact business application connectivity by potentially causing misconfigurations that either block legitimate traffic or allow unauthorized access,” says Chris Thomas, CRO of AlgoSec.

Role sprawl is another common scenario that contributes significantly to security debt, says Piyush Pandey, CEO at Pathlock. “Over time, organizations accumulate overly complex and outdated role definitions, resulting in orphaned accounts and unused entitlements,” Pandey says. “These unused roles and accounts increase the attack surface, providing potential entry points for attackers and complicating compliance efforts.”

This debt tends to accumulate when organizations lack effective change management policies, practices, and automation. “Effective change management and collaboration between teams ensure that connectivity requirements are accurately reflected in security policies, minimizing disruptions and maintaining application performance,” Thomas says.

Long-term strategies for effectively managing security technical debt

Ultimately, technical debt in the security stack accumulates fastest when CISOs lack a clear security tool strategy or roadmap to guide where, when, and how security tools are used. Getting this kind of north star set will take input from the people out in the field, but also a vision from security leadership and an ability to loop in budgetary and other business concerns, says Kim of Accenture.

“While it’s important for security engineers to have input into what tools they use, the business value and budgetary trade-offs must also be considered,” he says. “Security leaders should engage with IT investment review boards.”

Kamprianis says CISOs have got to be methodical and ruthless about their process of picking security solution winners and losers, which starts first by defining and establishing a security architecture.

“Define what security improvements you expect from every piece of your stack. Identify non-performing solutions and be decisive to identify if they can be quickly fixed to deliver or terminate them. Identify points of inconsistency and try to bring things to the same standard. Find the outliers (down) and either bring them up to your standard or terminate them. Finally, find out overlapping solutions and choose which ones you will keep and which ones to terminate.”

Using the savings from those terminations to renegotiate contracts and picking more modern and appropriate solutions will ensure that CISOs keep up to current demands without breaking the bank — or the faith with the CEO and board.