In a recent engagement, where I was helping an organization approach and execute more effective threat models, I was reminded of how difficult it can be to “threat model yourself”. Consider this: the idea of using a royal taster to ensure the king’s food isn’t poisoned seems like a great one…until the royal taster is the assassin. When an org’s security personnel carry out threat modeling exercises, they tend to make unconscious assumptions about the efficacy of their own security controls. This is dangerous.
For example, the security team are likely to focus on the known public interfaces that people are “supposed” to use, but overlook (or underestimate) how the misuse of the service could turn other elements into viable, maliciously manipulated, interfaces in their own right. The key is to approach the threat model from the perspective of the attacker, not from the perspective of the security architect/developer. It’s easier said than done.
A helpful illustration is the video above, where a developer watches helplessly as QA interact with a product in all the “wrong” ways. But it illustrates perfectly what can happen in threat modeling (or the identification and assessment of risk, in general): sometimes we forget that we cannot dictate the terms of an attacker’s engagement. They will put the rectangular block in the square hole, on purpose, just to see if you thought of that scenario.
The disconcerted developer in the video failed to anticipate this outcome, because that’s not how it is “supposed” to be used. The same thing happens in threat modeling, and in security in general, when we look at the world through our lens instead of the attacker’s.
Here’s a practical example: I’ve seen this happen in scenarios where only private endpoints were used in Azure environments. Since there are no public interfaces, there’s no way for traffic to reach the resources on the network from the internet. That means the resources are safe, and the network is trusted…right?
But this says nothing about the ability of an attacker to create a public interface later (after compromising a privileged account), or someone attacking it from the inside, because they bypassed this control entirely by compromising an endpoint. Because the lens was wrong, the assumptions were never identified or challenged; what was obvious from the attacker’s perspective was overlooked by the expert. As Oswald Chambers said, “an unguarded strength is a double weakness.”
Peter Drucker, Simon Sinek, and Brené Brown all talk (in their own way) about being “the dumbest person in the room”, i.e. learning to ask questions as someone who is, or is pretending to be, completely unfamiliar with the circumstance at hand. Frustratingly, I find that the more experienced I become, the more prone I am to making assumptions about what I (think) I know, and the more intentional I have to be about identifying those assumptions and biases.