“We receive over 5,000 incident reports every day.“
A CISO at a large university said this, not with pride, but with a mix of defiance and exhaustion. It’s a feeling we all know. The siren call of the SIEM, the flood of alerts from the IDS, the endless stream of data that promises clarity but often delivers chaos. We invest in processes, we build playbooks, which we call Incident Response Playbooks, we design what one academic researcher calls “scripts” for our teams to follow. But what happens when the script can’t keep up with the action?
A fascinating ethnographic study in the journal Science, Technology, & Human Values dives deep into this very question. Researcher Ashwin Jacob Mathew embedded himself with security teams to understand the gap between our structured plans and the messy, unpredictable reality of cybersecurity incident management.
The findings don’t reveal a new vulnerability or a silver-bullet technology. They reveal a truth about our work: our greatest strength isn’t our scripts, but our teams’ ability to improvise when they inevitably fail.
The Myth of the “Stable Script”
Why Your Incident Response Script is Inherently Unstable (and How Teams Thrive Anyway)
The article frames traditional incident response plans as “scripts”, a set of instructions designed to create a stable, predictable outcome. In a perfect world, an alert comes in, the analyst follows the script, and the incident is resolved.
But cybersecurity isn’t a perfect world. It’s a world defined by what the author calls “adversarial instability.” Unlike a faulty piece of machinery, our systems aren’t just breaking down; they are being actively broken by intelligent adversaries whose entire goal is to subvert our script. Attackers aren’t playing by our rules. They are re-writing them by exploiting “unscripted features” – vulnerabilities we never even knew existed. This fundamental conflict makes any security script inherently unstable.
The CISO with 5,000 daily alerts knows this. His team can’t possibly follow a rigid playbook for every single event. The sheer volume forces them to triage, to prioritize, and to rely on intuition. As one of his senior analysts put it, “I have no idea what to expect every day… the rest of the day I leave open because everything and anything will come by and you just kind of have to take it and go with it.“
Thriving in the Chaos: When “Unscripted Practices” Take Over
So, if the script is unstable, how does anything get done? The research points to the emergence of “unscripted practices”: the fluid, collaborative, and improvised actions that define elite security work.
The article highlights a perfect case study: a seemingly low-risk incident involving Mirai-infected cellular modems on a campus payment system. There was no playbook for this. The response required a fascinating dance of improvisation:
Collaborative Diagnosis:
The lead analyst, a former history teacher, leveraged his expertise in payment systems while relying on colleagues for their deeper technical knowledge of botnets.
Cross-Boundary Negotiation:
The team had to work with the external cellphone company, sharing knowledge to diagnose an issue in a system they didn’t own or control.
Collective Sense-Making:
They collectively assessed the risk, deciding the infection was confined and didn’t pose an immediate data threat, allowing them to keep the payment systems running while a patch was developed.
This wasn’t chaos; it was a highly coordinated, unscripted performance. The team’s strength was not in following a procedure, but in their ability to renegotiate relationships and build a solution in real-time. This is also where knowledge is truly built: not in a classroom, but in the trenches. As the analyst noted, “Thankfully, work has brought [the threats] to me.“
The Hidden Knowledge Network
This improvisational skill is fueled by something our org charts can’t capture: informal knowledge networks. The study found that analysts rely heavily on trusted, private groups of peers outside their own organizations.
One analyst described how he shares threat intelligence on a closed Slack channel with researchers from Cisco, Malwarebytes, and other independent experts. These are the backchannels where real, actionable intelligence is exchanged, far from the noise of public feeds. They are malleable, trust-based networks built on reputation, and they are essential for defending against novel threats.
A CISO’s Read: Insights and Soft Spots
As leaders, this research offers a powerful validation of what we often see anecdotally. It gives us a framework for articulating the value of our teams beyond tickets closed and procedures followed. The key insight is that we must lead for resilience, not just compliance. Our job is to create the conditions; the trust, the psychological safety, the collaborative tools, for these “unscripted practices” to flourish.
However, the research has its soft spots from a CISO’s perspective:
-
- It’s Descriptive, Not Prescriptive: It masterfully describes the problem but offers no clear guidance on how to manage it, especially when updating or creating new Incident Response Playbooks. How do we measure the performance of an “unscripted” team? How do we scale this improvisational excellence without introducing unacceptable risk or analyst burnout?
-
- The Academic Environment: The research was conducted at universities, which are notoriously complex and decentralized – “cesspools” where “new germs” breed, as one analyst joked. A CISO in a tightly regulated financial or healthcare environment might find it harder to justify such fluid, “unscripted” work to auditors who demand rigid, documented processes.
-
- It Romanticizes Improvisation: While celebrating the heroics of the analyst, the article doesn’t fully grapple with the downsides. Relying on improvisation can lead to inconsistent outcomes, single points of failure (if a key expert leaves), and immense pressure on individuals.
Yet despite these gaps, the article is a crucial read. It reminds us that our Incident Response Playbooks are the starting point, not the destination. The real work of security happens in the space between the lines, in the moments of uncertainty where our teams must improvise, collaborate, and learn together.
For Further Reading:
The insights in this post are based on the academic article: Mathew, A. J. (2024). “Unscripted Practices for Uncertain Events: Organizational Problems in Cybersecurity Incident Management.” Science, Technology, & Human Values, 49(4), 827-850.
