How Should Security Be Architected for Human Error?

#1757852: How Should Security Be Architected for Human Error?

Description:	We've all internalized that humans are the weak link in cybersecurity. Why do we not account for that when we're building our cybersecurity programs? Humans and automation checking each other The instinct to automate humans out of the loop misses the point. Pete Apple, formerly of Microsoft, said the real work is in the combination: "it's the combo of human/automation/AI that needs further development," since "humans on their own will occasionally foul up" and "automation/AI on its own unchecked certainly has." The answer, he said, is investing in a human and automation pairing "which audits each other more effectively, versus just being an outsource of activities to 'the agent.'" Kirk P. of Duolingo took that same logic to phishing simulations, arguing they're best used as measurement rather than punishment. "What the phishing exercise gets you is one strong measurement in an employee's risk score," he said. "We turn up other controls to accommodate, not shame that behavior." Assume the click happens Resilience planning starts by assuming prevention will fail. Donald McFarlane of Microsoft framed it as a mindset shift, saying, "resilience is an operational capability, not a product feature." His model starts with the click already happening. "Engineer the organization so it doesn't become a business crisis," he said. "Segment aggressively. Minimize privilege. Detect quickly. Contain automatically. Recover rapidly," then test those assumptions through "chaos engineering, adversarial emulation, tabletop exercises, and recovery drills." Sean Cassidy, CISO at Plaid, pushed the containment piece further with a specific mechanism: blast radius rate limiting. "Why do we let people send 50,000 emails or delete millions of rows from a database table before we notice and stop them?" he asked. People should hit "speed bumps when doing things far outside the norm," the same way fraud teams already do, "and this is extra important with agents doing god knows what at machine speed." Catch it in the moment Real-time intervention may do more than upfront prevention ever could. Britt Bohn of Ent argued that stopping and correcting a mistake as it happens beats trying to block it in advance. "If a mistake is made, intervene and stop, and more importantly educate on the fly," he said, adding that a click worth stopping should get stopped at the endpoint in real time either way. That kind of in-the-moment correction, he said, "makes muscle memory stronger" and stronger muscle memory "builds resilience." Dan Baeta, formerly of Coinbase, suggested that muscle memory should be assessed earlier, during hiring. "We already assess job aptitude during interviews, so why not evaluate basic security awareness as well?" he asked, especially for roles with elevated access. Take the manual step out entirely T.C. Niedzialkowski, former head of security at Opendoor, argued the most durable fix removes human error from the process rather than managing around it. Infrastructure as code and GitOps, he said, take "manual processes, proceduralize, document, and version control" them, and the payoff compounds when those changes run "through automated testing and deployment to gain assurance before making changes in production."
More info:	https://www.linkedin.com/pulse/how-should-security-architected-human-error-cisoseries-kwuac/

Description:

We've all internalized that humans are the weak link in cybersecurity. Why do we not account for that when we're building our cybersecurity programs?

Humans and automation checking each other
The instinct to automate humans out of the loop misses the point. Pete Apple, formerly of Microsoft, said the real work is in the combination: "it's the combo of human/automation/AI that needs further development," since "humans on their own will occasionally foul up" and "automation/AI on its own unchecked certainly has." The answer, he said, is investing in a human and automation pairing "which audits each other more effectively, versus just being an outsource of activities to 'the agent.'" Kirk P. of Duolingo took that same logic to phishing simulations, arguing they're best used as measurement rather than punishment. "What the phishing exercise gets you is one strong measurement in an employee's risk score," he said. "We turn up other controls to accommodate, not shame that behavior."

Assume the click happens
Resilience planning starts by assuming prevention will fail. Donald McFarlane of Microsoft framed it as a mindset shift, saying, "resilience is an operational capability, not a product feature." His model starts with the click already happening. "Engineer the organization so it doesn't become a business crisis," he said. "Segment aggressively. Minimize privilege. Detect quickly. Contain automatically. Recover rapidly," then test those assumptions through "chaos engineering, adversarial emulation, tabletop exercises, and recovery drills." Sean Cassidy, CISO at Plaid, pushed the containment piece further with a specific mechanism: blast radius rate limiting. "Why do we let people send 50,000 emails or delete millions of rows from a database table before we notice and stop them?" he asked. People should hit "speed bumps when doing things far outside the norm," the same way fraud teams already do, "and this is extra important with agents doing god knows what at machine speed."

Catch it in the moment
Real-time intervention may do more than upfront prevention ever could. Britt Bohn of Ent argued that stopping and correcting a mistake as it happens beats trying to block it in advance. "If a mistake is made, intervene and stop, and more importantly educate on the fly," he said, adding that a click worth stopping should get stopped at the endpoint in real time either way. That kind of in-the-moment correction, he said, "makes muscle memory stronger" and stronger muscle memory "builds resilience." Dan Baeta, formerly of Coinbase, suggested that muscle memory should be assessed earlier, during hiring. "We already assess job aptitude during interviews, so why not evaluate basic security awareness as well?" he asked, especially for roles with elevated access.

Take the manual step out entirely
T.C. Niedzialkowski, former head of security at Opendoor, argued the most durable fix removes human error from the process rather than managing around it. Infrastructure as code and GitOps, he said, take "manual processes, proceduralize, document, and version control" them, and the payoff compounds when those changes run "through automated testing and deployment to gain assurance before making changes in production."

More info:

https://www.linkedin.com/pulse/how-should-security-architected-human-error-cisoseries-kwuac/

Date added	Oct. 15, 2026, 10:15 p.m.
Source	Linkedin
Subjects	PodCasts / Webcast / Webinar / eSummit / Virtual Event etc. Security Management/Strategic Security/ROI/ROSI - CISO and Higher Level