Follow Me!

Sunday, February 26, 2017

An Excellent Example of Incident Response

https://blog.cloudflare.com/incident-report-on-memory-leak-caused-by-cloudflare-parser-bug/

This is an excellent example of how to respond to a software security incident.   Having worked at Microsoft for 5 years, I had the amazing experience of watching the Microsoft Security Response Center (MSRC) pioneer and then execute on Incident Response when software vulnerabilities are discovered, either by researchers or by attackers via zero days.

What struck me is not necessarily the analysis conducted about how the bug worked, but in the response procedures that were obviously defined and trained in the organization prior to this happening.  Admittedly, I don't know a lot about Cloudflare, so it's possible that they've developed this IR process only after some initial painful failures (that is usually the case for most organizations.) Regardless,  it is clear that they moved quickly and efficiently and were able to diagnose the root cause quickly.   Of course, it helped that an excellent security researcher like Tavis Ormandy of Google Project Zero provided good initial information.  

I was especially struck by their "global kill" flag that ships with features they deploy (excluding the Server Side Exclude feature which predate that kill switch)   This allows them to stop catastrophic security breaches.  Good foresight. It's unclear at this point what kind of disruption that might cause to operations - but I think given the severity of the bug, this appears to be the right call.  

So, to recap

1.  Immediate response to responsible disclosure
2.  The ability and wherewithal to contain the incident effectively (global kill flag) and the guts to use it.
3.  The tenacity and care to start following up on all the loose ends to notify customers and clean up the residual mess

The book is still being written on this one, but based on what we've seen so far, the response by Cloudflare has been very professional.

 

No comments:

Post a Comment