Measuring End-to-End Security Engineering
Davit Baghdasaryan, Garrett Held at AppSec USA 2017
This talk will introduce a new approach to SDL. At Twilio we call it End to End Security Engineering. It’s End-to-End because it covers the full product lifecycle, from Security Design to Monitoring and gives the ability to measure the state of security at each point.
The approach defines a ‘perfect secure system’ and produces metrics which tell us where we are relative to that perfect system.
The final state of the product’s security and risk depends on ‘collective understanding’ of threats and attacks as well as investments in building controls, tests and detections. Then we measure and adjust them to improve their effectiveness.
A perfect secure system is one for which:
- All products and features are threat modeled
- All possible attacks are identified
- All attacks are being monitored for
- All attacks are protected with controls
- All controls are covered with tests which check validity of implementation
- All controls are shown to be effective
To measure this we keep track of the following metrics:
- % of threat modeled products/features
- experience of threat modelers
- % of attacks being monitored
- % of attacks with implemented controls
- % of controls covered by full tests
- # of successful attacks
- # of prevented attacks after control has been put in place
Speakers
Davit Baghdasaryan
Principal Security Engineer, Twilio, Inc
Davit is a Principal Security Engineer at Twilio focusing on Product Security. He has been building security systems for more than 10 years, ranging from fingerprint sensors, mobile, web apps to backend systems. Davit is also one of the original designers of FIDO UAF protocols.
Garrett Held
Head of Product Security, Twilio
Garrett is the Head of Product Security at Twilio. He has been working in Information Security for more than 12 years as a Managing Application Security Consultant, Instructor, Principal Product Security Engineer, and Director of Security.
The approach defines a ‘perfect secure system’ and produces metrics which tell us where we are relative to that perfect system.
The final state of the product’s security and risk depends on ‘collective understanding’ of threats and attacks as well as investments in building controls, tests and detections. Then we measure and adjust them to improve their effectiveness.
A perfect secure system is one for which:
- All products and features are threat modeled
- All possible attacks are identified
- All attacks are being monitored for
- All attacks are protected with controls
- All controls are covered with tests which check validity of implementation
- All controls are shown to be effective
To measure this we keep track of the following metrics:
- % of threat modeled products/features
- experience of threat modelers
- % of attacks being monitored
- % of attacks with implemented controls
- % of controls covered by full tests
- # of successful attacks
- # of prevented attacks after control has been put in place
Speakers
Davit Baghdasaryan
Principal Security Engineer, Twilio, Inc
Davit is a Principal Security Engineer at Twilio focusing on Product Security. He has been building security systems for more than 10 years, ranging from fingerprint sensors, mobile, web apps to backend systems. Davit is also one of the original designers of FIDO UAF protocols.
Garrett Held
Head of Product Security, Twilio
Garrett is the Head of Product Security at Twilio. He has been working in Information Security for more than 12 years as a Managing Application Security Consultant, Instructor, Principal Product Security Engineer, and Director of Security.