Safely Deploying a 100K line Envoy YAML Configuration to Production

Jyoti Mahapatra, Lisa Lu at KubeCon + CloudNativeCon North America 2020

Have you ever caused a production incident due to an Envoy misconfiguration? You’re not alone! This talk is about how Lyft has built guardrails to prevent such failures. The presenters will share their experience operating Envoy configurations at scale. They will explore the challenges around handling constantly changing cluster and routing configurations & the tools used to guarantee accuracy and consistency in those changes. These tools empower service owners less familiar with Envoy to make configuration changes independently and quickly without approval barriers. This talk will introduce the audience to various Envoy configuration testing strategies: •Validating behavior for thousands of routes to avoid blackholing traffic •Auditing & safely removing unused routes and clusters •Safely deprecating fields between Envoy versions •Validating Envoy’s static & realtime configurations

https://sched.co/ekEq