{"id":253,"date":"2021-08-31T14:40:46","date_gmt":"2021-08-31T14:40:46","guid":{"rendered":"https:\/\/fde.cat\/?p=253"},"modified":"2021-08-31T14:40:46","modified_gmt":"2021-08-31T14:40:46","slug":"sre-weekly-issue-254","status":"publish","type":"post","link":"https:\/\/fde.cat\/index.php\/2021\/08\/31\/sre-weekly-issue-254\/","title":{"rendered":"SRE Weekly Issue #254"},"content":{"rendered":"

View on sreweekly.com<\/a><\/p>\n

\n

A message from our sponsor, StackHawk:<\/h2>\n

Need to run a standalone Kotlin app as a fat jar in a Gradle project? Check out how we handled that!
\nhttp:\/\/sthwk.com\/kotlin-with-gradle<\/a><\/p>\n<\/div>\n

Articles<\/h2>\n
\n
Coinbase Incident Post Mortem: January 6\u20137, 2021<\/a><\/div>\n
\n

This one\u2019s juicy. At one point, the front-end was blocked up, so the back-end saw less traffic and scaled down. Then when the traffic came flooding back, the back-end was ill-prepared. We can all learn from this.<\/p>\n

Coinbase<\/small><\/p>\n<\/div>\n<\/div>\n

\n
Soar: Simulation for Observability, reliAbility, and secuRity<\/a><\/div>\n
\n

Cloudflare has what amounts to a sophisticated staging environment for testing new code.<\/p>\n

Yan Zhai \u2014 Cloudflare<\/small><\/p>\n<\/div>\n<\/div>\n

\n
Failing to make progress under excess request load<\/a><\/div>\n
\n

Sometimes rolling back doesn\u2019t actually get you back to a good state, especially when there\u2019s pent-up demand.<\/p>\n

Rachel By the Bay<\/small><\/p>\n<\/div>\n<\/div>\n

\n
Google Cloud Issue Summary \u2014 Google Meet \u2014 2021-01-08<\/a><\/div>\n
\n

Here\u2019s Google\u2019s follow-up on a Google Meet outage earlier this month.<\/p>\n

Google<\/small><\/p>\n<\/div>\n<\/div>\n

\n
The Next Gen Database Servers Powering Let\u2019s Encrypt <\/a><\/div>\n
\n

Those are some seriously big database servers.<\/p>\n

Josh Aas and James Renken \u2014 Let\u2019s Encrypt<\/small><\/p>\n<\/div>\n<\/div>\n

\n
Incident Management in 2021: from Basics to Best Practices<\/a><\/div>\n
\n

A great general overview of all aspects of incident response, including definitions and best practices.<\/p>\n

Better Uptime<\/small><\/p>\n<\/div>\n<\/div>\n

\n
Using GPT-3 for plain language incident root cause from logs<\/a><\/div>\n
\n

Check out what happens when you unleash a generalized language model AI on some log messages related to an incident.<\/p>\n

Larry Lancaster \u2014 Zebrium<\/small><\/p>\n<\/div>\n<\/div>\n

\n
Taming Operational Load with VMware CRE<\/a><\/div>\n
\n

The CRE team at VMware undertook a project to find and reduce toil. Note that \u201cwith VMware CRE\u201d does not<\/em> mean \u201cwith some product named VMware CRE\u2122\u201d.<\/p>\n

Gustavo Franco \u2014 VMware<\/small><\/p>\n<\/div>\n<\/div>\n

\n
Slack RCA for outage on January 4, 2021<\/a><\/div>\n
\n

This is Slack\u2019s RCA for their outage earlier this month<\/a>. This is a great example of a complex incident with many contributing factors \u2014 certainly no single \u201croot cause\u201d here.<\/p>\n

Slack<\/small><\/p>\n<\/div>\n<\/div>\n

Outages<\/h2>\n