{"id":261,"date":"2021-08-31T14:40:46","date_gmt":"2021-08-31T14:40:46","guid":{"rendered":"https:\/\/fde.cat\/?p=261"},"modified":"2021-08-31T14:40:46","modified_gmt":"2021-08-31T14:40:46","slug":"sre-weekly-issue-256","status":"publish","type":"post","link":"https:\/\/fde.cat\/index.php\/2021\/08\/31\/sre-weekly-issue-256\/","title":{"rendered":"SRE Weekly Issue #256"},"content":{"rendered":"

View on sreweekly.com<\/a><\/p>\n

\n

A message from our sponsor, StackHawk:<\/h2>\n

Register now for the first-ever ZAPCon taking place March 9th. The free event will focus on OWASP ZAP and application security best practices. You wont want to miss it!
\nhttp:\/\/sthwk.com\/zapcon-sre-weekly<\/a><\/p>\n<\/div>\n

Articles<\/h2>\n
\n
Slack\u2019s Outage on January 4th 2021<\/a><\/div>\n
\n

Here\u2019s a blog post from Slack giving even more information about what went wrong on January 4. Bravo, Slack, there\u2019s a lot in here for us to learn from.<\/p>\n

Laura Nolan \u2014 Slack<\/small><\/p>\n<\/div>\n<\/div>\n

\n
Zero Downtime Release: Disruption-free Load Balancing of a Multi-Billion User Website<\/a><\/div>\n
\n

This academic paper from Facebook explains how they release code without disrupting active connections, even for a small number of users.<\/p>\n

Usama Naseer, Luca Niccolini, Udip Pant, Alan Frindell, Ranjeeth Dasineni, and Theophilus A. Benson \u2014 Facebook<\/small><\/p>\n<\/div>\n<\/div>\n

\n
NOTAM for SREs<\/a><\/div>\n
\n

Another lesson we can learn from aviation: have one place where engineers can find out about temporary infrastructure changes that are important.<\/p>\n

Bill Duncan<\/small><\/p>\n<\/div>\n<\/div>\n

\n
Incident Post Mortem: January 29, 2021 [Coinbase]<\/a><\/div>\n
\n

Coinbase posted this detailed analysis of their January 29th incident.<\/p>\n

Coinbase<\/small><\/p>\n<\/div>\n<\/div>\n

\n
Council Post: How Cloud Services Platform Teams Can Drive The Adoption Of Effective SRE Practices<\/a><\/div>\n
\n

Interesting thesis: a company moving into the cloud is in a unique position to adopt SRE practices \u2014 and better situated than cloud-first companies.<\/p>\n

Tina Huang (CTO, Transposit) \u2014 Forbes<\/small><\/p>\n<\/div>\n<\/div>\n

\n
\u201cI\u2019m Just Doing my Job,\u201d An SRE Myth<\/a><\/div>\n
\n

We need to push past surface-level mitigation of an incident and really dig in and learn.<\/p>\n

Darrell Pappa \u2014 Blameless<\/small><\/p>\n<\/div>\n<\/div>\n

\n
GitHub Availability Report: January 2021<\/a><\/div>\n
\n

GitHub\u2019s database failed in a manner that wasn\u2019t detected by their automated failover system.<\/p>\n

Keith Ballinger \u2014 GitHub<\/small><\/p>\n<\/div>\n<\/div>\n

\n
Open source update: School of SRE<\/a><\/div>\n
\n

LinkedIn published their SRE training documentation in the form of a full curriculum covering a range of topics.<\/p>\n

Akbar KM and Kalyanasundaram Somasundaram \u2014 LinkedIn<\/small><\/p>\n<\/div>\n<\/div>\n

\n
Push some big numbers through your system and look for bugs<\/a><\/div>\n
\n

Your code may be designed to handle 64-bit integers, but what if a library (such as a JSON decoder) converts them to floating point numbers?<\/p>\n

rachelbythebay<\/small><\/p>\n<\/div>\n<\/div>\n

Outages<\/h2>\n