{"id":858,"date":"2024-04-22T01:44:20","date_gmt":"2024-04-22T01:44:20","guid":{"rendered":"https:\/\/fde.cat\/index.php\/2024\/04\/22\/sre-weekly-issue-421\/"},"modified":"2024-04-22T01:44:20","modified_gmt":"2024-04-22T01:44:20","slug":"sre-weekly-issue-421","status":"publish","type":"post","link":"https:\/\/fde.cat\/index.php\/2024\/04\/22\/sre-weekly-issue-421\/","title":{"rendered":"SRE Weekly Issue #421"},"content":{"rendered":"

View on sreweekly.com<\/a><\/p>\n

Last week, I mistakenly attributed [an article](https:\/\/www.paigerduty.com\/sre-biggest-problem\/) to PagerDuty. Actually, it was by Paige Cruz<\/strong>, whose clever blog name I didn\u2019t pay anywhere near close enough attention to! Thanks to several readers that nudged me gently about my error.<\/p>\n

\n

A message from our sponsor, FireHydrant<\/a>:<\/h2>\n

FireHydrant is now AI-powered for faster, smarter incidents! Power up your incidents with auto-generated real-time summaries, retrospectives, and status page updates.
\nhttps:\/\/firehydrant.com\/blog\/ai-for-incident-management-is-here\/<\/a><\/p>\n<\/div>\n

\n
\n
\n
The problem with invariants is that they change over time<\/a><\/div>\n
\n

If you\u2019ve been in this business long enough, you\u2019ve almost certainly run into an incident where one of the contributors was an implicit invariant that was violated by a new change.<\/p>\n

Easily the majority of incidents I\u2019ve been in.<\/p>\n

\u00a0\u00a0Lorin Hochstein<\/small><\/p>\n<\/div>\n<\/div>\n

\n
The TwinSLO Proposal<\/a><\/div>\n
\n

This article is about trying to solve for this problem:<\/p>\n

a potentially significant number of customers or queries can be affected by an outage and this won\u2019t trigger an SLO violation.<\/p>\n

\u00a0\u00a0Niall Murphy
<\/small><\/p>\n<\/div>\n<\/div>\n

\n
An Anonymous Complaint\/Dr. Poston\u2019s Response<\/a><\/div>\n
\n

A surgeon struggles with the difficulties in building a culture of retrospectives and introspection in their surgical team, by running a fascinating retro on himself in this blog post.<\/p>\n

\u00a0\u00a0Robert Poston, MD<\/small><\/p>\n<\/div>\n<\/div>\n

\n
Incidents and the requirement of slowing down<\/a><\/div>\n
\n

An argument for buying yourself time to slow down and make decisions carefully, as a way of ultimately speeding up incident resolution.<\/p>\n

\u00a0\u00a0Shayon Mukherjee<\/small><\/p>\n<\/div>\n<\/div>\n

\n
Build your own role-playing game: the business continuity plan drill<\/a><\/div>\n
\n

Disasters threatening a business\u2019 ability to operate core functions don\u2019t occur that often (phew!), but we do want to ensure we are prepared to keep our business running if they do. To practice disaster response skills, we run business continuity drills, and you can too with our 10-step plan!<\/p>\n

\u00a0\u00a0Janna Brummel \u2014 WeTransfer<\/small><\/p>\n<\/div>\n<\/div>\n

\n
Availability Archetypes<\/a><\/div>\n
\n

How people think about reliability varies between companies. Which of the four different perspectives laid out int his article does your company fit into, if any?<\/p>\n

\u00a0\u00a0Ross Brodbeck<\/small><\/p>\n<\/div>\n<\/div>\n

\n
eu1 ingest and UI down<\/a><\/div>\n
\n

Honeycomb posted this followup on their April 9 outage, explaining what went wrong and how they\u2019re responding.<\/p>\n

\u00a0\u00a0Honeycomb<\/small><\/p>\n

\u00a0\u00a0Full disclosure: Honeycomb is my employer.<\/em><\/small><\/p>\n<\/div>\n<\/div>\n

\n
For an SRE, relationships and communication matter most: advice from SRE\u2019s<\/a><\/div>\n
\n

The author of this article posed a question on r\/sre<\/a>:<\/p>\n

What matters most for your success as an SRE?<\/p>\n

They share a summary of the answers they got, with their commentary.<\/p>\n

\u00a0\u00a0No\u010dnica Mellifera \u2014 Checkly<\/small><\/p>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n

SRE WEEKLY<\/p>","protected":false},"excerpt":{"rendered":"

View on sreweekly.com Last week, I mistakenly attributed [an article](https:\/\/www.paigerduty.com\/sre-biggest-problem\/) to PagerDuty. Actually, it was by Paige Cruz, whose clever blog name I didn\u2019t pay anywhere near close enough attention to! Thanks to several readers that nudged me gently about my error. A message from our sponsor, FireHydrant: FireHydrant is now AI-powered for faster, smarter… Continue reading SRE Weekly Issue #421<\/span><\/a><\/p>\n","protected":false},"author":0,"featured_media":0,"comment_status":"","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"spay_email":"","footnotes":""},"categories":[8],"tags":[],"class_list":["post-858","post","type-post","status-publish","format-standard","hentry","category-sre","entry"],"jetpack_featured_media_url":"","jetpack-related-posts":[{"id":855,"url":"https:\/\/fde.cat\/index.php\/2024\/04\/15\/sre-weekly-issue-420\/","url_meta":{"origin":858,"position":0},"title":"SRE Weekly Issue #420","date":"April 15, 2024","format":false,"excerpt":"View on sreweekly.com A message from our sponsor, FireHydrant: FireHydrant is now AI-powered for faster, smarter incidents! Power up your incidents with auto-generated real-time summaries, retrospectives, and status page updates. https:\/\/firehydrant.com\/blog\/ai-for-incident-management-is-here\/ 1.0 Launch Retrospective The game Last Epoch launched in February, and they had a rocky start. This huge retrospective\u2026","rel":"","context":"In "SRE"","img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":835,"url":"https:\/\/fde.cat\/index.php\/2024\/03\/11\/sre-weekly-issue-415\/","url_meta":{"origin":858,"position":1},"title":"SRE Weekly Issue #415","date":"March 11, 2024","format":false,"excerpt":"View on sreweekly.com A message from our sponsor, FireHydrant: Join FireHydrant and talk shop with your DevOps peers on March 28! You\u2019ll gain a better understanding of what makes a fatigue-free on-call culture and how to implement practices to improve yours at this free, virtual roundtable. https:\/\/app.livestorm.co\/firehydrant\/better-incidents-spring-bonfire-secrets-to-fatigue-free-on-call-in-2024 The Wrong Way\u2026","rel":"","context":"In "SRE"","img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":867,"url":"https:\/\/fde.cat\/index.php\/2024\/05\/20\/sre-weekly-issue-425\/","url_meta":{"origin":858,"position":2},"title":"SRE Weekly Issue #425","date":"May 20, 2024","format":false,"excerpt":"View on sreweekly.com A message from our sponsor, FireHydrant: FireHydrant is now AI-powered for faster, smarter incidents! Power up your incidents with auto-generated real-time summaries, retrospectives, and status page updates. https:\/\/firehydrant.com\/blog\/ai-for-incident-management-is-here\/ Presenting to Engineering Leadership Great practical advice for how to present reliability problems (and your proposed solutions) to e-staff.\u2026","rel":"","context":"In "SRE"","img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":663,"url":"https:\/\/fde.cat\/index.php\/2022\/12\/19\/sre-weekly-issue-352\/","url_meta":{"origin":858,"position":3},"title":"SRE Weekly Issue #352","date":"December 19, 2022","format":false,"excerpt":"View on sreweekly.com A message from our sponsor, Rootly: Manage incidents directly from Slack with Rootly\u00a0\ud83d\ude92. Rootly automates manual tasks like creating an incident channel, Jira ticket and Zoom rooms, inviting responders, creating statuspage updates, postmortem timelines and more. Want to see why companies like Canva and Grammarly love us?:\u2026","rel":"","context":"In "SRE"","img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":798,"url":"https:\/\/fde.cat\/index.php\/2023\/12\/04\/sre-weekly-issue-401\/","url_meta":{"origin":858,"position":4},"title":"SRE Weekly Issue #401","date":"December 4, 2023","format":false,"excerpt":"View on sreweekly.com A message from our sponsor, FireHydrant: Join FireHydrant Dec.14 for a conversation about on-call culture and its effect on engineering organizations, featuring special guests from Outreach and Udemy. Gain a better understanding of what makes excellent on-call culture and how to implement practices to improve yours. https:\/\/app.livestorm.co\/firehydrant\/better-incidents-winter-bonfire-inside-on-call?type=detailed\u2026","rel":"","context":"In "SRE"","img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":864,"url":"https:\/\/fde.cat\/index.php\/2024\/05\/13\/sre-weekly-issue-424\/","url_meta":{"origin":858,"position":5},"title":"SRE Weekly Issue #424","date":"May 13, 2024","format":false,"excerpt":"View on sreweekly.com A message from our sponsor, FireHydrant: FireHydrant is now AI-powered for faster, smarter incidents! Power up your incidents with auto-generated real-time summaries, retrospectives, and status page updates. https:\/\/firehydrant.com\/blog\/ai-for-incident-management-is-here\/ My Availability Investment Playbook Here\u2019s an ultra-practical guide to pushing for reliability investments at your company, formatted as a\u2026","rel":"","context":"In "SRE"","img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]}],"_links":{"self":[{"href":"https:\/\/fde.cat\/index.php\/wp-json\/wp\/v2\/posts\/858","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/fde.cat\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/fde.cat\/index.php\/wp-json\/wp\/v2\/types\/post"}],"replies":[{"embeddable":true,"href":"https:\/\/fde.cat\/index.php\/wp-json\/wp\/v2\/comments?post=858"}],"version-history":[{"count":0,"href":"https:\/\/fde.cat\/index.php\/wp-json\/wp\/v2\/posts\/858\/revisions"}],"wp:attachment":[{"href":"https:\/\/fde.cat\/index.php\/wp-json\/wp\/v2\/media?parent=858"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/fde.cat\/index.php\/wp-json\/wp\/v2\/categories?post=858"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/fde.cat\/index.php\/wp-json\/wp\/v2\/tags?post=858"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}