{"id":564,"date":"2022-04-18T13:03:27","date_gmt":"2022-04-18T13:03:27","guid":{"rendered":"https:\/\/fde.cat\/index.php\/2022\/04\/18\/sre-weekly-issue-318\/"},"modified":"2022-04-18T13:03:27","modified_gmt":"2022-04-18T13:03:27","slug":"sre-weekly-issue-318","status":"publish","type":"post","link":"https:\/\/fde.cat\/index.php\/2022\/04\/18\/sre-weekly-issue-318\/","title":{"rendered":"SRE Weekly Issue #318"},"content":{"rendered":"<p><a href=\"https:\/\/sreweekly.com\/sre-weekly-issue-318\/\" title=\"Permalink to SRE Weekly Issue #318\" class=\"email_only\">View on sreweekly.com<\/a><\/p>\n<div class=\"sreweekly-sponsor-message\">\n<h2>A message from our sponsor, <a href=\"https:\/\/rootly.com\/demo\/?utm_source=sreweekly\">Rootly<\/a>:<\/h2>\n<p>Manage incidents directly from Slack with Rootly \ud83d\ude92. Automate manual admin tasks like creating incident channel, Jira and Zoom, paging and adding responders, postmortem timeline, setting up reminders, and more. Book a demo (+ get a snazzy Rootly lego set):<br \/>\n<a href=\"https:\/\/rootly.com\/demo\/\">https:\/\/rootly.com\/demo\/<\/a><\/p>\n<\/div>\n<h2>Articles<\/h2>\n<div class=\"wp-block-group\">\n<div class=\"wp-block-group__inner-container\">\n<div class=\"sreweekly-entry\">\n<div class=\"sreweekly-title\"><a href=\"https:\/\/ferd.ca\/errors-are-constructed-not-discovered.html\" target=\"_blank\" rel=\"noopener\">Errors are constructed, not discovered<\/a><\/div>\n<div class=\"sreweekly-description\">\n<p>This talk summary explores the concept that \u201cerror\u201d is a concept applied to an event from the outside, rather than a simple fact.  What can this tell us about our after-incident investigation process?<\/p>\n<p>\u00a0\u00a0Fred Hebert<\/p>\n<\/div>\n<\/div>\n<div class=\"sreweekly-entry\">\n<div class=\"sreweekly-title\"><a href=\"https:\/\/blog.cloudflare.com\/pipefail-how-a-missing-shell-option-slowed-cloudflare-down\/\" target=\"_blank\" rel=\"noopener\">PIPEFAIL: How a missing shell option slowed Cloudflare down<\/a><\/div>\n<div class=\"sreweekly-description\">\n<p>Here\u2019s a deep dive into a performance degradation in Cloudflare last December that was related to missing error handling in a shell script.<\/p>\n<p>\u00a0\u00a0Alex Forster \u2014 Cloudflare<\/p>\n<\/div>\n<\/div>\n<div class=\"sreweekly-entry\">\n<div class=\"sreweekly-title\"><a href=\"https:\/\/newsletter.pragmaticengineer.com\/p\/scoop-atlassian?s=r\" target=\"_blank\" rel=\"noopener\">The Scoop: Inside the Longest Atlassian Outage of All Time<\/a><\/div>\n<div class=\"sreweekly-description\">\n<p>Atlassian is having a tough time.  It seems as if they deleted a few hundred customers\u2019 data and have to pull it out of their backups one at a time.<\/p>\n<p>Here\u2019s <a href=\"https:\/\/thenewstack.io\/the-atlassian-outage-just-keeps-going-and-going-and\/\">another article<\/a> about the outage (Steven J. Vaughan-Nichols \u2014 The New Stack).<\/p>\n<p>\u00a0\u00a0Gergely Orosz \u2014 Pragmatic Engineer<\/p>\n<\/div>\n<\/div>\n<div class=\"sreweekly-entry\">\n<div class=\"sreweekly-title\"><a href=\"https:\/\/ably.com\/blog\/message-durability-quality-of-service-distributed-system\" target=\"_blank\" rel=\"noopener\">Message durability and quality of service across a large-scale distributed system<\/a><\/div>\n<div class=\"sreweekly-description\">\n<p>Cool trick: their client library can fall back to a backup domain if DNS ably.io fails.<\/p>\n<p>\u00a0\u00a0Jo Stichbury \u2014 Ably<\/p>\n<\/div>\n<\/div>\n<div class=\"sreweekly-entry\">\n<div class=\"sreweekly-title\"><a href=\"https:\/\/www.datadoghq.com\/blog\/engineering\/grpc-dns-and-load-balancing-incident\/\" target=\"_blank\" rel=\"noopener\">It\u2019s always DNS . . . except when it\u2019s not: A deep dive through gRPC, Kubernetes, and AWS networking<\/a><\/div>\n<div class=\"sreweekly-description\">\n<p>It still wasn\u2019t quite DNS, it was an interesting situation with the Linux kernel\u2019s martian packet detection algorithm.<\/p>\n<p>\u00a0\u00a0Laurent Bernaille and David Lentz \u2014 DataDog<\/p>\n<\/div>\n<\/div>\n<div class=\"sreweekly-entry\">\n<div class=\"sreweekly-title\"><a href=\"https:\/\/www.scientificamerican.com\/article\/indias-inadvertent-missile-launch-underscores-the-risk-of-accidental-nuclear-warfare\/\" target=\"_blank\" rel=\"noopener\">India\u2019s Inadvertent Missile Launch Underscores the Risk of Accidental Nuclear Warfare<\/a><\/div>\n<div class=\"sreweekly-description\">\n<p>Aside from the terrifying risk of nuclear war, this sounds very similar to the kind of complex system failures SREs deal with routinely.<\/p>\n<p>\u00a0\u00a0Zia Mian, M. V. Ramana \u2014 Scientific American<\/p>\n<\/div>\n<\/div>\n<div class=\"sreweekly-entry\">\n<div class=\"sreweekly-title\"><a href=\"https:\/\/rootly.com\/blog\/the-pros-and-cons-of-embedded-sres?utm_source=dzone&amp;utm_medium=blog\" target=\"_blank\" rel=\"noopener\">The Pros and Cons of Embedded SREs<\/a><\/div>\n<div class=\"sreweekly-description\">\n<p>Both approaches have their pros and cons. The right strategy for your company or team depends, of course, on your needs and priorities.<\/p>\n<p>\u00a0\u00a0Quentin Rousseau \u2014 Rootly<br \/>\n<em>This article is published by my sponsor, Rootly, but their sponsorship did not influence its inclusion in this issue.<\/em><\/p>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<h2>Outages<\/h2>\n<p><a href=\"https:\/\/www.theverge.com\/2022\/4\/12\/23022541\/youtube-sidebar-outage-settings-account-switch-sign-in\">YouTube<\/a><br \/>\n<a href=\"https:\/\/staceyoniot.com\/insteon-is-down-and-may-not-be-coming-back\/\">Insteon<\/a><\/p>\n<p>Insteon is down and may not be coming back<\/p>\n<p><a href=\"https:\/\/www.thesun.co.uk\/money\/18280948\/amazon-website-app-down-shoppers-cant-check-out\/\">Amazon<\/a><br \/>\nSRE WEEKLY<\/p>","protected":false},"excerpt":{"rendered":"<p>View on sreweekly.com A message from our sponsor, Rootly: Manage incidents directly from Slack with Rootly \ud83d\ude92. Automate manual admin tasks like creating incident channel, Jira and Zoom, paging and adding responders, postmortem timeline, setting up reminders, and more. Book a demo (+ get a snazzy Rootly lego set): https:\/\/rootly.com\/demo\/ Articles Errors are constructed, not&hellip; <a class=\"more-link\" href=\"https:\/\/fde.cat\/index.php\/2022\/04\/18\/sre-weekly-issue-318\/\">Continue reading <span class=\"screen-reader-text\">SRE Weekly Issue #318<\/span><\/a><\/p>\n","protected":false},"author":0,"featured_media":0,"comment_status":"","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"spay_email":"","footnotes":""},"categories":[8],"tags":[],"class_list":["post-564","post","type-post","status-publish","format-standard","hentry","category-sre","entry"],"jetpack_featured_media_url":"","jetpack-related-posts":[{"id":579,"url":"https:\/\/fde.cat\/index.php\/2022\/05\/30\/sre-weekly-issue-324\/","url_meta":{"origin":564,"position":0},"title":"SRE Weekly Issue #324","date":"May 30, 2022","format":false,"excerpt":"View on sreweekly.com A message from our sponsor, Rootly: Manage incidents directly from Slack with Rootly \ud83d\ude92. Automate manual admin tasks like creating incident channel, Jira and Zoom, paging and adding responders, postmortem timeline, setting up reminders, and more. Book a demo (+ get a snazzy Rootly lego set): https:\/\/rootly.com\/demo\/\u2026","rel":"","context":"In &quot;SRE&quot;","img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":543,"url":"https:\/\/fde.cat\/index.php\/2022\/02\/21\/sre-weekly-issue-310\/","url_meta":{"origin":564,"position":1},"title":"SRE Weekly Issue #310","date":"February 21, 2022","format":false,"excerpt":"View on sreweekly.com A message from our sponsor, Rootly: Manage incidents directly from Slack with Rootly \ud83d\ude92. Automate manual admin tasks like creating incident channel, Jira and Zoom, paging the right team, postmortem timeline, setting up reminders, and more. Book a demo (+ get a snazzy Rootly shirt): https:\/\/rootly.com\/demo\/?utm_source=sreweekly Articles\u2026","rel":"","context":"In &quot;SRE&quot;","img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":594,"url":"https:\/\/fde.cat\/index.php\/2022\/06\/06\/sre-weekly-issue-325\/","url_meta":{"origin":564,"position":2},"title":"SRE Weekly Issue #325","date":"June 6, 2022","format":false,"excerpt":"View on sreweekly.com A message from our sponsor, Rootly: Manage incidents directly from Slack with Rootly \ud83d\ude92. Automate manual admin tasks like creating incident channel, Jira and Zoom, paging and adding responders, postmortem timeline, setting up reminders, and more. Book a demo (+ get a snazzy Rootly lego set): https:\/\/rootly.com\/demo\/\u2026","rel":"","context":"In &quot;SRE&quot;","img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":531,"url":"https:\/\/fde.cat\/index.php\/2022\/01\/17\/sre-weekly-issue-305\/","url_meta":{"origin":564,"position":3},"title":"SRE Weekly Issue #305","date":"January 17, 2022","format":false,"excerpt":"View on sreweekly.com A message from our sponsor, Rootly: Manage incidents directly from Slack with Rootly \ud83d\ude92. Automate manual admin tasks like creating incident channel, Jira and Zoom, paging the right team, postmortem timeline, setting up reminders, and more. Book a demo (+ get a snazzy Rootly shirt): https:\/\/rootly.com\/demo\/?utm_source=sreweekly Articles\u2026","rel":"","context":"In &quot;SRE&quot;","img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":535,"url":"https:\/\/fde.cat\/index.php\/2022\/01\/24\/sre-weekly-issue-306\/","url_meta":{"origin":564,"position":4},"title":"SRE Weekly Issue #306","date":"January 24, 2022","format":false,"excerpt":"View on sreweekly.com A message from our sponsor, Rootly: Manage incidents directly from Slack with Rootly \ud83d\ude92. Automate manual admin tasks like creating incident channel, Jira and Zoom, paging the right team, postmortem timeline, setting up reminders, and more. Book a demo (+ get a snazzy Rootly shirt): https:\/\/rootly.com\/demo\/?utm_source=sreweekly Articles\u2026","rel":"","context":"In &quot;SRE&quot;","img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":537,"url":"https:\/\/fde.cat\/index.php\/2022\/01\/31\/sre-weekly-issue-307\/","url_meta":{"origin":564,"position":5},"title":"SRE Weekly Issue #307","date":"January 31, 2022","format":false,"excerpt":"View on sreweekly.com A message from our sponsor, Rootly: Manage incidents directly from Slack with Rootly \ud83d\ude92. Automate manual admin tasks like creating incident channel, Jira and Zoom, paging the right team, postmortem timeline, setting up reminders, and more. Book a demo (+ get a snazzy Rootly shirt): https:\/\/rootly.com\/demo\/?utm_source=sreweekly Articles\u2026","rel":"","context":"In &quot;SRE&quot;","img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]}],"_links":{"self":[{"href":"https:\/\/fde.cat\/index.php\/wp-json\/wp\/v2\/posts\/564","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/fde.cat\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/fde.cat\/index.php\/wp-json\/wp\/v2\/types\/post"}],"replies":[{"embeddable":true,"href":"https:\/\/fde.cat\/index.php\/wp-json\/wp\/v2\/comments?post=564"}],"version-history":[{"count":0,"href":"https:\/\/fde.cat\/index.php\/wp-json\/wp\/v2\/posts\/564\/revisions"}],"wp:attachment":[{"href":"https:\/\/fde.cat\/index.php\/wp-json\/wp\/v2\/media?parent=564"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/fde.cat\/index.php\/wp-json\/wp\/v2\/categories?post=564"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/fde.cat\/index.php\/wp-json\/wp\/v2\/tags?post=564"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}