{"id":543,"date":"2022-02-21T03:54:57","date_gmt":"2022-02-21T03:54:57","guid":{"rendered":"https:\/\/fde.cat\/index.php\/2022\/02\/21\/sre-weekly-issue-310\/"},"modified":"2022-02-21T03:54:57","modified_gmt":"2022-02-21T03:54:57","slug":"sre-weekly-issue-310","status":"publish","type":"post","link":"https:\/\/fde.cat\/index.php\/2022\/02\/21\/sre-weekly-issue-310\/","title":{"rendered":"SRE Weekly Issue #310"},"content":{"rendered":"<p><a href=\"https:\/\/sreweekly.com\/sre-weekly-issue-310\/\" title=\"Permalink to SRE Weekly Issue #310\" class=\"email_only\">View on sreweekly.com<\/a><\/p>\n<div class=\"sreweekly-sponsor-message\">\n<h2>A message from our sponsor, <a href=\"https:\/\/rootly.com\/demo\/?utm_source=sreweekly\">Rootly<\/a>:<\/h2>\n<p>Manage incidents directly from Slack with Rootly \ud83d\ude92. Automate manual admin tasks like creating incident channel, Jira and Zoom, paging the right team, postmortem timeline, setting up reminders, and more. Book a demo (+ get a snazzy Rootly shirt):<br \/>\n<a href=\"https:\/\/rootly.com\/demo\/?utm_source=sreweekly\">https:\/\/rootly.com\/demo\/?utm_source=sreweekly<\/a><\/p>\n<\/div>\n<h2>Articles<\/h2>\n<div class=\"wp-block-group\">\n<div class=\"wp-block-group__inner-container\">\n<div class=\"sreweekly-entry\">\n<div class=\"sreweekly-title\"><a href=\"https:\/\/www.jeli.io\/incident-analysis-101-who-should-investigate\/\" target=\"_blank\" rel=\"noopener\">Incident Analysis 101: Who should investigate?<\/a><\/div>\n<div class=\"sreweekly-description\">\n<p>Here\u2019s the next incredibly useful article in Jeli\u2019s Incident Analysis 101 series.  This one covers the skills and traits of a good incident analyst, along with what <em>not<\/em> to look for.<\/p>\n<p>\u00a0\u00a0Laura Maguire \u2014 Jeli<\/p>\n<\/div>\n<\/div>\n<div class=\"sreweekly-entry\">\n<div class=\"sreweekly-title\"><a href=\"https:\/\/danluu.com\/cache-incidents\/\" target=\"_blank\" rel=\"noopener\">A decade of major cache incidents at Twitter<\/a><\/div>\n<div class=\"sreweekly-description\">\n<p>This article has a remarkable level of detail on 13 incidents at Twitter that were related to cache.  The authors open with an explanation of why they focused on cache-related incidents.<\/p>\n<p>\u00a0\u00a0Dan Luu and Yao Yue<\/p>\n<\/div>\n<\/div>\n<div class=\"sreweekly-entry\">\n<div class=\"sreweekly-title\"><a href=\"https:\/\/incident.io\/blog\/three-pillars-of-great-incident-response\" target=\"_blank\" rel=\"noopener\">The three pillars of great incident response<\/a><\/div>\n<div class=\"sreweekly-description\">\n<p>[\u2026] the same three pillars form the core of any good process, whether it\u2019s for the largest e-commerce giant or a scrappy SaaS startup.<\/p>\n<p>The three pillars are:<\/p>\n<p>Clarity<br \/>\nTransparency<br \/>\nCalm<\/p>\n<p>\u00a0\u00a0Lisa Karlin Curtis \u2014 incident.io<\/p>\n<\/div>\n<\/div>\n<div class=\"sreweekly-entry\">\n<div class=\"sreweekly-title\"><a href=\"https:\/\/incident.io\/blog\/designing-your-incident-severity-levels\" target=\"_blank\" rel=\"noopener\">Designing your incident severity levels<\/a><\/div>\n<div class=\"sreweekly-description\">\n<p>This one recommends doing away with \u201cP0\u201d and \u201cP5\u201d and instead using plain words like \u201cLow\u201d and \u201cHigh\u201d.<\/p>\n<p>\u00a0\u00a0Stephen Whitworth \u2014 incident.io<\/p>\n<\/div>\n<\/div>\n<div class=\"sreweekly-entry\">\n<div class=\"sreweekly-title\"><a href=\"https:\/\/rootly.com\/blog\/why-and-how-sres-can-benefit-from-feature-flags\" target=\"_blank\" rel=\"noopener\">Why and How SREs Can Benefit from Feature Flags<\/a><\/div>\n<div class=\"sreweekly-description\">\n<p>Feature flags can be a useful way to resolve user impact during an incident.<\/p>\n<p>\u00a0\u00a0Weihan Li \u2014 Rootly<br \/>\n<em>This article is published by my sponsor, Rootly, but their sponsorship did not influence its inclusion in this issue.<\/em><\/p>\n<\/div>\n<\/div>\n<div class=\"sreweekly-entry\">\n<div class=\"sreweekly-title\"><a href=\"https:\/\/engineering.hellofresh.com\/who-monitors-the-monitoring-system-is-my-prometheus-alive-at-all-2789fd3647b3\" target=\"_blank\" rel=\"noopener\">Who monitors the monitoring system?\u200a\u2014\u200aIs my Prometheus alive at all<\/a><\/div>\n<div class=\"sreweekly-description\">\n<p>Implementing a dead-switch for your alerting tool is really important so that you don\u2019t blissfully sleep through an outage.<\/p>\n<p>\u00a0\u00a0Chris Loukas \u2014 HelloFresh<\/p>\n<\/div>\n<\/div>\n<div class=\"sreweekly-entry\">\n<div class=\"sreweekly-title\"><a href=\"https:\/\/www.honeycomb.io\/blog\/how-we-define-sre-work\/\" target=\"_blank\" rel=\"noopener\">How We Define SRE Work<\/a><\/div>\n<div class=\"sreweekly-description\">\n<p>As SRE #1, the author of this article got to define the SRE role from the ground up.<\/p>\n<p>\u00a0\u00a0Fred Hebert \u2014 Honeycomb<\/p>\n<\/div>\n<\/div>\n<div class=\"sreweekly-entry\">\n<div class=\"sreweekly-title\"><a href=\"https:\/\/www.usenix.org\/publications\/loginonline\/lessons-learned-10-years-sre-part-1-starting-sre\" target=\"_blank\" rel=\"noopener\">Lessons Learned in 10 Years of SRE: Part 1 \u2013 Starting SRE<\/a><\/div>\n<div class=\"sreweekly-description\">\n<p>In this article, I will share five lessons I learned about starting SRE teams (or engagements, or organizations).<\/p>\n<p>This article is all about the shape of an SRE team, rather than technical details like SLOs and such.<\/p>\n<p>\u00a0\u00a0Andrea Spadaccini \u2014 USENIX ;login:<\/p>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<h2>Outages<\/h2>\n<p><a href=\"https:\/\/ocistatus.oraclecloud.com\/incidents\/tp4cbkqxr55t\">Oracle Cloud Infrastructure DNS<\/a><br \/>\n<a href=\"https:\/\/dnsimple.statuspage.io\/\">DNSimple<\/a><br \/>\n<a href=\"https:\/\/knowtechie.com\/red-alert-twitter-is-down-for-a-bunch-of-people-right-now\/\">Twitter<\/a><br \/>\n<a href=\"https:\/\/discord.statuspage.io\/incidents\/rn7lfvwzvzvh\">Discord<\/a><br \/>\nSRE WEEKLY<\/p>","protected":false},"excerpt":{"rendered":"<p>View on sreweekly.com A message from our sponsor, Rootly: Manage incidents directly from Slack with Rootly \ud83d\ude92. Automate manual admin tasks like creating incident channel, Jira and Zoom, paging the right team, postmortem timeline, setting up reminders, and more. Book a demo (+ get a snazzy Rootly shirt): https:\/\/rootly.com\/demo\/?utm_source=sreweekly Articles Incident Analysis 101: Who should&hellip; <a class=\"more-link\" href=\"https:\/\/fde.cat\/index.php\/2022\/02\/21\/sre-weekly-issue-310\/\">Continue reading <span class=\"screen-reader-text\">SRE Weekly Issue #310<\/span><\/a><\/p>\n","protected":false},"author":0,"featured_media":0,"comment_status":"","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"spay_email":"","footnotes":""},"categories":[8],"tags":[],"class_list":["post-543","post","type-post","status-publish","format-standard","hentry","category-sre","entry"],"jetpack_featured_media_url":"","jetpack-related-posts":[{"id":535,"url":"https:\/\/fde.cat\/index.php\/2022\/01\/24\/sre-weekly-issue-306\/","url_meta":{"origin":543,"position":0},"title":"SRE Weekly Issue #306","date":"January 24, 2022","format":false,"excerpt":"View on sreweekly.com A message from our sponsor, Rootly: Manage incidents directly from Slack with Rootly \ud83d\ude92. Automate manual admin tasks like creating incident channel, Jira and Zoom, paging the right team, postmortem timeline, setting up reminders, and more. Book a demo (+ get a snazzy Rootly shirt): https:\/\/rootly.com\/demo\/?utm_source=sreweekly Articles\u2026","rel":"","context":"In &quot;SRE&quot;","img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":579,"url":"https:\/\/fde.cat\/index.php\/2022\/05\/30\/sre-weekly-issue-324\/","url_meta":{"origin":543,"position":1},"title":"SRE Weekly Issue #324","date":"May 30, 2022","format":false,"excerpt":"View on sreweekly.com A message from our sponsor, Rootly: Manage incidents directly from Slack with Rootly \ud83d\ude92. Automate manual admin tasks like creating incident channel, Jira and Zoom, paging and adding responders, postmortem timeline, setting up reminders, and more. Book a demo (+ get a snazzy Rootly lego set): https:\/\/rootly.com\/demo\/\u2026","rel":"","context":"In &quot;SRE&quot;","img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":546,"url":"https:\/\/fde.cat\/index.php\/2022\/03\/07\/sre-weekly-issue-312\/","url_meta":{"origin":543,"position":2},"title":"SRE Weekly Issue #312","date":"March 7, 2022","format":false,"excerpt":"View on sreweekly.com A message from our sponsor, Rootly: Manage incidents directly from Slack with Rootly \ud83d\ude92. Automate manual admin tasks like creating incident channel, Jira and Zoom, paging the right team, postmortem timeline, setting up reminders, and more. Book a demo (+ get a snazzy Rootly shirt): https:\/\/rootly.com\/demo\/?utm_source=sreweekly Articles\u2026","rel":"","context":"In &quot;SRE&quot;","img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":537,"url":"https:\/\/fde.cat\/index.php\/2022\/01\/31\/sre-weekly-issue-307\/","url_meta":{"origin":543,"position":3},"title":"SRE Weekly Issue #307","date":"January 31, 2022","format":false,"excerpt":"View on sreweekly.com A message from our sponsor, Rootly: Manage incidents directly from Slack with Rootly \ud83d\ude92. Automate manual admin tasks like creating incident channel, Jira and Zoom, paging the right team, postmortem timeline, setting up reminders, and more. Book a demo (+ get a snazzy Rootly shirt): https:\/\/rootly.com\/demo\/?utm_source=sreweekly Articles\u2026","rel":"","context":"In &quot;SRE&quot;","img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":603,"url":"https:\/\/fde.cat\/index.php\/2022\/07\/04\/sre-weekly-issue-329\/","url_meta":{"origin":543,"position":4},"title":"SRE Weekly Issue #329","date":"July 4, 2022","format":false,"excerpt":"View on sreweekly.com A message from our sponsor, Rootly: Manage incidents directly from Slack with Rootly \ud83d\ude92. Automate manual admin tasks like creating incident channel, Jira and Zoom, paging and adding responders, postmortem timeline, setting up reminders, and more. Book a demo (+ get a snazzy Rootly lego set): https:\/\/rootly.com\/demo\/\u2026","rel":"","context":"In &quot;SRE&quot;","img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":521,"url":"https:\/\/fde.cat\/index.php\/2021\/12\/27\/sre-weekly-issue-302\/","url_meta":{"origin":543,"position":5},"title":"SRE Weekly Issue #302","date":"December 27, 2021","format":false,"excerpt":"View on sreweekly.com Happy holidays, for those that celebrate! I put this issue together in advance, so no Outages section this week. A message from our sponsor, Rootly: Manage incidents directly from Slack with Rootly \ud83d\ude92. Automate manual admin tasks like creating incident channel, Jira and Zoom, paging the right\u2026","rel":"","context":"In &quot;SRE&quot;","img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]}],"_links":{"self":[{"href":"https:\/\/fde.cat\/index.php\/wp-json\/wp\/v2\/posts\/543","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/fde.cat\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/fde.cat\/index.php\/wp-json\/wp\/v2\/types\/post"}],"replies":[{"embeddable":true,"href":"https:\/\/fde.cat\/index.php\/wp-json\/wp\/v2\/comments?post=543"}],"version-history":[{"count":0,"href":"https:\/\/fde.cat\/index.php\/wp-json\/wp\/v2\/posts\/543\/revisions"}],"wp:attachment":[{"href":"https:\/\/fde.cat\/index.php\/wp-json\/wp\/v2\/media?parent=543"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/fde.cat\/index.php\/wp-json\/wp\/v2\/categories?post=543"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/fde.cat\/index.php\/wp-json\/wp\/v2\/tags?post=543"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}