{"id":872,"date":"2024-06-03T01:46:45","date_gmt":"2024-06-03T01:46:45","guid":{"rendered":"https:\/\/fde.cat\/index.php\/2024\/06\/03\/sre-weekly-issue-427\/"},"modified":"2024-06-03T01:46:45","modified_gmt":"2024-06-03T01:46:45","slug":"sre-weekly-issue-427","status":"publish","type":"post","link":"https:\/\/fde.cat\/index.php\/2024\/06\/03\/sre-weekly-issue-427\/","title":{"rendered":"SRE Weekly Issue #427"},"content":{"rendered":"

View on sreweekly.com<\/a><\/p>\n

\n

A message from our sponsor, FireHydrant<\/a>:<\/h2>\n

We\u2019ve gone all out on our new integration with Microsoft Teams. If you\u2019re a MS Teams user, FireHydrant now supports the most comprehensive integration for incident management. Run the entire IM process without ever leaving the chat.<\/p>\n

https:\/\/firehydrant.com\/blog\/introducing-a-brand-new-microsoft-teams-integration\/<\/a><\/p>\n<\/div>\n

\n
\n
\n
Why didn\u2019t you status?<\/a><\/div>\n
\n

Written by a GitHub employee, this article seeks to answer the titular question, with discussions of noise reduction concerns and incidents that affect only a subset of customers.<\/p>\n

\u00a0\u00a0Ross Brodbeck<\/small><\/p>\n<\/div>\n<\/div>\n

\n
Google Cloud Incident Report: VPC Incident on May 16, 2024<\/a><\/div>\n
\n

Wow, this incident is a really great example of the idea that there is no one single root cause.<\/p>\n

\u00a0\u00a0Google<\/small><\/p>\n<\/div>\n<\/div>\n

\n
Trial by Fire: Tales from the SRE Frontlines\u200a\u2014\u200aEp2: The Scary ApplicationSet<\/a><\/div>\n
\n

Understand the safeguard configuration of the ArgoCD\u2019s ApplicationSet through the experience of our SRE who learned from an incident<\/p>\n

\u00a0\u00a0Tanat Lokejaroenlarb \u2014 Adevinta<\/small><\/p>\n<\/div>\n<\/div>\n

\n
Make Two Trips<\/a><\/div>\n
\n

Sometimes it\u2019s better to do something in multiple passes, even if it\u2019s less efficient. This applies to individual programs and major deployments alike.<\/p>\n

\u00a0\u00a0Thomas A. Limoncelli \u2014 ACM Queue<\/small><\/p>\n<\/div>\n<\/div>\n

\n
The problem with a root cause is that it explains too much<\/a><\/div>\n
\n

Another thought-provoking take on the argument that there is no one root cause.<\/p>\n

\u00a0\u00a0Lorin Hochstein<\/small><\/p>\n<\/div>\n<\/div>\n

\n
Kubernetes Tip: What Happens To Pods Running On Node That Become Unreachable?<\/a><\/div>\n
\n

I referenced this at work the other day, but the interesting bit is that the pod-eviction-timeout option has been removed in Kubernetes 1.27<\/a> and I\u2019ve had difficulty finding out what it was replaced by.<\/p>\n

\u00a0\u00a0Bhargav Bhikkaji<\/small><\/p>\n<\/div>\n<\/div>\n

\n
Incident Summaries using LLMs<\/a><\/div>\n
\n

How to use llama-2 7b to generate summaries of your incidents, using Cloudflare workers and Workers AI.<\/p>\n

It\u2019s a complete how-to using an open source LLM.<\/p>\n

\u00a0\u00a0Karl Stoney<\/small><\/p>\n<\/div>\n<\/div>\n

\n
Incident 2023-12-04: Data leak and loss in some free tier databases<\/a><\/div>\n
\n

Here\u2019s a great incident writeup from last December that I came across this week.<\/p>\n

By the way, if you see or write an incident followup post, I\u2019d be grateful if you sent a link my way!<\/p>\n

\u00a0\u00a0Turso<\/small><\/p>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n

SRE WEEKLY<\/p>","protected":false},"excerpt":{"rendered":"

View on sreweekly.com A message from our sponsor, FireHydrant: We\u2019ve gone all out on our new integration with Microsoft Teams. If you\u2019re a MS Teams user, FireHydrant now supports the most comprehensive integration for incident management. Run the entire IM process without ever leaving the chat. https:\/\/firehydrant.com\/blog\/introducing-a-brand-new-microsoft-teams-integration\/ Why didn\u2019t you status? Written by a GitHub… Continue reading SRE Weekly Issue #427<\/span><\/a><\/p>\n","protected":false},"author":0,"featured_media":0,"comment_status":"","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"spay_email":"","footnotes":""},"categories":[8],"tags":[],"class_list":["post-872","post","type-post","status-publish","format-standard","hentry","category-sre","entry"],"jetpack_featured_media_url":"","jetpack-related-posts":[{"id":899,"url":"https:\/\/fde.cat\/index.php\/2024\/07\/22\/sre-weekly-issue-434\/","url_meta":{"origin":872,"position":0},"title":"SRE Weekly Issue #434","date":"July 22, 2024","format":false,"excerpt":"View on sreweekly.com A message from our sponsor, FireHydrant: We\u2019ve gone all out on our new integration with Microsoft Teams. If you\u2019re a MS Teams user, FireHydrant now supports the most comprehensive integration for incident management. Run the entire IM process without ever leaving the chat. https:\/\/firehydrant.com\/blog\/introducing-a-brand-new-microsoft-teams-integration\/ Technical Details: Falcon\u2026","rel":"","context":"In "SRE"","img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":882,"url":"https:\/\/fde.cat\/index.php\/2024\/06\/17\/sre-weekly-issue-429\/","url_meta":{"origin":872,"position":1},"title":"SRE Weekly Issue #429","date":"June 17, 2024","format":false,"excerpt":"View on sreweekly.com A message from our sponsor, FireHydrant: We\u2019ve gone all out on our new integration with Microsoft Teams. If you\u2019re a MS Teams user, FireHydrant now supports the most comprehensive integration for incident management. Run the entire IM process without ever leaving the chat. https:\/\/firehydrant.com\/blog\/introducing-a-brand-new-microsoft-teams-integration\/ Virtualizing Our Storage\u2026","rel":"","context":"In "SRE"","img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":874,"url":"https:\/\/fde.cat\/index.php\/2024\/06\/10\/sre-weekly-issue-428\/","url_meta":{"origin":872,"position":2},"title":"SRE Weekly Issue #428","date":"June 10, 2024","format":false,"excerpt":"View on sreweekly.com A message from our sponsor, FireHydrant: We\u2019ve gone all out on our new integration with Microsoft Teams. If you\u2019re a MS Teams user, FireHydrant now supports the most comprehensive integration for incident management. Run the entire IM process without ever leaving the chat. https:\/\/firehydrant.com\/blog\/introducing-a-brand-new-microsoft-teams-integration\/ The Reverse Red\u2026","rel":"","context":"In "SRE"","img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":891,"url":"https:\/\/fde.cat\/index.php\/2024\/07\/08\/sre-weekly-issue-432\/","url_meta":{"origin":872,"position":3},"title":"SRE Weekly Issue #432","date":"July 8, 2024","format":false,"excerpt":"View on sreweekly.com A message from our sponsor, FireHydrant: We\u2019ve gone all out on our new integration with Microsoft Teams. If you\u2019re a MS Teams user, FireHydrant now supports the most comprehensive integration for incident management. Run the entire IM process without ever leaving the chat. https:\/\/firehydrant.com\/blog\/introducing-a-brand-new-microsoft-teams-integration\/ Investigating Mysterious Kafka\u2026","rel":"","context":"In "SRE"","img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":885,"url":"https:\/\/fde.cat\/index.php\/2024\/06\/24\/sre-weekly-issue-430\/","url_meta":{"origin":872,"position":4},"title":"SRE Weekly Issue #430","date":"June 24, 2024","format":false,"excerpt":"View on sreweekly.com A message from our sponsor, FireHydrant: We\u2019ve gone all out on our new integration with Microsoft Teams. If you\u2019re a MS Teams user, FireHydrant now supports the most comprehensive integration for incident management. Run the entire IM process without ever leaving the chat. https:\/\/firehydrant.com\/blog\/introducing-a-brand-new-microsoft-teams-integration\/ r\/sre: Senior SRE\u2026","rel":"","context":"In "SRE"","img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":889,"url":"https:\/\/fde.cat\/index.php\/2024\/07\/01\/sre-weekly-issue-431\/","url_meta":{"origin":872,"position":5},"title":"SRE Weekly Issue #431","date":"July 1, 2024","format":false,"excerpt":"View on sreweekly.com A message from our sponsor, FireHydrant: We\u2019ve gone all out on our new integration with Microsoft Teams. If you\u2019re a MS Teams user, FireHydrant now supports the most comprehensive integration for incident management. Run the entire IM process without ever leaving the chat. https:\/\/firehydrant.com\/blog\/introducing-a-brand-new-microsoft-teams-integration\/ Cloudflare incident on\u2026","rel":"","context":"In "SRE"","img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]}],"_links":{"self":[{"href":"https:\/\/fde.cat\/index.php\/wp-json\/wp\/v2\/posts\/872","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/fde.cat\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/fde.cat\/index.php\/wp-json\/wp\/v2\/types\/post"}],"replies":[{"embeddable":true,"href":"https:\/\/fde.cat\/index.php\/wp-json\/wp\/v2\/comments?post=872"}],"version-history":[{"count":0,"href":"https:\/\/fde.cat\/index.php\/wp-json\/wp\/v2\/posts\/872\/revisions"}],"wp:attachment":[{"href":"https:\/\/fde.cat\/index.php\/wp-json\/wp\/v2\/media?parent=872"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/fde.cat\/index.php\/wp-json\/wp\/v2\/categories?post=872"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/fde.cat\/index.php\/wp-json\/wp\/v2\/tags?post=872"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}