In the spring of 2019 — a full year before the COVID-19 pandemic caused the entire world to turn our living rooms into offices, schools, and gathering places — Mark Zuckerberg shared a plan to create the digital equivalent of the living room, where people could connect and hang out together. That work was already underway early… Continue reading Co-experiences: Hanging out in the digital living room
What it is: Infer# brings the Infer static analysis platform to developers who use Microsoft’s C# programming language. It can already detect null-pointer dereference and resource leak bugs, thanks to bi-abduction analysis. Detection of race conditions based on RacerD analysis is also in the works. Infer# has been used to analyze Microsoft software, including Roslyn,… Continue reading Infer powering Microsoft’s Infer#, a new static analyzer for C#
Co-authors: Walaa Eldin Moustafa, Wenye Zhang, Sushant Raikar, Raymond Lam, Ron Hu, Shardul Mahadik, Laura Chen, Khai Tran, Chris Chen, and Nagarathnam Muthusamy Introduction At LinkedIn, our big data compute infrastructure continually grows over time, not only to keep pace with the growth in the number of data applications, or their domains spanning data curation,… Continue reading Coral: A SQL translation, analysis, and rewrite engine for modern data lakehouses
Facebook’s codebase changes each day as engineers develop new features and optimizations for our apps. If not validated, each of these changes could potentially regress the functionality or reliability of our products for billions of people around the world. To mitigate this risk, we maintain an enormous suite of automated regression tests to cover various… Continue reading How do you test your tests?
Facebook’s services rely on fleets of servers in data centers all over the globe — all running applications and delivering the performance our services need. This is why we need to make sure our server hardware is reliable and that we can manage server hardware failures at our scale with as little disruption to our… Continue reading How Facebook keeps its large-scale infrastructure hardware up and running
When I started my journey at LinkedIn ten years ago, the company was just beginning to experience extreme growth in the volume, variety, and velocity of our data. Over the next few years, my colleagues and I in LinkedIn’s data infrastructure team built out foundational technology like Espresso, Databus, and Kafka, among others, to ensure… Continue reading DataHub: Popular metadata architectures explained
As part of our efforts to help expand connectivity around the world, Facebook Connectivity has been prototyping SuperCell, a wide-area coverage solution for increasing mobile connectivity in rural communities. Now, after working with telecom industry partners to conduct several trials and data analyses, we’re ready to share what we’ve learned. Rather than produce this solution… Continue reading SuperCell: Reaching new heights for wider connectivity
Co-authors: Xiang Zhang and Jingyu Zhu Introduction The Lambda architecture has become a popular architectural style that promises both speed and accuracy in data processing by using a hybrid approach of both batch processing and stream processing methods. But it also has some drawbacks, such as complexity and additional development/operational overheads. One of our features… Continue reading From Lambda to Lambda-less: Lessons learned