Category: Technology

What the research is: The Ribbon filter is a new data structure that is more space-efficient than the popular Bloom filters that are widely used for optimizing data retrieval. One of the ways that Bloom, and now Ribbon, filters solve real engineering problems is by providing smooth configurability unmatched by other filters. Bloom filters work… Continue reading Ribbon filter: Practically smaller than Bloom and Xor

Our infrastructure supports thousands of services that handle billions of requests per second. We’ve previously discussed how we built our service encryption infrastructure to keep these globally distributed services operating securely and performantly. This post discusses the system we designed to enforce encryption policies within our network and shares some of the lessons we learned… Continue reading Enforcing encryption at scale

For years, people have been asking us to create a true multi-device experience that allows people to use WhatsApp on other devices without requiring a smartphone connection. Today, we’re announcing the rollout of a limited public beta test for WhatsApp’s updated multi-device capability.  With this new capability, you can now use WhatsApp on your phone… Continue reading How WhatsApp enables multi-device capability

Training AI models at a large scale isn’t easy. Aside from the need for large amounts of computing power and resources, there is also considerable engineering complexity behind training very large models. At Facebook AI Research (FAIR) Engineering, we have been working on building tools and infrastructure to make training large AI models easier. Our… Continue reading Fully Sharded Data Parallel: faster AI training with fewer GPUs

Co-written by Alex Oscherov Up until recently, we, like many companies, built our data pipelines in any one of a handful of technologies using Java or Scala, including Apache Spark, Storm, and Kafka. But Java is a very verbose language, so writing these pipelines in Java involves a lot of boilerplate code. For example, simple bean… Continue reading Building Data Pipelines Using Kotlin

MySQL, an open source database developed by Oracle, powers some of Facebook’s most important workloads. We actively develop new features in MySQL to support our evolving requirements. These features change many different areas of MySQL, including client connectors, storage engine, optimizer, and replication. Each new major version of MySQL requires significant time and effort to… Continue reading Migrating Facebook to MySQL 8.0