{"id":167,"date":"2020-12-21T17:00:50","date_gmt":"2020-12-21T17:00:50","guid":{"rendered":"https:\/\/fde.cat\/index.php\/2020\/12\/21\/a-smaller-faster-video-calling-library-for-our-apps\/"},"modified":"2021-02-02T13:43:15","modified_gmt":"2021-02-02T13:43:15","slug":"a-smaller-faster-video-calling-library-for-our-apps","status":"publish","type":"post","link":"https:\/\/fde.cat\/index.php\/2020\/12\/21\/a-smaller-faster-video-calling-library-for-our-apps\/","title":{"rendered":"A smaller, faster video calling library for our apps"},"content":{"rendered":"<ul>\n<li><span>We are rolling out a new video calling library to all the relevant products across our apps and services, including Instagram, Messenger, Portal, Workplace chat, etc.\u00a0<\/span><\/li>\n<li><span>To create a library generic enough to support all these different use cases, we needed to rewrite our existing library from scratch using the latest version of the open source WebRTC library, an incredibly rare undertaking that involved engineers from across the company.<\/span><\/li>\n<li><span>Compared with the previous library, rsys is capable of supporting multiple platforms, including Android, iOS, MacOS, Windows, and Linux. It is approximately 20 percent smaller, which makes it easy to integrate into size-constrained platforms, such as Messenger Lite. Rsys has approximately 90 percent unit test coverage and a thorough integration testing framework that covers all our major calling scenarios.<\/span><\/li>\n<li><span>We accomplished this by optimizing libraries and architecture for binary size wherever possible by separating the pieces needed for calling into isolated, standalone modules, and leveraging cross-platform solutions that aren\u2019t reliant on the operating system and environment.<\/span><\/li>\n<\/ul>\n<p><span>Facebook\u2019s initial version of video calling was written on top of a seven-year-old fork of <\/span><a href=\"https:\/\/webrtc.org\/\"><span>WebRTC <\/span><\/a><span>specifically to enable native audio calling in Messenger. At that time, our goal was to build the most feature-rich experience possible for our users. Since then, we\u2019ve added video calling, group calling, video chat heads, and interactive AR effects. With millions of people using video calling every month, the full-featured library that looked simple on the surface had become far more complex behind the scenes. We had a large amount of Messenger-specific code, which made it hard to support apps like Portal and Instagram. We had separate signaling protocols for group calling and peer-to-peer calling, which required us to write features twice and created a large divide in the codebase. We were also spending much more time keeping our fork of WebRTC updated with the latest improvements from open source. Finally, we were falling behind in our desire to provide reliable service for low-powered devices and in low-bandwidth scenarios. <\/span><\/p>\n<p><span>In looking at how we could make video calling more efficient and easier to scale for the future, we realized that our best way forward was to redesign the library from the ground up and rewrite the entire library. The result is rsys, a video calling library that allows us to make use of significant advancements made in the video calling space since the original library was written in 2014. Compared to the previous version, rsys is approximately 20 percent smaller and available across all development platforms. With this new iteration, we\u2019re reimagining how we think about our video calling platform and starting from the ground up with a new client core and extensibility framework. This helps us advance our own state-of-the-art technologies and the new codebase is designed to be sustainable and scalable for the next decade, laying the foundation for remote presence and interoperability across apps.<\/span><\/p>\n<h2><span>Smaller and faster<\/span><\/h2>\n<p><span>A smaller library loads, updates, and starts faster for the person using it, regardless of the device type or network conditions. A small library is also easier to manage, update, test, and optimize. When we started thinking about this new version, our peak binary size had reached as high as 20 MB. We were able to get some reduction by editing a few sections of code but to get where we wanted, we realized we\u2019d need to rewrite the entire library from scratch.<\/span><\/p>\n<p><span>The simplest way to get a smaller library would have been to strip away many of the features we\u2019ve added over the years, but it was important to us to keep all the most used features, like AR effects. So we stepped back and looked at how we could apply what we\u2019ve learned over the past decade and what we know about the needs of the people using our apps today. After exploring our options, we decided we needed to look past the interface and dig into the infrastructure of the library itself.<\/span><\/p>\n<p><span>We made several architectural choices to optimize for size, introduced a plug-and-play framework using <\/span><a href=\"https:\/\/docs.bazel.build\/versions\/master\/be\/functions.html#select\"><span>selects<\/span><\/a><span> to compile features selectively into apps that need them, and introduced a generic framework for writing new features based on <\/span><a href=\"https:\/\/facebook.github.io\/flux\/\"><span>Flux<\/span><\/a><span> architecture. We also moved away from heavily templated generic libraries like <\/span><a href=\"https:\/\/github.com\/facebook\/folly\"><span>Folly<\/span><\/a><span> and towards more optimally sized libraries like <\/span><a href=\"https:\/\/boost-ext.github.io\/sml\/index.html\"><span>Boost.SML<\/span><\/a><span> to realize size gains throughout all apps.<\/span><\/p>\n<p><span>In the end, we reduced core binary size by around 20 percent, from approximately 9 MB to approximately 7 MB. We accomplished this by rebuilding our features to fit a simplified architecture and design. While we kept most of the features, we will continue to introduce more pluggable features over time. Fewer lines of code makes the library lighter, faster, and more reliable, and a streamlined code base means engineers can innovate more quickly.<\/span><\/p>\n<p><span>One of our main goals with this work was to minimize code complexity and eliminate redundancies. We knew a unified architecture would allow for global optimization (instead of having each feature focused on local optimizations) and allow code to be reused in smart ways. To build this unified architecture, we made a few major changes:<\/span><\/p>\n<ul>\n<li><b>Signaling: <\/b><span>We came up with a state machine architecture for the signaling stack that could unify the protocol semantics for both peer-to-peer and group calling. We were able to abstract out any protocol-specific details away from the rest of the library and provide a signaling component with the sole responsibility of negotiating shared state between call participants. By cutting duplicate code, we are able to write features once, allow making protocol changes easily as well as provide a unified user experience for peer-to-peer and group calling.<\/span><\/li>\n<li><b>Media: <\/b><span>We decided to re-use our state machine architecture and apply it to the media stack as well, but this time we captured the semantics of open source WebRTC APIs. In parallel, we also worked on replacing our forked version of WebRTC with the latest, keeping any product-specific optimizations. This gave us the ability to change the WebRTC version underneath the state machine as long as the semantics of the APIs themselves did not change significantly and we could set up regular pulls from the open source codebase. This allows us to easily update to the latest features without any downtime or delays.<\/span><\/li>\n<li><b>SDK: <\/b><span>In order to have feature-specific states, we used Flux architecture to manage data and provide an API for calling products that worked similarly to the React JS\u2013based applications familiar to web developers. Each API call results in specific actions being routed through a central dispatcher. These actions are then handled by specific reducer classes and emit out model objects based on the type of action. These model objects are sent to bridges that contain all the feature-specific business logic and result in subsequent actions to change the model. Finally, all model updates are sent to the UI, where they are converted into platform-specific view objects for rendering. This allows us to clearly define a feature comprising a reducer, bridge, actions, and models, which in turn allows us to make features configurable at runtime for different apps.<\/span><\/li>\n<li><b>OS:<\/b><span> To make our platform generic and scalable across various products, we decided to abstract away any functionality that directly depends on the OS. We knew that having platform-specific code for Android, iOS, etc. was necessary for certain functions like creating HW encoders, decoders, threading abstractions, etc., but we tried to create generic interfaces for these so that platforms such as MacOS and Windows could easily plug in by providing different implementations through proxy objects. We also heavily used <\/span><a href=\"https:\/\/buck.build\/rule\/cxx_library.html\"><span>cxx_library<\/span><\/a><span> in Buck to configure platform-specific libraries in an easy way for compiler flags, linker arguments, etc.<\/span><\/li>\n<\/ul>\n<figure aria-describedby=\"caption-attachment-17126\" class=\"wp-caption alignnone\"><img decoding=\"async\" loading=\"lazy\" class=\"size-full wp-image-17126\" src=\"https:\/\/i0.wp.com\/engineering.fb.com\/wp-content\/uploads\/2020\/12\/Rsys.jpg?resize=750%2C422&#038;ssl=1\" alt=\"Rsys architecture for calling.\" width=\"750\" height=\"422\" srcset=\"https:\/\/i0.wp.com\/engineering.fb.com\/wp-content\/uploads\/2020\/12\/Rsys.jpg?resize=750%2C422&#038;ssl=1 1920w, https:\/\/i0.wp.com\/engineering.fb.com\/wp-content\/uploads\/2020\/12\/Rsys.jpg?resize=750%2C422&#038;ssl=1?resize=580,326 580w, https:\/\/i0.wp.com\/engineering.fb.com\/wp-content\/uploads\/2020\/12\/Rsys.jpg?resize=750%2C422&#038;ssl=1?resize=916,516 916w, https:\/\/i0.wp.com\/engineering.fb.com\/wp-content\/uploads\/2020\/12\/Rsys.jpg?resize=750%2C422&#038;ssl=1?resize=768,432 768w, https:\/\/i0.wp.com\/engineering.fb.com\/wp-content\/uploads\/2020\/12\/Rsys.jpg?resize=750%2C422&#038;ssl=1?resize=1024,577 1024w, https:\/\/i0.wp.com\/engineering.fb.com\/wp-content\/uploads\/2020\/12\/Rsys.jpg?resize=750%2C422&#038;ssl=1?resize=1536,865 1536w, https:\/\/i0.wp.com\/engineering.fb.com\/wp-content\/uploads\/2020\/12\/Rsys.jpg?resize=750%2C422&#038;ssl=1?resize=96,54 96w, https:\/\/i0.wp.com\/engineering.fb.com\/wp-content\/uploads\/2020\/12\/Rsys.jpg?resize=750%2C422&#038;ssl=1?resize=192,108 192w\" sizes=\"auto, (max-width: 992px) 100vw, 62vw\" data-recalc-dims=\"1\"><figcaption class=\"wp-caption-text\">Rsys architecture for calling.<\/figcaption><\/figure>\n<h2><span>What\u2019s next<\/span><\/h2>\n<p><span>Today, our calling platform is significantly smaller and able to scale across many different use cases and platforms. We support calls that millions of people use every day. Our library is part of all our major calling apps, including Messenger, Instagram, Portal and Workplace chat. Building rsys has been a long journey, and yet, for the people using these apps, it won\u2019t look or feel much different. It will continue to be the same great calling experience that people have come to expect and love. But this is just the beginning.<\/span><\/p>\n<p><span>The work we\u2019ve put into rsys will allow us to continue innovating and scaling our calling experiences as we head into the future. In addition to building a library that\u2019s sustainable for the next decade or more, this work has laid the foundation for <\/span><a href=\"https:\/\/about.fb.com\/news\/2020\/09\/privacy-matters-cross-app-communication\/\"><span>cross-app calling<\/span><\/a><span> across all of our apps. It has also built the foundation we\u2019ll need for an environment centered around remote presence.<\/span><\/p>\n<p><i><span>This work was made possible in collaboration with the client platform teams. We\u2019d like to thank everyone who contributed to rsys, particularly Ed Munoz, Hani Atassi, Alice Meng, Shelly Willard, Val Kozina, Adam Hill, Matt Hammerly, Ondrej Lehecka, Eugene Agafonov, Michael Stella, Cameron Pickett, Ian Petersen, and Mudit Goel for their help with implementation and continued guidance and support.<\/span><\/i><\/p>\n<p>The post <a rel=\"nofollow\" href=\"https:\/\/engineering.fb.com\/2020\/12\/21\/video-engineering\/rsys\/\">A smaller, faster video calling library for our apps<\/a> appeared first on <a rel=\"nofollow\" href=\"https:\/\/engineering.fb.com\/\">Facebook Engineering<\/a>.<\/p>\n<p><a href=\"https:\/\/engineering.fb.com\/2020\/12\/21\/video-engineering\/rsys\/\" target=\"_blank\" rel=\"noopener\">Read More<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"<p>We are rolling out a new video calling library to all the relevant products across our apps and services, including Instagram, Messenger, Portal, Workplace chat, etc.\u00a0 To create a library generic enough to support all these different use cases, we needed to rewrite our existing library from scratch using the latest version of the open&hellip; <a class=\"more-link\" href=\"https:\/\/fde.cat\/index.php\/2020\/12\/21\/a-smaller-faster-video-calling-library-for-our-apps\/\">Continue reading <span class=\"screen-reader-text\">A smaller, faster video calling library for our apps<\/span><\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"spay_email":"","footnotes":""},"categories":[1,7],"tags":[],"class_list":["post-167","post","type-post","status-publish","format-standard","hentry","category-external","category-technology","entry"],"jetpack_featured_media_url":"","jetpack-related-posts":[{"id":166,"url":"https:\/\/fde.cat\/index.php\/2020\/12\/30\/2020-year-in-review-connectivity-innovations-faster-apps-and-progress-toward-net-zero\/","url_meta":{"origin":167,"position":0},"title":"2020 year in review: Connectivity innovations, faster apps, and progress toward net zero","date":"December 30, 2020","format":false,"excerpt":"It goes without saying that 2020 has been a challenging year, to put it lightly. But if anything, the COVID-19 pandemic has shined a light on our need to connect as people. For Facebook, that meant our work has become more important than ever. Whether it was finding new and\u2026","rel":"","context":"In &quot;External&quot;","img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":649,"url":"https:\/\/fde.cat\/index.php\/2022\/11\/09\/tulip-schematizing-metas-data-platform\/","url_meta":{"origin":167,"position":1},"title":"Tulip: Schematizing Meta\u2019s data platform","date":"November 9, 2022","format":false,"excerpt":"We\u2019re sharing Tulip, a binary serialization protocol supporting schema evolution.\u00a0 Tulip assists with data schematization by addressing protocol reliability and other issues simultaneously.\u00a0 It replaces multiple legacy formats used in Meta\u2019s data platform and has achieved significant performance and efficiency gains. There are numerous heterogeneous services, such as warehouse data\u2026","rel":"","context":"In &quot;Technology&quot;","img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":877,"url":"https:\/\/fde.cat\/index.php\/2024\/06\/11\/unlocking-the-power-of-mixed-reality-devices-with-mobileconfig\/","url_meta":{"origin":167,"position":2},"title":"Unlocking the power of mixed reality devices with MobileConfig","date":"June 11, 2024","format":false,"excerpt":"MobileConfig enables developers to centrally manage a mobile app\u2019s configuration parameters in our data centers. Once a parameter value is changed on our central server, billions of app devices automatically fetch and apply the new value without app updates. These remotely managed configuration parameters serve various purposes such as A\/B\u2026","rel":"","context":"In &quot;Technology&quot;","img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":321,"url":"https:\/\/fde.cat\/index.php\/2021\/08\/31\/meet-kats-a-one-stop-shop-for-time-series-analysis\/","url_meta":{"origin":167,"position":3},"title":"Meet Kats \u2014 a one-stop shop for time series analysis","date":"August 31, 2021","format":false,"excerpt":"What it is:\u00a0 A new library to analyze time series data. Kats is a lightweight, easy-to-use, and generalizable framework for generic time series analysis, including forecasting, anomaly detection, multivariate analysis, and feature extraction\/embedding. To the best of our knowledge, Kats is the first comprehensive Python library for generic time series\u2026","rel":"","context":"In &quot;Technology&quot;","img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":643,"url":"https:\/\/fde.cat\/index.php\/2022\/10\/24\/from-zero-to-10-million-lines-of-kotlin\/","url_meta":{"origin":167,"position":4},"title":"From zero to 10 million lines of Kotlin","date":"October 24, 2022","format":false,"excerpt":"We\u2019re sharing lessons learned from shifting our Android development from Java to Kotlin. Kotlin is a popular language for Android development and offers some key advantages over Java.\u00a0 As of today, our Android codebase contains over 10 million lines of Kotlin code. We\u2019re open sourcing various examples and utilities we\u2026","rel":"","context":"In &quot;Technology&quot;","img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":800,"url":"https:\/\/fde.cat\/index.php\/2023\/12\/07\/building-end-to-end-security-for-messenger\/","url_meta":{"origin":167,"position":5},"title":"Building end-to-end security for Messenger","date":"December 7, 2023","format":false,"excerpt":"We are beginning to upgrade people\u2019s personal conversations on Messenger to use end-to-end encryption (E2EE) by default Meta is publishing two technical white papers on end-to-end encryption: Our Messenger end-to-end encryption whitepaper describes the core cryptographic protocol for transmitting messages between clients. The Labyrinth encrypted storage protocol whitepaper explains our\u2026","rel":"","context":"In &quot;Technology&quot;","img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]}],"_links":{"self":[{"href":"https:\/\/fde.cat\/index.php\/wp-json\/wp\/v2\/posts\/167","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/fde.cat\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/fde.cat\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/fde.cat\/index.php\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/fde.cat\/index.php\/wp-json\/wp\/v2\/comments?post=167"}],"version-history":[{"count":1,"href":"https:\/\/fde.cat\/index.php\/wp-json\/wp\/v2\/posts\/167\/revisions"}],"predecessor-version":[{"id":209,"href":"https:\/\/fde.cat\/index.php\/wp-json\/wp\/v2\/posts\/167\/revisions\/209"}],"wp:attachment":[{"href":"https:\/\/fde.cat\/index.php\/wp-json\/wp\/v2\/media?parent=167"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/fde.cat\/index.php\/wp-json\/wp\/v2\/categories?post=167"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/fde.cat\/index.php\/wp-json\/wp\/v2\/tags?post=167"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}