{"id":169,"date":"2020-12-16T17:00:08","date_gmt":"2020-12-16T17:00:08","guid":{"rendered":"https:\/\/fde.cat\/index.php\/2020\/12\/16\/co-experiences-hanging-out-in-the-digital-living-room\/"},"modified":"2021-02-02T13:43:26","modified_gmt":"2021-02-02T13:43:26","slug":"co-experiences-hanging-out-in-the-digital-living-room","status":"publish","type":"post","link":"https:\/\/fde.cat\/index.php\/2020\/12\/16\/co-experiences-hanging-out-in-the-digital-living-room\/","title":{"rendered":"Co-experiences: Hanging out in the digital living room"},"content":{"rendered":"

In the spring of 2019 \u2014\u00a0a full year before the COVID-19 pandemic caused the entire world to turn our living rooms into offices, schools, and gathering places \u2014\u00a0Mark Zuckerberg shared a plan to create <\/span>the digital equivalent of the living room<\/span><\/a>, where people could connect and hang out together. That work was already underway early this year, when countries began to implement stay-at-home measures. As people began moving online <\/span>at a volume the internet could barely support<\/span><\/a>, we realized that plan needed to become a reality as quickly as possible. We shifted plans to accelerate work on co-experiences, a suite of digital shared experiences people can enjoy together.\u00a0<\/span><\/p>\n

What do we mean by co-experiences? Think about those moments in your own living room when you feel closer to friends and family. Those probably include sitting down for a long chat, but also experiences like watching sports or an emotional season finale on TV, sharing music by a new band you\u2019re into, or sharing old photos and the stories behind them. People bond over shared experiences, and we want to make video chat on Messenger and Instagram a space where people can share these same types of experiences in their virtual living rooms. To that end, we\u2019ve already rolled out <\/span>Messenger Rooms<\/span><\/a>, <\/span>cross-app communication<\/span><\/a> between Messenger and Instagram, as well as screen sharing, 360 backgrounds, and AR lighting. Our latest addition is <\/span>Watch Together<\/span><\/a>, which allows people to watch shows and videos together on their devices.<\/span><\/p>\n

<\/div>\n

To accomplish this, we needed to operate like a startup. Even though Messenger is an established product, we were building a new experience from scratch and had to make sure we were building something that people wanted and would actually use. It needed to be valuable and actually make video chatting more fun. This meant optimizing for learning, listening, and adapting. We challenged our assumptions early and often, and moved quickly. We also had to look at video calling as a whole in order to truly make this a hangout experience. We had to completely redesign the video call experience to create a natural, comfortable space for people to hang out by making the interface simple and intuitive enough that anyone could jump in quickly and find the options they need.<\/span><\/p>\n

Along the way, we built new platforms and solved the hard technical problems that come with building synchronous experiences, so that everyone can see the same thing at the same time no matter which device or network they\u2019re on. This presented additional technical challenges when done at Facebook scale.\u00a0For Watch Together, we needed to build a brand-new state synchronization platform over our video calling stack, fine-tune audio to optimize mixing and reduce echo, and build personalized video recommendation models, to name just a few.\u00a0<\/span><\/p>\n

Product engineering challenges\u00a0<\/b><\/h2>\n

We wanted to think of Watch Together as its own product and have a strong product\/market fit \u2014 a product that people would not only use but also want to share with others and use when hanging out. We knew that we probably wouldn\u2019t get things right on the first attempt, so we were prepared for trial and error and to learn quickly.\u00a0<\/span><\/p>\n

To optimize the features we were building and make sure it was the experience people wanted, we drew on both quantitative signals from our instrumentation and qualitative signals in the form of research feedback from real people using Watch Together: We ran numerous research studies and maintained a product influencer community with a diverse set of users. Thanks to feedback from these users, we completely redesigned the video calling experience to create an intuitive space for the things people do together. We designed a new swipe-up drawer; simplified the flows for selecting videos; and tweaked and iterated on ranking, the categories, and groupings and content available. We also tested multiple different call layouts to optimize the viewing experience so that on a person\u2019s device, they see a comfortable balance between the video being watched and their view of the other participants. We also heard the importance of audio balance, echo canceling, and independent volume control for the video, each of which we brought into the final experience.\u00a0<\/span><\/p>\n

\"\"<\/p>\n

We were able to draw on strong quantitative signals provided by instrumentation to understand what is happening and how the product is being used, and to validate what matches expectations and what doesn\u2019t. We leveraged tools like funnel analysis and drop rates, ranking performance, UX interactions, participation, and retention J-curves to inform where Watch Together needed improvement. We ran hundreds of individual experiments to get to the experience that exists today. All these UX patterns were built in a generic way so that we can leverage them in new experiences in the future.\u00a0<\/span><\/p>\n

Building a synchronous experience<\/b><\/h2>\n

One of the biggest challenges with co-experiences was enabling real-time, high-precision synchronization across various apps and devices to ensure that everyone is watching the same thing at the same time. In addition, we needed to build in control reliability to ensure that playback controls such as play, pause, rewind, or fast-forward occur simultaneously across devices. We also needed the audio stream from videos and real-time calls to be well synchronized so that people aren\u2019t dealing with lags and overlaps.<\/span><\/p>\n

To address this, we built our own state synchronization platform (state sync), which utilizes our multiway infrastructure (the servers we use to host individual and group calls) for real-time resolution of client-side state for all devices in a call. This is a significant improvement over previous technology. Prior to state sync, engineers jumped through many hoops to synchronize state between calls. The most common method was to use data messages, sending information to all clients for client-side resolution on each respective device. This led to individual Android, iOS, and web implementations of how to calculate the final state. But this strategy often led to inconsistencies between clients, and it was not always as reliable as we\u2019d like. With state sync, we can support web, Android, and iOS across both Messenger and Instagram, leveraging our own calling infrastructure.
\n<\/span><\/p>\n

<\/div>\n

We defined <\/span>Thrift<\/span><\/a> structures for both the input (the messages to be received on the server) and the output (the expected resolved output structure from the server). From there, state sync can be broken down into three main parts: subscription, input, and resolution.\u00a0<\/span><\/p>\n

    \n
  1. To begin sending and receiving state, the client must first subscribe to the state sync topic it wishes to interface with. This ensures we don\u2019t send Watch Together data to a client that doesn\u2019t support it.\u00a0<\/span><\/li>\n
  2. In the input stage, we use the predefined Thrift structures to send formatted messages for resolution. We use a snapshot-based model, such that the latest input from a client is the one used by the resolver, in case multiple inputs are sent in quick succession.\u00a0<\/span><\/li>\n
  3. All clients in a call hit the same server-side resolver, centralizing all inputs through the same resolver logic, eliminating the need for multiple message processing units on each client. This is another big benefit of state sync: We save processing power on the clients and maintain consistency.<\/span><\/li>\n<\/ol>\n

    State sync was built as a new, generic platform designed so that we can leverage it for future experiences. Today, it powers not only Watch Together but also other scenarios, such as screen-sharing moderation in Messenger Rooms, and <\/span>Rooms to Live<\/span><\/a>. The platform is also built with <\/span>cross-app communication<\/span><\/a> in mind, enabling friends and family to connect via Watch Together or any of the new <\/span>Messenger experiences on Instagram<\/span><\/a>.\u00a0<\/span><\/p>\n

    The future of co-experiences<\/b><\/h2>\n

    Thanks to the work we\u2019ve done to understand hangouts, build generic UX components that can be reused, and develop our state sync platform, we have a solid foundation in place on top of which we can continue to grow experiences like Watch Together, as well as build new co-experiences.<\/span><\/p>\n

    These tools will allow us to scale over time to support more experiences and to support them across all of our apps and services. In the future, we\u2019d love to see a world where people could use Messenger and Instagram as a virtual living room, to talk, share, watch, and much more. We are just getting started and are excited for the journey ahead.<\/span><\/p>\n

    The post Co-experiences: Hanging out in the digital living room<\/a> appeared first on Facebook Engineering<\/a>.<\/p>\n

    Read More<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"

    In the spring of 2019 \u2014\u00a0a full year before the COVID-19 pandemic caused the entire world to turn our living rooms into offices, schools, and gathering places \u2014\u00a0Mark Zuckerberg shared a plan to create the digital equivalent of the living room, where people could connect and hang out together. That work was already underway early… Continue reading Co-experiences: Hanging out in the digital living room<\/span><\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"spay_email":"","footnotes":""},"categories":[1,7],"tags":[],"class_list":["post-169","post","type-post","status-publish","format-standard","hentry","category-external","category-technology","entry"],"jetpack_featured_media_url":"","jetpack-related-posts":[{"id":690,"url":"https:\/\/fde.cat\/index.php\/2023\/03\/14\/how-is-salesforce-improving-everyday-developer-experiences-and-innovating-scalable-solutions\/","url_meta":{"origin":169,"position":0},"title":"How is Salesforce Improving Everyday Developer Experiences and Innovating Scalable Solutions?","date":"March 14, 2023","format":false,"excerpt":"In our \u201cEngineering Energizers\u201d Q&A series, we examine the life experiences and career paths that have shaped Salesforce engineering leaders. Meet Prianna Ahsan, a software engineering architect for MuleSoft\u2019s production engineering team. Prianna and her team enhance developer experiences by supporting cutting-edge projects, including the migration of MuleSoft onto Salesforce\u2019s\u2026","rel":"","context":"In "Technology"","img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":325,"url":"https:\/\/fde.cat\/index.php\/2021\/08\/31\/driving-towards-an-open-internet-ecosystem-to-help-tackle-the-digital-divide\/","url_meta":{"origin":169,"position":1},"title":"Driving towards an open internet ecosystem to help tackle the digital divide","date":"August 31, 2021","format":false,"excerpt":"Connectivity is an integral part of Facebook\u2019s mission to bring people closer together, and the COVID-19 pandemic has only heightened the demand for critical internet access. According to the latest edition of our Inclusive Internet Index, nearly 70 percent of people around the world believe that increased internet usage in\u2026","rel":"","context":"In "Technology"","img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":586,"url":"https:\/\/fde.cat\/index.php\/2022\/02\/15\/connor-thorsens-trip-of-a-lifetime\/","url_meta":{"origin":169,"position":2},"title":"Connor Thorsen\u2019s Trip of a Lifetime","date":"February 15, 2022","format":false,"excerpt":"Connor Thorsen had just moved to San Francisco to start his new full-time role at Salesforce when the pandemic called many employees back home. A former two-time summer intern with Salesforce Commerce Cloud in Massachusetts, Connor was hired as a software engineer on the Security UI team to build internal\u2026","rel":"","context":"In "Technology"","img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":582,"url":"https:\/\/fde.cat\/index.php\/2022\/03\/23\/how-this-journalist-turned-engineer-promotes-equality-and-inclusion-through-digital-allyship\/","url_meta":{"origin":169,"position":3},"title":"How This Journalist-Turned-Engineer Promotes Equality and Inclusion Through Digital Allyship","date":"March 23, 2022","format":false,"excerpt":"Crystal Preston-Watson is an Accessibility and Quality Engineer based in Denver, Colorado. She is the Senior Digital Accessibility Analyst at Salesforce. Crystal believes that accessibility is a civil and human right and is dedicated to making innovative, inclusive, and accessible applications for everyone. Crystal Preston-Watson\u2019s driving principle as a Senior\u2026","rel":"","context":"In "Technology"","img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":865,"url":"https:\/\/fde.cat\/index.php\/2024\/05\/14\/behind-the-scenes-of-threads-for-web\/","url_meta":{"origin":169,"position":4},"title":"Behind the scenes of Threads for web","date":"May 14, 2024","format":false,"excerpt":"When Threads first launched one of the top feature requests was for a web client. In this episode of the Meta Tech Podcast, Pascal Hartig (@passy) sits down with Ally C. and Kevin C., two engineers on the Threads Web Team that delivered the basic version of Threads for web\u2026","rel":"","context":"In "Technology"","img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":675,"url":"https:\/\/fde.cat\/index.php\/2023\/02\/07\/improving-metas-global-maps\/","url_meta":{"origin":169,"position":5},"title":"Improving Meta\u2019s global maps","date":"February 7, 2023","format":false,"excerpt":"A lot has changed since the initial launch of our basemap in late 2020. We\u2019re Meta now, but our mission remains the same: Giving people the power to build community and bring the world closer together. Across Meta, our family of applications (Facebook, Instagram, WhatsApp, among others) are using our\u2026","rel":"","context":"In "Technology"","img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]}],"_links":{"self":[{"href":"https:\/\/fde.cat\/index.php\/wp-json\/wp\/v2\/posts\/169","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/fde.cat\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/fde.cat\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/fde.cat\/index.php\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/fde.cat\/index.php\/wp-json\/wp\/v2\/comments?post=169"}],"version-history":[{"count":1,"href":"https:\/\/fde.cat\/index.php\/wp-json\/wp\/v2\/posts\/169\/revisions"}],"predecessor-version":[{"id":212,"href":"https:\/\/fde.cat\/index.php\/wp-json\/wp\/v2\/posts\/169\/revisions\/212"}],"wp:attachment":[{"href":"https:\/\/fde.cat\/index.php\/wp-json\/wp\/v2\/media?parent=169"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/fde.cat\/index.php\/wp-json\/wp\/v2\/categories?post=169"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/fde.cat\/index.php\/wp-json\/wp\/v2\/tags?post=169"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}