{"id":501,"date":"2021-11-09T17:30:40","date_gmt":"2021-11-09T17:30:40","guid":{"rendered":"https:\/\/fde.cat\/index.php\/2021\/11\/09\/ocp-summit-2021-open-networking-hardware-lays-the-groundwork-for-the-metaverse\/"},"modified":"2021-11-09T17:30:40","modified_gmt":"2021-11-09T17:30:40","slug":"ocp-summit-2021-open-networking-hardware-lays-the-groundwork-for-the-metaverse","status":"publish","type":"post","link":"https:\/\/fde.cat\/index.php\/2021\/11\/09\/ocp-summit-2021-open-networking-hardware-lays-the-groundwork-for-the-metaverse\/","title":{"rendered":"OCP Summit 2021: Open networking hardware lays the groundwork for the metaverse"},"content":{"rendered":"

Open infrastructure technologies and networking hardware will play an important role as we build new technologies for the <\/span>metaverse<\/span><\/a>, where billions of people will someday come together in virtual spaces. As we head toward the next major computing platform with a continued spirit of embracing openness and disaggregation, we\u2019re announcing two new milestones for our data centers: We\u2019re sharing our next-generation network hardware portfolio in our data centers, developed in close partnership with multiple vendors. And in conjunction with this, we\u2019ve migrated our data center network hardware to a standard and open API \u2014 the Open Compute Project (<\/span>OCP<\/span><\/a>) Switch Abstraction Interface (<\/span>SAI<\/span><\/a>).\u00a0<\/span><\/p>\n

We\u2019ve come a long way in the decade since we first decided to design and build our own data centers<\/a>. Even then, we knew they\u2019d be based on concepts of openness and disaggregation, with technologies that are modular to make upgrading easy and efficient. Since founding OCP in 2009<\/a>, we\u2019ve shared our data center and component designs, and open-sourced our network orchestration software, to spark new ideas both in our own data centers and across the industry.<\/span><\/p>\n

Today, those ideas have made Meta\u2019s data centers among the most sustainable and efficient in the world. Now, through OCP, we\u2019re bringing new open advanced network technologies to our data centers, and the wider industry, for emerging frontiers of computing \u2014 from advanced AI applications to the metaverse.<\/span><\/p>\n

Wedge 400\/400C: New TORs for more powerful open networks<\/span><\/h2>\n

The Wedge 400 is Meta\u2019s next-generation TOR switch.<\/p>\n

We\u2019ve partnered with Broadcom, our long-standing ASIC partner, and Cisco Systems, our newest ASIC partner, to use their ASICs in our two next-generation top-of-rack (TOR) switches \u2014 the Wedge 400 and 400C, the latest versions of our<\/span> Wedge<\/span><\/a> TOR. The Wedge 400 utilizes Broadcom\u2019s Tomahawk 3 ASIC, while the 400C uses Cisco\u2019s Silicon One \u2014 our first contribution using Cisco\u2019s new chip. Both TORs offer higher front panel port density and greater performance for AI and machine learning applications, while also enabling future expansions.\u00a0<\/span><\/p>\n

The Wedge 400 and 400C have already been deployed in our data centers and boast several improvements over the Wedge 100S, including 4x the switching capacity (upgraded from 3.2 Tbps to 12.8 Tbps), 8x the burst absorption performance, and a field-replaceable CPU subsystem. Both the Wedge400 and 400C are manufactured by Celestica and are<\/span> open platforms<\/span><\/a> that developers of any size, from startups to large ISPs, can utilize for their own projects.<\/span><\/p>\n

<\/div>\n

\u00a0<\/p>\n

FBOSS is now powered by SAI<\/span><\/h2>\n

In the past<\/span>, <\/span>FBOSS<\/span><\/a>, Meta\u2019s own network operating system for controlling network switches, has utilized the specific API provided by the ASIC vendor. Now, with FBOSS being adapted to OCP SAI and deployed at scale in the Meta network, we can work with more silicon vendors. Broadcom has partnered closely on our migration of FBOSS from OpenNSA to SAI. In addition, we\u2019ve worked with Cisco Systems to support FBOSS with SAI with their ASIC.\u00a0<\/span><\/p>\n

Adapting and migrating FBOSS to SAI means we can onboard multiple ASICS from multiple vendors more quickly and easily onboard new ones in the future. SAI\u2019s API lets engineers configure new networking hardware without needing to delve into the specifics of the underlying chipset\u2019s SDK. Furthermore, SAI has been extended to even the PHY layer, with Credo Semi supporting FBOSS with their own SAI implementation.<\/span><\/p>\n

With this hardware being shared through OCP, supporting SAI also means closer collaboration with and feedback from the wider industry. Developers and engineers from all over the world can work with this open hardware and contribute their own software that they, in turn, can use themselves and share with the wider industry. It all goes toward our goal of creating a future where networking is both <\/span>open and disaggregated<\/span><\/a>.<\/span><\/p>\n

Next-generation 200G and 400G fabrics<\/span><\/h2>\n

We\u2019ve already deployed 200G optics in our data centers, with plans to deploy 400G in the future.<\/p>\n

Meta\u2019s data center fabrics have evolved from 100 Gbps to the next-generation 200 Gbps\/400 Gbps. Meta has already deployed 200G-FR4 optics at scale and contributed to specifications for 400G-FR4 optics that will be deployed in the future.<\/span><\/p>\n

Meta has developed two next-generation 200G fabric switches, the Minipack2 (the latest version of<\/span> Minipack<\/span><\/a>, Meta\u2019s own modular network switch) and the Arista 7388X5, in partnership with Arista Networks. Both of which are also backward compatible with previous 100G switches and will support upgrades to 400G.<\/span><\/p>\n

The Minipack2 is based on the Broadcom Tomahawk4 25.6T switch ASIC and Broadcom retimer. The Arista 7388X5 is also based on the Broadcom Tomahawk4 25.6T switch ASIC, with versions of the 7388X5 also utilizing a Credo chipset. They\u2019re high-performance switches that transmit up to 25.6 Tbps and 10.6 Bpps with modular line cards. They support 128x 200G-FR4 QSFP56 optics modules and can maintain a consistent SerDes speed at the switch ASIC, the optics host interface, and on the optics line\/wavelength. They simplify connectivity without needing a gearbox to convert data streams. They also have significantly reduced power per bit compared with their previous models (the <\/span>OCP-accepted Meta Minipack and OCP-Inspired Arista 7368X4<\/span><\/a>, respectively).<\/span><\/p>\n

The Minipack2, Meta\u2019s own modular network switch, developed in partnership with Broadcom<\/p>\n

In addition to sharing key features of the Minipack2, the Arista 7388X5 offers hyperscale cloud scalability and flexible operating systems (it can support Arista EOS, FBOSS, and SONiC).\u00a0<\/span><\/p>\n

The Arista 7388X5 is a next-generation 200G fabric switch developed in partnership with Arista Networks.<\/p>\n

Looking toward the metaverse, and more<\/span><\/h2>\n

The metaverse will rely on many technologies, including advanced AI at scale. To deliver a diversity of new workloads that will be created as a result, we continue down the path of disaggregated global networks and data centers that will underpin all of this. The technologies that Meta and the wider industry will create will, of course, need to be fast and flexible, but more than that, they will need to operate efficiently and sustainably \u2014 from the data center all the way to edge devices. The only way to achieve this will be through collaboration through communities like OCP and other partnerships.\u00a0<\/span><\/p>\n

Open hardware drives the innovation necessary to reach these goals. And our collaborations with both long-standing and new vendors to create open designs for racks, servers, storage boxes, motherboards, and more will help push Meta and the wider industry onto the next major computing platform. We\u2019re only about one percent along on the journey, but the road to the metaverse will be paved with open advanced networking hardware.<\/span><\/p>\n

Acknowledgements<\/span><\/h2>\n

The authors would like to acknowledge the work across many teams within Meta, including the FBOSS, Network Hardware Engineering, DNE, and SOE teams. We would also like to thank our partners and their engineering teams for their close collaboration on these contributions.\u00a0<\/span><\/p>\n

The post OCP Summit 2021: Open networking hardware lays the groundwork for the metaverse<\/a> appeared first on Facebook Engineering<\/a>.<\/p>\n

Facebook Engineering<\/p>","protected":false},"excerpt":{"rendered":"

Open infrastructure technologies and networking hardware will play an important role as we build new technologies for the metaverse, where billions of people will someday come together in virtual spaces. As we head toward the next major computing platform with a continued spirit of embracing openness and disaggregation, we\u2019re announcing two new milestones for our… Continue reading OCP Summit 2021: Open networking hardware lays the groundwork for the metaverse<\/span><\/a><\/p>\n","protected":false},"author":0,"featured_media":0,"comment_status":"","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"spay_email":"","footnotes":""},"categories":[7],"tags":[],"class_list":["post-501","post","type-post","status-publish","format-standard","hentry","category-technology","entry"],"jetpack_featured_media_url":"","jetpack-related-posts":[{"id":641,"url":"https:\/\/fde.cat\/index.php\/2022\/10\/18\/ocp-summit-2022-open-hardware-for-ai-infrastructure\/","url_meta":{"origin":501,"position":0},"title":"OCP Summit 2022: Open hardware for AI infrastructure","date":"October 18, 2022","format":false,"excerpt":"At OCP Summit 2022, we\u2019re announcing Grand Teton, our next-generation platform for AI at scale that we\u2019ll contribute to the OCP community. We\u2019re also sharing new innovations designed to support data centers as they advance to support new AI technologies: A new, more efficient version of Open Rack. Our Air-Assisted\u2026","rel":"","context":"In "Technology"","img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":730,"url":"https:\/\/fde.cat\/index.php\/2023\/06\/29\/metas-evenstar-is-transitioning-to-ocp-to-accelerate-open-ran-adoption\/","url_meta":{"origin":501,"position":1},"title":"Meta\u2019s Evenstar is transitioning to OCP to accelerate open RAN adoption","date":"June 29, 2023","format":false,"excerpt":"Meta is transferring its IP for Evenstar, a program to accelerate the adoption of open RAN technologies, to the Open Compute Project (OCP). Meta will contribute Evenstar\u2019s radio unit design to OCP, giving the telecom industry its first open, white box radio unit solution. The TIP Open RAN community will\u2026","rel":"","context":"In "Technology"","img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":836,"url":"https:\/\/fde.cat\/index.php\/2024\/03\/12\/building-metas-genai-infrastructure\/","url_meta":{"origin":501,"position":2},"title":"Building Meta\u2019s GenAI Infrastructure","date":"March 12, 2024","format":false,"excerpt":"Marking a major investment in Meta\u2019s AI future, we are announcing two 24k GPU clusters. We are sharing details on the hardware, network, storage, design, performance, and software that help us extract high throughput and reliability for various AI workloads. We use this cluster design for Llama 3 training. We\u2026","rel":"","context":"In "Technology"","img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":597,"url":"https:\/\/fde.cat\/index.php\/2022\/06\/09\/under-the-hood-metas-cloud-gaming-infrastructure\/","url_meta":{"origin":501,"position":3},"title":"Under the hood: Meta\u2019s cloud gaming infrastructure","date":"June 9, 2022","format":false,"excerpt":"The promise of cloud gaming is a promise to democratize gaming. Anyone who loves games should be able to enjoy them and share the experience with their friends, no matter where they\u2019re located, and even if they don\u2019t have the latest, most expensive gaming hardware. Facebook launched its cloud gaming\u2026","rel":"","context":"In "Technology"","img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":654,"url":"https:\/\/fde.cat\/index.php\/2022\/11\/21\/ptp-timing-accuracy-and-precision-for-the-future-of-computing\/","url_meta":{"origin":501,"position":4},"title":"PTP: Timing accuracy and precision for the future of computing","date":"November 21, 2022","format":false,"excerpt":"Meta is deploying a timing protocol, Precision Time Protocol (PTP), that will offer new levels of accuracy and precision to our networks and data centers. We believe PTP will become the global standard for keeping time in computer networks. PTP will benefit today\u2019s products and services and will be a\u2026","rel":"","context":"In "Technology"","img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":879,"url":"https:\/\/fde.cat\/index.php\/2024\/06\/12\/how-meta-trains-large-language-models-at-scale\/","url_meta":{"origin":501,"position":5},"title":"How Meta trains large language models at scale","date":"June 12, 2024","format":false,"excerpt":"As we continue to focus our AI research and development on solving increasingly complex problems, one of the most significant and challenging shifts we\u2019ve experienced is the sheer scale of computation required to train large language models (LLMs). Traditionally, our AI model training has involved a training massive number of\u2026","rel":"","context":"In "Technology"","img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]}],"_links":{"self":[{"href":"https:\/\/fde.cat\/index.php\/wp-json\/wp\/v2\/posts\/501","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/fde.cat\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/fde.cat\/index.php\/wp-json\/wp\/v2\/types\/post"}],"replies":[{"embeddable":true,"href":"https:\/\/fde.cat\/index.php\/wp-json\/wp\/v2\/comments?post=501"}],"version-history":[{"count":0,"href":"https:\/\/fde.cat\/index.php\/wp-json\/wp\/v2\/posts\/501\/revisions"}],"wp:attachment":[{"href":"https:\/\/fde.cat\/index.php\/wp-json\/wp\/v2\/media?parent=501"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/fde.cat\/index.php\/wp-json\/wp\/v2\/categories?post=501"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/fde.cat\/index.php\/wp-json\/wp\/v2\/tags?post=501"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}