{"id":626,"date":"2022-08-31T12:00:07","date_gmt":"2022-08-31T12:00:07","guid":{"rendered":"https:\/\/fde.cat\/index.php\/2022\/08\/31\/introducing-velox-an-open-source-unified-execution-engine\/"},"modified":"2022-08-31T12:00:07","modified_gmt":"2022-08-31T12:00:07","slug":"introducing-velox-an-open-source-unified-execution-engine","status":"publish","type":"post","link":"https:\/\/fde.cat\/index.php\/2022\/08\/31\/introducing-velox-an-open-source-unified-execution-engine\/","title":{"rendered":"Introducing Velox: An open source unified execution engine"},"content":{"rendered":"

Meta is introducing Velox, an open source unified execution engine aimed at accelerating data management systems and streamlining their development.<\/span>
\nVelox is under active development. <\/span>Experimental results from our paper published at the International Conference on <\/span>Very Large Data Bases (VLDB) 2022 show how Velox improves efficiency and consistency in data management systems.<\/span>
\nVelox helps consolidate and unify data management systems in a manner we believe will be of benefit to the industry. We\u2019re hoping the larger open source community will join us in contributing to the project.<\/span><\/p>\n

Meta\u2019s infrastructure plays an important role in supporting our products and services. Our data infrastructure ecosystem is composed of dozens of specialized data computation engines, all focused on different workloads for a variety of use cases ranging from SQL analytics (batch and interactive) to transactional workloads, stream processing, data ingestion, and more. Recently, the rapid growth of artificial intelligence (AI) and machine learning (ML) use cases<\/span> within Meta\u2019s infrastructure<\/span><\/a> has led to additional engines and libraries targeted at feature engineering, data preprocessing, and other workloads for ML training and serving pipelines.\u00a0<\/span><\/p>\n

However, despite the similarities, these engines have largely evolved independently. This fragmentation has made maintaining and enhancing them difficult, especially considering that as workloads evolve, the hardware that executes these workloads also changes. Ultimately, this fragmentation results in systems with different feature sets and inconsistent semantics \u2014 reducing the productivity of data users that need to interact with multiple engines to finish tasks.<\/span><\/p>\n

In order to address these challenges and to create a stronger, more efficient data infrastructure for our own products and the world, Meta has created and open sourced <\/span>Velox<\/span><\/a>. It\u2019s a novel, state-of-the-art unified execution engine that aims to speed up data management systems as well as streamline their development. Velox unifies the common data-intensive components of data computation engines while still being extensible and adaptable to different computation engines. It democratizes optimizations that were previously implemented only in individual engines, providing a framework in which consistent semantics can be implemented. This reduces work duplication, promotes reusability, and improves overall efficiency and consistency.\u00a0\u00a0<\/span><\/p>\n

Velox is under active development, but it\u2019s already in various stages of integration with more than a dozen data systems at Meta, including Presto, Spark, and PyTorch (the latter through a data preprocessing library called TorchArrow), as well as other internal stream processing platforms, transactional engines, data ingestion systems and infrastructure, ML systems for feature engineering, and others.\u00a0<\/span><\/p>\n

Since it was first <\/span>uploaded to GitHub<\/span><\/a>, the Velox open source project has attracted more than 150 code contributors, including key collaborators such as Ahana, Intel, and Voltron Data, as well as various academic institutions. By open-sourcing and fostering a community for Velox, we believe we can accelerate the pace of innovation in the data management system\u2019s development industry. We hope more individuals and companies will join us in this effort.\u00a0<\/span><\/p>\n

An overview of Velox<\/span><\/h2>\n

While data computation engines may seem distinct at first, they are all composed of a similar set of logical components: a language front end, an intermediate representation (IR), an optimizer, an execution runtime, and an execution engine. Velox provides the building blocks required to implement execution engines, consisting of all data-intensive operations executed within a single host, such as expression evaluation, aggregation, sorting, joining, and more \u2014 also commonly referred to as the data plane. Therefore, Velox expects an optimized plan as input and efficiently executes it using the resources available in the local host.<\/span><\/p>\n

Data management systems like Presto and Spark typically have their own execution engines and other components. Velox can function as a common execution engine across different data management systems. (Diagram by Philip Bell.)<\/em><\/p>\n

Velox leverages numerous runtime optimizations, such as filter and conjunct reordering, key normalization for array and hash-based aggregations and joins, dynamic filter pushdown, and adaptive column prefetching. These optimizations provide optimal local efficiency given the available knowledge and statistics extracted from incoming batches of data. Velox is also designed from the ground up to efficiently support complex data types due to their ubiquity in modern workloads, and hence extensively relies on dictionary encoding for cardinality-increasing and cardinality-reducing operations such as joins and filtering, while still providing fast paths for primitive data types.<\/span><\/p>\n

The main components provided by Velox are:<\/span>
\n<\/span><\/p>\n

Type: a generic type system that allows developers to represent scalar, complex, and nested data types, including structs, maps, arrays, functions (lambdas), decimals, tensors, and more.<\/span><\/span><\/span><\/span><\/span>
\nVector: an Apache Arrow\u2013compatible columnar memory layout module supporting multiple encodings, such as flat, dictionary, constant, sequence\/RLE, and frame of reference, in addition to a lazy materialization pattern and support for out-of-order result buffer population.<\/span><\/span><\/span><\/span><\/span>
\nExpression Eval: a state-of-the-art vectorized expression evaluation engine built based on vector-encoded data, leveraging techniques such as common subexpression elimination, constant folding, efficient null propagation, encoding-aware evaluation, dictionary peeling, and memoization.<\/span><\/span><\/span><\/span><\/span>
\nFunctions: APIs that can be used by developers to build custom functions, providing a simple (row by row) and vectorized (batch by batch) interface for scalar functions and an API for aggregate functions.\u00a0<\/span><\/p>\n

A function package<\/span> compatible with the popular PrestoSQL dialect is also provided as part of the library.<\/span><\/span><\/p>\n

Operators: implementation of common SQL operators such as TableScan, Project, Filter, Aggregation, Exchange\/Merge, OrderBy, TopN, HashJoin, MergeJoin, Unnest, and more.<\/span><\/span><\/span><\/span><\/span>
\nI\/O: a set of APIs that allows Velox to be integrated in the context of other engines and runtimes, such as:<\/span><\/p>\n

Connectors: enables developers to specialize data sources and sinks for TableScan and TableWrite operators.<\/span>
\nDWIO: an extensible interface providing support for encoding\/decoding popular file formats such as Parquet, ORC, and DWRF.<\/span>
\nStorage adapters: a byte-based extensible interface that allows Velox to connect to storage systems such as <\/span>Tectonic<\/span><\/a>, S3, HDFS, and more.\u00a0<\/span>
\nSerializers: a serialization interface targeting network communication where different wire protocols can be implemented, supporting<\/span> PrestoPage<\/span><\/a> and Spark\u2019s UnsafeRow formats.<\/span><\/span><\/p>\n

Resource management: a collection of primitives for handling computational resources, such as CPU and memory management, spilling, and memory and SSD caching.<\/span><\/p>\n

Velox\u2019s main integrations and experimental results<\/span><\/h2>\n

Beyond efficiency gains, Velox provides value by unifying the execution engines across different data computation engines. The three most popular integrations are Presto, Spark, and TorchArrow\/PyTorch.<\/span><\/p>\n

Presto \u2014 Prestissimo\u00a0<\/span><\/h3>\n

Velox is being integrated into Presto as part of the<\/span> Prestissimo<\/span><\/a> project, where Presto Java workers are replaced by a C++ process based on Velox. The project was originally created by Meta in 2020 and is under continued development in collaboration with Ahana, along with other open source contributors.<\/span><\/p>\n

Prestissimo provides a C++ implementation of Presto\u2019s HTTP REST interface, including worker-to-worker exchange serialization protocol, coordinator-to-worker orchestration, and status reporting endpoints, thereby providing a drop-in C++ replacement for Presto workers. The main query workflow consists of receiving a Presto plan fragment from a Java coordinator, translating it into a Velox query plan, and handing it off to Velox for execution.<\/span><\/p>\n

We conducted two different experiments to explore the speedup provided by Velox in Presto. Our first experiment used the TPC-H benchmark and measured close to an order of magnitude speedup in some CPU-bound queries. We saw a more modest speedup (averaging 3-6x) for shuffle-bound queries.<\/span><\/p>\n

Although the TPC-H dataset is a standard benchmark, it\u2019s not representative of real workloads. To explore how Velox might perform in these scenarios, we created an experiment where we executed production traffic generated by a variety of interactive analytical tools found at Meta. In this experiment, we saw an average of 6-7x speedups in data querying, with some results increasing speedups by over an order of magnitude. You can learn more about the details of the experiments and their results in our research paper<\/a>.<\/span><\/p>\n

Prestissimo results on real analytic workloads. The histogram above shows relative speedup of Prestissimo over Presto Java. The y-axis indicates the number of queries (in thousands [K]). Zero on the x-axis means Presto Java is faster; 10 indicates that Prestissimo is at least 10 times faster than Presto Java.Prestissimo\u2019s codebase<\/span><\/a> is available on GitHub. <\/span>\u00a0<\/span><\/p>\n

Spark \u2014 Gluten<\/span><\/h3>\n

Velox is also being integrated into Spark as part of the<\/span> Gluten project<\/span><\/a> created by Intel. Gluten allows C++ execution engines (such as Velox) to be used within the Spark environment while executing Spark SQL queries. Gluten decouples the Spark JVM and execution engine by creating a JNI API based on the Apache Arrow data format and Substrait query plans, thus allowing Velox to be used within Spark by simply integrating with Gluten\u2019s JNI API.<\/span><\/p>\n

Gluten\u2019s codebase<\/span><\/a> is available on GitHub. <\/span>\u00a0<\/span><\/p>\n

TorchArrow<\/span><\/h3>\n

TorchArrow<\/span><\/a> is a dataframe Python library for data preprocessing in deep learning, and part of the PyTorch project. TorchArrow internally translates the dataframe representation into a Velox plan and delegates it to Velox for execution. In addition to converging the otherwise fragmented space of ML data preprocessing libraries, this integration allows Meta to consolidate execution-engine code between analytic engines and ML infrastructure. It provides a more consistent experience for ML end users, who are commonly required to interact with different computation engines to complete a particular task, by exposing the same set of functions\/UDFs and ensuring consistent behavior across engines.<\/span><\/p>\n

TorchArrow was recently released in<\/span> beta mode<\/span><\/a> on GitHub.<\/span><\/p>\n

The future of database system development<\/span><\/h2>\n

Velox demonstrates that it is possible to make data computation systems more adaptable by consolidating their execution engines into a single unified library. As we continue to integrate Velox into our own systems, we are committed to building a sustainable open source community to support the project as well as to speed up library development and industry adoption. We are also interested in continuing to blur the boundaries between ML infrastructure and traditional data management systems\u00a0by unifying function packages and semantics between these silos.<\/span><\/p>\n

Looking at the future, we believe Velox\u2019s unified and modular nature has the potential to be beneficial to industries that utilize, and especially those that develop, data management systems. It will allow us to partner with hardware vendors and proactively adapt our unified software stack as hardware advances. Reusing unified and highly efficient components will also allow us to innovate faster as data workloads evolve. We believe that modularity and reusability are the future of database system development, and we hope that data companies, academia, and individual database practitioners alike will join us in this effort.\u00a0<\/span><\/p>\n

In-depth documentation about Velox and these components can be found on our <\/span>website<\/span><\/a> and in our research paper \u201cVelox: Meta\u2019s unified execution engine<\/a>.\u201d<\/span><\/p>\n

Acknowledgements<\/span><\/h2>\n

We would like to thank all contributors to the Velox project. A special thank-you to Sridhar Anumandla, Philip Bell, Biswapesh Chattopadhyay, Naveen Cherukuri, Wei He, Jiju John, Jimmy Lu, Xiaoxuang Meng, Krishna Pai, Laith Sakka, Bikramjeet Vigand, Kevin Wilfong from the Meta team, and to countless community contributors, including Frank Hu, Deepak Majeti, Aditi Pandit, and Ying Su.<\/span><\/p>\n

The post Introducing Velox: An open source unified execution engine<\/a> appeared first on Engineering at Meta<\/a>.<\/p>\n

Engineering at Meta<\/p>","protected":false},"excerpt":{"rendered":"

Meta is introducing Velox, an open source unified execution engine aimed at accelerating data management systems and streamlining their development. Velox is under active development. Experimental results from our paper published at the International Conference on Very Large Data Bases (VLDB) 2022 show how Velox improves efficiency and consistency in data management systems. Velox helps… Continue reading Introducing Velox: An open source unified execution engine<\/span><\/a><\/p>\n","protected":false},"author":0,"featured_media":0,"comment_status":"","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"spay_email":"","footnotes":""},"categories":[7],"tags":[],"class_list":["post-626","post","type-post","status-publish","format-standard","hentry","category-technology","entry"],"jetpack_featured_media_url":"","jetpack-related-posts":[{"id":827,"url":"https:\/\/fde.cat\/index.php\/2024\/02\/20\/aligning-velox-and-apache-arrow-towards-composable-data-management\/","url_meta":{"origin":626,"position":0},"title":"Aligning Velox and Apache Arrow: Towards composable data management","date":"February 20, 2024","format":false,"excerpt":"We\u2019ve partnered with Voltron Data and the Arrow community to align and converge Apache Arrow with Velox, Meta\u2019s open source execution engine. Apache Arrow 15 includes three new format layouts developed through this partnership: StringView, ListView, and Run-End-Encoding (REE). This new convergence helps Meta and the larger community build data\u2026","rel":"","context":"In "Technology"","img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":869,"url":"https:\/\/fde.cat\/index.php\/2024\/05\/22\/composable-data-management-at-meta\/","url_meta":{"origin":626,"position":1},"title":"Composable data management at Meta","date":"May 22, 2024","format":false,"excerpt":"In recent years, Meta\u2019s data management systems have evolved into a composable architecture that creates interoperability, promotes reusability, and improves engineering efficiency.\u00a0 We\u2019re sharing how we\u2019ve achieved this, in part, by leveraging Velox, Meta\u2019s open source execution engine, as well as work ahead as we continue to rethink our data\u2026","rel":"","context":"In "Technology"","img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":658,"url":"https:\/\/fde.cat\/index.php\/2022\/11\/30\/enabling-static-analysis-of-sql-queries-at-meta\/","url_meta":{"origin":626,"position":2},"title":"Enabling static analysis of SQL queries at Meta","date":"November 30, 2022","format":false,"excerpt":"UPM is our internal standalone library to perform static analysis of SQL code and enhance SQL authoring.\u00a0 UPM takes SQL code as input and represents it as a data structure called a semantic tree. Infrastructure teams at Meta leverage UPM to build SQL linters, catch user mistakes in SQL code,\u2026","rel":"","context":"In "Technology"","img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":644,"url":"https:\/\/fde.cat\/index.php\/2022\/10\/24\/how-salesforce-built-a-cloud-native-task-execution-service\/","url_meta":{"origin":626,"position":3},"title":"How Salesforce Built a Cloud-Native Task Execution Service","date":"October 24, 2022","format":false,"excerpt":"If you\u2019re paying attention to Salesforce technology, you\u2019ve no doubt heard about\u00a0Hyperforce, our new approach to deploying Salesforce on public cloud providers. Start with\u00a0a look at Hyperforce\u2019s architecture. There are many compelling reasons to move to Hyperforce, both for us and our customers. We\u2019re excited to do it in the\u2026","rel":"","context":"In "Technology"","img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":753,"url":"https:\/\/fde.cat\/index.php\/2023\/08\/29\/scheduling-jupyter-notebooks-at-meta\/","url_meta":{"origin":626,"position":4},"title":"Scheduling Jupyter Notebooks at Meta","date":"August 29, 2023","format":false,"excerpt":"At Meta, Bento is our internal Jupyter notebooks platform that is leveraged by many internal users. Notebooks are also being used widely for creating reports and workflows (for example, performing data ETL) that need to be repeated at certain intervals. Users with such notebooks would have to remember to manually\u2026","rel":"","context":"In "Technology"","img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":544,"url":"https:\/\/fde.cat\/index.php\/2022\/02\/22\/the-unified-infrastructure-platform-behind-salesforce-hyperforce\/","url_meta":{"origin":626,"position":5},"title":"The Unified Infrastructure Platform Behind Salesforce Hyperforce","date":"February 22, 2022","format":false,"excerpt":"If you\u2019re paying attention to Salesforce technology at all, you\u2019ve no doubt heard about Hyperforce, our new approach to deploying Salesforce on public cloud providers. As with any big announcement, it can be a little hard to cut through the hyperbolic language and understand what\u2019s going\u00a0on. In this blog series,\u2026","rel":"","context":"In "Technology"","img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]}],"_links":{"self":[{"href":"https:\/\/fde.cat\/index.php\/wp-json\/wp\/v2\/posts\/626","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/fde.cat\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/fde.cat\/index.php\/wp-json\/wp\/v2\/types\/post"}],"replies":[{"embeddable":true,"href":"https:\/\/fde.cat\/index.php\/wp-json\/wp\/v2\/comments?post=626"}],"version-history":[{"count":0,"href":"https:\/\/fde.cat\/index.php\/wp-json\/wp\/v2\/posts\/626\/revisions"}],"wp:attachment":[{"href":"https:\/\/fde.cat\/index.php\/wp-json\/wp\/v2\/media?parent=626"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/fde.cat\/index.php\/wp-json\/wp\/v2\/categories?post=626"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/fde.cat\/index.php\/wp-json\/wp\/v2\/tags?post=626"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}