{"id":768,"date":"2023-10-05T16:00:32","date_gmt":"2023-10-05T16:00:32","guid":{"rendered":"https:\/\/fde.cat\/index.php\/2023\/10\/05\/meta-contributes-new-features-to-python-3-12\/"},"modified":"2023-10-05T16:00:32","modified_gmt":"2023-10-05T16:00:32","slug":"meta-contributes-new-features-to-python-3-12","status":"publish","type":"post","link":"https:\/\/fde.cat\/index.php\/2023\/10\/05\/meta-contributes-new-features-to-python-3-12\/","title":{"rendered":"Meta contributes new features to Python 3.12"},"content":{"rendered":"

Python 3.12 is out! It includes new features and performance improvements \u2013 some contributed by Meta \u2013 that we believe will benefit all Python users.<\/span>
\nWe\u2019re sharing details about these new features that we worked closely with the Python community to develop.<\/span><\/p>\n

This week\u2019s release of <\/span>Python 3.12<\/span><\/a> marks a milestone in our efforts to make our work developing and scaling Python for Meta\u2019s use cases <\/span>more accessible to the broader Python community<\/span><\/a>. Open source at Meta is an important part of how we work and share our learnings with the community. <\/span><\/p>\n

For several years, we have been sharing our work on Python and CPython through our open source Python runtime,<\/span> Cinder<\/span><\/a>. We have also been working closely with the Python community to introduce new features and optimizations to improve Python\u2019s performance and to allow third parties to experiment with Python runtime optimization more easily.<\/span><\/p>\n

For the Python 3.12 release, we collaborated with the Python community on several categories of features:<\/span><\/p>\n

Immortal Objects<\/span>
\nType system improvements<\/span>
\nPerformance optimizations<\/span>
\nNew benchmarks<\/span>
\nCinder hooks\u00a0<\/span><\/p>\n

Immortal Objects<\/span><\/h2>\n

Immortal Objects \u2013 PEP 683<\/span><\/a> makes it possible to create Python objects that don\u2019t participate in <\/span>reference counting<\/span><\/a>, and will live until Python interpreter shutdown. The<\/span> original motivation<\/span><\/a> for this feature was to reduce memory use in the forking Instagram web-server workload by reducing copy-on-writes triggered by reference-count updates.<\/span><\/p>\n

Immortal Objects are also an important step towards truly immutable Python objects that can be shared between Python interpreters with no need for locking, for example, via the global interpreter lock (GIL) This can enable improved Python single-process parallelism, whether via<\/span> multiple sub-interpreters<\/span><\/a> or<\/span> GIL-free multi-threading<\/span><\/a>.<\/span><\/p>\n

Type system improvements<\/span><\/h2>\n

The engineering team behind <\/span>Pyre<\/span><\/a>, an open source Python type-checker, authored and implemented <\/span>PEP 698<\/span><\/a> to add a <\/span>@typing.override<\/span> decorator, which helps avoid bugs when refactoring class inheritance hierarchies that use method overriding.\u00a0<\/span><\/p>\n

Python developers can apply this new decorator to a subclass method that overrides a method from a base class. As a result, static type checkers will be able to warn developers if the base class is modified such that the overridden method no longer exists. Developers can avoid accidentally turning a method override into dead code. This improves confidence in refactoring and helps keep the code more maintainable.\u00a0\u00a0<\/span><\/p>\n

Performance optimizations<\/span><\/h2>\n

Faster comprehensions<\/span><\/h3>\n

In previous Python versions, all comprehensions were compiled as nested functions, and every execution of a comprehension allocated and destroyed a single-use Python function object.<\/span><\/p>\n

In Python 3.12,<\/span> PEP 709<\/span><\/a> inlines all list, dict, and set comprehensions for better performance (up to two times better in the best case).<\/span><\/p>\n

The implementation and debugging of PEP 709 also uncovered a pre-existing bytecode compiler bug that could result in silently wrong code execution in Python 3.11, which we <\/span>fixed<\/span><\/a>.<\/span><\/p>\n

Eager asyncio tasks<\/span><\/h3>\n

While Python\u2019s asynchronous programming support enables single-process concurrency, it also has noticeable runtime overhead. Every call to an async function creates an extra coroutine object, and the standard asyncio library will often bring additional overhead in the form of<\/span> Task<\/span><\/a> objects and event loop scheduling.<\/span><\/p>\n

We observed that, in practice, in a fully async codebase, many async functions are often able to return a result immediately, with no need to suspend. (This may be due to memoization, for example.) In these cases, if the result of the function is immediately awaited (e.g., by <\/span>await some_async_func()<\/span>,<\/span> the most common way to call an async function), the coroutine\/Task objects and event loop scheduling can be unnecessary overhead.<\/span><\/p>\n

Cinder eliminates this overhead via eager async execution. If an async function call is awaited immediately, it is called with a flag set that allows it to return a result directly, if possible, without creating a coroutine object. If an<\/span> asyncio.gather()<\/span><\/a> is immediately awaited, and all the async functions it gathers are able to return immediately, there\u2019s no need to ever create a <\/span>Task<\/span>\u00a0 or schedule it to the event loop.\u00a0<\/span><\/p>\n

Fully eager async execution would be an invasive (and breaking) change to Python, and doesn\u2019t work as well with the new Python 3.11+<\/span> TaskGroup<\/span><\/a> API for managing concurrent tasks. So in Python 3.12 we added a simpler version of the feature:<\/span> eager asyncio tasks<\/span><\/a>. With eager tasks, coroutine and Task objects are still created when a result is available immediately, but we can sometimes avoid scheduling the task to the event loop and instead resolve it right away.<\/span><\/p>\n

This is more efficient, but it is a semantic change, so this feature is <\/span>opt-in via a custom task factory<\/span><\/a>.<\/span><\/p>\n

Other asyncio improvements<\/span><\/h3>\n

We also landed a faster<\/span> C implementation of asyncio.current_task<\/span><\/a> and an <\/span>optimization to async task creation<\/span><\/a> that shows a <\/span>win of up to 5 percent on asyncio benchmarks<\/span><\/a>.<\/span>\u00a0<\/span><\/p>\n

Faster <\/span>super()<\/span> calls<\/span><\/h3>\n

The new<\/span> LOAD_SUPER_ATTR opcode<\/span><\/a> optimizes code of the form <\/span>super().attr<\/span> and <\/span>super().method(\u2026)<\/span>. Such code previously had to allocate, and then throw away, a single-use \u201csuper\u201d object each time it ran. Now it has little more overhead than an ordinary method call or attribute access.<\/span><\/p>\n

Other performance optimizations<\/span><\/h3>\n

We also landed two<\/span> hasattr <\/span><\/a>optimizations<\/span><\/a> and a<\/span>\u00a03.8x performance improvement to unittest.mock.Mock<\/span><\/a>.<\/span><\/p>\n

New benchmarks<\/span><\/h2>\n

When we optimize Python for internal use at Meta, we are usually able to test and validate our optimizations directly against our real-world workloads. Optimization work on open-source Python doesn\u2019t have such a production workload to test against and needs to be effective (and avoid regression) on a variety of different workloads.<\/span><\/p>\n

The<\/span> Python Performance Benchmark suite<\/span><\/a> is the standard set of benchmarks used in open-source Python optimization work. During the 3.12 development cycle, we contributed several new benchmarks to it so that it more accurately represents workload characteristics we see at Meta.<\/span><\/p>\n

We added:<\/span><\/p>\n

A<\/span> set of async_tree benchmarks<\/span><\/a> that better model an asyncio-heavy workload.<\/span>
\nA pair of benchmarks that exercise<\/span> comprehensions<\/span><\/a> and<\/span> super()<\/span><\/a> more thoroughly, which were blind spots of the existing benchmark suite.<\/span><\/p>\n

Cinder hooks<\/span><\/h2>\n

Some parts of Cinder (our<\/span> JIT compiler<\/span><\/a> and<\/span> Static Python<\/span><\/a>) wouldn\u2019t make sense as part of upstream CPython (because of limited platform support, C versus C++, semantic changes, and just the size of the code), so our goal is to package these as an independent extension module, CinderX.<\/span><\/p>\n

This requires a number of new hooks in the core runtime. We landed many of these hooks in Python 3.12:<\/span><\/p>\n

An<\/span> API to set the vectorcall entrypoint for a Python function<\/span><\/a>. This gives the JIT an entry point to take over execution for a given function.<\/span>
\nWe added<\/span> dictionary watchers<\/span><\/a>,<\/span> type watchers<\/span><\/a>,<\/span> function watchers<\/span><\/a>, and<\/span> code object watchers<\/span><\/a>. All of these allow the Cinder JIT to be notified of dynamic changes that might invalidate its assumptions, so its fast path can remain as fast as possible.<\/span>
\nWe landed<\/span> extensibility in the code generator for CPython\u2019s core interpreter<\/span><\/a> that will allow Static Python to easily re-generate an interpreter with added Static Python opcodes, and a <\/span>C API to visit all GC-tracked objects<\/span><\/a>, which will allow the Cinder JIT to discover functions that were created before it was enabled.<\/span>
\nWe also added a <\/span>thread-safe API for writing to perf-map files<\/span><\/a>. Perf-map files allow the Linux perf profiler to give a human-readable name to dynamically-generated sections of machine code, e.g. from a JIT compiler. This API will allow the Cinder JIT to safely write to perf map files without colliding with other JITs or with the new Python 3.12<\/span> perf trampoline feature<\/span><\/a>.<\/span><\/p>\n

These improvements will be useful to anyone building a third party JIT compiler or runtime optimizer for CPython. There are also plans to use the watchers internally in core CPython.\u00a0<\/span><\/p>\n

Beyond Python 3.12<\/span><\/h2>\n

Python plays a significant role at Meta. It\u2019s an important part of our infrastructure, including the<\/span> Instagram server stack<\/span><\/a>. And it\u2019s the lingua franca for<\/span> our AI\/ML work<\/span><\/a>, highlighted by our development of<\/span> PyTorch<\/span><\/a>, a machine learning framework for a wide range of use cases including computer vision, natural language processing, and more.<\/span><\/p>\n

Our work with the Python community doesn\u2019t end with the 3.12 release. We are currently discussing a new proposal, <\/span>PEP 703<\/span><\/a>, with the Python Steering Council to remove the GIL and allow Python to run in multiple threads in parallel. This update could greatly help anyone using Python in a multi-threaded environment.\u00a0<\/span><\/p>\n

Meta\u2019s involvement with the Python community also goes beyond code. In 2023, we continued supporting the <\/span>Developer in Residence program for Python<\/span><\/a> and sponsored events like <\/span>PyCon US<\/span><\/a>. We also shared our learnings in talks like \u201c<\/span>Breaking Boundaries: Advancements in High-Performance AI\/ML through PyTorch\u2019s Python Compiler<\/span><\/a>\u201d and <\/span>posts on the Meta Engineering blog<\/span><\/a>.\u00a0<\/span><\/p>\n

We are grateful to be a part of this open source community and look forward to working together to move the Python programming language forward.<\/span><\/p>\n

Acknowledgements<\/span><\/h2>\n

The author would like to acknowledge the following people for their work in contributing to all of these new features: Eddie Elizondo, Vladimir Matveev, Itamar Oren, Steven Troxler, Joshua Xu, Shannon Zhu, Jacob Bower, Pranav Thulasiram Bhat, Ariel Lin, Andrew Frost, and Sam Gross.<\/span><\/p>\n

The post Meta contributes new features to Python 3.12<\/a> appeared first on Engineering at Meta<\/a>.<\/p>\n

Engineering at Meta<\/p>","protected":false},"excerpt":{"rendered":"

Python 3.12 is out! It includes new features and performance improvements \u2013 some contributed by Meta \u2013 that we believe will benefit all Python users. We\u2019re sharing details about these new features that we worked closely with the Python community to develop. This week\u2019s release of Python 3.12 marks a milestone in our efforts to… Continue reading Meta contributes new features to Python 3.12<\/span><\/a><\/p>\n","protected":false},"author":0,"featured_media":0,"comment_status":"","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"spay_email":"","footnotes":""},"categories":[7],"tags":[],"class_list":["post-768","post","type-post","status-publish","format-standard","hentry","category-technology","entry"],"jetpack_featured_media_url":"","jetpack-related-posts":[{"id":824,"url":"https:\/\/fde.cat\/index.php\/2024\/02\/12\/meta-loves-python\/","url_meta":{"origin":768,"position":0},"title":"Meta loves Python","date":"February 12, 2024","format":false,"excerpt":"By now you\u2019re already aware that Python 3.12 has been released. But did you know that several of its new features were developed by Meta? Meta engineer Pascal Hartig (@passy) is joined on the Meta Tech Podcast by Itamar Oren and Carl Meyer, two software engineers at Meta, to discuss\u2026","rel":"","context":"In "Technology"","img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":795,"url":"https:\/\/fde.cat\/index.php\/2023\/11\/21\/writing-and-linting-python-at-scale\/","url_meta":{"origin":768,"position":1},"title":"Writing and linting Python at scale","date":"November 21, 2023","format":false,"excerpt":"Python plays a big part at Meta. It powers Instagram\u2019s backend and plays an important role in our configuration systems, as well as much of our AI work. Meta even made contributions to Python 3.12, the latest version of Python. On this episode of the\u00a0Meta Tech Podcast, Meta engineer Pascal\u2026","rel":"","context":"In "Technology"","img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":748,"url":"https:\/\/fde.cat\/index.php\/2023\/08\/15\/introducing-immortal-objects-for-python\/","url_meta":{"origin":768,"position":2},"title":"Introducing Immortal Objects for Python","date":"August 15, 2023","format":false,"excerpt":"Instagram has introduced Immortal Objects \u2013 PEP-683 \u2013 to Python. Now, objects can bypass reference count checks and live throughout the entire execution of the runtime, unlocking exciting avenues for true parallelism. At Meta, we use Python (Django) for our frontend server within Instagram. To handle parallelism, we rely on\u2026","rel":"","context":"In "Technology"","img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":742,"url":"https:\/\/fde.cat\/index.php\/2023\/08\/07\/fixit-2-metas-next-generation-auto-fixing-linter\/","url_meta":{"origin":768,"position":3},"title":"Fixit 2: Meta\u2019s next-generation auto-fixing linter","date":"August 7, 2023","format":false,"excerpt":"Fixit is dead! Long live Fixit 2 \u2013 the latest version of our open-source auto-fixing linter. Fixit 2 allows developers to efficiently build custom lint rules and perform auto-fixes for their codebases. Fixit 2 is available today on PyPI. Python is one of the most popular languages in use at\u2026","rel":"","context":"In "Technology"","img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":176,"url":"https:\/\/fde.cat\/index.php\/2021\/01\/25\/smart-argument-suite-seamlessly-connecting-python-jobs\/","url_meta":{"origin":768,"position":4},"title":"Smart Argument Suite: Seamlessly connecting Python jobs","date":"January 25, 2021","format":false,"excerpt":"Co-authors: Jun Jia and Alice Wu Introduction It\u2019s a very common scenario that an AI solution involves composing different jobs, such as data processing and model training or evaluation, into workflows and then submitting them to an orchestration engine for execution. At large companies such as LinkedIn, there may be\u2026","rel":"","context":"In "External"","img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":566,"url":"https:\/\/fde.cat\/index.php\/2022\/04\/26\/sql-notebooks-combining-the-power-of-jupyter-and-sql-editors-for-data-analytics\/","url_meta":{"origin":768,"position":5},"title":"SQL Notebooks: Combining the power of Jupyter and SQL editors for data analytics","date":"April 26, 2022","format":false,"excerpt":"At Meta, our internal data tools are the main channel from our data scientists to our production engineers. As such, it\u2019s important for us to empower our scientists and engineers not only to use data to make decisions, but also to do so in a secure and compliant way. We\u2019ve\u2026","rel":"","context":"In "Technology"","img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]}],"_links":{"self":[{"href":"https:\/\/fde.cat\/index.php\/wp-json\/wp\/v2\/posts\/768","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/fde.cat\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/fde.cat\/index.php\/wp-json\/wp\/v2\/types\/post"}],"replies":[{"embeddable":true,"href":"https:\/\/fde.cat\/index.php\/wp-json\/wp\/v2\/comments?post=768"}],"version-history":[{"count":0,"href":"https:\/\/fde.cat\/index.php\/wp-json\/wp\/v2\/posts\/768\/revisions"}],"wp:attachment":[{"href":"https:\/\/fde.cat\/index.php\/wp-json\/wp\/v2\/media?parent=768"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/fde.cat\/index.php\/wp-json\/wp\/v2\/categories?post=768"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/fde.cat\/index.php\/wp-json\/wp\/v2\/tags?post=768"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}