{"id":748,"date":"2023-08-15T16:00:31","date_gmt":"2023-08-15T16:00:31","guid":{"rendered":"https:\/\/fde.cat\/index.php\/2023\/08\/15\/introducing-immortal-objects-for-python\/"},"modified":"2023-08-15T16:00:31","modified_gmt":"2023-08-15T16:00:31","slug":"introducing-immortal-objects-for-python","status":"publish","type":"post","link":"https:\/\/fde.cat\/index.php\/2023\/08\/15\/introducing-immortal-objects-for-python\/","title":{"rendered":"Introducing Immortal Objects for Python"},"content":{"rendered":"

Instagram has introduced Immortal Objects \u2013 <\/span>PEP-683<\/span><\/a> \u2013 to Python. Now, <\/span>objects can bypass reference count checks and live throughout the entire execution of the runtime, unlocking exciting avenues for true parallelism.<\/span><\/p>\n

At Meta, we use Python (Django) for our frontend server within Instagram. To handle parallelism, we rely on a multi-process architecture along with asyncio for per-process concurrency. However, our scale \u2013 both in terms of business logic and the volume of handled requests \u2013\u00a0 can cause an increase in memory pressure, leading to efficiency bottlenecks.<\/span><\/p>\n

To mitigate this effect, we rely on a pre-fork web server architecture to cache as many objects as possible and have each separate process use them as read-only structured through shared memory. While this greatly helps, upon closer inspection we saw that our processes\u2019 private memory usage grew over time while our shared memory decreased.<\/span><\/p>\n

By analyzing the Python heap, we found that while most of our Python Objects were practically immutable and lived throughout the entire execution of the runtime, it ended up still modifying these objects through reference counts and garbage collection (GC) operations that mutate the objects\u2019 metadata on every read and GC cycle \u2013\u00a0 thus, triggering a <\/span>copy on write<\/span><\/a> on the server process.\u00a0<\/span><\/p>\n

The effect of copy on writes is increasing private memory and a reduction of shared memory from the main process.<\/p>\n

Immortal Objects for Python\u00a0<\/span><\/h2>\n

This problem of state mutation of shared objects is at the heart of how the Python runtime works. Given that it relies on reference counting and cycle detection, the runtime requires modifying the core memory structure of the object, which is one of the reasons the language requires a global interpreter lock (GIL).<\/span><\/p>\n

To get around this issue, we introduced <\/span>Immortal Objects \u2013 PEP-683<\/span><\/a>. This creates an immortal object (an object for which the core object state will never change) by marking a special value in the object\u2019s reference count field. It allows the runtime to know when it can and can\u2019t mutate both the reference count fields and GC header.<\/span><\/p>\n

A comparison of standard objects versus immortal objects. With standard objects, a user can guarantee that it will not mutate its type and\/or its data. Immortality adds an extra guarantee that the runtime will not modify the reference count or the GC Header if present, enabling full object immutability.<\/p>\n

While implementing and releasing this within Instagram was a relatively straightforward process due to our relatively isolated environment, sharing this to the community was a long and arduous process. Most of this was due to the <\/span>solution\u2019s implementation<\/span><\/a>, which had to deal with a combination of problems such as backwards compatibility, platform compatibility, and performance degradation.<\/span><\/p>\n

First, the implementation had to guarantee that, even after changing the reference count implementation, applications wouldn\u2019t crash if some objects suddenly had different refcount values.<\/span><\/p>\n

Second, it changes the core memory representation of a Python object and how it increases its reference counts. It needed to work across all the different platforms (Unix, Windows, Mac), compilers (GCC, Clang, and MSVC), architectures (32-bit and 64-bit), and hardware types (little- and big-endian).<\/span><\/p>\n

Finally, the core implementation relies on adding explicit checks in the reference count increment and decrement routines, which are two of the hottest code paths in the entire execution of the runtime. This inevitably meant a performance degradation in the service. Fortunately, with the smart usage of register allocations, we managed to get this down to just a ~2 percent regression across every system, making it a reasonable regression for the benefits that it brings.\u00a0<\/span><\/p>\n

How Immortal Objects have impacted Instagram<\/span><\/h2>\n

For Instagram, our initial focus was to achieve improvements in both memory and CPU efficiency of handling our requests by reducing copy on writes. Through immortal objects, we managed to greatly reduce private memory by increasing shared memory usage.\u00a0<\/span><\/p>\n

Increasing shared memory usage through immortal Objects allows us to significantly reduce private memory. Reducing the number of copy on writes.<\/p>\n

However, the implications of these changes go far beyond Instagram and into the evolution of Python as a language. Until now, one of Python\u2019s limitations has been that it couldn\u2019t guarantee true immutability of objects on the heap. Both the GC and the reference count mechanism had unrestricted access to both of these fields.<\/span><\/p>\n

Contributing immortal objects into Python introduces true immutability guarantees for the first time ever. It helps objects bypass both reference counts and garbage collection checks. This means that we can now share immortal objects across threads without requiring the GIL to provide thread safety.<\/span><\/p>\n

This is an important building block towards a multi-core Python runtime. There are two proposals that leverage immortal objects to achieve this in different ways:<\/span><\/p>\n

PEP-684<\/span><\/a>: A Per-Interpreter GIL<\/span>
\nPEP-703<\/span><\/a>: Making the Global Interpreter Lock Optional in CPython<\/span><\/p>\n

Try Immortal Objects today<\/span><\/h2>\n

We invite the community to think of ways they can leverage immortalization in their applications as well as review the existing proposals to anticipate how to improve their applications for a multi-core environment. At Meta, we are excited about the direction in the language\u2019s development and we are ready to keep contributing externally while we keep experimenting and evolving Instagram.<\/span><\/p>\n

The post Introducing Immortal Objects for Python<\/a> appeared first on Engineering at Meta<\/a>.<\/p>\n

Engineering at Meta<\/p>","protected":false},"excerpt":{"rendered":"

Instagram has introduced Immortal Objects \u2013 PEP-683 \u2013 to Python. Now, objects can bypass reference count checks and live throughout the entire execution of the runtime, unlocking exciting avenues for true parallelism. At Meta, we use Python (Django) for our frontend server within Instagram. To handle parallelism, we rely on a multi-process architecture along with… Continue reading Introducing Immortal Objects for Python<\/span><\/a><\/p>\n","protected":false},"author":0,"featured_media":0,"comment_status":"","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"spay_email":"","footnotes":""},"categories":[7],"tags":[],"class_list":["post-748","post","type-post","status-publish","format-standard","hentry","category-technology","entry"],"jetpack_featured_media_url":"","jetpack-related-posts":[{"id":768,"url":"https:\/\/fde.cat\/index.php\/2023\/10\/05\/meta-contributes-new-features-to-python-3-12\/","url_meta":{"origin":748,"position":0},"title":"Meta contributes new features to Python 3.12","date":"October 5, 2023","format":false,"excerpt":"Python 3.12 is out! It includes new features and performance improvements \u2013 some contributed by Meta \u2013 that we believe will benefit all Python users. We\u2019re sharing details about these new features that we worked closely with the Python community to develop. This week\u2019s release of Python 3.12 marks a\u2026","rel":"","context":"In "Technology"","img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":824,"url":"https:\/\/fde.cat\/index.php\/2024\/02\/12\/meta-loves-python\/","url_meta":{"origin":748,"position":1},"title":"Meta loves Python","date":"February 12, 2024","format":false,"excerpt":"By now you\u2019re already aware that Python 3.12 has been released. But did you know that several of its new features were developed by Meta? Meta engineer Pascal Hartig (@passy) is joined on the Meta Tech Podcast by Itamar Oren and Carl Meyer, two software engineers at Meta, to discuss\u2026","rel":"","context":"In "Technology"","img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":174,"url":"https:\/\/fde.cat\/index.php\/2021\/01\/28\/taming-memory-fragmentation-in-venice-with-jemalloc\/","url_meta":{"origin":748,"position":2},"title":"Taming memory fragmentation in Venice with Jemalloc","date":"January 28, 2021","format":false,"excerpt":"Sometimes, an engineering problem arises that might make us feel like maybe we don't know what we're doing, or at the very least, forces us out of the comfort zone of our area of expertise. That day came for the Venice team at Linkedin when we began to notice that\u2026","rel":"","context":"In "External"","img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":632,"url":"https:\/\/fde.cat\/index.php\/2022\/09\/12\/memlab-an-open-source-framework-for-finding-javascript-memory-leaks\/","url_meta":{"origin":748,"position":3},"title":"MemLab: An open source framework for finding JavaScript memory leaks","date":"September 12, 2022","format":false,"excerpt":"We\u2019ve open-sourced MemLab, a JavaScript memory testing framework that automates memory leak detection. Finding and addressing the root cause of memory leaks is important for delivering a quality user experience on web applications. MemLab has helped engineers and developers at Meta improve user experience and make significant improvements in memory\u2026","rel":"","context":"In "Technology"","img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":176,"url":"https:\/\/fde.cat\/index.php\/2021\/01\/25\/smart-argument-suite-seamlessly-connecting-python-jobs\/","url_meta":{"origin":748,"position":4},"title":"Smart Argument Suite: Seamlessly connecting Python jobs","date":"January 25, 2021","format":false,"excerpt":"Co-authors: Jun Jia and Alice Wu Introduction It\u2019s a very common scenario that an AI solution involves composing different jobs, such as data processing and model training or evaluation, into workflows and then submitting them to an orchestration engine for execution. At large companies such as LinkedIn, there may be\u2026","rel":"","context":"In "External"","img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":570,"url":"https:\/\/fde.cat\/index.php\/2022\/05\/02\/how-the-cinder-jits-function-inliner-helps-us-optimize-instagram\/","url_meta":{"origin":748,"position":5},"title":"How the Cinder JIT\u2019s function inliner helps us optimize Instagram","date":"May 2, 2022","format":false,"excerpt":"Since Instagram runs one of the world\u2019s largest deployments of the Django web framework, we have natural interest in finding ways to optimize Python so we can speed up our production application. As part of this effort, we\u2019ve recently open-sourced Cinder, our Python runtime that is a fork of CPython.\u2026","rel":"","context":"In "Technology"","img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]}],"_links":{"self":[{"href":"https:\/\/fde.cat\/index.php\/wp-json\/wp\/v2\/posts\/748","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/fde.cat\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/fde.cat\/index.php\/wp-json\/wp\/v2\/types\/post"}],"replies":[{"embeddable":true,"href":"https:\/\/fde.cat\/index.php\/wp-json\/wp\/v2\/comments?post=748"}],"version-history":[{"count":0,"href":"https:\/\/fde.cat\/index.php\/wp-json\/wp\/v2\/posts\/748\/revisions"}],"wp:attachment":[{"href":"https:\/\/fde.cat\/index.php\/wp-json\/wp\/v2\/media?parent=748"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/fde.cat\/index.php\/wp-json\/wp\/v2\/categories?post=748"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/fde.cat\/index.php\/wp-json\/wp\/v2\/tags?post=748"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}