A few survivors keep Python's freed memory

I built 256 MiB of small Python objects, dropped all but a random few, and measured how much memory the process gave back. In the normal CPython 3.14 build, with 64-byte objects, keeping one object in 100,000 kept 36 MiB resident, and keeping one in 3,300 kept all 256 MiB. A one-line formula predicts it to within 7 MiB. The free-threaded build has finer-grained bookkeeping, so the same survivors should pin far less, but by default it gave back almost nothing at all, even when every object was freed, and it still hadn't ten minutes later.

Why survivors pin memory

The normal build allocates small objects (up to 512 bytes) from its own allocator, pymalloc. It gets memory from the OS in 1 MiB arenas, splits each arena into 16 KiB pools, and each pool holds objects of one size (sizes in pycore_obmalloc.h). In obmalloc.c an arena goes back to the OS only when every pool in it is empty. One live object anywhere in the arena keeps the whole megabyte.

Objects allocated one after another fill arenas in order. With 64-byte objects one arena held about 16,070 of them (63 usable pools of 255). If a fraction p of objects survive at random, an arena is released only if none of its k objects survived, which happens with probability (1 − p)k. With k = 16,070, that is already 85% at p = 1 in 100,000, and essentially zero at 1 in 3,000.

The test

Each run builds about 256 MiB of distinct bytes objects of one size (64, 128, 256 or 512 bytes as reported by sys.getsizeof), keeps a chosen fraction, deletes the rest, runs gc.collect(), waits 2.5 s while doing some light allocation of lists and dicts, and reads resident memory from /proc/self/statm. Survivors were picked at random (3 seeds each), as every n-th object, or as the oldest n objects. CPython 3.14.8, normal and free-threaded builds, on Linux arm64 with 4 KiB pages.

The normal build (blue) matches the arena model throughout. Bigger objects mean fewer per arena, so the same fraction pins less: at 1 in 10,000, 64-byte objects kept 202 MiB and 512-byte objects kept 45 MiB. Which objects survive matters as much as how many. Keeping the oldest 1% kept 5 MiB or less, because they sat together in the first few arenas. Keeping every 1,000th kept all of it, because every arena had one.

The kept memory isn't wasted for good. Empty pools inside a pinned arena can serve any object size, so when I then allocated 256 MiB of 128-byte objects, the process grew by only 73 MiB at 1 survivor in 1,000, 16 MiB of which was the new list. A pool that still holds a survivor serves only its own size, though, and at 1 in 100 nearly every pool has one: the same allocation grew the process by 251 MiB, so the program now held about twice what it was using.

The free-threaded build keeps everything

The free-threaded build doesn't use pymalloc. It allocates objects with mimalloc, whose small-object pages are 64 KiB, sixteen times smaller than an arena. By the same formula, the same survivors should pin far less. Instead, in its default configuration, it kept roughly 200 to 330 MiB resident (the objects plus the list that held them) at every survivor fraction, including zero. In a longer run, the process grew about 365 MiB, freed everything, and kept allocating small lists and dicts every 50 ms; ten minutes later 314 MiB was still resident.

The free-threading HOWTO warns about this: mimalloc defers returning freed memory to the OS, and setting MIMALLOC_PURGE_DELAY=0 shortens the delay. The source shows why "deferred" can mean indefinitely. When a page empties, CPython parks it on a per-thread list until it's safe to reuse; it's handed back to its 32 MiB segment (types.h) only during a later allocation. The segment then schedules a purge 10 ms out, and every further free in the burst pushes that out by another millisecond. Nothing runs the purge on a timer. The 10 ms and 1 ms are defaults in options.c. The purge happens when the program frees into or allocates from that same segment again (segment.c). gc.collect() doesn't trigger it. A program whose later allocations fit in pages it already has never touches those segments again, so their memory stays resident.

With MIMALLOC_PURGE_DELAY=0 (green in the chart), memory came back, and survivors pinned it by the 64 KiB page model: at 1 in 1,000, 512-byte objects kept 27 MiB instead of the normal build's 223. But it came back only once the program allocated again. A process that freed everything and then sat idle kept all of it, with or without the setting. In a crude timing test on a busy machine, the setting made a build-and-drop loop about 8% slower, and a small-object churn loop got through about 17% fewer rounds; treat those as rough.

None of this memory is lost to the program. After freeing 256 MiB, the free-threaded process could allocate 256 MiB of tuples, a different kind of object that lives in a separate mimalloc heap, and ended within about 12 MiB of a fresh process that allocated only the tuples. The difference is what the OS, a container's memory limit, or a dashboard sees: a free-threaded process's resident size behaves like a high-water mark.

What to do

In the normal build, if a big batch of objects goes away, it matters whether the ones you keep were created in the middle of it. A cache that retains a few entries from each batch, or a dict that keeps one in a thousand results, keeps the whole batch's memory. Long-lived objects created before or apart from the batch don't (the oldest-1% case). I didn't test whether copying survivors into fresh objects helps; pymalloc fills its fullest arenas first, so the copies may land right back in the pinned ones. In the free-threaded build, don't read resident memory as live data. If the number matters, for example under a container limit, try MIMALLOC_PURGE_DELAY=0 and measure what it costs your workload; even then, memory returns only while the program keeps allocating.

Versions: CPython 3.14.8 standalone builds (normal and free-threaded), Linux arm64, 4 KiB pages; source quoted from v3.14.0. All runs single-threaded. Sizes are MiB; "kept" is resident memory above where the run started, so the list holding the objects is included until it's freed.