Weak References and Garbage Collection (weakref, gc)

Updated

September 7, 2026

Weak References and Garbage Collection (`weakref`, `gc`)

After reading this chapter, you will master CPython’s hybrid memory management architecture (reference counting coupled with generational cyclic garbage collection), construct self-pruning caches using weakref.WeakValueDictionary and weakref.WeakKeyDictionary, guarantee resource reclamation with weakref.finalize without relying on hazardous __del__ methods, and diagnose reference cycles using the gc module.

Mental model

CPython manages memory primarily through reference counting: every object tracks how many references point to it (ob_refcnt). When ob_refcnt drops to zero, CPython immediately deallocates the object’s memory without waiting:

Reference Counting (Instant Deallocation):
  ref_a = Node()   ──▶ Node (refcount: 1)
  ref_b = ref_a    ──▶ Node (refcount: 2)
  del ref_a        ──▶ Node (refcount: 1)
  del ref_b        ──▶ Node (refcount: 0) ──▶ Deallocated immediately!

However, circular references defeat pure reference counting. If Object A references Object B, and Object B references Object A, their reference counts never drop to zero even after all external names are deleted:

Reference Cycle (Leaked Without Cyclic GC):
  [ Root Namespace ]
          │ (External variables deleted!)
          ▼
    ┌──────────┐  refers to  ┌──────────┐
    │ Object A │ ───────────▶ │ Object B │
    │ (refcnt=1)│ ◀─────────── │ (refcnt=1)│
    └──────────┘  refers to  └──────────┘

CPython solves this with a secondary generational garbage collector (gc module) that periodically inspects container objects (Gen 0, Gen 1, Gen 2), detects unreachable islands of cyclic references, and breaks them.

A weak reference (weakref) allows you to reference an object without incrementing its ob_refcnt. If all strong references disappear, the object is immediately deallocated, and the weak reference automatically invalidates itself:

Weak Reference Architecture:
  strong_ref = Node() ──(ob_refcnt = 1)──▶ [ Target Node ]
                                                 ▲
  weak_ref = weakref.ref(strong_ref) ────────────┘ (ob_refcnt remains 1!)

  del strong_ref  ──▶ Target Node deallocated instantly!
  weak_ref()      ──▶ Returns None (dead reference)

Minimal example

Save as weakref_overview.py:

# weakref_overview.py
import weakref

class HeavyModel:
    def __init__(self, model_id: str) -> None:
        self.model_id = model_id

    def __repr__(self) -> str:
        return f"HeavyModel({self.model_id})"

def main() -> None:
    # 1. Create a target object with one strong reference
    model = HeavyModel("transformer-v3")
    print(f"Created object: {model}")

    # 2. Create a weak reference; reference count is unchanged
    weak_handle = weakref.ref(model)
    print(f"Weak handle before deletion: {weak_handle}")
    print(f"Dereferenced value: {weak_handle()}")

    # 3. Delete the only strong reference
    print("\nDeleting strong reference 'model'...")
    del model

    # 4. The target object is immediately reclaimed; weak handle returns None
    print(f"Weak handle after deletion: {weak_handle}")
    print(f"Dereferenced value: {weak_handle()}")

if __name__ == "__main__":
    main()

Run via uv run python weakref_overview.py:

Created object: HeavyModel(transformer-v3)
Weak handle before deletion: <weakref at 0x...; to 'HeavyModel' at 0x...>
Dereferenced value: HeavyModel(transformer-v3)

Deleting strong reference 'model'...
Weak handle after deletion: <weakref at 0x...; dead>
Dereferenced value: None

Worked examples

Case 1: Self-Pruning In-Memory Entity Cache with `WeakValueDictionary`

Standard dictionaries keep cached objects alive forever unless explicitly deleted, leading to silent memory leaks in long-running services. A WeakValueDictionary automatically evicts entries as soon as no external client references the cached instance:

# entity_cache.py
import weakref

class TenantConfiguration:
    def __init__(self, tenant_id: str, quota: int) -> None:
        self.tenant_id = tenant_id
        self.quota = quota

    def __repr__(self) -> str:
        return f"TenantConfiguration(id={self.tenant_id}, quota={self.quota})"

class TenantRegistry:
    def __init__(self) -> None:
        # Cache values weakly; entries disappear when callers drop their reference
        self._cache: weakref.WeakValueDictionary[str, TenantConfiguration] = (
            weakref.WeakValueDictionary()
        )

    def get_or_load(self, tenant_id: str) -> TenantConfiguration:
        config = self._cache.get(tenant_id)
        if config is None:
            print(f"  [CACHE MISS] Fetching config for '{tenant_id}' from database...")
            config = TenantConfiguration(tenant_id=tenant_id, quota=10_000)
            self._cache[tenant_id] = config
        else:
            print(f"  [CACHE HIT ] Returning cached config for '{tenant_id}'")
        return config

    def active_cached_keys(self) -> list[str]:
        return list(self._cache.keys())

def main() -> None:
    registry = TenantRegistry()

    # Client A requests tenant-01
    client_a_handle = registry.get_or_load("tenant-01")
    print(f"Active cache keys: {registry.active_cached_keys()}")

    # Client B requests tenant-01 (cache hit)
    client_b_handle = registry.get_or_load("tenant-01")

    # Client A drops its reference; Client B still holds one
    print("\nClient A releasing reference...")
    del client_a_handle
    print(f"Active cache keys: {registry.active_cached_keys()}")

    # Client B drops its reference; no strong references remain!
    print("Client B releasing reference...")
    del client_b_handle
    print(f"Active cache keys after all clients disconnect: {registry.active_cached_keys()}")

    # Next request triggers a fresh load from the database
    print("\nSubsequent request:")
    _ = registry.get_or_load("tenant-01")

if __name__ == "__main__":
    main()

Run:

uv run python entity_cache.py

Output:

  [CACHE MISS] Fetching config for 'tenant-01' from database...
Active cache keys: ['tenant-01']
  [CACHE HIT ] Returning cached config for 'tenant-01'

Client A releasing reference...
Active cache keys: ['tenant-01']
Client B releasing reference...
Active cache keys after all clients disconnect: []

Subsequent request:
  [CACHE MISS] Fetching config for 'tenant-01' from database...

Case 2: Deterministic OS Resource Cleanup with `weakref.finalize`

Relying on __del__ for resource cleanup is notoriously fragile: exceptions raised inside __del__ are silently swallowed, execution timing during interpreter shutdown is undefined, and resurrection bugs can corrupt state. weakref.finalize provides a safe, deterministic alternative:

# safe_resource_reaper.py
import os
import tempfile
import weakref

def _release_temp_file(filepath: str) -> None:
    """Standalone cleanup function. Must NOT hold reference to the owner object."""
    if os.path.exists(filepath):
        os.remove(filepath)
        print(f"[CLEANUP] Deleted temporary file from disk: {filepath}")

class EphemeralBuffer:
    def __init__(self, prefix: str) -> None:
        fd, self.path = tempfile.mkstemp(prefix=prefix)
        os.close(fd)
        print(f"[INIT] Allocated scratch file: {self.path}")

        # Register finalizer without passing 'self' to the callback
        self._finalizer = weakref.finalize(self, _release_temp_file, self.path)

    @property
    def is_alive(self) -> bool:
        return self._finalizer.alive

    def close(self) -> None:
        """Manual explicit cleanup. Can be called safely multiple times."""
        self._finalizer()

def main() -> None:
    # 1. Automatic cleanup when object goes out of scope
    print("--- Scope 1: Implicit deallocation ---")
    def worker():
        buf = EphemeralBuffer("task_worker_")
        print(f"Buffer is active: {buf.is_alive}")

    worker()
    print("Worker function exited.\n")

    # 2. Explicit manual cleanup
    print("--- Scope 2: Explicit manual close ---")
    buf2 = EphemeralBuffer("manual_task_")
    print(f"Buffer alive before close: {buf2.is_alive}")
    buf2.close()
    print(f"Buffer alive after close: {buf2.is_alive}")
    # Calling close again is a safe no-op
    buf2.close()

if __name__ == "__main__":
    main()

Run:

uv run python safe_resource_reaper.py

Output:

--- Scope 1: Implicit deallocation ---
[INIT] Allocated scratch file: /tmp/task_worker_...
Buffer is active: True
[CLEANUP] Deleted temporary file from disk: /tmp/task_worker_...
Worker function exited.

--- Scope 2: Explicit manual close ---
[INIT] Allocated scratch file: /tmp/manual_task_...
Buffer alive before close: True
[CLEANUP] Deleted temporary file from disk: /tmp/manual_task_...
Buffer alive after close: False

Case 3: Diagnosing and Breaking Cyclic Reference Leaks with `gc`

When developing complex graph structures or event subscription architectures, circular references prevent reference counting from freeing memory. The gc module exposes tools to inspect and break these cycles:

# cycle_diagnostics.py
import gc
from typing import Optional

class GraphNode:
    def __init__(self, label: str) -> None:
        self.label = label
        self.neighbor: Optional["GraphNode"] = None

    def __repr__(self) -> str:
        return f"GraphNode({self.label})"

def inspect_cycle() -> None:
    # 1. Create a circular reference inside local scope
    node_a = GraphNode("NodeA")
    node_b = GraphNode("NodeB")
    node_a.neighbor = node_b
    node_b.neighbor = node_a

    print("Created circular reference: A <───▶ B")

    # 2. Inspect referrers
    referrers = gc.get_referrers(node_a)
    referrer_types = [type(r).__name__ for r in referrers]
    print(f"Referrers to NodeA: {referrer_types}")

def main() -> None:
    gc.collect()

    inspect_cycle()

    # When inspect_cycle() returns, its stack frame is popped.
    # NodeA and NodeB have refcount 1 due to each other, but cannot be reached!
    print("\nLocal frame popped. Cycle is now orphaned.")

    # 3. Trigger explicit cyclic garbage collection
    collected = gc.collect()
    print(f"gc.collect() successfully reaped {collected} unreachable cyclic objects.")

if __name__ == "__main__":
    main()

Run:

uv run python cycle_diagnostics.py

Output:

Created circular reference: A <───▶ B
Referrers to NodeA: ['GraphNode']

Local frame popped. Cycle is now orphaned.
gc.collect() successfully reaped 2 unreachable cyclic objects.

Pitfalls

Pitfall 1: Attempting to Weakly Reference Built-in Types Directly

CPython’s core built-in types (dict, list, int, str, tuple) optimize memory by omitting the __weakref__ pointer slot from their C structures. Calling weakref.ref() on them raises a TypeError:

# THE TRAP:
import weakref

raw_data = {"status": "ok", "retries": 3}
try:
    ref = weakref.ref(raw_data)
except TypeError as err:
    print(f"Caught: {err}")

Output:

Caught: cannot create weak reference to 'dict' object

The Fix: Subclass the built-in type, or wrap the data in a custom class, which adds the __weakref__ attribute automatically:

# THE FIX:
class TrackedDict(dict):
    pass

tracked = TrackedDict({"status": "ok"})
valid_ref = weakref.ref(tracked)
print(f"Successfully referenced: {valid_ref()['status']}")

Pitfall 2: Capturing `self` in a Finalizer Callback

Passing a bound method (such as self.cleanup) or a closure capturing self into weakref.finalize creates a strong reference from the finalizer back to the object. Because the finalizer holds self, the object never dies, and the finalizer never executes!

# THE TRAP (Memory leak!):
class LeakyResource:
    def __init__(self):
        # self.cleanup is a BOUND METHOD that holds a strong reference to self!
        self._finalizer = weakref.finalize(self, self.cleanup)

    def cleanup(self):
        print("Cleaning up...")

# THE FIX:
class SafeResource:
    def __init__(self):
        # Pass a static method or module-level function and only the primitive handle
        self._finalizer = weakref.finalize(self, SafeResource._cleanup_static, "raw_handle")

    @staticmethod
    def _cleanup_static(handle: str):
        print(f"Safely cleaning up {handle}")

Exercises

Build a parent-child tree node structure where each parent maintains a list of strong references to its children, while each child maintains a weakref.ref back to its parent to prevent circular reference memory leaks.
Implement a custom @memoize_weak decorator that caches function results using a WeakValueDictionary, ensuring that cached results are evicted as soon as caller code drops them.
Use gc.get_objects() to write a diagnostic utility that prints the top 5 classes with the highest number of live instances in memory.
Benchmark the latency of creating and destroying 100,000 objects with pure reference counting versus 100,000 objects chained in reference cycles collected by gc.collect().
Implement a SharedLock resource that uses weakref.finalize to guarantee that if a thread crashes or an object is garbage-collected without explicitly releasing its lock, the lock is automatically freed.

Weak References and Garbage Collection (weakref, gc)

Mental model

Minimal example

Worked examples

Case 1: Self-Pruning In-Memory Entity Cache with WeakValueDictionary

Case 2: Deterministic OS Resource Cleanup with weakref.finalize

Case 3: Diagnosing and Breaking Cyclic Reference Leaks with gc

Pitfalls

Pitfall 1: Attempting to Weakly Reference Built-in Types Directly

Pitfall 2: Capturing self in a Finalizer Callback

Exercises

Further reading

Weak References and Garbage Collection (`weakref`, `gc`)

Case 1: Self-Pruning In-Memory Entity Cache with `WeakValueDictionary`

Case 2: Deterministic OS Resource Cleanup with `weakref.finalize`

Case 3: Diagnosing and Breaking Cyclic Reference Leaks with `gc`

Pitfall 2: Capturing `self` in a Finalizer Callback