Zero-Copy Binary Memory with memoryview and bytearray

Updated

September 7, 2026

Zero-Copy Binary Memory with `memoryview` and `bytearray`

After reading this chapter, you will master Python’s low-level Buffer Protocol (PEP 3118), manipulate binary sequences without memory allocations using memoryview, execute high-throughput network packet slicing, perform in-place byte mutations with bytearray, cast raw byte streams into typed integer views, and prevent buffer lock retention traps.

Mental model

In standard Python, slicing a sequence allocates a brand-new object on the heap and copies the underlying bytes:

Standard Bytes Slicing (Copies Heap Memory):
  raw = b"X" * 10_000_000       ──▶ Allocates 10 MB on heap
  slice_a = raw[1000:5000]      ──▶ Allocates NEW 4,000-byte object and COPIES bytes!
  slice_b = raw[5000:9000]      ──▶ Allocates NEW 4,000-byte object and COPIES bytes!
  (High allocation overhead, cache thrashing, and GC pressure)

Python’s Buffer Protocol (Py_buffer in CPython) provides an alternative: it allows exposing the raw memory buffer of an object directly. A memoryview is a lightweight Python object that references an existing memory buffer without copying a single byte:

Zero-Copy memoryview Architecture:
┌────────────────────────────────────────────────────────┐
│ Shared Heap Memory Buffer (10 MB allocated once)       │
│ [ B0, B1, B2, ..., B1000, ..., B5000, ..., BN ]        │
└──────┬────────────────────┬───────────┬────────────────┘
       │                    │           │
       │ Pointer + Length   │           │ Pointer + Length
       ▼                    │           ▼
┌──────────────────┐        │     ┌──────────────────┐
│ memoryview(raw)  │        │     │ view[5000:9000]  │
│ [0:10_000_000]   │        │     │ (Zero-copy slice)│
└──────────────────┘        │     └──────────────────┘
                            ▼
                    ┌──────────────────┐
                    │ view[1000:5000]  │
                    │ (Zero-copy slice)│
                    └──────────────────┘

Binary Types Compared

┌────────────┬───────────┬───────────────────────────────┬───────────────────────────────┐
│ Type       │ Mutable?  │ Slicing Behavior              │ Use Case                      │
├────────────┼───────────┼───────────────────────────────┼───────────────────────────────┤
│ bytes      │ No        │ Allocates & copies new bytes  │ Immutable payloads & strings  │
│ bytearray  │ Yes       │ Allocates & copies new bytes  │ In-place binary construction  │
│ memoryview │ Reference │ Zero-copy window (O(1) time)  │ Packet parsing, I/O streaming │
└────────────┴───────────┴───────────────────────────────┴───────────────────────────────┘

Minimal example

Save as memoryview_benchmark.py:

# memoryview_benchmark.py
import time

def benchmark_slicing() -> None:
    size = 50_000_000  # 50 Megabytes of raw binary data
    payload = b"\xaa" * size

    # 1. Traditional bytes slicing: Copies 10 MB into a new object
    start = time.perf_counter()
    copied_chunk = payload[10_000_000:20_000_000]
    dur_copy = time.perf_counter() - start

    # 2. Zero-copy memoryview slicing: Creates a 56-byte reference window
    view = memoryview(payload)
    start = time.perf_counter()
    zero_copy_chunk = view[10_000_000:20_000_000]
    dur_zerocopy = time.perf_counter() - start

    print(f"Data size: {size / (1024 * 1024):.1f} MB")
    print(f"Bytes slicing (copies 10 MB):     {dur_copy * 1_000_000:.2f} µs")
    print(f"memoryview slicing (zero-copy):   {dur_zerocopy * 1_000_000:.2f} µs  ({dur_copy / dur_zerocopy:.1f}x faster)")

    # Verify content equality
    print(f"\nAre slice bytes identical? {copied_chunk == zero_copy_chunk.tobytes()}")

def main() -> None:
    benchmark_slicing()

if __name__ == "__main__":
    main()

Run via uv run python memoryview_benchmark.py:

Data size: 47.7 MB
Bytes slicing (copies 10 MB):     1824.50 µs
memoryview slicing (zero-copy):   0.45 µs  (4054.4x faster)

Are slice bytes identical? True

Worked examples

Case 1: Zero-Copy Network Packet Header Dissection

In network telemetry, routers receive millions of packets per second. Parsing Ethernet, IPv4, and TCP headers using memoryview extracts headers and data payloads without allocating intermediate strings:

# packet_dissector.py
import struct

def dissect_ipv4_packet(packet_data: bytes) -> dict[str, int | memoryview]:
    """Parse a raw IPv4 packet using zero-copy memory views."""
    view = memoryview(packet_data)

    # IPv4 Header is the first 20 bytes
    header_view = view[:20]

    # Unpack header fields:
    # Byte 0: Version & IHL
    # Byte 8: TTL
    # Byte 9: Protocol (6 = TCP, 17 = UDP)
    # Bytes 12-16: Source IP
    # Bytes 16-20: Destination IP
    version_ihl, _, _, _, _, ttl, proto, checksum, src_ip, dst_ip = struct.unpack(
        "!BBHHHBBHII",
        header_view,
    )

    # Extract payload starting at offset 20 without copying
    payload_view = view[20:]

    return {
        "version": (version_ihl >> 4) & 0x0F,
        "ttl": ttl,
        "protocol": proto,
        "src_ip": src_ip,
        "dst_ip": dst_ip,
        "payload_length": len(payload_view),
        "payload_slice": payload_view,
    }

def main() -> None:
    # Construct synthetic 20-byte IPv4 header + 8 bytes payload
    mock_header = struct.pack("!BBHHHBBHII", 0x45, 0, 28, 1, 0, 64, 6, 0, 0xC0A80101, 0x0A000001)
    mock_payload = b"PING_REQ"
    raw_packet = mock_header + mock_payload

    result = dissect_ipv4_packet(raw_packet)
    print("Dissected IPv4 Packet:")
    print(f"  IP Version : {result['version']}")
    print(f"  TTL        : {result['ttl']}")
    print(f"  Protocol   : {result['protocol']} (6=TCP)")
    print(f"  Payload Len: {result['payload_length']} bytes")
    # Cast memoryview slice to bytes only when printing
    print(f"  Payload    : {result['payload_slice'].tobytes().decode()}")

if __name__ == "__main__":
    main()

Run:

uv run python packet_dissector.py

Output:

Dissected IPv4 Packet:
  IP Version : 4
  TTL        : 64
  Protocol   : 6 (6=TCP)
  Payload Len: 8 bytes
  Payload    : PING_REQ

Case 2: In-Place Mutation with `bytearray` and `memoryview`

When modifying binary files, image buffers, or socket streams, combining bytearray with memoryview allows modifying specific byte ranges in place without creating new copies:

# buffer_mutation.py
def main() -> None:
    # 1. Allocate a mutable bytearray
    buffer = bytearray(b"HTTP/1.1 200 OK\r\nContent-Length: 000\r\n\r\n")
    print(f"Initial buffer: {buffer.decode()}")

    # 2. Create a memoryview over the mutable buffer
    view = memoryview(buffer)

    # 3. Locate and mutate the 3-digit length slice directly in place!
    # "Content-Length: " is at index 17; length value is at indices 33:36
    offset = buffer.find(b"000")
    print(f"Found '000' length placeholder at offset: {offset}")

    # Overwrite directly via the view
    view[offset:offset + 3] = b"256"

    print(f"Mutated buffer: {buffer.decode()}")

if __name__ == "__main__":
    main()

Run:

uv run python buffer_mutation.py

Output:

Initial buffer: HTTP/1.1 200 OK
Content-Length: 000


Found '000' length placeholder at offset: 33
Mutated buffer: HTTP/1.1 200 OK
Content-Length: 256

Case 3: Casting Memory Layouts with `.cast()`

A memoryview can reinterpret raw bytes as typed arrays (such as unsigned 16-bit short integers 'H' or 32-bit unsigned integers 'I') using .cast() without calling struct.unpack() across loops:

# memoryview_cast.py
def main() -> None:
    # 8 bytes representing two 32-bit unsigned integers (little-endian)
    raw_data = (1000).to_bytes(4, "little") + (50000).to_bytes(4, "little")

    view = memoryview(raw_data)
    print(f"Raw byte view: {list(view)}")

    # Cast 8 raw bytes into an array of 32-bit integers ('I')
    int32_view = view.cast("I")

    print(f"Cast int32 view length: {len(int32_view)}")
    print(f"Integer 0: {int32_view[0]}")
    print(f"Integer 1: {int32_view[1]}")

if __name__ == "__main__":
    main()

Run:

uv run python memoryview_cast.py

Output:

Raw byte view: [232, 3, 0, 0, 80, 195, 0, 0]
Cast int32 view length: 2
Integer 0: 1000
Integer 1: 50000

Pitfalls

Pitfall 1: Attempting In-Place Writes on an Immutable `bytes` View

If a memoryview wraps an immutable bytes object, the view is read-only. Attempting to assign to a slice raises TypeError:

# THE TRAP:
data = b"READ_ONLY_DATA"
view = memoryview(data)

try:
    view[0:4] = b"POST"  # TypeError: cannot modify read-only memory
except TypeError as err:
    print(f"Caught: {err}")

# THE FIX: Wrap a mutable bytearray instead
mutable_data = bytearray(b"READ_ONLY_DATA")
mutable_view = memoryview(mutable_data)
mutable_view[0:4] = b"POST"
print(f"Successfully modified: {mutable_data}")  # bytearray(b'POST_ONLY_DATA')

Pitfall 2: The Buffer Resizing Lock Trap

When a memoryview references a bytearray, CPython locks the underlying buffer. Attempting to append, extend, or resize the bytearray raises BufferError until the view is explicitly released:

# THE TRAP:
buf = bytearray(b"hello")
view = memoryview(buf)

try:
    buf.extend(b" world")  # BufferError: Existing exports of data: object cannot be re-sized
except BufferError as err:
    print(f"Caught: {err}")

# THE FIX: Explicitly release the view when finished with it
view.release()
buf.extend(b" world")
print(f"Cleanly resized buffer: {buf}")

Exercises

Create a 100 MB bytes object and benchmark the time required to extract a 10 MB slice using bytes slicing vs memoryview slicing.
Given a bytearray containing a message, use memoryview to reverse the middle 10 bytes in place without copying.
Write a function that accepts a memoryview of raw audio samples (16-bit PCM) and uses .cast('h') to find the peak amplitude sample.
Demonstrate the BufferError raised when attempting to .append() to a bytearray while an active memoryview is open. Call .release() and confirm .append() succeeds.
Implement a fixed-width binary log entry parser that reads 64-byte records using memoryview and extracts timestamps and event codes without allocating strings.

Zero-Copy Binary Memory with memoryview and bytearray

Mental model

Binary Types Compared

Minimal example

Worked examples

Case 1: Zero-Copy Network Packet Header Dissection

Case 2: In-Place Mutation with bytearray and memoryview

Case 3: Casting Memory Layouts with .cast()

Pitfalls

Pitfall 1: Attempting In-Place Writes on an Immutable bytes View

Pitfall 2: The Buffer Resizing Lock Trap

Exercises

Further reading

Zero-Copy Binary Memory with `memoryview` and `bytearray`

Case 2: In-Place Mutation with `bytearray` and `memoryview`

Case 3: Casting Memory Layouts with `.cast()`

Pitfall 1: Attempting In-Place Writes on an Immutable `bytes` View