Structured Data Serialization: JSON, CSV, and TOML

Updated

September 7, 2026

Structured Data Serialization: JSON, CSV, and TOML

After reading this chapter, you will master Python’s built-in serialization libraries: encoding and decoding JSON payloads with custom type hooks, processing structured tabular datasets with csv.DictReader and csv.DictWriter, and parsing configuration files using the built-in tomllib parser.

Mental model

Serialization translates in-memory Python objects (dictionaries, lists, primitives) into standardized byte or text streams suitable for disk persistence or network transmission:

Python Memory              Standard Library Module         Serialized Format
─────────────────────────────────────────────────────────────────────────────
dict / list / primitives   ──▶ json.dumps()           ──▶ JSON text string
tabular record list        ──▶ csv.DictWriter()       ──▶ Comma-Separated Values
TOML configuration text    ──▶ tomllib.loads()        ──▶ Python dict

While JSON supports only a subset of primitives (str, int, float, bool, list, dict, None), Python provides extension hooks (default= and object_hook=) to encode domain types like datetime, UUID, or set.

Minimal example

Save as serialization_showcase.py:

# serialization_showcase.py
import csv
import io
import json
import tomllib

def main() -> None:
    # 1. JSON Serialization
    cluster_state = {
        "cluster": "k8s-prod-us",
        "nodes": 3,
        "active": True,
        "tags": ["web", "api"],
    }
    json_str = json.dumps(cluster_state, indent=2)
    print("--- Serialized JSON ---")
    print(json_str)

    # 2. TOML Parsing (Built-in via tomllib in Python 3.11+)
    raw_toml = """
    [server]
    host = "0.0.0.0"
    port = 8080

    [database]
    pool_size = 10
    timeout_seconds = 5.5
    """
    config_dict = tomllib.loads(raw_toml)
    print("\n--- Parsed TOML Configuration ---")
    print(f"Host: {config_dict['server']['host']}:{config_dict['server']['port']}")
    print(f"DB Timeout: {config_dict['database']['timeout_seconds']}s")

    # 3. CSV Tabular Generation with DictWriter
    csv_buffer = io.StringIO()
    fieldnames = ["hostname", "ip", "status"]
    writer = csv.DictWriter(csv_buffer, fieldnames=fieldnames)
    writer.writeheader()
    writer.writerow({"hostname": "node-01", "ip": "10.0.1.10", "status": "UP"})
    writer.writerow({"hostname": "node-02", "ip": "10.0.1.11", "status": "UP"})

    print("\n--- Generated CSV ---")
    print(csv_buffer.getvalue().strip())

if __name__ == "__main__":
    main()

Run via uv run python serialization_showcase.py:

--- Serialized JSON ---
{
  "cluster": "k8s-prod-us",
  "nodes": 3,
  "active": true,
  "tags": [
    "web",
    "api"
  ]
}

--- Parsed TOML Configuration ---
Host: 0.0.0.0:8080
DB Timeout: 5.5s

--- Generated CSV ---
hostname,ip,status
node-01,10.0.1.10,UP
node-02,10.0.1.11,UP

Worked examples

Case 1: Custom JSON Serialization for datetime, UUID, and set

Attempting to serialize non-native types raises TypeError. Supplying a custom default= callable serializes extended types cleanly:

# custom_json_encoder.py
import json
from datetime import datetime, timezone
from uuid import UUID, uuid4
from typing import Any

def custom_json_serializer(obj: Any) -> Any:
    if isinstance(obj, (datetime)):
        return obj.isoformat()
    if isinstance(obj, UUID):
        return str(obj)
    if isinstance(obj, set):
        return sorted(list(obj))
    raise TypeError(f"Object of type {type(obj).__name__} is not JSON serializable")

if __name__ == "__main__":
    telemetry_payload = {
        "event_id": uuid4(),
        "timestamp": datetime.now(timezone.utc),
        "tags": {"prod", "us-east-1", "critical"},
        "metric": 98.4,
    }

    serialized = json.dumps(telemetry_payload, indent=2, default=custom_json_serializer)
    print("Serialized payload with custom types:")
    print(serialized)

Run:

uv run python custom_json_encoder.py

Output:

Serialized payload with custom types:
{
  "event_id": "...",
  "timestamp": "2026-09-07T...",
  "tags": [
    "critical",
    "prod",
    "us-east-1"
  ],
  "metric": 98.4
}

Case 2: Reading CSV with DictReader and Type Conversion

csv.reader returns raw string lists (["10.0.1.1", "8080"]). DictReader binds values to header column names:

# parse_nodes_csv.py
import csv
import io

RAW_CSV_DATA = """hostname,port,healthy
app-node-01,8080,true
app-node-02,8080,false
db-primary,5432,true
"""

def parse_node_inventory(csv_text: str) -> list[dict[str, str | int | bool]]:
    stream = io.StringIO(csv_text.strip())
    reader = csv.DictReader(stream)
    nodes = []

    for row in reader:
        nodes.append({
            "hostname": row["hostname"],
            "port": int(row["port"]),
            "healthy": row["healthy"].strip().lower() == "true",
        })
    return nodes

if __name__ == "__main__":
    inventory = parse_node_inventory(RAW_CSV_DATA)
    print("Parsed and typed node inventory:")
    for node in inventory:
        print(f"  {node['hostname']:15} | Port: {node['port']:5d} | Healthy: {node['healthy']}")

Run:

uv run python parse_nodes_csv.py

Output:

Parsed and typed node inventory:
  app-node-01     | Port:  8080 | Healthy: True
  app-node-02     | Port:  8080 | Healthy: False
  db-primary      | Port:  5432 | Healthy: True

Pitfalls

Pitfall 1: Writing CSVs on Windows Without newline=""

When opening a file for csv.writer, failing to pass newline="" causes Python and the CSV module to double-write carriage returns (\r\r\n), resulting in blank lines between every record:

# THE BUG:
with open("output.csv", "w") as f:
    writer = csv.writer(f)

# THE FIX:
with open("output.csv", "w", newline="", encoding="utf-8") as f:
    writer = csv.writer(f)

Pitfall 2: tomllib is Read-Only

The standard library tomllib module (PEP 680) provides high-performance TOML parsing only (loads and load). If your application needs to write TOML files, use the third-party package tomli_w or serialize to YAML/JSON.

Exercises

  1. Write a script that reads a JSON file, modifies a nested key, and writes it back atomically using formatted indentation (indent=2).
  2. Create a custom JSON decoder using object_hook that automatically converts ISO-8601 string timestamps back into datetime objects.
  3. Parse a CSV file containing user IDs, filter out inactive users, and write the remaining users to a new CSV using csv.DictWriter.
  4. Parse a pyproject.toml file with tomllib.loads() and print the project name and dependencies list.

Further reading

  • PEP 680: tomllib: Support for Parsing TOML in the Standard Library.
  • Python Standard Library: json, csv, and tomllib modules.
  • JSON Specification: RFC 8259.