encoding/binary, gob, and pem

Updated

September 8, 2026

encoding/binary, gob, and pem

Overview

Beyond JSON/CSV/XML (chapter 985), the stdlib encodes binary layouts, Go-native gob streams, and PEM blocks (certs/keys). Pick the format for the consumer: wire protocols → binary; Go-to-Go caches → gob (carefully); TLS material → pem + x509.

encoding/binary

Read and write numbers in a fixed endianness—file headers, network protocols, length-prefixed frames.

var buf bytes.Buffer
// write
if err := binary.Write(&buf, binary.BigEndian, uint32(42)); err != nil {
    return err
}
if err := binary.Write(&buf, binary.BigEndian, int16(-3)); err != nil {
    return err
}

// read
var u uint32
var s int16
r := bytes.NewReader(buf.Bytes())
if err := binary.Read(r, binary.BigEndian, &u); err != nil {
    return err
}
if err := binary.Read(r, binary.BigEndian, &s); err != nil {
    return err
}

Fixed size helpers

b := make([]byte, 8)
binary.BigEndian.PutUint64(b, 0xdeadbeef)
n := binary.BigEndian.Uint64(b)
_ = n
Endian Common use
binary.BigEndian Network byte order, many file formats
binary.LittleEndian x86-native dumps, some formats
binary.NativeEndian Same-machine only (Go 1.19+)

Length-prefixed message

func writeFrame(w io.Writer, payload []byte) error {
    if err := binary.Write(w, binary.BigEndian, uint32(len(payload))); err != nil {
        return err
    }
    _, err := w.Write(payload)
    return err
}

func readFrame(r io.Reader, max int) ([]byte, error) {
    var n uint32
    if err := binary.Read(r, binary.BigEndian, &n); err != nil {
        return nil, err
    }
    if int(n) > max {
        return nil, fmt.Errorf("frame %d exceeds max %d", n, max)
    }
    buf := make([]byte, n)
    _, err := io.ReadFull(r, buf)
    return buf, err
}

Always bound max—never trust remote lengths blindly.

encoding/gob

Gob is Go’s built-in binary serialization for Go types (including interfaces with registration). Great for same-version Go services and caches; not a long-term public API format.

type Item struct {
    SKU   string
    Count int
}

var buf bytes.Buffer
enc := gob.NewEncoder(&buf)
if err := enc.Encode(Item{SKU: "go-book", Count: 2}); err != nil {
    log.Fatal(err)
}
dec := gob.NewDecoder(&buf)
var out Item
if err := dec.Decode(&out); err != nil {
    log.Fatal(err)
}

Interfaces

gob.Register(Item{}) // once; before encode/decode of interface values

Caveats

Prefer gob when Avoid gob when
Go ↔︎ Go, controlled versions Public multi-language APIs
Short-lived cache blobs Long-term archival formats
Internal RPC experiments Anything needs schema evolution guarantees

For public APIs prefer JSON, Protobuf, or similar.

encoding/pem

PEM is the ASCII armor around DER bytes (-----BEGIN ...-----).

block, rest := pem.Decode(pemBytes)
if block == nil {
    return fmt.Errorf("no PEM block")
}
_ = rest
// block.Type e.g. "CERTIFICATE", "RSA PRIVATE KEY"
// block.Bytes is DER

Encode:

var buf bytes.Buffer
_ = pem.Encode(&buf, &pem.Block{Type: "CERTIFICATE", Bytes: der})

Pair with crypto/x509:

cert, err := x509.ParseCertificate(block.Bytes)

Never log private key PEM in application logs.

encoding/hex and base64 (recap)

h := hex.EncodeToString(sum[:])
raw, err := hex.DecodeString(h)

s := base64.StdEncoding.EncodeToString(data)
// URL-safe: base64.RawURLEncoding

Chapter 985 covers these in the JSON/CSV track; they pair naturally with binary digests and tokens.

Rules of thumb

Do Don’t
Cap length-prefixed frames Allocate make([]byte, n) from network without max
Document endianness Mix Big/Little without a spec
Use gob only in-process/versioned Go Expose gob as a mobile client protocol
Keep PEM keys off disk world-readable Commit private keys into git

Try next

  1. Implement writeFrame/readFrame over a bytes.Buffer and fuzz lengths.
  2. Gob-encode a slice of structs and decode into a new value.
  3. PEM-decode a test certificate and print Subject via x509.ParseCertificate.