Encoding and Serialization

Updated

September 13, 2026

Encoding and Serialization

The desk already has structs. Disk and the network want bytes. The boring default is encoding/json with struct tags for objects, encoding/csv for tables, and encoding/binary only when you are writing a fixed-width number. Do not invent a private text format if one of these three fits.

Mental model

JSON maps object keys to struct fields through tags: `json:"id"`. Exported fields are included. Unexported fields are omitted — silently. omitempty drops zeros and empty strings. json.Marshal produces bytes. json.Unmarshal fills a pointer.

CSV is rows of strings. encoding/csv quotes fields that need it. You still decide what each column means.

encoding/binary writes integers in big-endian or little-endian form. Use it for a length prefix or a packed id, not for a ticket with a note.

Unknown JSON keys are ignored by default. Missing keys leave the field at its zero value. That is convenient and easy to miss.

Worked examples

Case 1: JSON marshal and unmarshal

Save as ticket_json.go.

// ticket_json.go
package main

import (
    "encoding/json"
    "fmt"
)

type Ticket struct {
    ID    int    `json:"id"`
    Table int    `json:"table"`
    Note  string `json:"note"`
}

func main() {
    t := Ticket{ID: 7, Table: 12, Note: "no onions"}
    raw, err := json.Marshal(t)
    if err != nil {
        fmt.Println(err)
        return
    }
    fmt.Println(string(raw))

    var got Ticket
    if err := json.Unmarshal(raw, &got); err != nil {
        fmt.Println(err)
        return
    }
    fmt.Printf("id=%d table=%d note=%q\n", got.ID, got.Table, got.Note)
}

Run:

go run ticket_json.go

Output:

{"id":7,"table":12,"note":"no onions"}
id=7 table=12 note="no onions"

Marshal needs a value. Unmarshal needs a pointer. Field order in the JSON follows the struct.

Case 2: Tags that rename and omit

Save as ticket_tags.go. The Go field is Table; the JSON key is table_n. Empty notes disappear.

// ticket_tags.go
package main

import (
    "encoding/json"
    "fmt"
)

type Ticket struct {
    ID    int    `json:"id"`
    Table int    `json:"table_n"`
    Note  string `json:"note,omitempty"`
}

func main() {
    a, err := json.Marshal(Ticket{ID: 7, Table: 12, Note: "no onions"})
    if err != nil {
        fmt.Println(err)
        return
    }
    b, err := json.Marshal(Ticket{ID: 8, Table: 3})
    if err != nil {
        fmt.Println(err)
        return
    }
    fmt.Println(string(a))
    fmt.Println(string(b))
}

Run:

go run ticket_tags.go

Output:

{"id":7,"table_n":12,"note":"no onions"}
{"id":8,"table_n":3}

table: 0 would still appear: 0 is omitted only if you add omitempty on Table. Decide per field.

Case 3: CSV

Save as tickets_csv.go. Header row, two tickets, stdout.

// tickets_csv.go
package main

import (
    "encoding/csv"
    "fmt"
    "os"
    "strings"
)

func main() {
    var b strings.Builder
    w := csv.NewWriter(&b)
    if err := w.Write([]string{"id", "table", "note"}); err != nil {
        fmt.Println(err)
        return
    }
    if err := w.Write([]string{"7", "12", "no onions"}); err != nil {
        fmt.Println(err)
        return
    }
    if err := w.Write([]string{"8", "3", "extra napkins"}); err != nil {
        fmt.Println(err)
        return
    }
    w.Flush()
    if err := w.Error(); err != nil {
        fmt.Println(err)
        return
    }

    r := csv.NewReader(strings.NewReader(b.String()))
    rows, err := r.ReadAll()
    if err != nil {
        fmt.Println(err)
        return
    }
    for i, row := range rows {
        if i == 0 {
            continue
        }
        fmt.Printf("ticket %s table %s (%s)\n", row[0], row[1], row[2])
    }
    fmt.Fprint(os.Stdout, b.String())
}

Run:

go run tickets_csv.go

Output:

ticket 7 table 12 (no onions)
ticket 8 table 3 (extra napkins)
id,table,note
7,12,no onions
8,3,extra napkins

Flush before you read the builder. ReadAll is fine for a small sheet. For a large file, Read in a loop.

Case 4: Binary id

Save as ticket_id.go. Four bytes, big-endian.

// ticket_id.go
package main

import (
    "encoding/binary"
    "fmt"
)

func main() {
    var buf [4]byte
    binary.BigEndian.PutUint32(buf[:], 7)
    fmt.Printf("%x\n", buf)
    id := binary.BigEndian.Uint32(buf[:])
    fmt.Println(id)
}

Run:

go run ticket_id.go

Output:

00000007
7

That is the whole trick. A note string does not belong here. If you need a self-describing blob, use JSON.

The trap

Save as silent_json.go. Lowercase fields are invisible to encoding/json.

// silent_json.go
package main

import (
    "encoding/json"
    "fmt"
)

type ticket struct {
    id    int
    Table int `json:"table"`
}

func main() {
    raw, err := json.Marshal(ticket{id: 7, Table: 12})
    if err != nil {
        fmt.Println(err)
        return
    }
    fmt.Println(string(raw))
}

Run:

go run silent_json.go

Output:

{"table":12}

No error. The id is gone. The fix is an exported field (ID) and a tag. The same silence happens if you unmarshal into a struct with the wrong key names: zeros, no complaint. Compare against a golden JSON in a test when the payload matters.

The boring rule

  • JSON for structured records. Tags on every exported field you intend to ship.
  • Export the field. Tags do not override unexported.
  • CSV for rectangular tables of strings. Flush. Check Error.
  • binary for integers you control both ends of. Pick an endian and keep it.
  • Unmarshal into a pointer. Check the error. Then check that required fields are non-zero if zero would be wrong.
  • Do not fmt.Sprintf your own JSON.

Try this

  1. In ticket_json.go, unmarshal {"id":7} into a Ticket and print Table and Note. They are zero.
  2. Add `json:"table,omitempty"` to Case 2’s Table and marshal Ticket{ID: 1}. Confirm table_n is gone — then remember you renamed the key; the tag must say table_n,omitempty.
  3. In tickets_csv.go, put a comma inside a note (no onions, extra ice). Confirm the writer quotes the field and the reader still returns one cell.
  4. In ticket_id.go, write 7 with binary.LittleEndian and print %x. Compare with the big-endian line.