compress and archive

Updated

September 8, 2026

compress and archive

Overview

The standard library covers everyday compression and packaging: gzip, zlib, flate, bzip2 (reader), lzw, plus zip and tar archives. Use these for log shipping, backup tools, HTTP Content-Encoding, and release artifacts—without pulling a third-party zipper for common cases.

Package map

Package Role
compress/gzip .gz streams (HTTP, logs)
compress/zlib zlib wrappers
compress/flate raw DEFLATE
compress/bzip2 bzip2 reader only
archive/zip ZIP read/write
archive/tar tar read/write (often + gzip)

Gzip: compress a file

func gzipFile(src, dst string) error {
    in, err := os.Open(src)
    if err != nil {
        return err
    }
    defer in.Close()

    out, err := os.Create(dst)
    if err != nil {
        return err
    }
    defer out.Close()

    zw := gzip.NewWriter(out)
    zw.Name = filepath.Base(src)
    if _, err := io.Copy(zw, in); err != nil {
        _ = zw.Close()
        return err
    }
    return zw.Close() // flush trailer; check this error
}

Decompress

func gunzip(src io.Reader, dst io.Writer) error {
    zr, err := gzip.NewReader(src)
    if err != nil {
        return err
    }
    defer zr.Close()
    _, err = io.Copy(dst, zr)
    return err
}

HTTP Content-Encoding sketch

// response: write gzip when client accepts it
func gzipMiddleware(next http.Handler) http.Handler {
    return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
        if !strings.Contains(r.Header.Get("Accept-Encoding"), "gzip") {
            next.ServeHTTP(w, r)
            return
        }
        w.Header().Set("Content-Encoding", "gzip")
        gz := gzip.NewWriter(w)
        defer gz.Close()
        gzw := gzipResponseWriter{Writer: gz, ResponseWriter: w}
        next.ServeHTTP(gzw, r)
    })
}

(Production middleware also handles Content-Length, status codes, and Vary: Accept-Encoding—keep this as a pattern, not a drop-in.)

tar + gzip (.tar.gz)

func untarGz(r io.Reader, destDir string) error {
    gr, err := gzip.NewReader(r)
    if err != nil {
        return err
    }
    defer gr.Close()
    tr := tar.NewReader(gr)

    for {
        hdr, err := tr.Next()
        if errors.Is(err, io.EOF) {
            return nil
        }
        if err != nil {
            return err
        }
        // Prevent path traversal: clean + ensure under destDir
        target, err := safeJoin(destDir, hdr.Name)
        if err != nil {
            return err
        }
        switch hdr.Typeflag {
        case tar.TypeDir:
            if err := os.MkdirAll(target, 0o755); err != nil {
                return err
            }
        case tar.TypeReg:
            if err := os.MkdirAll(filepath.Dir(target), 0o755); err != nil {
                return err
            }
            f, err := os.OpenFile(target, os.O_CREATE|os.O_WRONLY|os.O_TRUNC, hdr.FileInfo().Mode())
            if err != nil {
                return err
            }
            if _, err := io.Copy(f, io.LimitReader(tr, hdr.Size)); err != nil {
                f.Close()
                return err
            }
            f.Close()
        }
    }
}

Path traversal: never write hdr.Name that escapes destDir (../ tricks). Always clean and verify a common root.

ZIP

// read
r, err := zip.OpenReader("site.zip")
defer r.Close()
for _, f := range r.File {
    rc, err := f.Open()
    // io.Copy to safe path under dest; then rc.Close()
}

// write
f, err := os.Create("out.zip")
zw := zip.NewWriter(f)
w, err := zw.Create("readme.txt")
_, err = io.WriteString(w, "hello")
err = zw.Close()

Same traversal rules as tar when extracting.

Streaming vs memory

  good:  io.Copy(gzipWriter, file)     // bounded memory
  bad:   gzip.Compress whole []byte     // for huge inputs

Prefer stream composition (io.Reader → compressor → io.Writer).

Rules of thumb

Do Don’t
Check Close on gzip writers Ignore close errors (corrupt archives)
Limit extracted sizes Trust archive headers blindly
Sanitize entry paths Extract zip-slip paths
Stream large payloads ReadAll multi-GB archives

Try next

  1. Gzip a log file and gunzip it; compare SHA-256.
  2. Build a tiny tar.gz of a directory listing.
  3. Attempt a zip-slip path and ensure your extractor rejects it.