os, filepath, io/fs, and embed

os, filepath, io/fs, and embed

Overview

These packages cover process environment and files:

Package	Role
os	Files, env, args, process exit, working directory
path/filepath	OS-aware path manipulation
io/fs	Abstract filesystem interface
embed	Ship files inside the binary

Path safety and portable tools depend on getting this layer right.

os basics

data, err := os.ReadFile("config.json")     // Go 1.16+
err = os.WriteFile("out.txt", data, 0o644)
f, err := os.OpenFile(path, os.O_RDWR|os.O_CREATE, 0o644)
info, err := os.Stat(path)
os.MkdirAll("a/b/c", 0o755)
os.Remove(path)
os.RemoveAll(dir) // careful

Env and args

os.Args          // argv; Args[0] is program name
os.Getenv("HOME")
os.LookupEnv("PORT") // value, ok
os.Exit(1)       // skips deferred cleanup in other goroutines

Prefer returning errors to main and a single os.Exit there.

filepath: always OS-aware

Use path/filepath for local disk paths. Use path (slash-only) for URL-like paths.

filepath.Join("a", "b", "c.txt")
filepath.Clean("../etc/passwd")
filepath.Abs(".")
filepath.Ext("main.go")      // ".go"
filepath.Base("/tmp/x")      // "x"
filepath.Dir("/tmp/x")       // "/tmp"
filepath.WalkDir(root, fn)   // preferred over Walk

Prevent path traversal

func safeJoin(root, name string) (string, error) {
    clean := filepath.Clean("/" + name) // force absolute-like clean
    rel := strings.TrimPrefix(clean, string(filepath.Separator))
    full := filepath.Join(root, rel)
    rootAbs, err := filepath.Abs(root)
    if err != nil {
        return "", err
    }
    fullAbs, err := filepath.Abs(full)
    if err != nil {
        return "", err
    }
    sep := string(os.PathSeparator)
    if fullAbs != rootAbs && !strings.HasPrefix(fullAbs, rootAbs+sep) {
        return "", fmt.Errorf("path escapes root")
    }
    return fullAbs, nil
}

io/fs abstraction

var fsys fs.FS = os.DirFS("/data")
b, err := fs.ReadFile(fsys, "hello.txt")
entries, err := fs.ReadDir(fsys, ".")
err = fs.WalkDir(fsys, ".", func(path string, d fs.DirEntry, err error) error {
    return err
})

Benefits: tests can swap fstest.MapFS; embed.FS implements fs.FS.

embed

import "embed"

//go:embed static/*
var staticFS embed.FS

//go:embed VERSION
var version string

http.Handle("/static/", http.FileServer(http.FS(staticFS)))

Rules:

• Paths are relative to the source file.
• No .. in embed patterns.
• Embedded content is read-only at runtime.

See also codegen / embed.

Runnable example

go mod init example
go run .

package main

import (
    "fmt"
    "io/fs"
    "os"
    "path/filepath"
    "testing/fstest"
)

func main() {
    dir, err := os.MkdirTemp("", "stdlib-fs-*")
    if err != nil {
        panic(err)
    }
    defer os.RemoveAll(dir)

    path := filepath.Join(dir, "note.txt")
    if err := os.WriteFile(path, []byte("hello fs\n"), 0o644); err != nil {
        panic(err)
    }

    data, _ := os.ReadFile(path)
    fmt.Printf("disk=%q\n", data)

    // Walk
    _ = filepath.WalkDir(dir, func(p string, d fs.DirEntry, err error) error {
        if err != nil {
            return err
        }
        rel, _ := filepath.Rel(dir, p)
        fmt.Println("walk:", rel, "dir?", d.IsDir())
        return nil
    })

    // Abstract FS (great for tests)
    mapFS := fstest.MapFS{
        "a.txt":  {Data: []byte("A")},
        "b/c.txt": {Data: []byte("C")},
    }
    b, _ := fs.ReadFile(mapFS, "b/c.txt")
    fmt.Printf("mapfs=%q\n", b)

    fmt.Println("join:", filepath.Join("var", "log", "app.log"))
    fmt.Println("clean:", filepath.Clean("a//b/../c"))
}

Expected output (temp path names vary):

disk="hello fs\n"
walk: . dir? true
walk: note.txt dir? false
mapfs="C"
join: var/log/app.log
clean: a/c

What to notice: - DirFS / MapFS / embed.FS share one fs.FS API. - WalkDir reports the root as "." with IsDir()==true. - Cleaning alone does not make a path safe relative to a root — join + prefix check.

Try next: Serve embed.FS with http.FileServer(http.FS(...)) and verify a missing file returns 404.