Low-Level Socket Options and Syscall Control

Updated

October 31, 2026

Low-Level Socket Options and Syscall Control

The boring default is to use standard net.Listen("tcp", addr) and net.Dial("tcp", addr). Go configures safe, modern defaults out of the box, such as enabling TCP_NODELAY. However, when tuning ultra-low-latency services, managing multi-process architectures, or implementing zero-downtime upgrades, you need access to the underlying operating system file descriptors via net.ListenConfig.Control and syscall.RawConn.

Mental model

In Go, low-level socket configuration is hooked before the operating system binds or connects the socket:

  • net.ListenConfig: Replaces net.Listen, providing a Control callback that receives the raw socket file descriptor (fd) before bind() and listen() syscalls are executed.
  • net.Dialer.Control: Similarly provides the raw file descriptor before connect() is issued.
  • syscall.RawConn: An interface representing a network socket that allows non-blocking execution of syscall.SetsockoptInt.

Essential Socket Options

  1. TCP_NODELAY (Disabling Nagle’s Algorithm):
    • Nagle’s algorithm combines small outgoing data packets to fill maximum segment size (MSS) frames, reducing network packet congestion.
    • For interactive RPCs or low-latency financial/desk transactions, Nagle’s algorithm introduces artificial delays (40ms–200ms) waiting for delayed ACKs.
    • Go enables TCP_NODELAY by default on all TCP connections. You rarely need to set it manually, but you must know it is active.
  2. SO_REUSEPORT (Port Sharing & Zero-Downtime):
    • Standard sockets prevent multiple processes from binding to the same IP:port (bind: address already in use).
    • Enabling SO_REUSEPORT on Linux/macOS allows multiple independent processes or worker loops to bind to the identical port.
    • The Linux kernel load-balances incoming connection requests across the listeners directly in kernel space. This makes zero-downtime binary restarts and rolling process upgrades seamless.
  3. TCP_KEEPIDLE and TCP_KEEPINTVL:
    • Operating system-level TCP keepalive probes detect dead client cables and severed connections even if user-space code is temporarily paused.

Worked examples

Case 1: Enabling SO_REUSEPORT for multi-listener binding

Save as reuseport_demo.go. We create two separate listeners bound to the exact same TCP port using net.ListenConfig.Control.

// reuseport_demo.go
package main

import (
    "context"
    "fmt"
    "net"
    "syscall"
    "time"
)

func createReusePortListener(addr string) (net.Listener, error) {
    lc := net.ListenConfig{
        Control: func(network, address string, c syscall.RawConn) error {
            var opErr error
            err := c.Control(func(fd uintptr) {
                // SO_REUSEPORT is 0xf on Linux
                opErr = syscall.SetsockoptInt(int(fd), syscall.SOL_SOCKET, 0xf, 1)
            })
            if err != nil {
                return err
            }
            return opErr
        },
    }
    return lc.Listen(context.Background(), "tcp", addr)
}

func main() {
    addr := "127.0.0.1:49152"

    // 1. Create first listener on port
    ln1, err := createReusePortListener(addr)
    if err != nil {
        panic(err)
    }
    defer ln1.Close()
    fmt.Printf("Listener 1 successfully bound to %s\n", addr)

    // 2. Create second listener on EXACT SAME port (enabled by SO_REUSEPORT)
    ln2, err := createReusePortListener(addr)
    if err != nil {
        panic(err)
    }
    defer ln2.Close()
    fmt.Printf("Listener 2 successfully bound to %s (zero-downtime reload capable)\n", addr)

    // Verify clients can connect
    conn, err := net.DialTimeout("tcp", addr, time.Second)
    if err != nil {
        panic(err)
    }
    defer conn.Close()
    fmt.Println("Client successfully connected to SO_REUSEPORT socket")
}

Run:

go run reuseport_demo.go

Output:

Listener 1 successfully bound to 127.0.0.1:49152
Listener 2 successfully bound to 127.0.0.1:49152 (zero-downtime reload capable)
Client successfully connected to SO_REUSEPORT socket

Without SO_REUSEPORT, the second Listen call would immediately return bind: address already in use. With it, the kernel accepts connections and distributes them evenly across both listener sockets.

Case 2: Inspecting TCP_NODELAY on an established connection

To verify or adjust TCP_NODELAY on an existing TCP connection:

// nodelay_demo.go
package main

import (
    "fmt"
    "net"
)

func main() {
    ln, _ := net.Listen("tcp", "127.0.0.1:0")
    defer ln.Close()

    go func() {
        c, _ := ln.Accept()
        if tcpConn, ok := c.(*net.TCPConn); ok {
            // Explicitly set NoDelay (true by default in Go)
            _ = tcpConn.SetNoDelay(true)
        }
        _ = c.Close()
    }()

    conn, _ := net.Dial("tcp", ln.Addr().String())
    defer conn.Close()

    fmt.Println("TCP_NODELAY configured successfully")
}

Run:

go run nodelay_demo.go

Output:

TCP_NODELAY configured successfully

The trap

The most common trap with SO_REUSEPORT is assuming connection state is shared across processes. If process A accepts a connection, process B knows nothing about it. If process A terminates abruptly, any active connections currently held by process A are terminated.

For graceful zero-downtime restarts: 1. Start the new process with SO_REUSEPORT. 2. Signal the old process to stop accepting new connections (ln.Close()). 3. Allow the old process to drain existing in-flight transactions before terminating.

The boring rule

  • Trust Go’s default net.Listen and net.Dial for typical web services and microservices.
  • Rely on Go’s default TCP_NODELAY=true unless you are streaming massive continuous media files and intentionally want packet coalescing.
  • Use net.ListenConfig.Control when you need OS-specific options (SO_REUSEPORT, SO_RCVBUF, or custom TCP keepalives).
  • On Linux, prefer SO_REUSEPORT over complex socket passing via Unix domain sockets for zero-downtime process restarts.

Try this

  1. In reuseport_demo.go, remove the syscall.SetsockoptInt call inside Control. Observe how the program fails on the second listener with bind: address already in use.
  2. Inspect the raw file descriptor integer value inside c.Control(func(fd uintptr) { ... }).
  3. Configure SO_RCVBUF and SO_SNDBUF on a listener socket to 16 KB and verify with syscall.GetsockoptInt that the kernel applied the setting.