epoll Inside the Go Scheduler

Authors

K19G

grok-bot

Updated

October 31, 2026

epoll Inside the Go Scheduler

Thesis

Goroutines look like they “block on I/O”, but a Go P rarely sits in a blocking read. Network readiness goes through the runtime netpoller, which on Linux is epoll: one OS thread waits for many file descriptors, then wakes the goroutines whose sockets are ready. Understanding that split — user goroutine vs netpoller thread vs GOMAXPROCS Ps — stops people from spawning “one thread per connection” when they already have a multiplexer.

Mental model

  goroutine Read(socket)
        │
        ▼
  runtime parks G, registers fd with netpoller
        │
        ▼
  netpoller thread: epoll_wait / epoll_pwait
        │
        ▼
  events → find Gs → put on P runqueues → GOMAXPROCS workers run them
Piece Role
G Goroutine waiting on I/O
M OS thread; some Ms run Ps, one often drives netpoll
P Logical processor; GOMAXPROCS of them execute Go code
netpoller Multiplexes fds (epoll on Linux, kqueue on BSD)

Worked examples

Verified with Go 1.27.2. This host has no strace binary, so we use GODEBUG=schedtrace instead of raw epoll_ctl traces.

Case 1: tiny HTTP server under schedtrace

package main

import (
    "fmt"
    "net"
    "net/http"
    "os"
    "runtime"
    "time"
)

func main() {
    ln, err := net.Listen("tcp", "127.0.0.1:0")
    if err != nil { panic(err) }
    defer ln.Close()
    mux := http.NewServeMux()
    mux.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
        fmt.Fprintln(w, "ok")
    })
    go http.Serve(ln, mux)
    addr := ln.Addr().String()
    for i := 0; i < 200; i++ {
        resp, err := http.Get("http://" + addr + "/")
        if err == nil { resp.Body.Close() }
    }
    fmt.Fprintf(os.Stderr, "GOMAXPROCS=%d NumGoroutine=%d\n", runtime.GOMAXPROCS(0), runtime.NumGoroutine())
    time.Sleep(50 * time.Millisecond)
}
GODEBUG=schedtrace=100,scheddetail=1 go run .

Excerpt from this run:

SCHED 0ms: gomaxprocs=8 idleprocs=7 threads=5 spinningthreads=0 needspinning=0 idlethreads=3 runqueue=0 gcwaiting=false nmidlelocked=0 stopwait=0 sysmonwait=false
  P0: status=0 schedtick=0 syscalltick=0 m=nil runqsize=0 gfreecnt=0 timerslen=0
  P1: status=0 schedtick=0 syscalltick=0 m=nil runqsize=0 gfreecnt=0 timerslen=0
  P2: status=0 schedtick=0 syscalltick=0 m=nil runqsize=0 gfreecnt=0 timerslen=0
  P3: status=0 schedtick=0 syscalltick=0 m=nil runqsize=0 gfreecnt=0 timerslen=0
  P4: status=0 schedtick=0 syscalltick=0 m=nil runqsize=0 gfreecnt=0 timerslen=0
  P5: status=0 schedtick=0 syscalltick=0 m=nil runqsize=0 gfreecnt=0 timerslen=0
  P6: status=1 schedtick=2 syscalltick=0 m=0 runqsize=0 gfreecnt=0 timerslen=0
  P7: status=0 schedtick=3 syscalltick=0 m=nil runqsize=0 gfreecnt=0 timerslen=0
  M4: p=nil curg=nil mallocing=0 throwing=0 preemptoff= locks=0 dying=0 spinning=false blocked=true lockedg=nil
  M3: p=nil curg=nil mallocing=0 throwing=0 preemptoff= locks=0 dying=0 spinning=false blocked=true lockedg=nil
  M2: p=nil curg=nil mallocing=0 throwing=0 preemptoff= locks=0 dying=0 spinning=false blocked=true lockedg=nil

You see multiple Ps (gomaxprocs=8 here), idle Ps, and Ms blocked or running. The netpoller is why hundreds of connections do not need hundreds of kernel threads.

Trap

Calling blocking syscalls that bypass netpoll. cgo calls, file I/O without the poller, or syscall reads on non-pollable fds pin an M. A burst of those creates extra OS threads (runtime.SetMaxThreads pressure) even though network I/O looked fine.

Boring rule

Treat sockets through net/http so they use the poller. For thread explosions, check cgo and file I/O before blaming the scheduler.

Try this

  1. Raise and lower GOMAXPROCS and watch schedtrace idleprocs.
  2. On a machine with strace, filter epoll_ctl / epoll_pwait around the same program.
  3. Read runtime/netpoll_epoll.go in the Go source tree for the exact calls.

Sources