Worker pools in Go: bounding concurrency without reaching for a library

Go makes starting a goroutine so cheap that it’s tempting to just loop over a slice and fire off go process(item) for every element, and for small lists that’s genuinely fine. It stops being fine the moment the list’s length depends on user input or external data instead of a number you control, and I learned exactly where that line sits from an incident I’d rather not have had.

The incident that taught me the lesson

We had a CSV import feature: a customer uploads a file, each row gets enriched by calling a third-party API, and the enriched rows get written back to Postgres. The first version of that job looked almost exactly like the naive pattern above, one goroutine per row, no limit.

// don't do this
for _, row := range rows {
    row := row
    go func() {
        enriched, err := enrichRow(ctx, row)
        if err != nil {
            log.Error("enrich failed", "err", err)
            return
        }
        saveRow(ctx, enriched)
    }()
}

It worked in every test we ran, because every test file had a few dozen rows. Then a customer uploaded a file with about twelve thousand rows, and within a second or two we had roughly twelve thousand goroutines all trying to open an outbound HTTP connection to the same third-party API at once. Two things failed at the same moment. The third party’s own rate limiter saw a burst that looked exactly like abuse and started returning 429s and, once we kept retrying, temporary IP-level blocks. And our own Postgres connection pool, sized for ordinary request traffic, got twelve thousand near-simultaneous write attempts queued against a pool of twenty connections, which meant every unrelated request hitting the database during that window started timing out too.

Nothing about that code was wrong Go. Goroutines really are cheap, in the sense that spinning up twelve thousand of them didn’t crash the process. The problem was entirely downstream: an unbounded number of concurrent callers hitting two resources, a third-party rate limit and a connection pool, that were never designed to absorb an unbounded number of concurrent callers. Fixing it meant putting an actual ceiling on how many rows could be in flight at once, which is what a worker pool is for.

The simplest possible bound: a semaphore

Before reaching for a full worker pool, the smallest fix that would have prevented the incident is a semaphore, a buffered channel used purely to cap concurrency, with nothing waiting to be read out of it.

func enrichAll(ctx context.Context, rows []Row, maxConcurrency int) {
    sem := make(chan struct{}, maxConcurrency)
    var wg sync.WaitGroup

    for _, row := range rows {
        row := row
        wg.Add(1)
        sem <- struct{}{} // blocks once maxConcurrency slots are full

        go func() {
            defer wg.Done()
            defer func() { <-sem }()

            enriched, err := enrichRow(ctx, row)
            if err != nil {
                log.Error("enrich failed", "err", err)
                return
            }
            saveRow(ctx, enriched)
        }()
    }

    wg.Wait()
}

sem <- struct{}{} blocks as soon as maxConcurrency goroutines are already running, which means the loop itself stalls on that send until a slot frees up, and no more than maxConcurrency rows are ever in flight at once. This is enough for a lot of real cases: you already have the full list of work up front, you don’t need long-lived workers, and you just need a hard cap. It would have fully prevented the incident above with a single number, maxConcurrency: 20, chosen to match what the third-party API and our own connection pool could actually absorb.

When you actually want a worker pool instead

The semaphore version has one limitation: it still creates one goroutine per row, it just limits how many run concurrently. That’s fine at twelve thousand rows. It stops being fine if the number of items is unbounded or very large, because you’re still allocating a goroutine, plus its stack and the closure state, for every single item even while most of them are blocked waiting for a semaphore slot.

A worker pool flips that around: you start a fixed, small number of long-lived goroutines up front, and feed them work over a channel. No goroutine gets created per item, only per worker.

func enrichAll(ctx context.Context, rows []Row, workers int) error {
    jobs := make(chan Row)
    g, ctx := errgroup.WithContext(ctx)

    for i := 0; i < workers; i++ {
        g.Go(func() error {
            for row := range jobs {
                enriched, err := enrichRow(ctx, row)
                if err != nil {
                    return err
                }
                if err := saveRow(ctx, enriched); err != nil {
                    return err
                }
            }
            return nil
        })
    }

    g.Go(func() error {
        defer close(jobs)
        for _, row := range rows {
            select {
            case jobs <- row:
            case <-ctx.Done():
                return ctx.Err()
            }
        }
        return nil
    })

    return g.Wait()
}

errgroup.WithContext gives two things a hand-rolled sync.WaitGroup doesn’t: error propagation and cancellation that actually reach every worker. If any worker’s enrichRow or saveRow returns an error, that error cancels the shared context, and the producer goroutine feeding jobs picks that up on its next select and stops sending instead of blocking forever on a channel nobody’s reading anymore. Every worker’s for row := range jobs loop exits cleanly once jobs closes. g.Wait() blocks until every worker has actually returned and hands back the first error, if there was one.

This is the version I’d reach for now, both for the CSV job and for context cancellation done right, a topic that an earlier post covered on its own: the context created by errgroup.WithContext is exactly the context that should get threaded into every downstream call inside a worker, enrichRow(ctx, row) and saveRow(ctx, enriched), so that cancelling the group actually cancels the in-flight HTTP calls and database writes too, not just the loop that’s dispatching new work.

Fail-fast versus collect-everything

The pattern above is fail-fast: the first error cancels the whole group, which is right for a job where one row failing means the rest of the batch should stop too, an import where a schema mismatch on row one implies every row will fail the same way. It’s wrong for a job where you want to process every row independently and report which ones failed at the end, a bulk enrichment where row 4,001 timing out shouldn’t stop rows 4,002 through 12,000 from completing.

For that case, don’t cancel on error, collect errors instead, under a mutex, and let every worker keep running to completion:

var mu sync.Mutex
var errs []error

g.Go(func() error {
    for row := range jobs {
        if _, err := enrichRow(ctx, row); err != nil {
            mu.Lock()
            errs = append(errs, fmt.Errorf("row %s: %w", row.ID, err))
            mu.Unlock()
            continue // keep going, don't return
        }
    }
    return nil
})

The difference is one word, return err versus continue, but it changes the entire failure semantics of the batch, and it’s worth deciding on purpose rather than by whichever example you copied first.

Picking a number for maxConcurrency

The number itself matters more than the pattern, and it isn’t a Go question at all, it’s a question about the resource on the other end. Twenty was the right number for us because that’s roughly what our Postgres pool and the third party’s rate limit could sustain together, not because twenty is a good default for anything else. The honest way to set it is to look at the actual constraint, connection pool size, a documented rate limit, a downstream service’s own concurrency headroom, and set the worker count to comfortably fit under that ceiling, then load test the actual number rather than guessing round numbers that happen to look reasonable.

What actually changed in how I write fan-out code now

I stopped treating “how many goroutines” as a question Go’s runtime would sort out on its own, because it will, cheerfully, right up until something outside the process, a rate limit, a connection pool, a downstream service’s own capacity, can’t sort it out the same way. The pattern I reach for by default now is errgroup plus a fixed worker count sized to the actual downstream constraint, with a deliberate choice about fail-fast versus collect-all made up front instead of discovered later during an incident review.