Golang Fan-Out Fan-In Pattern: Goroutines, Channels, and Worker Pools

This guide explains fan-out and fan-in in Go for developers who already know goroutines and channels. It separates the vocabulary from worker pools, walks through a small merge pipeline, then covers errors, cancellation, ordering, how many workers to run, and channel ownership—without treating every program as a concurrency exercise.

Tested with Go 1.24 on Linux.

Quick answer: fan-out distributes work, fan-in merges results

Fan-out means handing work to many goroutines so independent tasks run at the same time—often several workers reading from one jobs channel. Fan-in means taking those parallel outputs and combining them into one stream—often one goroutine merging many channels into one. Use the pattern when tasks are independent and you can merge or consume results as they finish; decide error and ordering rules before you wire the channels.

What is the fan-out fan-in pattern?

In Go, fan-out/fan-in is usually built from goroutines, channels, and coordination primitives such as sync.WaitGroup or errgroup (extension). One producer or stage creates work; many goroutines process it in parallel (fan-out); a merge or collector stage presents a single outbound stream to the rest of the program (fan-in). That shape is common for parallel file or API work, validation batches, and queue consumers.

Why it is useful

Throughput rises when work is embarrassingly parallel: each unit does not depend on others finishing first. The cost is design complexity—closing channels correctly, handling errors, and avoiding leaks when something stops early.

Fan-out vs fan-in

Fan-out increases parallelism; fan-in brings results back together for one place to read, log, or aggregate.

Visual flow

+----------+
        | producer |
        +----+-----+
             |
             v
      (jobs channel)
             |
    +--------+---------+
    |        |         |
    v        v         v
 worker   worker     worker     <- fan-out (same code, parallel)
    |        |         |
    +--------+---------+
             |
      (merge / multiplex)
             |
             v
        +----+----+
        | consumer |                    <- fan-in (single outbound)
        +----------+

Where this fits in Go concurrency

For broader concurrency background, see concurrency in Go, WaitGroup, and context. Cooperative shutdown patterns overlap with stopping goroutines.

Basic fan-out fan-in flow

1. Create the input work (slice, iterator, or generator goroutine sending on a channel).
2. Start workers or parallel branches that read jobs and write results.
3. Optionally multiplex many result channels with a merge that copies into one chan T.
4. When all senders on a channel are done, close that channel so range exits.
5. After all merge copies finish, close the shared output channel so the final consumer’s range exits.

The program below is a classic pipeline: two generator branches feed cube stages, and merge fan-in multiplexes both cubes streams into one channel. Run it with go run .; you should see six cubed integers (values 1³ through 6³). The line order can change because the merge interleaves goroutines—that is normal fan-in behavior.

package main

import (
	"fmt"
	"sync"
)

func generator(nums ...int) <-chan int {
	out := make(chan int)
	go func() {
		defer close(out)
		for _, n := range nums {
			out <- n
		}
	}()
	return out
}

func cube(in <-chan int) <-chan int {
	out := make(chan int)
	go func() {
		defer close(out)
		for n := range in {
			out <- n * n * n
		}
	}()
	return out
}

func merge(in ...<-chan int) <-chan int {
	var wg sync.WaitGroup
	out := make(chan int)
	output := func(c <-chan int) {
		defer wg.Done()
		for n := range c {
			out <- n
		}
	}
	wg.Add(len(in))
	for _, c := range in {
		go output(c)
	}
	go func() {
		wg.Wait()
		close(out)
	}()
	return out
}

func main() {
	c1 := generator(1, 2, 3)
	c2 := generator(4, 5, 6)
	out := merge(cube(c1), cube(c2))
	for n := range out {
		fmt.Println(n)
	}
}

Fan-out fan-in vs worker pool

A worker pool is often how you implement fan-out with a cap on concurrency. Fan-in is how you merge or serialize results unless each worker writes to its own sink.

The terms overlap in real code. The important question is whether you need a hard limit on concurrent goroutines. If yes, use a worker-pool style fan-out. If not, the pattern may simply be a logical "split work, run in parallel, merge results" flow.

For example:

One subtle point: in many Go examples, fan-out means workers share the input and each item is processed by one worker. It does not always mean every worker receives every item.

Real-world scenarios

Use the pattern when work items are independent and you can clearly define how results and errors are combined.

Avoid using fan-out/fan-in only because concurrency looks attractive. It adds coordination overhead, so it is most useful when parallelism improves throughput, hides I/O latency, or keeps worker count bounded.

Handling errors

Channels of plain values do not carry errors. Common approaches:

Decide the policy before you fix the channel topology—fail-fast needs a way to unblock workers stuck on send (cancel or close paths), while “collect all” needs a bounded way to store errors without blocking forever.

A small pattern is to fan-in a Result value (or a (T, error) pair) on one channel so the consumer sees both data and errors in completion order:

type Result struct {
	Path string
	Size int64
	Err  error
}

// Workers send Result{Path: p, Err: err} on a shared or per-worker channel;
// merger copies into one stream; consumer checks res.Err per message.

For fail-fast across many goroutines, errgroup.Group is a common choice: the first non-nil error cancels the group context so sibling workers exit cleanly.

Cancellation and timeouts

If the consumer stops reading early but workers keep sending on a full or unbuffered result channel, sends can block forever—a goroutine leak. Tie shutdown to context.Context: workers select on ctx.Done() alongside real work; the producer stops enqueueing when the context is canceled; the merger exits and closes the output channel once workers finish.

A worker should also check cancellation before sending the result:

select {
case out <- result:
    // result delivered
case <-ctx.Done():
    return
}

This prevents the worker from getting stuck when the downstream consumer has already stopped reading.

The same idea applies when reading jobs:

select {
case job, ok := <-jobs:
    if !ok {
        return
    }
    // process job
case <-ctx.Done():
    return
}

The key rule is:

Every blocking send or receive should have a cancellation path when early exit is possible.

If you cancel, avoid sending on closed channels and avoid closing a channel more than once. When in doubt, use one goroutine that owns close on each channel end.

Ordering of results

Fan-out/fan-in usually does not preserve input order. Whichever worker finishes first sends its result first.

For example, assume the input order is:

job-1, job-2, job-3

But workers may finish like this:

job-2, job-3, job-1

That means the fan-in output may look different from the input order.

This is fine when order does not matter, such as:

• health checks
• independent file processing
• parallel API calls
• validation jobs
• background queue work

If order matters, attach a sequence index to each job before fan-out:

type Job struct {
    Index int
    Value string
}

type Result struct {
    Index int
    Value string
    Err   error
}

Workers can process jobs in any order, but each result still carries the original index.

After collecting results, sort or reorder them by Index before returning them to the caller.

Short rule:

Fan-out improves concurrency, but result order follows completion order.
If input order matters, carry an index and reorder after fan-in.

Another option is to use a single worker when strict order is required and parallelism is not worth the extra coordination.

Choosing how many workers

Fan-out does not mean "unbounded goroutines per item". Large inputs need a pool, a semaphore, or chunked scheduling.

Example:

Short rule:

Use enough workers to keep the bottleneck busy, not enough to overload the system.

For CPU-heavy work, the bottleneck is usually CPU cores.
For I/O-heavy work, the bottleneck may be network latency, file descriptors, database limits, API rate limits, or memory.

Channel closing rules

A simple ownership flow looks like this:

producer owns close(jobs)

jobs channel
    ↓
workers read jobs and send results
    ↓
merger waits for all workers
    ↓
merger owns close(results)

Only the sender should close a channel, and only when no more values will be sent.

This prevents:

• panic from sending on a closed channel
• result channel closing before all workers finish
• consumers waiting forever
• confusing range behavior

Example mistake:

worker-1 closes results
worker-2 still tries to send results
program panics

Better pattern:

workers only send results
one coordinator waits for all workers
coordinator closes the shared result channel

Short rule:

If multiple goroutines send to the same channel, none of those workers should close it individually.
Use one coordinator goroutine to close it after all senders are done.

Mistakes to avoid

Assuming output order matches input order—completion order wins unless you add sequence metadata.

Spawning one goroutine per row of a huge dataset with no bound—use a pool or semaphore.

Ignoring cancellation when a handler or CLI exits early—stop producers and unblock blocked sends.

Closing the merged result channel before all merge writers finish—consumers may panic or see incomplete data.

Using fan-out everywhere—sometimes a simple loop is clearer and fast enough.

Treating worker fan-out like broadcast—normal queue fan-out gives each job to one worker, not every worker.

Fan-out fan-in cheat sheet

Summary

Fan-out distributes work across goroutines; fan-in merges their outputs into one channel or consumer. Go expresses this with channels, small stage functions, and WaitGroup (or libraries like errgroup) so merges close the outbound stream only after all sources finish. Worker pools are a bounded style of fan-out. Plan for out-of-order completion, channel ownership, cancellation to avoid leaks, and explicit error policy—those choices matter as much as the sample pipeline.

References

• Go Blog: Pipelines and cancellation
• Package sync (WaitGroup)
• Package errgroup (extension library for fan-out with first-error cancel)