Concurrency

Fan-Out/Fan-In

Distributes work across multiple goroutines and collects results.

Parallel API callsData processingMap-reduce

Understanding Fan-Out/Fan-In

The Fan-Out/Fan-In pattern distributes work across multiple goroutines (fan-out) and then collects all their results into a single channel (fan-in). Fan-out means starting multiple goroutines to process data from a single source; fan-in means merging the output of all those goroutines into one channel. In Go, this leverages channels andsync.WaitGroup to coordinate the merge. This pattern is ideal for parallel API calls, data processing pipelines, and any scenario where work can be split and results combined.

Key Concepts

  • •Fan-out — distributing work from one source channel to multiple goroutines for parallel processing
  • •Fan-in — merging results from multiple goroutine channels into a single output channel
  • •sync.WaitGroup — used to track when all fan-out goroutines have completed before closing the merged channel
  • •Channel merging — a dedicated goroutine reads from all worker channels and writes to the single output channel

When to Use

✅ Use when
  • • Multiple independent tasks can run in parallel (API calls, file processing)
  • • You need to parallelize a map-reduce style computation
  • • A single data source needs to be processed by multiple consumers
  • • You want to aggregate results from concurrent operations
⚠️ Avoid when
  • • Tasks have sequential dependencies and can\'t run in parallel
  • • The fan-in merge adds complexity without meaningful speedup
  • • A simple worker pool provides the same benefit with less coordination code

Structure

Source Channel
↓ fan-out to N workers
Worker 1 / Worker 2 / Worker N
↓ fan-in (merge)
Merged Output Channel

How It Works

Input
Stream
1
2
3
4
Worker 1
Worker 2
Worker 3
Merged Output
Results
1
4
9
16
Single input stream
1

Input Stream

Single stream of data to process.

1 / 4

Basic Implementation

Number squaring with fan-out/fan-in:

main.go
Loading editor...

Real-World Example: Document Processing

Parallel document analysis:

main.go
Loading editor...