Go Concurrency Patterns

Go Concurrency Patterns

Production patterns for Go concurrency including goroutines, channels, synchronization primitives, and context management

Category: design Source: wshobson/agents

What Is This

The "Go Concurrency Patterns" skill equips developers with a practical understanding and toolbox of production-ready concurrency techniques in Go (Golang). This skill covers the core concurrency primitives provided by Go, including goroutines, channels, synchronization primitives such as mutexes and wait groups, and context management for cancellation and timeouts. It provides actionable patterns for designing robust, scalable, and maintainable concurrent programs, focusing on idiomatic Go solutions.

Concurrency is a central feature of the Go language. Go’s lightweight goroutines, powerful channel-based communication, and built-in synchronization tools enable developers to write high-performance, parallel applications with less complexity compared to traditional multithreading. Mastery of Go concurrency patterns is essential for building reliable distributed systems, processing pipelines, web servers, and any application that must handle multiple tasks simultaneously.

Why Use It

Concurrent programming is notoriously difficult in many languages due to the challenges of shared memory, race conditions, deadlocks, and resource leaks. Go was designed to simplify these challenges without sacrificing performance. The Go concurrency model encourages developers to structure programs around communicating sequential processes, using channels for safe message passing.

The "Go Concurrency Patterns" skill is valuable because it:

  • Enables safe, concurrent execution using goroutines and channels, reducing the risk of race conditions and deadlocks
  • Provides synchronization primitives like mutexes and wait groups for managing shared resources and goroutine lifecycles
  • Teaches use of context for cancellation, timeouts, and request scoping, which is critical for graceful shutdown and resource cleanup
  • Offers idiomatic design patterns, such as worker pools and pipelines, that are proven in production environments
  • Improves debugging and maintainability by encouraging clear concurrency boundaries and communication mechanisms

By mastering these patterns, developers can build scalable, responsive, and correct Go applications suitable for demanding production workloads.

How to Use It

To apply Go concurrency patterns, you will commonly use the following primitives and techniques:

Goroutines

Goroutines are lightweight threads managed by the Go runtime. Spawn a goroutine using the go keyword:

go func() {
    fmt.Println("Hello from a goroutine")
}()

Channels

Channels provide a safe way to communicate between goroutines:

ch := make(chan int)
go func() {
    ch <- 42 // Send value to channel
}()
value := <-ch // Receive value from channel
fmt.Println(value)

Select Statement

The select statement allows a goroutine to wait on multiple communication operations:

select {
case msg := <-ch1:
    fmt.Println("Received", msg)
case <-time.After(time.Second):
    fmt.Println("Timeout")
}

Synchronization Primitives

Use sync.WaitGroup to wait for a collection of goroutines to finish:

var wg sync.WaitGroup
wg.Add(2)
go func() {
    defer wg.Done()
    // work
}()
go func() {
    defer wg.Done()
    // more work
}()
wg.Wait()

Use sync.Mutex to protect shared data:

var mu sync.Mutex
var counter int
go func() {
    mu.Lock()
    counter++
    mu.Unlock()
}()

Context Management

The context package enables cancellation and timeouts:

ctx, cancel := context.WithTimeout(context.Background(), 2*time.Second)
defer cancel()

select {
case <-time.After(3 * time.Second):
    fmt.Println("Finished work")
case <-ctx.Done():
    fmt.Println("Timeout or cancellation")
}

Example: Worker Pool Pattern

A common concurrency pattern is the worker pool, where multiple goroutines process tasks from a shared channel:

func worker(id int, jobs <-chan int, results chan<- int, wg *sync.WaitGroup) {
    defer wg.Done()
    for job := range jobs {
        results <- job * 2 // Process job
    }
}

func main() {
    jobs := make(chan int, 5)
    results := make(chan int, 5)
    var wg sync.WaitGroup

    for w := 1; w <= 3; w++ {
        wg.Add(1)
        go worker(w, jobs, results, &wg)
    }

    for j := 1; j <= 5; j++ {
        jobs <- j
    }
    close(jobs)
    wg.Wait()
    close(results)
    for result := range results {
        fmt.Println(result)
    }
}

When to Use It

Apply Go concurrency patterns in the following scenarios:

  • Building concurrent Go applications that require processing multiple tasks in parallel
  • Implementing worker pools, pipelines, or fan-out/fan-in designs
  • Managing lifecycles of goroutines for web servers, background jobs, or periodic tasks
  • Using channels for safe communication between goroutines, avoiding shared memory
  • Debugging and preventing race conditions or deadlocks in concurrent code
  • Implementing graceful shutdown and cancellation in long-running services using context

Important Notes

  • Always ensure goroutines are properly synchronized and do not leak. Use wait groups, context cancellation, or channel signaling for clean shutdown.
  • Avoid sharing memory between goroutines unless protected by synchronization primitives like mutexes.
  • Prefer using channels for communication rather than shared memory, following Go’s concurrency mantra: "Don’t communicate by sharing memory; share memory by communicating."
  • Use buffered channels thoughtfully - they can help prevent blocking, but may hide synchronization issues if not managed carefully.
  • Be vigilant about race conditions. Use the Go race detector (go run -race) during development and testing.
  • Proper use of context is crucial for building reliable, cancelable, and timeout-aware concurrent operations.

By following these patterns and best practices, you can write robust, maintainable, and production-grade concurrent Go applications.