Goroutines

Summary: in this tutorial, you will learn how to use goroutines to develop concurrent applications.

Processes and Threads

When you launch a Go Program, the operating system (OS) creates a new process. A process is a container that holds program resources, including memory and threads:

Each process contains at least one thread known as the main thread. The main thread is the entry point for the application. When the main thread terminates, the application exits.

When a program has a single thread, it is called a single-threaded program. A process may contain multiple threads. In this case, the program is known as a multi-threaded program.

To execute multiple threads, the OS uses a scheduler to schedule the threads to run:

If the computer has a single CPU, the scheduler switches between threads so fast that often gives us an illusion that they are running simultaneously. This is called concurrency.

Concurrency

Concurrency is about to deal with multiple tasks at the same time. It doesn’t necessarily mean the tasks are running at the same time:

On the other hand, parallelism is about executing multiple tasks at the same time. This requires multiple processors or cores.

Nowadays, computers have more than one core. Therefore, the OS scheduler can run multiple threads at the same time, which creates a true parallelism:

To achieve concurrency, other programming languages such as C# or Java use multi-threading. However, developing multi-threaded programs is hard because we need to synchronize between threads carefully or they will break.

To make concurrency easy, Go provides a built-in language feature called goroutines.

Goroutines

Goroutines are much lighter than OS threads. They are known as lightweight threads and are managed by the Go runtime scheduler.

A lightweight thread does not like an operating system (OS) thread. It uses less memory than the OS thread. One OS thread may include multiple lightweight threads.

The Go runtime scheduler is sitting on top of the OS layer and is in charge of managing the execution of goroutines:

The Go runtime scheduler allows goroutines to run concurrently on a single logical processor or in parallel on multiple logical processors. This enables efficient use of system resources and improves performance.

The Go runtime scheduler is designed to handle tens or even hundreds of thousands of goroutines efficiently, making Go well-suited for highly concurrent applications.

Sequential programs

Typically, a Go program executes code in sequence line by line from top to bottom. For example:

package main

import (
    "fmt"
    "time"
)

func task(name string) {
    fmt.Println(name, "started...")
    time.Sleep(1 * time.Second)
    fmt.Println(name, "finished")
}

func main() {
    // measure the execution time
    start := time.Now()
    defer func() {
        elapsed := time.Since(start)
        fmt.Printf("It took %d ms\n", elapsed.Milliseconds())
    }()

    task("Task 1")
    task("Task 2")
    task("Task 3")

    fmt.Println("Done")
}Code language: Go (go)

How it works.

First, define a function task() that takes 1 second to finish:

func task(name string) {
    fmt.Println(name, "started...")
    time.Sleep(1 * time.Second)
    fmt.Println(name, "finished")
}Code language: Go (go)

The task() function uses the Sleep() function from the time module to pause the execution for one second.

In real-world applications, the task() function may call an external API or make a database connection instead of using the Sleep() function.

Second, measure the execution time of the main() function:

start := time.Now()
defer func() {
    elapsed := time.Since(start)
    fmt.Printf("It took %d ms\n", elapsed.Milliseconds())
}()Code language: Go (go)

Third, call the task() function three times with different string arguments:

task("Task 1")
task("Task 2")
task("Task 3")Code language: Go (go)

Finally, display the Done message:

fmt.Println("Done")Code language: Go (go)

Output:

Task 1 started...
Task 1 finished
Task 2 started...
Task 2 finished
Task 3 started...
Task 3 finished
Done
It took 3002 msCode language: Go (go)

The output shows two important pieces of information:

• The tasks started and finished in sequence.
• Each task function call took one second to finish. Since we called the task() function three times, the whole program took about 3 seconds to complete.

To speed up the program, we can use goroutines to run the task function concurrently.

Creating Goroutines

To start a goroutine, you use the go keyword followed by a function call that you want to run in a concurrently:

go fn()Code language: Go (go)

The go keyword instructs the Go runtime scheduler to spawn a new lightweight thread and run the fn() function.

For example, we can use goroutines to run the task function concurrently as follows:

package main

import (
    "fmt"
    "time"
)

func task(name string) {
    fmt.Println(name, "started...")
    time.Sleep(1 * time.Second)
    fmt.Println(name, "finished")
}

func main() {
    // measure the execution time
    start := time.Now()
    defer func() {
        elapsed := time.Since(start)
        fmt.Printf("It took %d ms\n", elapsed.Milliseconds())
    }()

    go task("Task 1")
    go task("Task 2")
    go task("Task 3")

    fmt.Println("Done")
}Code language: Go (go)

Output:

Done
It took 0 msCode language: Go (go)

It seems that the task() function was not executed at all. The reason is that the main() function ran so fast, that the task() function did not get a chance to execute:

To allow the task() function to run, the main() function must wait for the goroutines to run to complete.

Go provides synchronization primitives to do it:

• Using WaitGroup objects from the sync module.
• Using Channels.

We’ll focus on the WaitGroup in this tutorial.

Go WaitGroup

In Go, WaitGroup is used to wait for a group of goroutines to complete execution.

A WaitGroup works like a counter:

• First, set the initial counter value which represents the number of goroutines to wait for.
• Second, decrease the counter value each time a goroutine is completed.
• Third, wait until the counter becomes zero.

Using a WaitGroup

Here are the steps for using WaitGroup:

First, declare a variable with the sync.WaitGroup type:

var wg sync.WaitGroupCode language: Go (go)

Second, set the counter for the WaitGroup to n, which represents the number of goroutines to wait for:

wg.Add(n)Code language: Go (go)

Third, call the Done() function in the goroutine to decrement the counter once the goroutine completes:

defer wg.Done()Code language: Go (go)

Finally, call the wg.Wait() function to wait until all goroutine completes:

wg.Wait()Code language: Go (go)

Go WaitGroup example

The following example shows how to use a WaitGroup to wait for all the goroutines to complete:

package main

import (
    "fmt"
    "sync"
    "time"
)

var wg sync.WaitGroup

func main() {
    // measure the execution time
    start := time.Now()
    defer func() {
        elapsed := time.Since(start)
        fmt.Printf("It took %d ms\n", elapsed.Milliseconds())
    }()

    wg.Add(3)

    go task("Task 1")
    go task("Task 2")
    go task("Task 3")

    wg.Wait()

    fmt.Println("Done")
}

func task(name string) {
    defer wg.Done()

    fmt.Println(name, "started...")
    time.Sleep(1 * time.Second)
    fmt.Println(name, "finished")
}Code language: Go (go)

Output:

Task 3 started...
Task 1 started...
Task 2 started...
Task 2 finished
Task 1 finished
Task 3 finished
Done
It took 1000 msCode language: Go (go)

The output shows two important things:

• First, goroutines did not execute in any order.
• Second, the program took roughly 1 second to complete. It’s three times faster than the synchronous program on our computer.

How it works.

First, declare a variable with the type WaitGroup:

var wg sync.WaitGroupCode language: Go (go)

Second, set the WaitGroup‘s counter to three to wait for three goroutines to complete before creating goroutines:

wg.Add(3)Code language: Go (go)

Third, launch three goroutines:

go task("Task 1")
go task("Task 2")
go task("Task 3")Code language: Go (go)

Fifth, call the Wait() function to wait for all the goroutines to complete:

wg.Wait()Code language: Go (go)

Sixth, when executing the task() function, decrement the counter of the WaitGroup by one when it completes:

defer wg.Done()Code language: Go (go)

Goroutine practical example

The following example shows how to use goroutines to check the status of multiple URLs:

package main

import (
    "fmt"
    "net/http"
    "sync"
)

func check(url string, wg *sync.WaitGroup) {
    defer wg.Done()
    _, err := http.Get(url)
    if err != nil {
        fmt.Printf("The %s is down\n", url)
    } else {
        fmt.Printf("The %s is up\n", url)
    }
}

func main() {
    var wg sync.WaitGroup
    urls := []string{
        "https://www.google.com",
        "https://www.bing.com",
        "https://www.duckduckgo.com/",
    }

    wg.Add(len(urls))

    for _, url := range urls {
        go check(url, &wg)
    }
    wg.Wait()
}Code language: Go (go)

How it works.

Step 1. Define the check() function that checks the status of a URL and displays whether the URL is up or down:

func check(url string, wg *sync.WaitGroup) {
    defer wg.Done()
    _, err := http.Get(url)
    if err != nil {
        fmt.Printf("The %s is down\n", url)
    } else {
        fmt.Printf("The %s is up\n", url)
    }
}Code language: Go (go)

Step 2. Define a variable for the WaitGroup type

var wg sync.WaitGroupCode language: Go (go)

Step 3. Declare and initialize a slice of strings, which hold a list of URLs to check:

urls := []string{
    "https://www.google.com",
    "https://www.bing.com",
    "https://www.duckduckgo.com/",
}Code language: Go (go)

Step 4. Set the counter of the WaitGroup to the size of the urls slice:

wg.Add(len(urls))Code language: Go (go)

Step 5. iterate over the URLs and create each goroutine to check

for _, url := range urls {
    go check(url, &wg)
}Code language: Go (go)

Step 6. Wait for all goroutines to complete:

wg.Wait()Code language: Go (go)

Here’s the output of the program:

The https://www.google.com is up
The https://www.bing.com is up
The https://www.duckduckgo.com/ is upCode language: Go (go)

Please note that the sequence of the URLs may vary.

Summary

• Goroutines are lightweight threads managed by Go runtime.
• Use go keyword followed by a function call to run the function asynchronously.
• Use WaitGroup to wait for a collection of goroutines to finish executing.