A package manager is a tool for automating the process of building our code, as well as downloading, updating, and removing project dependencies in a consistent manner. It can determine whether a specific version of a package is installed on a project, and then install or upgrade the package, typically from a remote host.
Package managers have been around since long time, initially they were used in operating systems and later in the programming languages. While there are differences between both system and language level package managers, there is a significant overlap in how they work. In a language context, a package manager makes it easier to work with first-party and third-party libraries by helping us define and download project dependencies and pin down versions or version ranges so that we can, in theory, upgrade our dependencies without fear of breaking things. Most package managers also support a lock file through which they guarantee the reproducibility of builds in any environment.
Source: Dependency Injection in Go
For a long time, the package management strategy in Go was inadequate, as there was no official way to version dependencies. Although third-party solutions (glide, dep) did exist, integration with the language wasn't seamless, and adoption varied from project to project. Finally, in 2018, the Go team finally introduced Go modules with the aim of plugging this gaping hole in the ecosystem. The feature initially landed in Go 1.11 but wasn't enabled by default. As of Go 1.16, the latest version at the time of writing, modules are now enabled by default and continue to see increasing adoption in the community.

Third-Party Solutions

  • A third-party tool needs to be explicitly downloaded and installed
  • Problem with managing inter-dependency
  • Most of them are deprecated now

Go Module

  • Out of the box solution from the Go developer and community is moving towards it to manage dependencies.
  • It is the future of dependency management in Go.
  • Go community is actively working on it, many new features are added after first release.
  • Since it is embedded in the language, it comes very handy to implement, no need to install any third-party tool like glide or dep.
  • Provides a go.mod file to manage dependency, very descriptive to understand dependency hierarchy.
  • Upgrading / downgrading a dependency to a specific version is easier.
  • Provides feature for warming caches, which reduces deployment time.
Analysing the pros and cons of these solutions, my personal preference is to use ‘Go Module’ as a package manager.
In this post, we will discuss about Go modules working and basic use cases, so that knowledge of modules will help us get things done faster and help save hours of trial and error.

Semantic Versioning

Semantic versioning is the main versioning system that Go modules.
Major
Denotes API changes that make the current version backward-incompatible with the former version.
Minor
Connotes the addition of backward-compatible features/functionalities to the version.
Patch
Denotes the addition of backward-compatible bug fixes.
Pseudo-versions
A pseudo-version is a specially formatted pre-release version that encodes information about a specific revision in a version control repository. For example, v0.0.0-20191109021931-daa7c04131f5 is a pseudo-version.

Let’s Understand Basics

Package
Collection of one or more Go source (.go extension) files that reside in one directory.
Module
Collection of one or more packages and has a version associated with it.
Repository
Collection of one or more modules, at least in the context of Go modules.
Module Path
The module path defines the location of a module. It can be derived from version control system (like Git) metadata by the go command or set explicitly by the creator of the module.
Module Root
The module root corresponds to the module path. It is the directory in which Go module exists. It is the root directory in which the go.mod and go.sum files are located and contains a tree of Go source files.
Source packages
A module, in essence, is just a collection of Go packages. All other features of modules like versioning and dependency management exist to facilitate the building of these packages and their consumption.
Dependencies
Dependencies are the set of all modules that are required to build and test the main module (module in which go command is run).
Build List
A build list is the set of the main module and all its required dependencies that provide packages to a build of the main module. A build list is the result of the go command converting the go.mod file to a list of specific modules when it builds the main module. Only specific versions of dependencies are used. If multiple versions of a dependency exist, then the go command only adds the most recent version of it to the build list.
Go Module File
Sample Code

Go Mod File

go.mod is a file that is located within module root. It is automatically created when a module is initialized using the Go command go mod init.
Go Module File

Go Sum File

This file lists down the checksum of direct and indirect dependency required along with the version. It is to be mentioned that the go.mod file is enough for a successful build.
Why go.sum file is needed?
  • The checksum present in go.sum file is used to validate the checksum of each of direct and indirect dependency to confirm that none of them has been modified.
  • It ensures that builds are repeatable.
  • No malicious or accidental changes have been added to the dependencies.
Go Sum File

Addition of Dependencies in to Module

We can add dependencies to module by adding a dependency directly, in the source code. The go or module commands automatically determines what requirements are requested in the source code but are missing. It then adds them to the go.mod and downloads them when the main module is built. A dependency can also be added to module’s go.mod using go get on the command line.

Module Aware Commands

Module commands outside a module - Module aware commands normally run in the context of a main module defined by a go.mod file in the working directory or a parent directory. Some commands may be run in module-aware mode without a go.mod file. Such commands when run, search through the source code and add missing build dependencies to the go.mod file and download them to the local cache or vendor directory. However, they do not remove unused dependencies> or delete anything from the go.mod or go.sum. These commands are: go build, go clean, go fix, go fmt, go generate, go get, go install, go list, go run, go test and go vet.
go clean -modcache
The -modcache flag causes go clean to remove the entire module cache, including unpacked source code of versioned dependencies. This is usually the best way to remove the module cache. By default, most files and directories in the module cache are read-only to prevent tests and editors from unintentionally changing files after they've been authenticated. Unfortunately, this causes commands like rm -r to fail, since files can't be removed without first making their parent directories writable.

Module Command

The go mod command is used to perform operations on modules listed in the go.mod file. The eight commands are: go mod init, go mod vendor, go mod download, go mod verify, go mod why, go mod graph, go mod tidy.
Initialize Module
go mod init or go mod init [path] Initializes and writes a new go.mod file in the current directory, in effect creating a new module rooted at the current directory. The go.mod file must not already exist.
Download Packages
go mod download [-json] [-x] [modules] Downloads a module to local cache (GOPATH/pkg/mod/cache).
The -json flag causes download to print a sequence of JSON objects to standard output, describing each downloaded module (or failure).
The -x flag causes download to print the commands download executes to standard error.
Package Management
go mod tidy [-e] [-v] Determines missing and unused module dependencies ensures that the go.mod file matches the source code in the module. It adds any missing module requirements necessary to build the current module's packages and dependencies, and it removes requirements on modules that don't provide any relevant packages. It also adds any missing entries to go.sum and removes unnecessary entries. then adds or removes.
The -v flag for a detailed overview of this command’s results.
The -e flag (added in Go 1.16) causes go mod tidy to attempt to proceed despite errors encountered while loading packages.
Vendoring Packages
go mod vendor [-e] [-v] Constructs a directory named vendor in the main module's root directory that contains copies of all packages needed to support builds and tests of packages in the main module.
The -v flag for a detailed overview of this command’s results.
The -e flag (added in Go 1.16) causes go mod vendor to attempt to proceed despite errors encountered while loading packages.
Package Graph
go mod graph Prints a text version of the module requirement graph which is a list of our module’s direct and indirect dependencies.
Package Verification
go mod verify Checks that dependencies of the main module stored in the module cache have not been modified since they were downloaded.
Why Package Required
go mod why [-m] [-vendor] packages... Shows how and where packages or modules are needed in the main module. It does this by showing us the shortest path in the module’s dependency graph between our module and a specified package or module.
The -m flag causes go mod why to treat its arguments as a list of modules.
The -vendor flag causes go mod why to ignore imports in tests of packages outside the main module (as go mod vendor does). By default, go mod why considers the graph of packages matched by the all pattern.
Editing and Formatting
go mod edit [editing flags] [-fmt|-print|-json] [go.mod] Used to edit a go.mod file. It reads the go.mod file then writes changes to the same file or another specified file. It is mostly useful for tools or scripts.
Edit Commands

Environment Variables

GO111MODULE
Controls whether the go command runs in module-aware or GOPATH mode. Three values are recognized:
  • off The go command ignores go.mod files and runs in GOPATH mode.
  • on (or unset) The go command runs in module-aware mode, even when no go.mod file is present.
  • auto The go command runs in module-aware mode if a go.mod file is present in the current directory or any parent directory. In Go 1.15 and lower, this was the default.
GOMODCACHE
The directory where the go command will store downloaded modules and related files. See Module cache for details on the structure of this directory. If GOMODCACHE is not set, it defaults to $GOPATH/pkg/mod.
GOINSECURE
Comma-separated list of glob patterns of module path prefixes that may always be fetched in an insecure manner. Only applies to dependencies that are being fetched directly. Unlike the -insecure flag on go get, GOINSECURE does not disable module checksum database validation. Make use of GOPRIVATE or GONOSUMDB to disable module checksum.
GONOPROXY
Comma-separated list of glob patterns of module path prefixes that should always be fetched directly from version control repositories, not from module proxies. If GONOPROXY is not set, it defaults to GOPRIVATE
GONOSUMDB
Comma-separated list of glob patterns of module path prefixes for which the go should not verify checksums using the checksum database. If GONOSUMDB is not set, it defaults to GOPRIVATE.
GOPATH
In GOPATH mode, the GOPATH variable is a list of directories that may contain Go code. In module-aware mode, the module cache is stored in the pkg/mod subdirectory of the first GOPATH directory.
GOPRIVATE
  • Comma-separated list of glob patterns of module path prefixes that should be considered private.
  • GOPRIVATE is a default value for GONOPROXY and GONOSUMDB.
  • GOPRIVATE also determines whether a module is considered private for GOVCS.
GOPROXY
List of module proxy URLs, separated by commas , or pipes |. When the go command looks up information about a module, it contacts each proxy in the list in sequence until it receives a successful response or a terminal error. A proxy may respond with a 404 (Not Found) or 410 (Gone) status to indicate the module is not available on that server.
GOSUMDB
Identifies the name of the checksum database to use and optionally its public key and URL.
GOVCS
Controls the set of version control tools the go command may use to download public and private modules or other modules matching a glob pattern.

VS Code Go Extension

VS Code Go extension now enables the gopls language server by default, to deliver more robust IDE features and better support for Go modules.
gopls provides IDE features, such as as intelligent autocompletion, signature help, refactoring, and workspace symbol search.

Final Thought

Go Module is the future of Go dependency management. Go modules adoption is nearly universal with 77% satisfaction, but respondents also highlight a need for improved docs.