Using a separate Go module for your tools.go

In Managing your Go tool versions with go.mod and a tools.go, I mentioned how you can use a tools.go to track any dependencies your project needs for i.e. linting or go generates.

This is a very useful pattern and until the proposal to add a // tool directive in go.mod lands, is in my opinion the best way to do this, which is also why it's our recommendation in oapi-codegen.

However, sometimes that can lead to a fair bit of pollution to your top-level go.mod, as you now have a load of dependencies which aren't technically needed by the project for production usage, only for build/test.

As we found with oapi-codegen, it can be useful to consider your dependency graph as an interface you provide to your consumers, and even though Go performs pruning of the dependency graph, it can be nice to have the option not to declare dependencies you don't need.

Example setup

Let's say that we have a project with a tools.go:

//go:build tools
// +build tools

package main

import (
	_ "github.com/99designs/gqlgen"
	_ "github.com/deepmap/oapi-codegen/v2/cmd/oapi-codegen"
	_ "github.com/sqlc-dev/sqlc/cmd/sqlc"
)

This is then invoked i.e. via:

package db

//go:generate go run github.com/sqlc-dev/sqlc/cmd/sqlc generate

Migrating to a multi-module project

Multi-module Go projects are fairly straightforward to create, and "just" need a new go.mod created, with the relevant module path.

(The difficult of working with multi-module projects comes when trying to do releases)

To do this, we can first create a tools/go.mod:

module the.module.name/tools

We then need to:

# move `tools.go` to our new sub-module
git mv tools.go tools/tools.go

# make sure we've fetched the dependencies + cleaned up `go.mod`
sh -c 'cd tools && go mod tidy'
# ... + clean up our top-level module
go mod tidy

It's worth verifying that the new module's versions for dependencies are aligned with the versions you previously used in the parent version, as creating the module from scratch loses any version pinning you previously had.

Finally, we need to modify any go run references like so, adding the relative path to the tools/go.mod:

 package db

-//go:generate go run github.com/sqlc-dev/sqlc/cmd/sqlc generate
+//go:generate go run -modfile=../../tools/go.mod github.com/sqlc-dev/sqlc/cmd/sqlc generate

And remember that any CI checks that verify go mod tidy also needs to apply to your tools/go.mod too!

You can see this in action in a real-world project here.

If you're finding that you're not enjoying the overhead of the relative paths, you could go install via the go.mod, but that means someone can't i.e. just run go generate, which is most convenient.