Go runs package tests in parallel with other packages tests

Software
,
1

Learned something new today … when running go test ./... (test all packages in current dir), Go builds a binary for every package and then runs these tests in parallel. This is nice as it’s quite fast.

In SimpleIoT, we are starting to use our TestServer in more tests. We can fire up the entire SIOT stack in a test and then tear it down at the end of the test. This makes end-to-end testing very convenient, and has had the side effect of flushing out lots of bugs in the app life-cycle management. You have to have a well-behaved app to do this kind of thing.

Recently, I noticed if a ran tests on a clean directory, I was getting strange crashes and race conditions in the Nats Server component. The only seemed to happen when I cleaned the test cache (go clean -testcache). If I ran it a second time, it passed. Through the process of elimination, I learned that if tests that used TestServer were in two different directories, there were problems. A little bit of searching turned up this:

By default, execution of test code using the testing package will be done sequentially. However, note that it is only the tests within a given package that run sequentially.

If tests from multiple packages are specified, the tests will be run in parallel at the package level. For example, imagine there are two packages, package a and package b. The test code in package a will be run sequentially, and the test code in package b will also be run sequentially. However, the tests for package a and package b will be run in parallel.

The Go documentation also has this:

-p n
	the number of programs, such as build commands or
	test binaries, that can be run in parallel.
	The default is GOMAXPROCS, normally the number of CPUs available.

The solution in my cases was to force the testing of packages to run sequentially by adding the -p=1 argument:

go test -p=1 ./...

This slows things down quite a bit, but is a reasonable workaround for now as you rarely need to execute all tests during development except in CI, as most tests are cached from previous runs. Only if code changes are tests re-run.

Long term there are no doubt ways to allow multiple TestServers to run, but I’ll leave that for another day.