Skip to content

Running tests with -update fails unless autogold referenced in every test package #52

Open
@jamestelfer

Description

@jamestelfer

When a codebase has packages that do not use autogold, running go test ./... -update causes the test run to fail with a usage message. The usage message is shown for each package that does not have the flag registered.

This issue was introduced by #45 and appears to result from the late resolution of command line flags.

There are four work arounds:

  1. Use autogold in all packages :)
  2. Add a discard import for autogold in each package, so the flag is registered (but not used). This is similar to (1).
  3. Register the flag separately as a dummy value (clunky)
  4. Use build tags such that tests with autogold snapshots can be easily filtered on the command line.

I can understand that those who use autogold's assertions will likely not come across this. However, I expect that others are looking for the excellent golden file support and not for the rest of the assertions. Being able to either (a) only use autogold for the gold or (b) gradually transition from another tool is an advantage.

I don't know what would be required to resolve this in such a way that it doesn't step on the toes of other tooling, but I'd be happy to have a go at it if someone pointed me in the right direction.

The following is an example of the usage message shown per package:

flag provided but not defined: -update
Usage of /var/folders/29/7b6qj7_d3156p1r2njv0_h8m0000gn/T/go-build95459731/b001/foobar.test:
  -test.bench regexp
    	run only benchmarks matching regexp
  -test.benchmem
    	print memory allocations for benchmarks
  -test.benchtime d
    	run each benchmark for duration d or N times if `d` is of the form Nx (default 1s)
  -test.blockprofile file
    	write a goroutine blocking profile to file
  -test.blockprofilerate rate
    	set blocking profile rate (see runtime.SetBlockProfileRate) (default 1)
  -test.count n
    	run tests and benchmarks n times (default 1)
  -test.coverprofile file
    	write a coverage profile to file
  -test.cpu list
    	comma-separated list of cpu counts to run each test with
  -test.cpuprofile file
    	write a cpu profile to file
  -test.failfast
    	do not start new tests after the first test failure
  -test.fullpath
    	show full file names in error messages
  -test.fuzz regexp
    	run the fuzz test matching regexp
  -test.fuzzcachedir string
    	directory where interesting fuzzing inputs are stored (for use only by cmd/go)
  -test.fuzzminimizetime value
    	time to spend minimizing a value after finding a failing input (default 1m0s)
  -test.fuzztime value
    	time to spend fuzzing; default is to run indefinitely
  -test.fuzzworker
    	coordinate with the parent process to fuzz random values (for use only by cmd/go)
  -test.gocoverdir string
    	write coverage intermediate files to this directory
  -test.list regexp
    	list tests, examples, and benchmarks matching regexp then exit
  -test.memprofile file
    	write an allocation profile to file
  -test.memprofilerate rate
    	set memory allocation profiling rate (see runtime.MemProfileRate)
  -test.mutexprofile string
    	write a mutex contention profile to the named file after execution
  -test.mutexprofilefraction int
    	if >= 0, calls runtime.SetMutexProfileFraction() (default 1)
  -test.outputdir dir
    	write profiles to dir
  -test.paniconexit0
    	panic on call to os.Exit(0)
  -test.parallel n
    	run at most n tests in parallel (default 10)
  -test.run regexp
    	run only tests and examples matching regexp
  -test.short
    	run smaller test suite to save time
  -test.shuffle string
    	randomize the execution order of tests and benchmarks (default "off")
  -test.skip regexp
    	do not list or run tests matching regexp
  -test.testlogfile file
    	write test action log to file (for use only by cmd/go)
  -test.timeout d
    	panic test binary after duration d (default 0, timeout disabled)
  -test.trace file
    	write an execution trace to file
  -test.v
    	verbose: print additional output
FAIL	github.com/organization/foobar/baz	2.004s
FAIL

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions