A tool to help identify, triage and keep track of flaky tests.
snowflake consists of four key components:
- reporter - a ginkgo reporter, which sends the results of test runs to the snowflake server.
- server - serves the snowflake API.
- web - provides a web UI.
- snowgauge - the package responsible for actually detecting flaky tests.
To use snowflake you must first install the snowflake server and web, which can be done as foll 6F00 ows:
go get github.com/teddyking/snowflake
cd $GOPATH/src/github.com/teddyking/snowflake
make runserver
make runweb
Then you need to configure your ginkgo test suites with the snowflake reporter, for example:
# example_suite_test.go
func TestExamplesuite(t *testing.T) {
RegisterFailHandler(Fail)
importPath := "github.com/teddyking/snowflake"
gitCommitOutput, _ := exec.Command("git", "rev-parse", "HEAD").Output()
gitCommit := strings.Trim(string(gitCommitOutput), "\n")
conn, err := grpc.Dial(":2929", grpc.WithInsecure())
if err != nil {
log.Fatalf("could not connect to server: %s", err.Error())
}
reporterClient := api.NewReporterClient(conn)
snowflakeReporter := snowflake.NewReporter(importPath, gitCommit, reporterClient)
RunSpecsWithDefaultAndCustomReporters(t, "Examplesuite Suite", []Reporter{snowflakeReporter})
}
For a full example see the examples/examplesuite.
Then finally run your test suites. Flaky tests will appear in the web UI, which by default is accessible at http://localhost:8080.
The tests can be run as follows:
make test # run all tests
make testunit # run unit tests only
make testintegration # run integration tests only
make teste2e # run end-to-end tests only