Commit Graph

204 Commits (master)

Author SHA1 Message Date
Gopher Robot f862e5ec8b go.mod: update golang.org/x dependencies
Update golang.org/x dependencies to their latest tagged versions.

Change-Id: Id66ffedb32a6a9c37c1ca23720e1f92dd8779afd
Reviewed-on: https://go-review.googlesource.com/c/benchmarks/+/583535
Reviewed-by: Than McIntosh <thanm@google.com>
Auto-Submit: Gopher Robot <gobot@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2024-05-06 16:45:57 +00:00
Michael Anthony Knyszek d78363a64b sweet: add GOMAXPROCS suffix to benchmark names
This change adds the GOMAXPROCS suffix to benchmark names the same way
that `go test` does.

Change-Id: I3ae168620de716c0aa8b642d099ba5308c54f48b
Reviewed-on: https://go-review.googlesource.com/c/benchmarks/+/580837
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
2024-04-25 02:55:21 +00:00
Gopher Robot d10bd6dc28 go.mod: update golang.org/x dependencies
Update golang.org/x dependencies to their latest tagged versions.

Change-Id: Ib8e9fdec286a9961998b6b03084dff7b368fb283
Reviewed-on: https://go-review.googlesource.com/c/benchmarks/+/576615
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Than McIntosh <thanm@google.com>
Auto-Submit: Gopher Robot <gobot@golang.org>
2024-04-04 20:44:30 +00:00
Gopher Robot e936ae472e go.mod: update golang.org/x dependencies
Update golang.org/x dependencies to their latest tagged versions.

Change-Id: I7e320c467f4e21e9b2cd0e2ec2363c756e7520c4
Reviewed-on: https://go-review.googlesource.com/c/benchmarks/+/569015
Auto-Submit: Gopher Robot <gobot@golang.org>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Than McIntosh <thanm@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2024-03-05 16:02:44 +00:00
David Chase e81011366a bench,gc_latency: use -short to limit gc_latency output
There's only a few measurements we really care about, and
they also seem to actually be less noisy.  Even if they
weren't, this reduces the number of measurements reported
by about 80%.

Change-Id: Id1a93c97d74f27eb2b61f6a95bf2452578d4e8c4
Reviewed-on: https://go-review.googlesource.com/c/benchmarks/+/568295
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
2024-03-04 21:11:03 +00:00
Aleksey Markin 6bceba257e cmd/bent: add pgo benchmarking scenario
How to use.

The configuration for PGO measurements is located in the configurations-pgo.toml
file. This file contains two configurations, and you will need both of them.
The first configuration "pgo-generate" is enabled by default. It has a new
field "PgoGen", than contains a name of a sub-directory, where the .prof files
will be put. The second configuration "pgo-use" is disabled by default. This
configuration is needed to use the .prof files that were created during the
first phase. It has a new field "PgoUse", with a sub-directory path to .prof
files.

The usages scenario is the following:

1. Run bent with the configuration "pgo-generate" to create .prof files
2. Disable the configuration "pgo-generate" and enable "pgo-use"
3. Run bent with this configuration. This will be pgo-guided run with
   better performance

Notes: we need two new fields, because PgoGen do not add any option,
but is enables environment variables for cpuprofile script. The PgoUse
option adds the -pgo=<path to prof file> to the build line. Probably we
could avoid a step with enabling/disabling configuration for our comfort,
but I do not know how to set the particular order of configuration launches.

Change-Id: I04adf494a01e62b491cc9afce7eed49b19f9fae5
Reviewed-on: https://go-review.googlesource.com/c/benchmarks/+/565357
Reviewed-by: Than McIntosh <thanm@google.com>
Reviewed-by: David Chase <drchase@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2024-02-27 17:18:23 +00:00
Aleksey Markin 4309afc65a cmd/bent: inherit http_proxy variable
Change-Id: I6d22487c8967cdc92b84a31f622c9d6c3eface1f
Reviewed-on: https://go-review.googlesource.com/c/benchmarks/+/565635
Reviewed-by: Than McIntosh <thanm@google.com>
Reviewed-by: David Chase <drchase@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2024-02-27 17:01:47 +00:00
Gopher Robot adbfebb2c3 go.mod: update golang.org/x dependencies
Update golang.org/x dependencies to their latest tagged versions.

Change-Id: I0ee6b7d0af6f03855414b94ed18ea73a937f072c
Reviewed-on: https://go-review.googlesource.com/c/benchmarks/+/562350
Auto-Submit: Gopher Robot <gobot@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2024-02-08 14:31:16 +00:00
Gopher Robot 921b81ae98 go.mod: update golang.org/x dependencies
Update golang.org/x dependencies to their latest tagged versions.

Change-Id: I63c16716ecfa39faa3986de15fd3a8f601de496e
Reviewed-on: https://go-review.googlesource.com/c/benchmarks/+/554678
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Gopher Robot <gobot@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Than McIntosh <thanm@google.com>
2024-01-08 19:14:35 +00:00
Rob Findley e622c6e863 cmd/bench: update location of gopls benchmarks
Change-Id: I5a86759412beb2e824160323d43fd7e063e0b359
Reviewed-on: https://go-review.googlesource.com/c/benchmarks/+/545817
Reviewed-by: Alan Donovan <adonovan@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2023-11-30 19:44:00 +00:00
Gopher Robot 85f25f372b go.mod: update golang.org/x dependencies
Update golang.org/x dependencies to their latest tagged versions.

Change-Id: I24fb83843e5b60cf0baa941cac39d18b1cef9727
Reviewed-on: https://go-review.googlesource.com/c/benchmarks/+/545255
Reviewed-by: Heschi Kreinick <heschi@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Auto-Submit: Gopher Robot <gobot@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2023-11-27 18:10:43 +00:00
Gopher Robot 444331c978 go.mod: update golang.org/x dependencies
Update golang.org/x dependencies to their latest tagged versions.

Change-Id: I42155ac31847f65ff8c98dc5035055458772b4ed
Reviewed-on: https://go-review.googlesource.com/c/benchmarks/+/540819
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Auto-Submit: Gopher Robot <gobot@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Heschi Kreinick <heschi@google.com>
Reviewed-by: Carlos Amedee <carlos@golang.org>
2023-11-08 21:52:33 +00:00
Dmitri Shuralyov f8359292dd all: add test skips for GOARCH=wasm
These changes are enough to get 'GOOS=js GOARCH=wasm go test ./...'
to pass.

For golang/go#61104.

Change-Id: I78a2978a83a1026adeecc81b31bf7c2aa2db2155
Cq-Include-Trybots: luci.golang.try:x_benchmarks-gotip-js-wasm
Reviewed-on: https://go-review.googlesource.com/c/benchmarks/+/533695
Reviewed-by: Michael Pratt <mpratt@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Dmitri Shuralyov <dmitshur@golang.org>
2023-10-09 18:52:12 +00:00
Gopher Robot f6c13af26d go.mod: update golang.org/x dependencies
Update golang.org/x dependencies to their latest tagged versions.

Change-Id: Id93593830fd4b902636c5f253687a31dfdde83c6
Reviewed-on: https://go-review.googlesource.com/c/benchmarks/+/533375
Reviewed-by: Carlos Amedee <carlos@golang.org>
Auto-Submit: Gopher Robot <gobot@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
2023-10-06 13:46:23 +00:00
Rob Findley 1082dde546 cmd/bench: increase timeout for gopls tests to 180m
With recently added benchmarks, gopls benchmarks are starting to time
out.

Increase to 180m for now; we'll revisit if the benchmarks can be made
faster.

Change-Id: I2f60a8b606bebaaffac347173bdd064646e2a5fc
Reviewed-on: https://go-review.googlesource.com/c/benchmarks/+/530915
Auto-Submit: Robert Findley <rfindley@google.com>
Reviewed-by: Nooras Saba‎ <saba@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2023-09-25 14:46:45 +00:00
Gopher Robot 84f2805de5 go.mod: update golang.org/x dependencies
Update golang.org/x dependencies to their latest tagged versions.

Change-Id: I873f91eddce5db2e5464bfb6d929ba066bba2821
Reviewed-on: https://go-review.googlesource.com/c/benchmarks/+/526135
Reviewed-by: Heschi Kreinick <heschi@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Auto-Submit: Gopher Robot <gobot@golang.org>
2023-09-06 15:02:54 +00:00
Dmitri Shuralyov 3be94ae703 all: update go directive to 1.18
Done with:

go get go@1.18
go mod tidy
go fix ./...

Using go1.21.0.

Change-Id: Iab1ad7bd5cc5840c19b64ed8f02cc56de9371d04
Reviewed-on: https://go-review.googlesource.com/c/benchmarks/+/525757
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Heschi Kreinick <heschi@google.com>
Auto-Submit: Dmitri Shuralyov <dmitshur@golang.org>
2023-09-06 14:22:06 +00:00
David Chase 3cfa5422aa benchmarks/gc_latency: add new microbenchmark for tricky mutator latency issues
Gc_latency is a modified version of a program that tickled multiple
latency glitches in the Go GC/runtime.  This version reports the time
of the worst observed glitches so that they can be easily located in
a trace file and debugged.  This program can also be run as a benchmark
to allow easier automated performance monitoring; by default the benchmark doesn't report worst case times because those are too noisy.

Updates golang/go#27732.

Change-Id: I19b9060f24cda1547b8d75f762316dd5271e32c6
Reviewed-on: https://go-review.googlesource.com/c/benchmarks/+/372256
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
Run-TryBot: David Chase <drchase@google.com>
2023-08-28 20:24:59 +00:00
Michael Anthony Knyszek 6a4c432b9b bent: write out go.mod by hand and always pass go 1.21
Starting with Go 1.21, if a "too new" language version is discovered in
a go.mod, the go command will fail. This is a problem in bent, because
bent may use a newer dev toolchain to create a go.mod that can't be used
by older Go toolchains.

Since Go 1.21 is where this change happened (and older toolchains will
happily ignore "too new" language versions) use that as the default
minimum go.mod version and write out the go.mod directly.

Change-Id: If510968c62c5155eb692b5e03ffbd34afd8959d4
Reviewed-on: https://go-review.googlesource.com/c/benchmarks/+/521820
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Michael Knyszek <mknyszek@google.com>
Reviewed-by: David Chase <drchase@google.com>
2023-08-22 18:01:05 +00:00
cui fliter c7f59e451c sweet/common/diagnostics: gofmt format
Change-Id: Ia7734b408fa24d3abee6f25d47dfa85715084ed9
Reviewed-on: https://go-review.googlesource.com/c/benchmarks/+/520737
Run-TryBot: Ian Lance Taylor <iant@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
2023-08-18 22:04:20 +00:00
David Chase 1c7bc14bf2 cmd/bent: add -X flag for benchmark/library-modification experiments
Normally, for no-surprises reproducibility, bent always starts
the build directory from scratch.  This change adds a flag that
turns off that clean-start behavior, so that local edits to
build/*/go.mod for various benchmarks are not reverted.  For
example, if a change to Go breaks a library used by a benchmark,
replace directives in go.mod allow use of a modified library.

Change-Id: Ida8d3776eae081cc4fae63a0489cbb0579b7c5a8
Reviewed-on: https://go-review.googlesource.com/c/benchmarks/+/470935
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: David Chase <drchase@google.com>
Reviewed-by: Austin Clements <austin@google.com>
2023-08-16 14:54:49 +00:00
Guoqi Chen 354f4ce956 all: update go.etcd.io/bbolt to v1.3.7
go get go.etcd.io/bbolt@v1.3.7
go mod tidy

This brings in loong64 support.

Fixes golang/go#58306.

Change-Id: I8efd537441ca1334cd7f81e6b149aa39fd83a442
Reviewed-on: https://go-review.googlesource.com/c/benchmarks/+/492535
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Dmitri Shuralyov <dmitshur@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
2023-08-10 19:12:36 +00:00
Michael Anthony Knyszek 6468fc126e sweet: make tile38 and go-build with -short even shorter
-short is not meant to be used for benchmarking; just for testing and
debugging.

Change tile38 to run with a bare (empty) database, since a substantial
amount of time is spent on loading the database from disk. The lack of a
database also means that the tile38 server uses substantially less
memory, so we can let it run in parallel with more tasks in the
end-to-end test.

Change go-build to not do a first build of the target package during
Sweet's build step: it means the results won't be as useful since
they'll include downloading packages, but in -short mode it doesn't
matter.

Change-Id: I9e3559c4077fff696a374b97f91491783a26e5ac
Reviewed-on: https://go-review.googlesource.com/c/benchmarks/+/506559
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Michael Knyszek <mknyszek@google.com>
Auto-Submit: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
2023-06-27 19:24:32 +00:00
Michael Anthony Knyszek dc2b26307f sweet: reduce length of TestSweetEndToEnd
This test is failing on the LUCI builders because it's running out the
default timeout, and timeouts for subrepos haven't been ported there
yet.

Surprisingly, this is the only repository that runs out the default
clock. This prompted me to look into where time is spent in this test,
and I noticed that the etcd benchmark didn't use the -short flag at all
(!).

In addition to making use of -short in the etcd benchmark, this change
also fiddles around with other benchmarks' use of the -short flag to
make the test slightly shorter. The net decrease in time spent on my
local machine is 25% (5m -> 3m45s).

The test is also tweaked to allow more of the small benchmarks to run in
parallel during the test.

Finally, this change adds timing logs to the test, so when it next times
out we have a better idea as to why.

Change-Id: I7e394e675fd767d13778d934584c12ce23675c01
Reviewed-on: https://go-review.googlesource.com/c/benchmarks/+/506556
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Michael Pratt <mpratt@google.com>
Run-TryBot: Michael Knyszek <mknyszek@google.com>
2023-06-27 17:47:22 +00:00
Rob Findley 5444cb2fa8 cmd/bench: increase timeout for gopls tests to 60m
The benchmarks are regularly timing out due to slow network and
builder congestion.

Change-Id: Ia2ac5a7c39e07717a0e61a28e86ab223aed2d17a
Reviewed-on: https://go-review.googlesource.com/c/benchmarks/+/503515
Reviewed-by: Michael Pratt <mpratt@google.com>
Run-TryBot: Robert Findley <rfindley@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
2023-06-14 21:11:50 +00:00
Michael Pratt 73f971c9d6 Revert "cmd/bench: add PGO sweet runs"
This reverts CL 498278.

Reason for revert: This causes -perf builders to run out of disk space. 
Revert until we can reduce usage or add more disk.

Change-Id: Ibc64e69be3c73744cebe690db320a14300cccf47
Reviewed-on: https://go-review.googlesource.com/c/benchmarks/+/503417
Run-TryBot: Michael Pratt <mpratt@google.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Auto-Submit: Michael Pratt <mpratt@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
2023-06-14 20:07:49 +00:00
Michael Pratt be7df72229 cmd/bench: add PGO sweet runs
Sweet represents PGO runs as a unique configuration (using
`name+".pgo"`), effectively a unique "toolchain". But it is really a
separate dimension. So when reporting results, we use the same
"toolchain" for both and add the new "pgo" key that reports whether PGO
was enabled or not.

CL 498335 adds backend parsing of the results and bumps the builder
timeout for the extra runtime.

Change-Id: Iea1c4fe81b7d2a35bc0cf15c1a1be87c87f2a90b
Reviewed-on: https://go-review.googlesource.com/c/benchmarks/+/498278
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Run-TryBot: Michael Pratt <mpratt@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
2023-06-13 20:27:23 +00:00
Michael Pratt 4682cbf96c sweet: force -pgo=off for non-PGO builds
As of CL 497455, cmd/compile has a default.pgo that is applied
automatically. This means that `sweet run -pgo -run go-build` is
actually going to compare default.pgo vs a new profile, rather than no
pgo vs a new profile, which is what folks will expect.

Thus when not in a PGO build, force PGO off.

Change-Id: Ica1da2d2d8a4348cc3b8fa2e7b5bdbe9ac00e2ee
Reviewed-on: https://go-review.googlesource.com/c/benchmarks/+/498276
Reviewed-by: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Michael Pratt <mpratt@google.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
2023-06-13 20:27:16 +00:00
Michael Anthony Knyszek 018588a256 sweet: don't fetch large github repos for go-build benchmark if short
Currently the go-build benchmark fetches k8s and istio even if -short is
set. It won't build these repositories in this case, just fetch them. It
appears that fetching them is somewhat flaky, probably GitHub not liking
the fact that we're cloning such large repositories so often. (It's hard
to say why exactly it's failing, but every recently failure in the Sweet
integration test (#56958) is in exactly one of two places: cloning k8s
or istio from GitHub. It's also clear this cloning step is unnecessary
for this test.)

This CL modifies the harness' Get method to accept a struct (aligning
with the other harness methods) that now includes a Short parameter, the
same parameter passed to Build and Run. Now that "short" is plumbed down
into Get, we can skip cloning these repositories.

Fixes #56958.

Change-Id: Ia9e3749ac82608bc24a4a4791955d3a1f546faa1
Reviewed-on: https://go-review.googlesource.com/c/benchmarks/+/498284
Run-TryBot: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
2023-05-31 15:34:51 +00:00
David Chase d861a6ead4 cmd/bent: freshen cmpjob.sh and cmpcl.sh
Added optional running of localfix for better behavior on
processors than need that (e.g., arm64 Big.little), and added
more shell quoting just-in-case.

Change-Id: Ic55c0a81b2c1f2ecbae75abcb6f965a1cc862867
Reviewed-on: https://go-review.googlesource.com/c/benchmarks/+/485395
Reviewed-by: Michael Pratt <mpratt@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: David Chase <drchase@google.com>
2023-05-31 15:14:23 +00:00
Andrew Polukhin c4c5b3d18f all: remove unused code
Change-Id: I5bec6094d9dc5d16c685b8cbbecc7e1bb278fde9
Reviewed-on: https://go-review.googlesource.com/c/benchmarks/+/487775
Reviewed-by: David Chase <drchase@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Heschi Kreinick <heschi@google.com>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
2023-05-24 17:54:05 +00:00
Michael Anthony Knyszek 439d592ebe sweet: remove etcd benchmark output spew
This causes the upstream log to be truncated, and it looks like
benchmark results are now missing for some benchmarks on some commits.

Change-Id: I29f434870a21d4ccdd0dad7d8825e5bb0fc4dbdc
Reviewed-on: https://go-review.googlesource.com/c/benchmarks/+/493920
Run-TryBot: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
Auto-Submit: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
2023-05-10 17:32:10 +00:00
Michael Anthony Knyszek 0aa8cc8f77 sweet: make etcd a default benchmark
Change-Id: I91204b6500bac3d45e71f44eca553c5798289bb9
Reviewed-on: https://go-review.googlesource.com/c/benchmarks/+/493919
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Michael Pratt <mpratt@google.com>
Auto-Submit: Michael Knyszek <mknyszek@google.com>
2023-05-10 17:32:08 +00:00
Michael Anthony Knyszek 36b0653437 etcd: add benchmark
This change adds a benchmark that spins up a local cluster of 3 etcd
instances and runs etcd's benchmark tool against it.

For golang/go#59672.

Change-Id: I1c23439bad7bb16e055e689ddd4df8bf8d4b5288
Reviewed-on: https://go-review.googlesource.com/c/benchmarks/+/492776
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Auto-Submit: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
2023-05-09 17:57:03 +00:00
Michael Anthony Knyszek dce5bf9b3b sweet: merge tile38 benchmarks into one benchmark with all 3 queries
This change merges the 3 Tile38*Request benchmarks into Tile38QueryLoad.
The purpose of this change is three-fold:
- The existing benchmarks were more microbenchmarks since they really
  just hammered a single codepath. Here we at least hammer 3, which is
  nice for e.g. PGO experimentation.
- The Tile38 server is now only started up once for a benchmark run.
  Previously it was started up 3 times, each time taking around 30
  seconds to load state from disk, causing benchmarking to be really
  slow.
- This change lets us afford to run a single Tile38 server for longer,
  which should help reduce GC-related noise by adding more cycles.

This change increases the number of benchmark iterations from 20*50000
to 40*50000, a 2x increase. While intuitively that seems like it
should make the benchmark only slightly faster, since we're only
spinning up one server for all those iterations, we actually save a lot
more time.

20*50000 iterations was usually ~30 seconds. It took ~30 seconds to
start up the server. For 1 full run it took 3 minutes.

After this change, starting up the server still takes ~30 seconds, but
we only do it once, and the iterations itself only take ~60 seconds.
Now, 1 full run only takes 1 minute 30 seconds, or half as long.

For golang/go#59672.

Change-Id: I5694737e8dde977cbc6013f1d879d0a742a6c895
Reviewed-on: https://go-review.googlesource.com/c/benchmarks/+/485375
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
2023-04-28 16:23:15 +00:00
Robert Findley 2652dad2ab cmd/bench: update gopls benchmarks to filter to -short
This fixes a breakage due to renaming of BenchmarkGoToDefinition, and
provides a means for us to configure the benchmarks run by x/benchmark
in the future.

Also add some documentation, and refactor somewhat. Remove -v for gopls
benchmarks, as that prints noisy results.

Fixes golang/go#58875

Change-Id: I862ace4af029b8042d50467567bf314b2e5521ae
Reviewed-on: https://go-review.googlesource.com/c/benchmarks/+/473756
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Robert Findley <rfindley@google.com>
Auto-Submit: Robert Findley <rfindley@google.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
2023-03-07 14:28:52 +00:00
Michael Anthony Knyszek a97ee2e8b4 cmd/bench: clean build cache in between bent and sweet
Currently /workdir/gocache uses up 9.3 GiB of 16 GiB of available
space in /workdir, causing Sweet to not have enough room to copy
tile38's assets over by the time we get to that.

Thing is, in these benchmarks we basically don't care about the build
cache at all since we only build everything once. Clean the cache at
least once in between bent and Sweet to let Sweet complete.

Fixes golang/go#58765.

Change-Id: I6f10a7eee0218ca2d2b0358a11cfd41af00b0667
Reviewed-on: https://go-review.googlesource.com/c/benchmarks/+/472235
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Auto-Submit: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
Reviewed-by: Than McIntosh <thanm@google.com>
2023-03-02 01:47:03 +00:00
David Chase 1177310668 cmd/bent: don't clean cache, use -a always instead.
Build caching changed in 1.20, and building after a cleaned cache now
takes much much longer, no longer reflects typical user experience,
and is incomparable with earlier releases.  This should have been the
choice all along.

Change-Id: Idc3200d0b5d9cee4afd017abc111b2a9051aa0a9
Reviewed-on: https://go-review.googlesource.com/c/benchmarks/+/452595
Run-TryBot: David Chase <drchase@google.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
2023-02-23 23:03:26 +00:00
David Chase c42c638f13 cmd/bent: fix overlapping output lines that broke "shortname:"
This fix also separates benchmark runs with blank lines, which looks
a whole darn lot better.

Change-Id: I39b4ef42c742b253ced4294872f9a07de022f136
Reviewed-on: https://go-review.googlesource.com/c/benchmarks/+/462717
Reviewed-by: Michael Pratt <mpratt@google.com>
Run-TryBot: David Chase <drchase@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
2023-02-23 23:02:49 +00:00
Michael Anthony Knyszek b6edf46ca4 bent: remove noisy and low signal benchmarks from benchmarks-50
This is the configuration used by the benchmarking pipeline, and these
benchmarks appear to be low-signal.

Fixes #58534.

Change-Id: Ic49d72081809c68913dcf339d40421e4184d8d08
Reviewed-on: https://go-review.googlesource.com/c/benchmarks/+/468558
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Michael Pratt <mpratt@google.com>
Auto-Submit: Michael Knyszek <mknyszek@google.com>
Reviewed-by: David Chase <drchase@google.com>
2023-02-16 18:24:39 +00:00
Michael Anthony Knyszek 66f6a68fc0 third_party: clean up unused fogleman-* packages
For #58534.

Change-Id: I4e54affc59eda65343b4ccd32aab6291c609f3dc
Reviewed-on: https://go-review.googlesource.com/c/benchmarks/+/468559
Run-TryBot: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
Auto-Submit: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
2023-02-15 22:31:12 +00:00
Michael Anthony Knyszek 394af22c68 reflect: remove the Map benchmarks
They were initially added just as a test, but they've proven to be noisy
and low signal. Remove them.

This CL explicitly leaves behind the "go test" mode of cmd/bench so we
can easily add other "go test" benchmarks back. Right now it just won't
find anything, which is fine.

For #58534.

Change-Id: I60aa8fee3157d83dbabe58e8cefd28aeb2dbf6ec
Reviewed-on: https://go-review.googlesource.com/c/benchmarks/+/468557
Run-TryBot: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
2023-02-15 21:02:35 +00:00
Michael Anthony Knyszek a30118b749 sweet: remove bleve-query, fogleman-fauxgl, fogleman-pt
These three benchmarks have been fairly low signal and noisy.
Furthermore the fogleman-* benchmarks have some portability issues.

Fixes #58533.
For #58534.

Change-Id: I98b028a61eff2531fa4921852c40aab05f0af2d7
Reviewed-on: https://go-review.googlesource.com/c/benchmarks/+/468556
Run-TryBot: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
Auto-Submit: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
2023-02-15 20:57:00 +00:00
Michael Anthony Knyszek 2874b5b824 sweet: add support for execution traces and refactor profile plumbing
This change does a lot at once, but it's mostly refactoring. First, it
moves most of the profile abstraction out of benchmarks/internal/driver
and into a new shared package called diagnostics. It also renames
profiles to diagnostics to better capture the breadth of what this
mechanism collects. Then, it adds support for turning on diagnostics
from configuration files. Next, it adds support for generating
additional configurations to capture the overhead of collecting
diagnostics, starting with CPU profiling. Lastly, it adds support for
the new Trace diagnostic.

(This change also fixes a bug in go-build where Linux perf flags weren't
being propagated.)

In the future, core dumps could easily be folded into this new
diagnostics abstraction.

For golang/go#57175.

Change-Id: I999773e8be28c46fb5d4f6a79a94d542491e3754
Reviewed-on: https://go-review.googlesource.com/c/benchmarks/+/459095
Run-TryBot: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
2023-02-14 23:00:39 +00:00
cui fliter 8520151a72 all: fix some comments
Change-Id: I8a332489815651067a36c84690db9c6cb540882e
Reviewed-on: https://go-review.googlesource.com/c/benchmarks/+/465998
Run-TryBot: Ian Lance Taylor <iant@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: David Chase <drchase@google.com>
2023-02-08 14:50:34 +00:00
Michael Pratt 565f2d429f cmd/bent: add flag to disable build time reporting
We do not want these benchmarks in cmd/bench because we only run builds
once and thus do not get statistically significant results anyway.

For golang/go#57770.

Change-Id: I75987416458f826ee0a3e71c69f36fd48206eb77
Reviewed-on: https://go-review.googlesource.com/c/benchmarks/+/463196
Run-TryBot: Michael Pratt <mpratt@google.com>
Auto-Submit: Michael Pratt <mpratt@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: David Chase <drchase@google.com>
2023-01-26 18:16:47 +00:00
Michael Pratt 77c7737ef3 cmd/bent: move toolchain key print to lower layer
CL 462956 added logging of the toolchain key to properly annotate build
/ after-build benchmarks. This worked fine for the perf dashboard, but
results also go to a file, which this logging did not do. Additionally,
it logged even in non-verbose mode, when benchmark results are not.

Move this logging down near the benchmarks themselves to fix these
inconsistencies.

For golang/go#57770.

Change-Id: I6502092bd93a2dd6932018989c69fba5b51adff8
Reviewed-on: https://go-review.googlesource.com/c/benchmarks/+/463195
Reviewed-by: David Chase <drchase@google.com>
Run-TryBot: Michael Pratt <mpratt@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Auto-Submit: Michael Pratt <mpratt@google.com>
2023-01-26 18:16:45 +00:00
Michael Pratt 33cb330661 cmd/bent: clean up benchsize
1. Remove the section from the benchmark names. It is already in the
   unit, so this will provide better grouping.

2. Capitalize the first letter in the name. This makes the benchmark
   name match the name used for the build time benchmark.

3. Change section size parsing to use exact matches.

The existing substring match could match multiple lines. e.g.,
`zdebug_loc` matching `.zdebug_loc` and `.zdebug_loclists`. When this
happens, $zdebug_loc contains a string like "12345 123", which breaks
the later addition in `expr`.

Fix this by requiring an exact match of the section name field with the
desired section name. Rather than making a more complex regexp, just
include this as a condition in the awk command. Darwin and GNU size has
different formats, so they each require their own format.

For golang/go#57770.

Change-Id: I482bb90b29578179a3b59637c265580f06becaf8
Reviewed-on: https://go-review.googlesource.com/c/benchmarks/+/462957
Auto-Submit: Michael Pratt <mpratt@google.com>
Run-TryBot: Michael Pratt <mpratt@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
2023-01-20 21:10:30 +00:00
Michael Pratt 3269e10758 cmd/bent: log toolchain prior to compilation
Compilation may emit compilation benchmark results, so we need to log
the toolchain in use first.

For golang/go#57770.

Change-Id: Ia1fdbbabba9a556ff6be720fcd7455de23d3d147
Reviewed-on: https://go-review.googlesource.com/c/benchmarks/+/462956
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Auto-Submit: Michael Pratt <mpratt@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Michael Pratt <mpratt@google.com>
2023-01-20 21:03:59 +00:00
Michael Pratt 8d1b8c5ff6 cmd/bench: benchmark binary size and build time via bent
benchsize adds binary size benchmark results for each package in bent.
bent doesn't log these to stdout by default, we need to add -v for that.

It turns out that bent also has build time benchmarks that it doesn't
log without -v. So now we get those for free too.

Updates golang/go#57770.

Change-Id: I2b714a4f2e73842fe79d5d227a397efeca794d06
Reviewed-on: https://go-review.googlesource.com/c/benchmarks/+/462718
Run-TryBot: Michael Pratt <mpratt@google.com>
Auto-Submit: Michael Pratt <mpratt@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Reviewed-by: David Chase <drchase@google.com>
2023-01-18 22:14:08 +00:00