Tags: linmingren/cockroach
Tags
Merge cockroachdb#26615 26615: release-2.0: sql,compactor: rate limit clear range requests r=bdarnell,petermattis a=benesch Backports cockroachdb#26449. I'm running a clearrange roachtest with this diff applied tonight. If it passes we're good to go. ```diff diff --git a/pkg/cmd/roachtest/clearrange.go b/pkg/cmd/roachtest/clearrange.go index ea5bcdf..2b244af6d 100644 --- a/pkg/cmd/roachtest/clearrange.go +++ b/pkg/cmd/roachtest/clearrange.go @@ -30,19 +30,9 @@ func registerClearRange(r *registry) { // thoroughly brick the cluster. Stable: false, Run: func(ctx context.Context, t *test, c *cluster) { - t.Status(`downloading store dumps`) - // Created via: - // roachtest --cockroach cockroach-v2.0.1 store-gen --stores=10 bank \ - // --payload-bytes=10240 --ranges=0 --rows=65104166 - fixtureURL := `gs://cockroach-fixtures/workload/bank/version=1.0.0,payload-bytes=10240,ranges=0,rows=65104166,seed=1` - location := storeDirURL(fixtureURL, c.nodes, "2.0") + t.Status(`waiting for compactions to disappear`) + time.Sleep(90 * time.Minute) - // Download this store dump, which measures around 2TB (across all nodes). - if err := downloadStoreDumps(ctx, c, location, c.nodes); err != nil { - t.Fatal(err) - } - - c.Put(ctx, cockroach, "./cockroach") c.Start(ctx) // Also restore a much smaller table. We'll use it to run queries against @@ -81,7 +71,7 @@ func registerClearRange(r *registry) { // above didn't brick the cluster. // // Don't lower this number, or the test may pass erroneously. - const minutes = 60 + const minutes = 120 t.WorkerStatus("repeatedly running COUNT(*) on small table") for i := 0; i < minutes; i++ { after := time.After(time.Minute) ``` --- Now that DBCompactRange no longer attempts to compact the entire database (cockroachdb#26355), sending ClearRange requests in sequential batches of 50is enough to prevent a large DROP TABLE from bricking a cluster. They're slow enough that the compaction queue can keep up and purge range deletion tombstones before enough pile up to wedge the cluster. This is a partial fix for cockroachdb#24029. Release note (bug fix): The performance impact of dropping a large table has been substantially reduced. Co-authored-by: Nikhil Benesch <nikhil.benesch@gmail.com>
storage: Tick all replicas, not just unquiesced ones This reverts the main effect of cockroachdb#24956. The supporting infrastructure is left in place to minimize merge conflicts and to aid in diagnosing why the maps get out of sync. Fixes cockroachdb#26257 Release note: None
Merge cockroachdb#25307 25307: release-2.0: importccl: check node health and compatibility during IMPORT planning r=mjibson a=mjibson Backport 1/1 commits from cockroachdb#25162. /cc @cockroachdb/release --- Simplify the LoadCSV signature by taking just a PlanHookState for any argument that can be fetched from it. Determine the node list using this new health check function. We can remove the rand.Shuffle call because the map iteration should produce some level of randomness. Fixes cockroachdb#12876 Release note (bug fix): Fix problems with imports sometimes failing after node decommissioning. Co-authored-by: Matt Jibson <matt.jibson@gmail.com>
Merge cockroachdb#25126 25126: opt: support for EXPLAIN r=RaduBerinde a=RaduBerinde #### opt: introduce and build ExplainOp The ExplainOp must remember the physical props required by the underlying query. For now we only support `EXPLAIN (VERBOSE)`. Release note: None #### opt: execbuild support for EXPLAIN Release note: None #### opt: remove exec-explain Remove the `exec-explain` directive and convert all the tests to exec an EXPLAIN statement. Add `hide-columns` option to suppress the header with the column names and types, for cleaner output (and to avoid a ton of diffs in this change). Release note: None Co-authored-by: Radu Berinde <radu@cockroachlabs.com>
Merge cockroachdb#25126 25126: opt: support for EXPLAIN r=RaduBerinde a=RaduBerinde #### opt: introduce and build ExplainOp The ExplainOp must remember the physical props required by the underlying query. For now we only support `EXPLAIN (VERBOSE)`. Release note: None #### opt: execbuild support for EXPLAIN Release note: None #### opt: remove exec-explain Remove the `exec-explain` directive and convert all the tests to exec an EXPLAIN statement. Add `hide-columns` option to suppress the header with the column names and types, for cleaner output (and to avoid a ton of diffs in this change). Release note: None Co-authored-by: Radu Berinde <radu@cockroachlabs.com>
Merge cockroachdb#24889 24889: cherrypick-1.1: build: Update etcd r=bdarnell a=bdarnell Picks up a cherry-picked version of etcd-io/etcd#9073, to fix cockroachdb#18601 Release note (bug fix): Fixes potential cluster unavailability after raft logs grow too large. Co-authored-by: Ben Darnell <ben@cockroachlabs.com>
Merge cockroachdb#24825 24825: cherrypick-2.0: distsql: clean up lookup join internal columns r=pbardea a=pbardea Cherry-picks cockroachdb#24697 and makes a few changes from the original PR: - passing `nil` as the flowCtx in`processorBase.init` call in joinreader.go - the output of the `EXPLAIN (VERBOSE)` test in the `lookup_join` logic test Fixes cockroachdb#24401. The distsql physical planner was not properly forming the post processor spec according to the internal columns of lookup joins. For the hash and merge joiners, the internal columns depend on the streamToColMap of its upstream nodes. However since the lookup join's internal columns consist of the entire right side of table columns, the planner must modify the post processor spec and various maps. This simplifies code inside the lookup joiner. Release note (bug fix): Fix a bug causing some lookup join queries to report incorrect type errors. Co-authored-by: Paul Bardea <pbardea@gmail.com>
Merge cockroachdb#24594 24594: ts: Initial TimeSeries TestModel. r=mrtracy a=mrtracy Creates a new "testmodel" package under "ts" which contains an in-memory test model designed to be much more conceptually simple than the optimized query engine actually used by time series. The new test model stores time series in sorted arrays as full-fidelity nanosecond data points. It then uses a highly decomposed set of functions to process that data; these functions have been designed to be individually tested. The "Query" method shows an example of how simple this system is when compared to the actual time series system, which uses iterators and breaks up queries into multiple chunks. This model has not yet been hooked up to time series tests; it will soon be used to replace the existing test model, which is *not* easier to understand than the actual time series database. Co-authored-by: Matt Tracy <matt@cockroachlabs.com>
Merge pull request cockroachdb#24458 from petermattis/pmattis/2.0-dum… …my-commit cli: remove a superfluous TODO
Merge pull request cockroachdb#24393 from mberhault/backport2.0-24390 backport-2.0: sql: fill in required privileges at Validate time.
PreviousNext