8000 Sentry: catch.go:24: checking for BACKUP-LOCK file: fail to list destination: file-table-storage-list: runtime error: index out of range [6] with length 6 (1) Wraps: (2) attached stack trace -- stac... · Issue #148319 · cockroachdb/cockroach · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content
Sentry: catch.go:24: checking for BACKUP-LOCK file: fail to list destination: file-table-storage-list: runtime error: index out of range [6] with length 6 (1) Wraps: (2) attached stack trace -- stac... #148319
New issue
New issue
Closed
Closed
Sentry: catch.go:24: checking for BACKUP-LOCK file: fail to list destination: file-table-storage-list: runtime error: index out of range [6] with length 6 (1) Wraps: (2) attached stack trace -- stac...#148319
Labels
C-bugCode not up to spec/doc, specs & docs deemed correct. Solution expected to change code/behavior.O-sentryOriginated from an in-the-wild panic report.T-sql-queriesSQL Queries TeamX-blathers-triagedblathers was able to find an ownerbranch-release-25.2
@cockroach-sentry

Description

@cockroach-sentry
cockroach-sentry
opened on Jun 13, 2025

This issue was auto filed by Sentry. It represents a crash or reported error on a live cluster with telemetry enabled.

Sentry Link: https://cockroach-labs.sentry.io/issues/6678613012/?referrer=webhooks_plugin

Panic Message:

catch.go:24: checking for BACKUP-LOCK file: fail to list destination: file-table-storage-list: runtime error: index out of range [6] with length 6
(1)
Wraps: (2) attached stack trace
  -- stack trace:
  | github.com/cockroachdb/cockroach/pkg/backup/backupinfo.CheckForPreviousBackup
  | 	pkg/backup/backupinfo/manifest_handling.go:1310
  | github.com/cockroachdb/cockroach/pkg/backup.maybeWriteBackupLock
  | 	pkg/backup/backup_job.go:2050
  | github.com/cockroachdb/cockroach/pkg/backup.(*backupResumer).Resume
  | 	pkg/backup/backup_job.go:594
  | github.com/cockroachdb/cockroach/pkg/jobs.(*Registry).stepThroughStateMachine.func2
  | 	pkg/jobs/registry.go:1720
  | github.com/cockroachdb/cockroach/pkg/jobs.(*Registry).stepThroughStateMachine
  | 	pkg/jobs/registry.go:1721
  | github.com/cockroachdb/cockroach/pkg/jobs.(*Registry).runJob
  | 	pkg/jobs/adopt.go:410
  | github.com/cockroachdb/cockroach/pkg/jobs.(*StartableJob).Start.func2
  | 	pkg/jobs/jobs.go:799
  | github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2
  | 	pkg/util/stop/stopper.go:500
Wraps: (3) checking for BACKUP-LOCK file
Wraps: (4) attached stack trace
  -- stack trace:
  | github.com/cockroachdb/cockroach/pkg/cloud/userfile.(*fileTableStorage).List
  | 	pkg/cloud/userfile/file_table_storage.go:251
  | github.com/cockroachdb/cockroach/pkg/cloud.(*esWrapper).List
  | 	pkg/cloud/impl_registry.go:420
  | github.com/cockroachdb/cockroach/pkg/backup/backupinfo.CheckForPreviousBackup
  | 	pkg/backup/backupinfo/manifest_handling.go:1292
  | github.com/cockroachdb/cockroach/pkg/backup.maybeWriteBackupLock
  | 	pkg/backup/backup_job.go:2050
  | github.com/cockroachdb/cockroach/pkg/backup.(*backupResumer).Resume
  | 	pkg/backup/backup_job.go:594
  | github.com/cockroachdb/cockroach/pkg/jobs.(*Registry).stepThroughStateMachine.func2
  | 	pkg/jobs/registry.go:1720
  | github.com/cockroachdb/cockroach/pkg/jobs.(*Registry).stepThroughStateMachine
  | 	pkg/jobs/registry.go:1721
  | github.com/cockroachdb/cockroach/pkg/jobs.(*Registry).runJob
  | 	pkg/jobs/adopt.go:410
  | github.com/cockroachdb/cockroach/pkg/jobs.(*StartableJob).Start.func2
  | 	pkg/jobs/jobs.go:799
  | github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2
  | 	pkg/util/stop/stopper.go:500
Wraps: (5) fail to list destination
Wraps: (6) secondary error attachment
  | file-table-storage-list: runtime error: index out of range [6] with length 6
  | (1) attached stack trace
  |   -- stack trace:
  |   | github.com/cockroachdb/cockroach/pkg/sql.(*InternalExecutor).execInternal.func1.1
  |   | 	pkg/sql/internal.go:1260
  |   | github.com/cockroachdb/cockroach/pkg/sql.(*rowsIterator).Close
  |   | 	pkg/sql/internal.go:608
  |   | github.com/cockroachdb/cockroach/pkg/sql.(*rowsIterator).Next
  |   | 	pkg/sql/internal.go:584
  |   | github.com/cockroachdb/cockroach/pkg/cloud/userfile/filetable.(*FileToTableSystem).ListFiles
  |   | 	pkg/cloud/userfile/filetable/file_table_read_writer.go:338
  |   | github.com/cockroachdb/cockroach/pkg/cloud/userfile.(*fileTableStorage).List
  |   | 	pkg/cloud/userfile/file_table_storage.go:249
  |   | github.com/cockroachdb/cockroach/pkg/cloud.(*esWrapper).List
  |   | 	pkg/cloud/impl_registry.go:420
  |   | github.com/cockroachdb/cockroach/pkg/backup/backupinfo.CheckForPreviousBackup
  |   | 	pkg/backup/backupinfo/manifest_handling.go:1292
  |   | github.com/cockroachdb/cockroach/pkg/backup.maybeWriteBackupLock
  |   | 	pkg/backup/backup_job.go:2050
  |   | github.com/cockroachdb/cockroach/pkg/backup.(*backupResumer).Resume
  |   | 	pkg/backup/backup_job.go:594
  |   | github.com/cockroachdb/cockroach/pkg/jobs.(*Registry).stepThroughStateMachine.func2
  |   | 	pkg/jobs/registry.go:1720
  |   | github.com/cockroachdb/cockroach/pkg/jobs.(*Registry).stepThroughStateMachine
  |   | 	pkg/jobs/registry.go:1721
  |   | github.com/cockroachdb/cockroach/pkg/jobs.(*Registry).runJob
  |   | 	pkg/jobs/adopt.go:410
  |   | [...repeated from below...]
  | Wraps: (2) file-table-storage-list
  | Wraps: (3) assertion failure
  | Wraps: (4) attached stack trace
  |   -- stack trace:
  |   | github.com/cockroachdb/cockroach/pkg/util/errorutil.ShouldCatch
  |   | 	pkg/util/errorutil/catch.go:24
  |   | github.com/cockroachdb/cockroach/pkg/sql/opt/exec/execbuilder.(*Builder).build.func1
  |   | 	pkg/sql/opt/exec/execbuilder/builder.go:356
  |   | runtime.gopanic
  |   | 	GOROOT/src/runtime/panic.go:791
  |   | runtime.goPanicIndex
  |   | 	GOROOT/src/runtime/panic.go:115
  |   | github.com/cockroachdb/cockroach/pkg/sql.(*optTable).Statistic
  |   | 	pkg/sql/opt_catalog.go:1441
  |   | github.com/cockroachdb/cockroach/pkg/sql/opt/exec/execbuilder.(*Builder).maybeAnnotateWithEstimates
  |   | 	pkg/sql/opt/exec/execbuilder/relational.go:418
  |   | github.com/cockroachdb/cockroach/pkg/sql/opt/exec/execbuilder.(*Builder).buildRelational
  |   | 	pkg/sql/opt/exec/execbuilder/relational.go:383
  |   | github.com/cockroachdb/cockroach/pkg/sql/opt/exec/execbuilder.(*Builder).build
  |   | 	pkg/sql/opt/exec/execbuilder/builder.go:380
  |   | github.com/cockroachdb/cockroach/pkg/sql/opt/exec/execbuilder.(*Builder).Build
  |   | 	pkg/sql/opt/exec/execbuilder/builder.go:298
  |   | github.com/cockroachdb/cockroach/pkg/sql.(*optPlanningCtx).runExecBuilder
  |   | 	pkg/sql/plan_opt.go:935
  |   | github.com/cockroachdb/cockroach/pkg/sql.(*planner).runExecBuild
  |   | 	pkg/sql/plan_opt.go:344
  |   | github.com/cockroachdb/cockroach/pkg/sql.(*planner).makeOptimizerPlan
  |   | 	pkg/sql/plan_opt.go:267
  |   | github.com/cockroachdb/cockroach/pkg/sql.(*connExecutor).makeExecPlan
  |   | 	pkg/sql/conn_executor_exec.go:3320
  |   | github.com/cockroachdb/cockroach/pkg/sql.(*connExecutor).dispatchToExecutionEngine
  |   | 	pkg/sql/conn_executor_exec.go:2867
  |   | github.com/cockroachdb/cockroach/pkg/sql.(*connExecutor).execStmtInOpenState
  |   | 	pkg/sql/conn_executor_exec.go:1085
  |   | github.com/cockroachdb/cockroach/pkg/sql.(*connExecutor).execStmt.func2
  |   | 	pkg/sql/conn_executor_exec.go:170
  |   | github.com/cockroachdb/cockroach
AD31
/pkg/sql.(*connExecutor).execWithProfiling
  |   | 	pkg/sql/conn_executor_exec.go:4476
  |   | github.com/cockroachdb/cockroach/pkg/sql.(*connExecutor).execStmt
  |   | 	pkg/sql/conn_executor_exec.go:169
  |   | github.com/cockroachdb/cockroach/pkg/sql.(*connExecutor).execPortal
  |   | 	pkg/sql/conn_executor_exec.go:320
  |   | github.com/cockroachdb/cockroach/pkg/sql.(*connExecutor).execCmd.func2
  |   | 	pkg/sql/conn_executor.go:2556
  |   | github.com/cockroachdb/cockroach/pkg/sql.(*connExecutor).execCmd
  |   | 	pkg/sql/conn_executor.go:2558
  |   | github.com/cockroachdb/cockroach/pkg/sql.(*connExecutor).run
  |   | 	pkg/sql/conn_executor.go:2366
  |   | github.com/cockroachdb/cockroach/pkg/sql.(*InternalExecutor).runWithEx.func2
  |   | 	pkg/sql/internal.go:253
  |   | github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2
  |   | 	pkg/util/stop/stopper.go:500
  |   | runtime.goexit
  |   | 	src/runtime/asm_arm64.s:1223
  | Wraps: (5) runtime error: index out of range [6] with length 6
  |   | -- cause hidden behind barrier
  |   | runtime error: index out of range [6] with length 6
  |   | (1) runtime error: index out of range [6] with length 6
  |   | Error types: (1) runtime.boundsError
  | Error types: (1) *withstack.withStack (2) *errutil.withPrefix (3) *assert.withAssertionFailure (4) *withstack.withStack (5) *barriers.barrierErr
Wraps: (7) attached stack trace
  -- stack trace:
  | github.com/cockroachdb/cockroach/pkg/sql.(*InternalExecutor).execInternal.func1.1
  | 	pkg/sql/internal.go:1260
  | github.com/cockroachdb/cockroach/pkg/sql.(*rowsIterator).Close
  | 	pkg/sql/internal.go:608
  | github.com/cockroachdb/cockroach/pkg/sql.(*rowsIterator).Next
  | 	pkg/sql/internal.go:584
  | github.com/cockroachdb/cockroach/pkg/cloud/userfile/filetable.(*FileToTableSystem).ListFiles
  | 	pkg/cloud/userfile/filetable/file_table_read_writer.go:338
  | github.com/cockroachdb/cockroach/pkg/cloud/userfile.(*fileTableStorage).List
  | 	pkg/cloud/userfile/file_table_storage.go:249
  | github.com/cockroachdb/cockroach/pk...
Stacktrace (expand for inline code snippets):

src/runtime/asm_arm64.s#L1222-L1224
pkg/util/stop/stopper.go#L499-L501
pkg/jobs/jobs.go#L798-L800
pkg/jobs/adopt.go#L409-L411
pkg/jobs/registry.go#L1720-L1722
pkg/jobs/registry.go#L1719-L1721
pkg/backup/backup_job.go#L593-L595
pkg/backup/backup_job.go#L2049-L2051
pkg/backup/backupinfo/manifest_handling.go#L1309-L1311
src/runtime/asm_arm64.s#L1222-L1224
pkg/util/stop/stopper.go#L499-L501
pkg/jobs/jobs.go#L798-L800
pkg/jobs/adopt.go#L409-L411
pkg/jobs/registry.go#L1720-L1722
pkg/jobs/registry.go#L1719-L1721
pkg/backup/backup_job.go#L593-L595
pkg/backup/backup_job.go#L2049-L2051
pkg/backup/backupinfo/manifest_handling.go#L1291-L1293
pkg/cloud/impl_registry.go#L419-L421
pkg/cloud/userfile/file_table_storage.go#L250-L252
src/runtime/asm_arm64.s#L1222-L1224
pkg/util/stop/stopper.go#L499-L501
pkg/jobs/jobs.go#L798-L800
pkg/jobs/adopt.go#L409-L411
pkg/jobs/registry.go#L1720-L1722
pkg/jobs/registry.go#L1719-L1721
pkg/backup/backup_job.go#L593-L595
pkg/backup/backup_job.go#L2049-L2051
pkg/backup/backupinfo/manifest_handling.go#L1291-L1293
pkg/cloud/impl_registry.go#L419-L421
pkg/cloud/userfile/file_table_storage.go#L248-L250
pkg/cloud/userfile/filetable/file_table_read_writer.go#L337-L339
pkg/sql/internal.go#L583-L585
pkg/sql/internal.go#L607-L609
pkg/sql/internal.go#L1259-L1261
src/runtime/asm_arm64.s#L1222-L1224
pkg/util/stop/stopper.go#L499-L501
pkg/sql/internal.go#L252-L254
pkg/sql/conn_executor.go#L2365-L2367
pkg/sql/conn_executor.go#L2557-L2559
pkg/sql/conn_executor.go#L2555-L2557
pkg/sql/conn_executor_exec.go#L319-L321
pkg/sql/conn_executor_exec.go#L168-L170
pkg/sql/conn_executor_exec.go#L4475-L4477
pkg/sql/conn_executor_exec.go#L169-L171
pkg/sql/conn_executor_exec.go#L1084-L1086
pkg/sql/conn_executor_exec.go#L2866-L2868
pkg/sql/conn_executor_exec.go#L3319-L3321
pkg/sql/plan_opt.go#L266-L268
pkg/sql/plan_opt.go#L343-L345
pkg/sql/plan_opt.go#L934-L936
pkg/sql/opt/exec/execbuilder/builder.go#L297-L299
pkg/sql/opt/exec/execbuilder/builder.go#L379-L381
pkg/sql/opt/exec/execbuilder/relational.go#L382-L384
pkg/sql/opt/exec/execbuilder/relational.go#L417-L419
pkg/sql/opt_catalog.go#L1440-L1442
GOROOT/src/runtime/panic.go#L114-L116
GOROOT/src/runtime/panic.go#L790-L792
pkg/sql/opt/exec/execbuilder/builder.go#L355-L357
pkg/util/errorutil/catch.go#L23-L25

src/runtime/asm_arm64.s in runtime.goexit at line 1223
pkg/util/stop/stopper.go in pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2 at line 500
pkg/jobs/jobs.go in pkg/jobs.(*StartableJob).Start.func2 at line 799
pkg/jobs/adopt.go in pkg/jobs.(*Registry).runJob at line 410
pkg/jobs/registry.go in pkg/jobs.(*Registry).stepThroughStateMachine at line 1721
pkg/jobs/registry.go in pkg/jobs.(*Registry).stepThroughStateMachine.func2 at line 1720
pkg/backup/backup_job.go in pkg/backup.(*backupResumer).Resume at line 594
pkg/backup/backup_job.go in pkg/backup.maybeWriteBackupLock at line 2050
pkg/backup/backupinfo/manifest_handling.go in pkg/backup/backupinfo.CheckForPreviousBackup at line 1310
src/runtime/asm_arm64.s in runtime.goexit at line 1223
pkg/util/stop/stopper.go in pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2 at line 500
pkg/jobs/jobs.go in pkg/jobs.(*StartableJob).Start.func2 at line 799
pkg/jobs/adopt.go in pkg/jobs.(*Registry).runJob at line 410
pkg/jobs/registry.go in pkg/jobs.(*Registry).stepThroughStateMachine at line 1721
pkg/jobs/registry.go in pkg/jobs.(*Registry).stepThroughStateMachine.func2 at line 1720
pkg/backup/backup_job.go in pkg/backup.(*backupResumer).Resume at line 594
pkg/backup/backup_job.go in pkg/backup.maybeWriteBackupLock at line 2050
pkg/backup/backupinfo/manifest_handling.go in pkg/backup/backupinfo.CheckForPreviousBackup at line 1292
pkg/cloud/impl_registry.go in pkg/cloud.(*esWrapper).List at line 420
pkg/cloud/userfile/file_table_storage.go in pkg/cloud/userfile.(*fileTableStorage).List at line 251
src/runtime/asm_arm64.s in runtime.goexit at line 1223
pkg/util/stop/stopper.go in pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2 at line 500
pkg/jobs/jobs.go in pkg/jobs.(*StartableJob).Start.func2 at line 799
pkg/jobs/adopt.go in pkg/jobs.(*Registry).runJob at line 410
pkg/jobs/registry.go in pkg/jobs.(*Registry).stepThroughStateMachine at line 1721
pkg/jobs/registry.go in pkg/jobs.(*Registry).stepThroughStateMachine.func2 at line 1720
pkg/backup/backup_job.go in pkg/backup.(*backupResumer).Resume at line 594
pkg/backup/backup_job.go in pkg/backup.maybeWriteBackupLock at line 2050
pkg/backup/backupinfo/manifest_handling.go in pkg/backup/backupinfo.CheckForPreviousBackup at line 1292
pkg/cloud/impl_registry.go in pkg/cloud.(*esWrapper).List at line 420
pkg/cloud/userfile/file_table_storage.go in pkg/cloud/userfile.(*fileTableStorage).List at line 249
pkg/cloud/userfile/filetable/file_table_read_writer.go in pkg/cloud/userfile/filetable.(*FileToTableSystem).ListFiles at line 338
pkg/sql/internal.go in pkg/sql.(*rowsIterator).Next at line 584
pkg/sql/internal.go in pkg/sql.(*rowsIterator).Close at line 608
pkg/sql/internal.go in pkg/sql.(*InternalExecutor).execInternal.func1.1 at line 1260
src/runtime/asm_arm64.s in runtime.goexit at line 1223
pkg/util/stop/stopper.go in pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2 at line 500
pkg/sql/internal.go in pkg/sql.(*InternalExecutor).runWithEx.func2 at line 253
pkg/sql/conn_executor.go in pkg/sql.(*connExecutor).run at line 2366
pkg/sql/conn_executor.go in pkg/sql.(*connExecutor).execCmd at line 2558
pkg/sql/conn_executor.go in pkg/sql.(*connExecutor).execCmd.func2 at line 2556
pkg/sql/conn_executor_exec.go in pkg/sql.(*connExecutor).execPortal at line 320
pkg/sql/conn_executor_exec.go in pkg/sql.(*connExecutor).execStmt at line 169
pkg/sql/conn_executor_exec.go in pkg/sql.(*connExecutor).execWithProfiling at line 4476
pkg/sql/conn_executor_exec.go in pkg/sql.(*connExecutor).execStmt.func2 at line 170
pkg/sql/conn_executor_exec.go in pkg/sql.(*connExecutor).execStmtInOpenState at line 1085
pkg/sql/conn_executor_exec.go in pkg/sql.(*connExecutor).dispatchToExecutionEngine at line 2867
pkg/sql/conn_executor_exec.go in pkg/sql.(*connExecutor).makeExecPlan at line 3320
pkg/sql/plan_opt.go in pkg/sql.(*planner).makeOptimizerPlan at line 267
pkg/sql/plan_opt.go in pkg/sql.(*planner).runExecBuild at line 344
pkg/sql/plan_opt.go in pkg/sql.(*optPlanningCtx).runExecBuilder at line 935
pkg/sql/opt/exec/execbuilder/builder.go in pkg/sql/opt/exec/execbuilder.(*Builder).Build at line 298
pkg/sql/opt/exec/execbuilder/builder.go in pkg/sql/opt/exec/execbuilder.(*Builder).build at line 380
pkg/sql/opt/exec/execbuilder/relational.go in pkg/sql/opt/exec/execbuilder.(*Builder).buildRelational at line 383
pkg/sql/opt/exec/execbuilder/relational.go in pkg/sql/opt/exec/execbuilder.(*Builder).maybeAnnotateWithEstimates at line 418
pkg/sql/opt_catalog.go in pkg/sql.(*optTable).Statistic at line 1441
GOROOT/src/runtime/panic.go in runtime.goPanicIndex at line 115
GOROOT/src/runtime/panic.go in runtime.gopanic at line 791
pkg/sql/opt/exec/execbuilder/builder.go in pkg/sql/opt/exec/execbuilder.(*Builder).build.func1 at line 356
pkg/util/errorutil/catch.go in pkg/util/errorutil.ShouldCatch at line 24

Tags

Tag Value
Command start-single-node
Environment v25.2.1
Go Version go1.23.7 X:nocoverageredesign
Platform linux arm64
Distribution CCL
Cockroach Release v25.2.1
Cockroach SHA f9f183d
# of CPUs 6
# of Goroutines 422

Jira issue: CRDB-51539

Metadata

Metadata

Assignees

No one assigned

    Labels

    C-bugCode not up to spec/doc, specs & docs deemed correct. Solution expected to change code/behavior.O-sentryOriginated from an in-the-wild panic report.T-sql-queriesSQL Queries TeamX-blathers-triagedblathers was able to find an ownerbranch-release-25.2

    Type

    No type

    Projects

    SQL Queries

    Status

    Done

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions
      0