storage: avoid opening store with too new of a cluster version

We persist the cluster version within a flat file in the data directory. Before attempting to open Pebble, we assert that the currently persisted version is sufficiently high (guarding against 'skipping' an unskippable version):

cockroach/pkg/storage/pebble.go

Lines 929 to 954 in 53814aa

    
           if minVerFileExists { 
        
           	// Avoid running a binary too new for this store. This is what you'd catch 
        
           	// if, say, you restarted directly from v21.2 into v22.2 (bumping the min 
        
           	// version) without going through v22.1 first. 
        
           	// 
        
           	// Note that "going through" above means that v22.1 successfully upgrades 
        
           	// all existing stores. If v22.1 crashes half-way through the startup 
        
           	// sequence (so now some stores have v21.2, but others v22.1) you are 
        
           	// expected to run v22.1 again (hopefully without the crash this time) which 
        
           	// would then rewrite all the stores. 
        
           	if v := cfg.settings.Version; storeClusterVersion.Less(v.MinSupportedVersion()) { 
        
           		if storeClusterVersion.Major < clusterversion.DevOffset && v.LatestVersion().Major >= clusterversion.DevOffset { 
        
           			return nil, errors.Errorf( 
        
           				"store last used with cockroach non-development version v%s "+ 
        
           					"cannot be opened by development version v%s", 
        
           				storeClusterVersion, v.LatestVersion(), 
        
           			) 
        
           		} 
        
           		return nil, errors.Errorf( 
        
           			"store last used with cockroach version v%s "+ 
        
           				"is too old for running version v%s (which requires data from v%s or later)", 
        
           			storeClusterVersion, v.LatestVersion(), v.MinSupportedVersion(), 
        
           		) 
        
           	} 
        
           	cfg.opts.ErrorIfNotExists = true 
        
           } else {

We should also guard against rolling back from a newer, finalized cluster version here. I believe we have such protection later after the Engine has started although I haven't gone searching for it yet. But if we make backwards incompatible changes to the encryption-at-rest, validating after the engine has started is too late.

We should move handling of the cluster version file into the storage/fs package during fs.Env initialization to ensure that it fully encompasses encryption-at-rest initialization. And we should assert that the persisted cluster version is neither too high nor too low.

Jira issue: CRDB-51481

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

	if minVerFileExists {
	// Avoid running a binary too new for this store. This is what you'd catch
	// if, say, you restarted directly from v21.2 into v22.2 (bumping the min
	// version) without going through v22.1 first.
	//
	// Note that "going through" above means that v22.1 successfully upgrades
	// all existing stores. If v22.1 crashes half-way through the startup
	// sequence (so now some stores have v21.2, but others v22.1) you are
	// expected to run v22.1 again (hopefully without the crash this time) which
	// would then rewrite all the stores.
	if v := cfg.settings.Version; storeClusterVersion.Less(v.MinSupportedVersion()) {
	if storeClusterVersion.Major < clusterversion.DevOffset && v.LatestVersion().Major >= clusterversion.DevOffset {
	return nil, errors.Errorf(
	"store last used with cockroach non-development version v%s "+
	"cannot be opened by development version v%s",
	storeClusterVersion, v.LatestVersion(),
	)
	}
	return nil, errors.Errorf(
	"store last used with cockroach version v%s "+
	"is too old for running version v%s (which requires data from v%s or later)",
	storeClusterVersion, v.LatestVersion(), v.MinSupportedVersion(),
	)
	}
	cfg.opts.ErrorIfNotExists = true
	} else {

Description

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions