cometbft · mzabaluev · Nov 15, 2023 · Nov 14, 2023 · Nov 14, 2023
diff --git a/.changelog/unreleased/breaking-changes/1412-rpc-hardcode-websocket.md b/.changelog/unreleased/breaking-changes/1412-rpc-hardcode-websocket.md
@@ -0,0 +1,3 @@
+- `[rpc/client]` Hard-code the `/websocket` endpoint path such that it is
+  no longer configurable, removing the related client constructor parameter
+  ([\#1412](https://github.com/cometbft/cometbft/pull/1412))
diff --git a/.changelog/unreleased/breaking-changes/1556-return-meta-with-load-block.md b/.changelog/unreleased/breaking-changes/1556-return-meta-with-load-block.md
@@ -0,0 +1,2 @@
+- `[store]` Make the `LoadBlock` method also return block metadata
+  ([\#1556](https://github.com/cometbft/cometbft/issues/1556))
diff --git a/.changelog/unreleased/bug-fixes/1529-indexer-respect-height-params-on-query.md b/.changelog/unreleased/bug-fixes/1529-indexer-respect-height-params-on-query.md
@@ -0,0 +1,2 @@
+- `[state/indexer]` Respect both height params while querying for events
+   ([\#1529](https://github.com/cometbft/cometbft/pull/1529))
diff --git a/.changelog/unreleased/features/1512-metric-mempool-size-bytes.md b/.changelog/unreleased/features/1512-metric-mempool-size-bytes.md
@@ -0,0 +1,2 @@
+- `[metrics]` Add metric for mempool size in bytes `SizeBytes`.
+  ([\#1512](https://github.com/cometbft/cometbft/pull/1512))
diff --git a/...leased/enhancements/1096-config-pruner.md → ...leased/improvements/1096-config-pruner.md b/...leased/enhancements/1096-config-pruner.md → ...leased/improvements/1096-config-pruner.md
diff --git a/...released/enhancements/1096-node-pruner.md → ...released/improvements/1096-node-pruner.md b/...released/enhancements/1096-node-pruner.md → ...released/improvements/1096-node-pruner.md
diff --git a/...nhancements/1096-state-abcires-pruning.md → ...mprovements/1096-state-abcires-pruning.md b/...nhancements/1096-state-abcires-pruning.md → ...mprovements/1096-state-abcires-pruning.md
diff --git a/...ncements/1096-state-background-pruning.md → ...ovements/1096-state-background-pruning.md b/...ncements/1096-state-background-pruning.md → ...ovements/1096-state-background-pruning.md
diff --git a/.changelog/unreleased/improvements/1412-rpc-versioning.md b/.changelog/unreleased/improvements/1412-rpc-versioning.md
@@ -0,0 +1,3 @@
+- `[rpc]` The RPC API is now versioned, with all existing endpoints accessible
+  via `/v1/*` as well as `/*`
+  ([\#1412](https://github.com/cometbft/cometbft/pull/1412))
diff --git a/.changelog/unreleased/improvements/1518-reduce-default-maxbytes_increase_maxgas b/.changelog/unreleased/improvements/1518-reduce-default-maxbytes_increase_maxgas
@@ -0,0 +1,2 @@
+- `[consensus]` Reduce the default MaxBytes to 4MB and increase MaxGas to 10 million
+  ([\#1518](https://github.com/cometbft/cometbft/pull/1518))
diff --git a/.changelog/unreleased/improvements/1558-experimental-gossip-limiting.md b/.changelog/unreleased/improvements/1558-experimental-gossip-limiting.md
@@ -0,0 +1,9 @@
+- `[mempool]` Add experimental feature to limit the number of persistent peers and non-persistent
+  peers to which the node gossip transactions.
+  ([\#1558](https://github.com/cometbft/cometbft/pull/1558))
+  ([\#1584](https://github.com/cometbft/cometbft/pull/1584))
+- `[config]` Add mempool parameters `experimental_max_gossip_connections_to_persistent_peers` and
+  `experimental_max_gossip_connections_to_non_persistent_peers` for limiting the number of peers to
+  which the node gossip transactions. 
+  ([\#1558](https://github.com/cometbft/cometbft/pull/1558))
+  ([\#1584](https://github.com/cometbft/cometbft/pull/1584))
diff --git a/.changelog/unreleased/improvements/1579-e2e-disable-pex-reactor.md b/.changelog/unreleased/improvements/1579-e2e-disable-pex-reactor.md
@@ -0,0 +1,2 @@
+- `[e2e]` Allow disabling the PEX reactor on all nodes in the testnet
+  ([\#1579](https://github.com/cometbft/cometbft/pull/1579))
diff --git a/.github/workflows/proto-lint.yml b/.github/workflows/proto-lint.yml
@@ -16,7 +16,7 @@ jobs:
     timeout-minutes: 5
     steps:
       - uses: actions/checkout@v4
-      - uses: bufbuild/buf-setup-action@v1.27.1
+      - uses: bufbuild/buf-setup-action@v1.28.0
       - uses: bufbuild/buf-lint-action@v1
         with:
           input: 'proto'
diff --git a/UPGRADING.md b/UPGRADING.md
@@ -13,18 +13,20 @@ The minimum Go version has been bumped to [v1.21][go121].
 * The `Mempool` interface was modified on the following methods. Note that this
   interface is meant for internal use only, so you should be aware of these
   changes only if you happen to call these methods directly.
-  - `CheckTx`'s signature changed from
+    * `CheckTx`'s signature changed from
     `CheckTx(tx types.Tx, cb func(*abci.ResponseCheckTx), txInfo TxInfo) error`
     to `CheckTx(tx types.Tx) (abcicli.ReqRes, error)`.
-    - The method used to take a callback function `cb` to be applied to the ABCI
+        * The method used to take a callback function `cb` to be applied to the ABCI
     `CheckTx` response. Now `CheckTx` returns the ABCI response of type
     `abcicli.ReqRes`, on which the callback must be applied manually. For
     example:
+
       ```golang
       reqRes, err := CheckTx(tx)
       cb(reqRes.Response.GetCheckTx())
       ```
-    - The second parameter was `txInfo`, which essentially contained information
+
+        * The second parameter was `txInfo`, which essentially contained information
     about the sender of the transaction. Now that information is stored in the
     mempool reactor instead of the data structure, so it is no longer needed in
     this method.
@@ -41,6 +43,24 @@ The minimum Go version has been bumped to [v1.21][go121].
 * Removed the `replay` and `replay-console` subcommands
   ([\#1170](https://github.com/cometbft/cometbft/pull/1170))
 
+### RPC API
+
+* The RPC API is now versioned.
+  Although invoking methods without specifying the version is still supported for now,
+  support will be dropped in future releases and users are urged to use the versioned
+  approach.
+  For example, instead of `curl localhost:26657/block?height=5`, use
+  `curl localhost:26657/v1/block?height=5`.
+
+* The `/websocket` endpoint path is no longer configurable in the client or
+  server. Creating an RPC client now takes the form:
+
+  ```golang
+  // The WebSocket endpoint in the following example is assumed to be available
+  // at http://localhost:26657/v1/websocket
+  rpcClient, err := client.New("http://localhost:26657/v1")
+  ```
+
 ## v0.38.0
 
 This release introduces state machine-breaking changes, as well as substantial changes

diff --git a/blocksync/reactor.go b/blocksync/reactor.go
@@ -190,7 +190,7 @@ func (bcR *Reactor) RemovePeer(peer p2p.Peer, _ interface{}) {
 // respondToPeer loads a block and sends it to the requesting peer,
 // if we have it. Otherwise, we'll respond saying we don't have it.
 func (bcR *Reactor) respondToPeer(msg *bcproto.BlockRequest, src p2p.Peer) (queued bool) {
-	block := bcR.store.LoadBlock(msg.Height)
+	block, _ := bcR.store.LoadBlock(msg.Height)
 	if block == nil {
 		bcR.Logger.Info("Peer asking for a block we don't have", "src", src, "height", msg.Height)
 		return src.TrySend(p2p.Envelope{

diff --git a/blocksync/reactor_test.go b/blocksync/reactor_test.go
@@ -214,7 +214,7 @@ func TestNoBlockResponse(t *testing.T) {
 	assert.Equal(t, maxBlockHeight, reactorPairs[0].reactor.store.Height())
 
 	for _, tt := range tests {
-		block := reactorPairs[1].reactor.store.LoadBlock(tt.height)
+		block, _ := reactorPairs[1].reactor.store.LoadBlock(tt.height)
 		if tt.existent {
 			assert.True(t, block != nil)
 		} else {

diff --git a/cmd/cometbft/commands/debug/debug.go b/cmd/cometbft/commands/debug/debug.go
@@ -32,8 +32,8 @@ func init() {
 	DebugCmd.PersistentFlags().StringVar(
 		&nodeRPCAddr,
 		flagNodeRPCAddr,
-		"tcp://localhost:26657",
-		"the CometBFT node's RPC address (<host>:<port>)",
+		"tcp://localhost:26657/v1",
+		"the CometBFT node's RPC address (<host>:<port>) and version",
 	)
 
 	DebugCmd.AddCommand(killCmd)

diff --git a/cmd/cometbft/commands/debug/dump.go b/cmd/cometbft/commands/debug/dump.go
@@ -58,7 +58,7 @@ func dumpCmdHandler(_ *cobra.Command, args []string) error {
 		}
 	}
 
-	rpc, err := rpchttp.New(nodeRPCAddr, "/websocket")
+	rpc, err := rpchttp.New(nodeRPCAddr)
 	if err != nil {
 		return fmt.Errorf("failed to create new http client: %w", err)
 	}

diff --git a/cmd/cometbft/commands/debug/kill.go b/cmd/cometbft/commands/debug/kill.go
@@ -43,7 +43,7 @@ func killCmdHandler(_ *cobra.Command, args []string) error {
 		return errors.New("invalid output file")
 	}
 
-	rpc, err := rpchttp.New(nodeRPCAddr, "/websocket")
+	rpc, err := rpchttp.New(nodeRPCAddr)
 	if err != nil {
 		return fmt.Errorf("failed to create new http client: %w", err)
 	}

diff --git a/cmd/cometbft/commands/reindex_event.go b/cmd/cometbft/commands/reindex_event.go
@@ -150,7 +150,7 @@ func eventReIndex(cmd *cobra.Command, args eventReIndexArgs) error {
 		case <-cmd.Context().Done():
 			return fmt.Errorf("event re-index terminated at height %d: %w", height, cmd.Context().Err())
 		default:
-			block := args.blockStore.LoadBlock(height)
+			block, _ := args.blockStore.LoadBlock(height)
 			if block == nil {
 				return fmt.Errorf("not able to load block at height %d from the blockstore", height)
 			}

diff --git a/cmd/cometbft/commands/reindex_event_test.go b/cmd/cometbft/commands/reindex_event_test.go
@@ -135,9 +135,9 @@ func TestReIndexEvent(t *testing.T) {
 	mockBlockStore.
 		On("Base").Return(base).
 		On("Height").Return(height).
-		On("LoadBlock", base).Return(nil).Once().
-		On("LoadBlock", base).Return(&types.Block{Data: types.Data{Txs: types.Txs{make(types.Tx, 1)}}}).
-		On("LoadBlock", height).Return(&types.Block{Data: types.Data{Txs: types.Txs{make(types.Tx, 1)}}})
+		On("LoadBlock", base).Return(nil, nil).Once().
+		On("LoadBlock", base).Return(&types.Block{Data: types.Data{Txs: types.Txs{make(types.Tx, 1)}}}, &types.BlockMeta{}).
+		On("LoadBlock", height).Return(&types.Block{Data: types.Data{Txs: types.Txs{make(types.Tx, 1)}}}, &types.BlockMeta{})
 
 	abciResp := &abcitypes.FinalizeBlockResponse{
 		TxResults: []*abcitypes.ExecTxResult{

diff --git a/config/config.go b/config/config.go
@@ -876,6 +876,19 @@ type MempoolConfig struct {
 	// Including space needed by encoding (one varint per transaction).
 	// XXX: Unused due to https://github.com/tendermint/tendermint/issues/5796
 	MaxBatchBytes int `mapstructure:"max_batch_bytes"`
+	// Experimental parameters to limit gossiping txs to up to the specified number of peers.
+	// We use two independent upper values for persistent peers and for non-persistent peers.
+	// Unconditional peers are not affected by this feature.
+	// If we are connected to more than the specified number of persistent peers, only send txs to
+	// the first ExperimentalMaxGossipConnectionsToPersistentPeers of them. If one of those
+	// persistent peers disconnects, activate another persistent peer. Similarly for non-persistent
+	// peers, with an upper limit of ExperimentalMaxGossipConnectionsToNonPersistentPeers.
+	// If set to 0, the feature is disabled for the corresponding group of peers, that is, the
+	// number of active connections to that group of peers is not bounded.
+	// For non-persistent peers, if enabled, a value of 10 is recommended based on experimental
+	// performance results using the default P2P configuration.
+	ExperimentalMaxGossipConnectionsToPersistentPeers    int `mapstructure:"experimental_max_gossip_connections_to_persistent_peers"`
+	ExperimentalMaxGossipConnectionsToNonPersistentPeers int `mapstructure:"experimental_max_gossip_connections_to_non_persistent_peers"`
 }
 
 // DefaultMempoolConfig returns a default configuration for the CometBFT mempool
@@ -890,6 +903,8 @@ func DefaultMempoolConfig() *MempoolConfig {
 		MaxTxsBytes: 1024 * 1024 * 1024, // 1GB
 		CacheSize:   10000,
 		MaxTxBytes:  1024 * 1024, // 1MB
+		ExperimentalMaxGossipConnectionsToNonPersistentPeers: 0,
+		ExperimentalMaxGossipConnectionsToPersistentPeers:    0,
 	}
 }
 
@@ -925,6 +940,12 @@ func (cfg *MempoolConfig) ValidateBasic() error {
 	if cfg.MaxTxBytes < 0 {
 		return cmterrors.ErrNegativeField{Field: "max_tx_bytes"}
 	}
+	if cfg.ExperimentalMaxGossipConnectionsToPersistentPeers < 0 {
+		return cmterrors.ErrNegativeField{Field: "experimental_max_gossip_connections_to_persistent_peers"}
+	}
+	if cfg.ExperimentalMaxGossipConnectionsToNonPersistentPeers < 0 {
+		return cmterrors.ErrNegativeField{Field: "experimental_max_gossip_connections_to_non_persistent_peers"}
+	}
 	return nil
 }
 

diff --git a/config/toml.go b/config/toml.go
@@ -425,6 +425,20 @@ max_tx_bytes = {{ .Mempool.MaxTxBytes }}
 # XXX: Unused due to https://github.com/tendermint/tendermint/issues/5796
 max_batch_bytes = {{ .Mempool.MaxBatchBytes }}
 
+# Experimental parameters to limit gossiping txs to up to the specified number of peers.
+# We use two independent upper values for persistent peers and for non-persistent peers.
+# Unconditional peers are not affected by this feature.
+# If we are connected to more than the specified number of persistent peers, only send txs to
+# the first experimental_max_gossip_connections_to_persistent_peers of them. If one of those
+# persistent peers disconnects, activate another persistent peer. Similarly for non-persistent
+# peers, with an upper limit of experimental_max_gossip_connections_to_non_persistent_peers.
+# If set to 0, the feature is disabled for the corresponding group of peers, that is, the
+# number of active connections to that group of peers is not bounded.
+# For non-persistent peers, if enabled, a value of 10 is recommended based on experimental
+# performance results using the default P2P configuration.
+experimental_max_gossip_connections_to_persistent_peers = {{ .Mempool.ExperimentalMaxGossipConnectionsToPersistentPeers }}
+experimental_max_gossip_connections_to_non_persistent_peers = {{ .Mempool.ExperimentalMaxGossipConnectionsToNonPersistentPeers }}
+
 #######################################################
 ###         State Sync Configuration Options        ###
 #######################################################

diff --git a/consensus/reactor_test.go b/consensus/reactor_test.go
@@ -431,8 +431,8 @@ func TestReactorRecordsVotesAndBlockParts(t *testing.T) {
 	// Get peer state
 	ps := peer.Get(types.PeerStateKey).(*PeerState)
 
-	assert.Equal(t, true, ps.VotesSent() > 0, "number of votes sent should have increased")
-	assert.Equal(t, true, ps.BlockPartsSent() > 0, "number of votes sent should have increased")
+	assert.Greater(t, ps.VotesSent(), 0, "number of votes sent should have increased")
+	assert.Greater(t, ps.BlockPartsSent(), 0, "number of votes sent should have increased")
 }
 
 //-------------------------------------------------------------

diff --git a/consensus/replay.go b/consensus/replay.go
@@ -474,7 +474,7 @@ func (h *Handshaker) replayBlocks(
 		}
 
 		h.logger.Info("Applying block", "height", i)
-		block := h.store.LoadBlock(i)
+		block, _ := h.store.LoadBlock(i)
 		// Extra check to ensure the app was not changed in a way it shouldn't have.
 		if len(appHash) > 0 {
 			assertAppHashEqualsOneFromBlock(appHash, block)
@@ -503,8 +503,7 @@ func (h *Handshaker) replayBlocks(
 
 // ApplyBlock on the proxyApp with the last block.
 func (h *Handshaker) replayBlock(state sm.State, height int64, proxyApp proxy.AppConnConsensus) (sm.State, error) {
-	block := h.store.LoadBlock(height)
-	meta := h.store.LoadBlockMeta(height)
+	block, meta := h.store.LoadBlock(height)
 
 	// Use stubs for both mempool and evidence pool since no transactions nor
 	// evidence are needed here - block already exists.

diff --git a/consensus/replay_test.go b/consensus/replay_test.go
@@ -554,7 +554,8 @@ func setupChainWithChangingValidators(t *testing.T, name string, nBlocks int) (*
 	chain := []*types.Block{}
 	extCommits := []*types.ExtendedCommit{}
 	for i := 1; i <= nBlocks; i++ {
-		chain = append(chain, css[0].blockStore.LoadBlock(int64(i)))
+		block, _ := css[0].blockStore.LoadBlock(int64(i))
+		chain = append(chain, block)
 		extCommits = append(extCommits, css[0].blockStore.LoadBlockExtendedCommit(int64(i)))
 	}
 	return config, chain, extCommits, genesisState
@@ -1183,13 +1184,17 @@ func newMockBlockStore(t *testing.T, config *cfg.Config, params types.ConsensusP
 	}
 }
 
-func (bs *mockBlockStore) Height() int64                       { return int64(len(bs.chain)) }
-func (bs *mockBlockStore) Base() int64                         { return bs.base }
-func (bs *mockBlockStore) Size() int64                         { return bs.Height() - bs.Base() + 1 }
-func (bs *mockBlockStore) LoadBaseMeta() *types.BlockMeta      { return bs.LoadBlockMeta(bs.base) }
-func (bs *mockBlockStore) LoadBlock(height int64) *types.Block { return bs.chain[height-1] }
-func (bs *mockBlockStore) LoadBlockByHash([]byte) *types.Block {
-	return bs.chain[int64(len(bs.chain))-1]
+func (bs *mockBlockStore) Height() int64                  { return int64(len(bs.chain)) }
+func (bs *mockBlockStore) Base() int64                    { return bs.base }
+func (bs *mockBlockStore) Size() int64                    { return bs.Height() - bs.Base() + 1 }
+func (bs *mockBlockStore) LoadBaseMeta() *types.BlockMeta { return bs.LoadBlockMeta(bs.base) }
+func (bs *mockBlockStore) LoadBlock(height int64) (*types.Block, *types.BlockMeta) {
+	return bs.chain[height-1], bs.LoadBlockMeta(height)
+}
+
+func (bs *mockBlockStore) LoadBlockByHash([]byte) (*types.Block, *types.BlockMeta) {
+	height := int64(len(bs.chain))
+	return bs.chain[height-1], bs.LoadBlockMeta(height)
 }
 func (bs *mockBlockStore) LoadBlockMetaByHash([]byte) *types.BlockMeta { return nil }
 func (bs *mockBlockStore) LoadBlockMeta(height int64) *types.BlockMeta {

diff --git a/docs/architecture/README.md b/docs/architecture/README.md
@@ -60,3 +60,4 @@ numbering our ADRs from 100 onwards.
 ### Rejected
 
 - [ADR-100: Data companion push API](./adr-100-data-companion-push-api.md)
+- [ADR-110: Remote mempool](./adr-110-remote-mempool.md)
Original file line number	Diff line number	Diff line change
		@@ -0,0 +1,2 @@
		- `[store]` Make the `LoadBlock` method also return block metadata
		([\#1556](https://github.com/cometbft/cometbft/issues/1556))
Original file line number	Diff line number	Diff line change
		@@ -0,0 +1,2 @@
		- `[state/indexer]` Respect both height params while querying for events
		([\#1529](https://github.com/cometbft/cometbft/pull/1529))
Original file line number	Diff line number	Diff line change
		@@ -0,0 +1,2 @@
		- `[metrics]` Add metric for mempool size in bytes `SizeBytes`.
		([\#1512](https://github.com/cometbft/cometbft/pull/1512))
Original file line number	Diff line number	Diff line change
		@@ -0,0 +1,2 @@
		- `[consensus]` Reduce the default MaxBytes to 4MB and increase MaxGas to 10 million
		([\#1518](https://github.com/cometbft/cometbft/pull/1518))
Original file line number	Diff line number	Diff line change
		@@ -0,0 +1,2 @@
		- `[e2e]` Allow disabling the PEX reactor on all nodes in the testnet
		([\#1579](https://github.com/cometbft/cometbft/pull/1579))
Original file line number	Diff line number	Diff line change
Expand Up		@@ -60,3 +60,4 @@ numbering our ADRs from 100 onwards.
		### Rejected

		- [ADR-100: Data companion push API](./adr-100-data-companion-push-api.md)
		- [ADR-110: Remote mempool](./adr-110-remote-mempool.md)