8000 mempool/lanes: Validate MVP implementation on 200-nodes testnet · Issue #4168 · cometbft/cometbft · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

mempool/lanes: Validate MVP implementation on 200-nodes testnet #4168

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
Tracked by #2803
hvanz opened this issue Sep 24, 2024 · 1 comment
Closed
Tracked by #2803

mempool/lanes: Validate MVP implementation on 200-nodes testnet #4168

hvanz opened this issue Sep 24, 2024 · 1 comment
Assignees
@hvanz
Labels
mempool qa Quality assurance
Milestone
2024-Q3

Comments

@hvanz
Copy link
Member
hvanz commented Sep 24, 2024 •
edited
Loading

When the MVP implementation is finished in the feature branch, and before merging it into the main branch, we need to validate in our usual 200-nodes testnet that the performance of the new code is not degraded with respect to the main branch.
@hvanz hvanz added mempool qa Quality assurance labels Sep 24, 2024
@hvanz hvanz added this to the 2024-Q3 milestone Sep 24, 2024
@hvanz hvanz added this to CometBFT Sep 24, 2024
@hvanz hvanz mentioned this issue Sep 24, 2024
Closed
42 tasks
@hvanz
Copy link
Member Author
hvanz commented Sep 24, 2024 •
edited
Loading

Experiments on 200 nodes

The metrics below are a comparison between three CometBFT branches:

  1. v1.0.0-alpha2, commit e42f62b68, date: 2024-03-25.
    • This tag is the last QA'd version, see full report.
    • The metrics shown below are the result of the experiments performed for the QA process of v1.0.0-alpha2.
  2. main branch, commit 1323b1cdd, date: 2024-09-20
  3. feature/mempool-qos branch, commit fa9ae0c44, date: 2024-09-20
    • This branch contains the MVP implementation of Lanes.

We have enabled latency emulation in all cases.

Transaction load

Load: rate: 400 tx/s, tx size: 1kB, connections: 1, duration: 180s

In all experiments we injected a transaction load of 400 tx/s. Until v1.0.0-alpha2, we used this load as the saturation point of the system. In main and feature/mempool-qos, it seems that the saturation point could now be higher, but more tests are needed to corroborate that.

Results

  • main and feature/mempool-qos not only do not degrade the general performance compared against v1.0.0-alpha2 but they also improve on almost all metrics.
  • The metrics of feature/mempool-qos are overall equivalent to those of main. This was expected because in this test scenario system resources never reached their limits, which is when we would expect Lanes to kick in and show better results than main.

Therefore, the implementation in feature/mempool-qos has passed the test.

Metrics

v1.0.0-alpha2 (e42f62b68) main (1323b1cdd) feature/mempool-qos (fa9ae0c44)
number of txs in chain
total_txs total_txs total_txs
rate of tx processing (txs/min)
total_txs_rate total_txs_rate total_txs_rate
rate of block creation (blocks/min)
block_rate block_rate block_rate
block size
block_size_bytes block_size_bytes block_size_bytes
number of blocks
blocks blocks blocks
average mempool size
avg_mempool_size avg_mempool_size avg_mempool_size
maximum mempool size
mempool_size_max mempool_size_max mempool_size_max
number of rounds
rounds rounds rounds
number of peers
peers peers peers
CPU load
cpu cpu cpu
memory usage
memory memory memory

@hvanz hvanz changed the title mempool/lanes: Validate MVP implementation on 200 nodes mempool/lanes: Validate MVP implementation on 200-nodes testnet Sep 24, 2024
@hvanz hvanz self-assigned this Sep 24, 2024
@hvanz hvanz moved this from Todo to Done in CometBFT Sep 24, 2024
@hvanz hvanz closed this as completed Sep 25, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@hvanz hvanz

Labels
mempool qa Quality assurance
Projects
No open projects
CometBFT
Status: Done
Milestone
2024-Q3
Development

No branches or pull requests
1 participant
@hvanz
0