mempool: Add push-pull gossip protocol (CAT) #1472
Conversation
|
These are the results of some preliminary experiments run on a laptop using the e2e framework. For more thorough results we would still need to run these experiments with around 200 nodes in the cloud, as we do for the QA tests, probably also with a different network topology. Here, each experiment instance has:
Instances are defined by the permutation of transaction load rate (
For each We can see in the above graphs that:
|
|
These are the same experiments as above but with the following topology, which is more realistic, where node 1 receives all the load: These graphs look similar to the graphs of the above experiment. The main difference is that now the bandwidth of Flood is 2 times higher than that of CAT, instead of 3 to 6 times higher. This was expected, as now the push-pull protocol needs to transmit the transaction in various steps from node 1 to node 8. |
|
This pull request has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions. |
|
but I must kindly ask that we backport this. It is badly needed. |
Agree. Thanks Jacob! Is there a specific app-chain team that is waiting on it? If so, my bad, I was not aware! Even more importantly, are Celestia mainnet nodes employing the CAT mempool? Our last chat with their team (Oct/Nov) we agreed we'd push CAT over the finish line if they give us the green light they will use it in their mainnet, and the Comet+Celestia teams will do complementary testing of CAT. But I have not re-checked since then so my info is very stale. We should get an update. cc @cmwaters |
|
Then deprecate 1971 in favor of present PR? Not sure 1971 still has anything worth keeping, let me know if so. cc @faddat BTW thanks for bringing the present PR up-to-date with |
|
Big thanks from me too. 😍 I'll gladly close the other. |
|
For the moment deprioritized in favor of #3297 |
Closes: #2027.
Relates to #1058.
The CAT (for Content-Addressable Transaction) pool is a gossip protocol for the mempool originally implemented by Celestia. CAT is a push-pull protocol in contrast to CometBFT's default push protocol.
The code in this PR was ported from Celestia's feature/cat branch. The original protocol is built on top of the priority mempool implementation (aka
v1), which existed in CometBFT until v0.37. The current code was ported on top of CometBFT's default mempool implementation (CListMempool), so we had to make to some changes to adapt it to the different underlying implementation.PR checklist
.changelog(we use unclog to manage our changelog)docs/orspec/) and code comments