8000 [consensus] Add `consensus::aggregation` by BrendanChou · Pull Request #974 · commonwarexyz/monorepo · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

[consensus] Add consensus::aggregation #974

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 41 commits into
base: main
Choose a base branch
from
Open

Conversation

BrendanChou
Copy link
Collaborator
@BrendanChou BrendanChou commented May 21, 2025

Fixes #814

TODO:

@BrendanChou BrendanChou force-pushed the bc/814-aggregation branch 3 times, most recently from bcc3600 to 4321ffb Compare May 28, 2025 23:18
@BrendanChou BrendanChou force-pushed the bc/814-aggregation branch 3 times, most recently from 3e34e05 to 0bbdbea Compare June 10, 2025 20:31
use commonware_utils::{max_faults, Array};
use std::collections::{btree_map, BTreeMap, HashMap, HashSet};

/// A data structure that keeps track of the reported tip for each validator.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is an awesome data structure.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wonder if it makes sense to give it a catchier name given so (and publicly export it maybe in utils)?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agree it could be useful in utils but would rather not add stuff to the public API there until it gets more use and we can settle more on solidified behavior/functions (if it's used in at least one more place)

@BrendanChou BrendanChou force-pushed the bc/814-aggregation branch 7 times, most recently from 83bc6d4 to 01039b6 Compare June 18, 2025 15:04
@patrick-ogrady
Copy link
Contributor
patrick-ogrady commented Jun 18, 2025

I think this one will realllllly benefit from #1002 ❤️

(sorry I thought this would go out as a review comment lol)

@@ -0,0 +1,1093 @@
//! Threshold signature aggregation for agreeing on externally-finalized hashes.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: Recover threshold signatures over an externally synchronized stream of digests.

I suppose the "aggregation" here is really what we'd call recovery (and it would be aggregation in a multi-signature case). 🤔

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

...now second guessing the name aggregation lol

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In other consensus crates, we combine types and wire. Curious if you feel it is better to separate (I've taken the position that anything in types may be serialized somewhere and sent over the wire so delineation doesn't add anything).

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it's better to keep them separate. Just easier to see the direct types that you are expected to send over the wire or not

digest: D,
sender: &mut WrappedSender<NetS, PeerAck<V, D>>,
) -> Result<(), Error> {
// Entry must be `Pending::Unverified`, or return early
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we not just store digest in Verified immediately (and skip Unverified in this case)?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Didn't quite understand this comment. I think it's good to check that the existing value is Unverified, otherwise we can completely skip any other work here since it has already been verified before (and is still in verified state, or was completed)

//!
//! # Design Decisions
//!
//! ## Missing Signature Resolution
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could add a recommendation to use resolver for this backfilling (if you are missing something).

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Makes sense but commonware_resolver is not currently imported by the commonware_consensus crate and so there is not a great way to link it in rustdoc. Will leave it out for now

@BrendanChou BrendanChou force-pushed the bc/814-aggregation branch 4 times, most recently from 77f9b4c to 29ba1c5 Compare July 1, 2025 14:56
@BrendanChou BrendanChou force-pushed the bc/814-aggregation branch from ecfa97e to a14e04d Compare July 2, 2025 21:44
@BrendanChou BrendanChou marked this pull request as ready for review July 2, 2025 23:43
Copy link
codecov bot commented Jul 3, 2025

Codecov Report

Attention: Patch coverage is 91.63327% with 167 lines in your changes missing coverage. Please review.

Project coverage is 90.57%. Comparing base (ba2a872) to head (921acd6).
Report is 1 commits behind head on main.

Files with missing lines Patch % Lines
consensus/src/aggregation/engine.rs 84.44% 63 Missing ⚠️
consensus/src/aggregation/mocks/application.rs 31.66% 41 Missing ⚠️
consensus/src/aggregation/mod.rs 96.60% 25 Missing ⚠️
consensus/src/aggregation/mocks/reporter.rs 81.66% 22 Missing ⚠️
consensus/src/aggregation/mocks/monitor.rs 68.96% 9 Missing ⚠️
consensus/src/aggregation/mocks/supervisor.rs 85.41% 7 Missing ⚠️
@@            Coverage Diff             @@
##             main     #974      +/-   ##
==========================================
- Coverage   91.39%   90.57%   -0.83%     
==========================================
  Files         218      228      +10     
  Lines       59455    54654    -4801     
==========================================
- Hits        54338    49502    -4836     
- Misses       5117     5152      +35     
Files with missing lines Coverage Δ
consensus/src/aggregation/metrics.rs 100.00% <100.00%> (ø)
consensus/src/aggregation/safe_tip.rs 100.00% <100.00%> (ø)
consensus/src/aggregation/types.rs 100.00% <100.00%> (ø)
consensus/src/aggregation/wire.rs 100.00% <100.00%> (ø)
consensus/src/ordered_broadcast/engine.rs 88.06% <100.00%> (-1.60%) ⬇️
utils/src/lib.rs 100.00% <100.00%> (ø)
consensus/src/aggregation/mocks/supervisor.rs 85.41% <85.41%> (ø)
consensus/src/aggregation/mocks/monitor.rs 68.96% <68.96%> (ø)
consensus/src/aggregation/mocks/reporter.rs 81.66% <81.66%> (ø)
consensus/src/aggregation/mod.rs 96.60% <96.60%> (ø)
... and 2 more

... and 167 files with indirect coverage changes


Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update ba2a872...921acd6. Read the comment docs.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[consensus] Create consensus::aggregation dialect
3 participants
0