8000 Feat(syscall): add altbn128 g1 & g2 compression by ananas-block · Pull Request #32870 · solana-labs/solana · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content
This repository was archived by the owner on Jan 22, 2025. It is now read-only.

Feat(syscall): add altbn128 g1 & g2 compression #32870

Merged

Conversation

ananas-block
Copy link
Contributor
@ananas-block ananas-block commented Aug 17, 2023

Problem

  • Groth16 Proofs take up 256 bytes of instruction data per proof (2 g1 64 bytes each and one g2 element of 128 bytes).
  • For PSPs we currently need to verify two Groth16 proofs.
  • With g1 and g2 compression we can half the bytes per proof to 128.
  • Elliptic curve points consist of X and Y coordinates for every X there are two possible Y coordinates, one positive and one negative. Thus, you can send the X coordinate and recover the corresponding Y coordinates. You can use a bitmask on the X coordinate to convey whether to pick the positive or the negative Y coordinate.
  • ark-works 0.4. features functions to compress and decompress g1 and g2 elements as described above

Summary of Changes

Implementation Notes

  • This code is repetitive to enable different sized output fixed size arrays. I am using fixed sized arrays to avoid vectors since we have been repeatedly running low on heap memory in our programs.

Performance

CU calculations are based on 1 cu per 33ns compute time.

g1 compress: 1us * 1000 / 33 = 30 CU
g1 decompress: 13.154 us * 1000 / 33 = 398 CU
g2 compress: 2.8543 us * 1000 / 33 = 86 CU
g2 decompress: 449.13 us * 1000 / 33 = 13610 CU

See this branch for the bench implementation.

g1_decompress time: [13.089 us 13.154 us 13.237 us]
change: [-10.850% -10.279% -9.6768%] (p = 0.00 < 0.05)
Performance has improved.
Found 11 outliers among 100 measurements (11.00%)
1 (1.00%) low mild
3 (3.00%) high mild
7 (7.00%) high severe

g1_compress time: [999.77 ns 1.0049 us 1.0110 us]
change: [-3.6976% -3.0299% -2.3047%] (p = 0.00 < 0.05)
Performance has improved.
Found 12 outliers among 100 measurements (12.00%)
1 (1.00%) low mild
2 (2.00%) high mild
9 (9.00%) high severe

g2_decompress time: [447.25 us 449.13 us 451.51 us]
change: [-14.401% -13.723% -13.047%] (p = 0.00 < 0.05)
Performance has improved.
Found 15 outliers among 100 measurements (15.00%)
8 (8.00%) high mild
7 (7.00%) high severe

g2_compress time: [2.8443 us 2.8543 us 2.8695 us]
change: [-6.2482% -5.4817% -4.7391%] (p = 0.00 < 0.05)
Performance has improved.
Found 15 outliers among 100 measurements (15.00%)
5 (5.00%) high mild
10 (10.00%) high severe

@mergify mergify bot added community Community contribution need:merge-assist labels Aug 17, 2023
@mergify mergify bot requested a review from a team August 17, 2023 12:47
@ananas-block ananas-block force-pushed the feat-altbn128-add-compression branch from 02912d7 to c496f7a Compare August 17, 2023 12:56
@samkim-crypto samkim-crypto added the CI Pull Request is ready to enter CI label Aug 18, 2023
@solana-grimes solana-grimes removed the CI Pull Request is ready to enter CI label Aug 18, 2023
@samkim-crypto samkim-crypto self-requested a review August 18, 2023 08:01
@samkim-crypto samkim-crypto added the CI Pull Request is ready to enter CI label Aug 18, 2023
@solana-grimes solana-grimes removed the CI Pull Request is ready to enter CI label Aug 18, 2023
@codecov
Copy link
codecov bot commented Aug 18, 2023

Codecov Report

Merging #32870 (37d4cdd) into master (1b15464) will decrease coverage by 0.1%.
The diff coverage is 60.8%.

@@            Coverage Diff            @@
##           master   #32870     +/-   ##
=========================================
- Coverage    81.9%    81.9%   -0.1%     
=========================================
  Files         797      798      +1     
  Lines      216347   216572    +225     
=========================================
+ Hits       177369   177499    +130     
- Misses      38978    39073     +95     

@ananas-block ananas-block force-pushed the feat-altbn128-add-compression branch from 6670a6e to bf4fc37 Compare August 24, 2023 09:23
@vadorovsky vadorovsky force-pushed the feat-altbn128-add-compression branch 2 times, most recently from 58dec71 to 2d73be6 Compare September 4, 2023 17:21
@github-actions github-actions bot added the stale [bot only] Added to stale content; results in auto-close after a week. label Sep 19, 2023
@samkim-crypto samkim-crypto removed the stale [bot only] Added to stale content; results in auto-close after a week. label Sep 20, 2023
Copy link
Contributor
@samkim-crypto samkim-crypto left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Everything is all straightforward and looks fine to me. Some small nits below. I'd also prefer if alt_bn128 and alt_bn128_compression modules in the sdk are refactored into their own separate module.
alt_bn128.rs --> alt_bn128/mod.rs
alt_bn128.rs --> alt_bn128/compression.rs

I haven't tested the CU units yet, but I will do so tomorrow and approve it if there are no issues found. Please rebase when you get a chance. Thanks!

type G1 = ark_bn254::g1::G1Affine;
type G2 = ark_bn254::g2::G2Affine;

pub fn alt_bn128_compression_g1_decompress(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: Let's keep this simple and use alt_bn128_g1_decompress. I think the compression part will be evident from the module this function is imported from. Same with all the functions below.

#[error("Unexpected error")]
UnexpectedError,
#[error("Failed to decompress g1")]
DecompressingG1Failed,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: This is really minor, but can we name it G1DecompressionFailed instead to make it slightly more natural? Same with the error variants below.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

agreed, done

@samkim-crypto samkim-crypto added the CI Pull Request is ready to enter CI label Sep 20, 2023
@solana-grimes solana-grimes removed the CI Pull Request is ready to enter CI label Sep 20, 2023
@ananas-block ananas-block force-pushed the feat-altbn128-add-compression branch from 2d73be6 to 790e136 Compare September 22, 2023 01:35
@ananas-block
Copy link
Contributor Author

Thanks for the review!
I have adapted the following as requested:

  • shortened variable and function names by removing compression
  • moved alt_bn128 and alt_bn128_compression into a directory, alt_bn128 with the files mod.rs and compression.rs
  • adapted error naming

@samkim-crypto samkim-crypto added the CI Pull Request is ready to enter CI label Sep 23, 2023
@solana-grimes solana-grimes removed the CI Pull Request is ready to enter CI label Sep 23, 2023
@samkim-crypto
Copy link
Contributor
samkim-crypto commented Sep 23, 2023

Looks much cleaner now. Thanks! Unfortunately, there seems to be conflict in feature_set due to my late approval (sorry! :pray). I will approve as soon as you rebase one more time and the CI is green.

@SwenSchaeferjohann SwenSchaeferjohann force-pushed the feat-altbn128-add-compression branch 4 times, most recently from 35b1409 to 790e136 Compare September 23, 2023 15:02
@samkim-crypto samkim-crypto added the CI Pull Request is ready to enter CI label Sep 23, 2023
@solana-grimes solana-grimes removed the CI Pull Request is ready to enter CI label Sep 23, 2023
@samkim-crypto
Copy link
Contributor
samkim-crypto commented Sep 23, 2023

Hey! It seems like you are removing the feature Ffswd3egL3tccB6Rv3XY6oqfdzn913vUcjCSnpvCKpfx every time you resolve conflict during the rebase. See here. This creates a conflict with master, but this should be easily addressed.

still fixing tests for point of infinity

feat: proof compression syscall working

add rust test to ci

remove prints

added c test

added sycall pricing
@ananas-block ananas-block force-pushed the feat-altbn128-add-compression branch from 790e136 to 37d4cdd Compare September 23, 2023 22:01
@ananas-block
Copy link
Contributor Author
ananas-block commented Sep 23, 2023

Hey, @samkim-crypto it's rebased.
Thanks for reviewing! :)

@samkim-crypto samkim-crypto added the CI Pull Request is ready to enter CI label Sep 23, 2023
@solana-grimes solana-grimes removed the CI Pull Request is ready to enter CI label Sep 23, 2023
@samkim-crypto
Copy link
Contributor

Thanks for all the changes! I want to avoid merging during the weekend if possible when other engineers are resting. I will merge the PR early on Monday. Thank you so much for the patience!

@samkim-crypto samkim-crypto merged commit 997aa0a into solana-labs:master Sep 25, 2023
jeffwashington pushed a commit to jeffwashington/solana that referenced this pull request Sep 25, 2023
* solana-program - altbn128: add g1 & g2 compression

still fixing tests for point of infinity

feat: proof compression syscall working

add rust test to ci

remove prints

added c test

added sycall pricing

* fixed ci checks

* refactored altbn128 and compression
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
community Community contribution need:merge-assist
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants
0