8000 ci-bench: low-noise benchmarks with rustls-fuzzing-provider by ctz · Pull Request #2483 · rustls/rustls · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

ci-bench: low-noise benchmarks with rustls-fuzzing-provider #2483

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 3 commits into from
Jun 12, 2025
Merged

ci-bench: low-noise benchmarks with rustls-fuzzing-provider #2483

merged 3 commits into from
Jun 12, 2025

Conversation

ctz
Copy link
Member
@ctz ctz commented Jun 10, 2025

ci-bench has unfortunately become very noisy due to mlkem rejection sampling (which is variable-time, and happens during key generation and encapsulation -- so affects both the client and server uses).

This PR introduces these new benchmark cases (this is a run-to-run baseline):

Instruction count differences

Scenario Baseline Candidate Diff
transfer_no_resume_1.3_no_crypto_client 117161101 117161110 ⚠️ 9 (0.00%)
transfer_no_resume_1.3_no_crypto_server 104963700 104963695 ✅ -5 (-0.00%)
transfer_no_resume_1.2_no_crypto_server 104948964 104948962 ✅ -2 (-0.00%)
transfer_no_resume_1.2_no_crypto_client 117116968 117116966 ✅ -2 (-0.00%)
handshake_no_resume_1.2_no_crypto_client 95818 95818 0 (0.00%)
handshake_no_resume_1.2_no_crypto_server 82064 82064 0 (0.00%)
handshake_no_resume_1.3_no_crypto_server 99515 99515 0 (0.00%)
handshake_session_id_1.2_no_crypto_client 1066966 1066966 0 (0.00%)
handshake_tickets_1.2_no_crypto_server 1164833 1164833 0 (0.00%)
handshake_session_id_1.3_no_crypto_client 2003919 2003919 0 (0.00%)
handshake_tickets_1.2_no_crypto_client 1151875 1151875 0 (0.00%)
handshake_tickets_1.3_no_crypto_server 1998864 1998864 0 (0.00%)
handshake_tickets_1.3_no_crypto_client 2022130 2022130 0 (0.00%)
handshake_session_id_1.2_no_crypto_server 1010033 1010033 0 (0.00%)
handshake_no_resume_1.3_no_crypto_client 108043 108043 0 (0.00%)
handshake_session_id_1.3_no_crypto_server 2027472 2027472 0 (0.00%)

Obviously these are deeply unrepresentative, and will magnify the effect any minor improvements we make.

@rustls-benchmarking Rustls Benchmarking
Copy link
rustls-benchmarking bot commented Jun 10, 2025 •
edited
Loading

Benchmark results

Instruction counts

⚠️ Missing benchmarks

The following benchmark scenarios are present in the candidate but not in the baseline:

  • handshake_no_resume_1.3_no_crypto_client

  • handshake_tickets_1.3_no_crypto_server

  • handshake_tickets_1.2_no_crypto_client

  • transfer_no_resume_1.3_no_crypto_server

  • handshake_tickets_1.3_no_crypto_client

  • handshake_no_resume_1.2_no_crypto_server

  • handshake_no_resume_1.3_no_crypto_server

  • handshake_session_id_1.3_no_crypto_client

  • handshake_tickets_1.2_no_crypto_server

  • handshake_session_id_1.3_no_crypto_server

  • handshake_session_id_1.2_no_crypto_server

  • handshake_session_id_1.2_no_crypto_client

  • handshake_no_resume_1.2_no_crypto_client

  • transfer_no_resume_1.3_no_crypto_client

  • transfer_no_resume_1.2_no_crypto_client

  • transfer_no_resume_1.2_no_crypto_server

Significant differences

⚠️ There are significant instruction count differences

Click to expand
Scenario Baseline Candidate Diff Threshold
handshake_no_resume_aws_lc_rs_1.3_ecdsap256_chacha_server 1770111 1795391 ⚠️ 25280 (1.43%) 0.20%
handshake_no_resume_aws_lc_rs_1.3_ecdsap256_aes_server 1768080 1792840 ⚠️ 24760 (1.40%) 1.25%
handshake_no_resume_aws_lc_rs_1.3_ecdsap256_chacha_client 4278418 4331558 ⚠️ 53140 (1.24%) 0.20%
handshake_no_resume_aws_lc_rs_1.3_ecdsap256_aes_client 4276181 4328563 ⚠️ 52382 (1.22%) 1.01%
handshake_tickets_aws_lc_rs_1.3_rsa_chacha_client 64601638 64756539 ⚠️ 154901 (0.24%) 0.20%
handshake_tickets_aws_lc_rs_1.3_rsa_aes_client 64592049 64743193 ⚠️ 151144 (0.23%) 0.20%

Other differences

Click to expand
Scenario Baseline Candidate Diff Threshold
handshake_no_resume_aws_lc_rs_1.3_ecdsap384_chacha_client 9464908 9512111 47203 (0.50%) 0.56%
handshake_no_resume_aws_lc_rs_1.3_ecdsap384_aes_client 9512724 9483046 -29678 (-0.31%) 1.21%
handshake_no_resume_aws_lc_rs_1.3_ecdsap384_aes_server 2679644 2673788 -5856 (-0.22%) 0.84%
handshake_no_resume_aws_lc_rs_1.3_rsa_aes_server 11313538 11335561 22023 (0.19%) 1.56%
handshake_no_resume_aws_lc_rs_1.3_rsa_chacha_server 11344653 11366172 21519 (0.19%) 1.15%
handshake_session_id_aws_lc_rs_1.3_ecdsap256_chacha_client 64395079 64288439 -106640 (-0.17%) 0.20%
handshake_session_id_aws_lc_rs_1.3_ecdsap384_chacha_client 64401456 64295265 -106191 (-0.16%) 0.20%
handshake_tickets_aws_lc_rs_1.3_ecdsap256_aes_client 64734374 64632108 -102266 (-0.16%) 0.20%
handshake_tickets_aws_lc_rs_1.3_rsa_chacha_server 49559489 49634805 75316 (0.15%) 0.20%
handshake_tickets_aws_lc_rs_1.3_rsa_aes_server 49582866 49656381 73515 (0.15%) 0.20%
handshake_no_resume_ring_1.3_ecdsap256_chacha_client 3307697 3303448 -4249 (-0.13%) 0.28%
handshake_session_id_aws_lc_rs_1.3_ecdsap256_chacha_server 48241876 48189412 -52464 (-0.11%) 0.20%
handshake_session_id_aws_lc_rs_1.3_ecdsap384_chacha_server 48241867 48190112 -51755 (-0.11%) 0.20%
handshake_tickets_aws_lc_rs_1.3_ecdsap256_aes_server 49660782 49611120 -49662 (-0.10%) 0.20%
handshake_tickets_aws_lc_rs_1.3_ecdsap384_chacha_client 64596973 64534942 -62031 (-0.10%) 0.23%
handshake_session_id_aws_lc_rs_1.3_rsa_aes_client 64583729 64526926 -56803 (-0.09%) 0.25%
handshake_no_resume_aws_lc_rs_1.2_rsa_aes_server 10470508 10479216 8708 (0.08%) 1.10%
handshake_session_id_aws_lc_rs_1.3_rsa_chacha_client 64361787 64309157 -52630 (-0.08%) 0.20%
handshake_session_id_aws_lc_rs_1.3_ecdsap384_aes_client 64508614 64462029 -46585 (-0.07%) 0.23%
handshake_tickets_aws_lc_rs_1.3_ecdsap384_aes_client 64620599 64663156 42557 (0.07%) 0.20%
handshake_tickets_aws_lc_rs_1.3_ecdsap256_chacha_client 64589973 64629307 39334 (0.06%) 0.20%
handshake_tickets_aws_lc_rs_1.3_ecdsap384_chacha_server 49565473 49535784 -29689 (-0.06%) 0.20%
handshake_session_id_aws_lc_rs_1.3_rsa_aes_server 48373532 48347243 -26289 (-0.05%) 0.20%
handshake_session_id_aws_lc_rs_1.3_rsa_chacha_server 48213384 48187922 -25462 (-0.05%) 0.20%
handshake_session_id_aws_lc_rs_1.3_ecdsap384_aes_server 48344887 48322787 -22100 (-0.05%) 0.20%
handshake_tickets_aws_lc_rs_1.3_ecdsap384_aes_server 49605504 49627936 22432 (0.05%) 0.20%
transfer_no_resume_aws_lc_rs_1.3_rsa_aes_client 58265723 58240958 -24765 (-0.04%) 0.20%
handshake_no_resume_aws_lc_rs_1.3_rsa_aes_client 3109521 3110841 1320 (0.04%) 1.22%
handshake_tickets_aws_lc_rs_1.3_ecdsap256_chacha_server 49560494 49581368 20874 (0.04%) 0.20%
transfer_no_resume_ring_1.3_rsa_aes_client 58157965 58133497 -24468 (-0.04%) 0.20%
transfer_no_resume_ring_1.2_rsa_aes_client 58037987 58016862 -21125 (-0.04%) 0.20%
transfer_no_resume_aws_lc_rs_1.3_rsa_chacha_client 92727157 92702402 -24755 (-0.03%) 0.20%
transfer_no_resume_ring_1.3_rsa_chacha_client 92694282 92669823 -24459 (-0.03%) 0.20%
handshake_no_resume_ring_1.3_ecdsap256_chacha_server 1298759 1299021 262 (0.02%) 0.20%
handshake_no_resume_ring_1.3_ecdsap256_aes_server 1297589 1297829 240 (0.02%) 0.20%
handshake_session_id_aws_lc_rs_1.3_ecdsap256_aes_client 64405649 64415236 9587 (0.01%) 0.24%
handshake_session_id_aws_lc_rs_1.3_ecdsap256_aes_server 48296291 48302022 5731 (0.01%) 0.20%
handshake_no_resume_ring_1.3_ecdsap256_aes_client 3306797 3306413 -384 (-0.01%) 0.26%
handshake_no_resume_aws_lc_rs_1.2_rsa_aes_client 1720136 1719962 -174 (-0.01%) 0.20%
transfer_no_resume_aws_lc_rs_1.3_ecdsap256_aes_client 58240904 58236244 -4660 (-0.01%) 0.20%
handshake_no_resume_ring_1.2_rsa_aes_client 2246517 2246343 -174 (-0.01%) 0.20%
handshake_no_resume_ring_1.3_rsa_aes_client 2334232 2334058 -174 (-0.01%) 0.20%
handshake_no_resume_ring_1.3_rsa_chacha_client 2339881 2339707 -174 (-0.01%) 0.20%
transfer_no_resume_aws_lc_rs_1.3_ecdsap384_aes_client 58243131 58239121 -4010 (-0.01%) 0.20%
transfer_no_resume_ring_1.3_ecdsap256_aes_client 58126352 58122518 -3834 (-0.01%) 0.20%
transfer_no_resume_ring_1.3_ecdsap384_aes_client 58133568 58129735 -3833 (-0.01%) 0.20%
handshake_session_id_aws_lc_rs_1.2_rsa_aes_client 3879061 3879266 205 (0.01%) 0.20%
transfer_no_resume_aws_lc_rs_1.3_ecdsap256_chacha_client 92701066 92696401 -4665 (-0.01%) 0.20%
transfer_no_resume_aws_lc_rs_1.3_ecdsap384_chacha_client 92703292 92698640 -4652 (-0.01%) 0.20%
handshake_tickets_aws_lc_rs_1.2_rsa_aes_client 4221126 4221310 184 (0.00%) 0.20%
transfer_no_resume_ring_1.3_ecdsap256_chacha_client 92662682 92658838 -3844 (-0.00%) 0.20%
transfer_no_resume_ring_1.3_ecdsap384_chacha_client 92671813 92667974 -3839 (-0.00%) 0.20%
transfer_no_resume_ring_1.3_ecdsap256_aes_server 46289082 46287177 -1905 (-0.00%) 0.20%
transfer_no_resume_aws_lc_rs_1.3_rsa_aes_server 46457821 46455912 -1909 (-0.00%) 0.20%
transfer_no_resume_aws_lc_rs_1.2_rsa_aes_server 46401605 46399700 -1905 (-0.00%) 0.20%
transfer_no_resume_ring_1.2_rsa_aes_server 46189285 46187393 -1892 (-0.00%) 0.20%
transfer_no_resume_ring_1.3_rsa_aes_server 46284273 46282378 -1895 (-0.00%) 0.20%
transfer_no_resume_ring_1.3_ecdsap384_aes_server 46294220 46292325 -1895 (-0.00%) 0.20%
transfer_no_resume_aws_lc_rs_1.3_ecdsap256_aes_server 46464401 46462507 -1894 (-0.00%) 0.20%
transfer_no_resume_aws_lc_rs_1.3_ecdsap384_aes_server 46453381 46451493 -1888 (-0.00%) 0.20%
handshake_session_id_ring_1.2_rsa_aes_client 4307710 4307862 152 (0.00%) 0.20%
handshake_no_resume_ring_1.3_ecdsap384_chacha_server 7222921 7223170 249 (0.00%) 0.20%
handshake_no_resume_ring_1.3_ecdsap384_aes_server 7220971 7221211 240 (0.00%) 0.20%
transfer_no_resume_aws_lc_rs_1.2_rsa_aes_client 58163732 58162010 -1722 (-0.00%) 0.20%
transfer_no_resume_ring_1.3_ecdsap256_chacha_server 80538689 80536778 -1911 (-0.00%) 0.20%
transfer_no_resume_ring_1.3_ecdsap384_chacha_server 80543181 80541279 -1902 (-0.00%) 0.20%
transfer_no_resume_aws_lc_rs_1.3_rsa_chacha_server 80638298 80636397 -1901 (-0.00%) 0.20%
transfer_no_resume_aws_lc_rs_1.3_ecdsap256_chacha_server 80644244 80642352 -1892 (-0.00%) 0.20%
transfer_no_resume_ring_1.3_rsa_chacha_server 80533862 80531979 -1883 (-0.00%) 0.20%
transfer_no_resume_aws_lc_rs_1.3_ecdsap384_chacha_server 80633218 80631333 -1885 (-0.00%) 0.20%
handshake_tickets_ring_1.2_rsa_aes_client 4571298 4571193 -105 (-0.00%) 0.20%
handshake_no_resume_aws_lc_rs_1.3_rsa_chacha_client 3116245 3116315 70 (0.00%) 0.20%
handshake_session_id_ring_1.3_ecdsap256_chacha_server 31988072 31987840 -232 (-0.00%) 0.20%
handshake_session_id_ring_1.3_ecdsap384_chacha_server 31988175 31987943 -232 (-0.00%) 0.20%
handshake_session_id_ring_1.3_ecdsap256_aes_server 32109722 32109490 -232 (-0.00%) 0.20%
handshake_session_id_ring_1.3_ecdsap384_aes_server 32109825 32109593 -232 (-0.00%) 0.20%
handshake_no_resume_aws_lc_rs_1.3_ecdsap384_chacha_server 2676490 2676505 15 (0.00%) 0.20%
handshake_tickets_ring_1.2_rsa_aes_server 4758624 4758640 16 (0.00%) 0.20%
handshake_tickets_ring_1.3_rsa_chacha_client 31160512 31160433 -79 (-0.00%) 0.20%
handshake_tickets_ring_1.3_rsa_aes_client 31231783 31231704 -79 (-0.00%) 0.20%
handshake_session_id_ring_1.3_rsa_chacha_client 30841748 30841826 78 (0.00%) 0.20%
handshake_tickets_ring_1.3_rsa_chacha_server 32448813 32448895 82 (0.00%) 0.20%
handshake_session_id_ring_1.3_rsa_aes_client 30933038 30933116 78 (0.00%) 0.20%
handshake_tickets_ring_1.3_rsa_aes_server 32550768 32550850 82 (0.00%) 0.20%
handshake_tickets_ring_1.3_ecdsap384_chacha_client 31153169 31153247 78 (0.00%) 0.20%
handshake_tickets_ring_1.3_ecdsap384_aes_client 31224419 31224497 78 (0.00%) 0.20%
handshake_no_resume_ring_1.3_ecdsap384_chacha_client 34737978 34738062 84 (0.00%) 0.20%
handshake_tickets_ring_1.3_ecdsap384_chacha_server 32451580 32451502 -78 (-0.00%) 0.20%
handshake_tickets_ring_1.3_ecdsap384_aes_server 32553490 32553412 -78 (-0.00%) 0.20%
handshake_session_id_ring_1.3_ecdsap256_chacha_client 30837694 30837636 -58 (-0.00%) 0.20%
handshake_session_id_ring_1.3_ecdsap256_aes_client 30928984 30928926 -58 (-0.00%) 0.20%
handshake_no_resume_ring_1.3_ecdsap384_aes_client 34736308 34736353 45 (0.00%) 0.20%
handshake_session_id_aws_lc_rs_1.2_rsa_aes_server 3870591 3870587 -4 (-0.00%) 0.20%
handshake_tickets_aws_lc_rs_1.2_rsa_aes_server 5011840 5011844 4 (0.00%) 0.20%
handshake_session_id_ring_1.3_ecdsap384_chacha_client 30834745 30834761 16 (0.00%) 0.20%
handshake_session_id_ring_1.3_ecdsap384_aes_client 30926035 30926051 16 (0.00%) 0.20%
handshake_no_resume_ring_1.2_rsa_aes_server 10996575 10996571 -4 (-0.00%) 0.20%
handshake_no_resume_ring_1.3_rsa_aes_server 11124634 11124630 -4 (-0.00%) 0.20%
handshake_no_resume_ring_1.3_rsa_chacha_server 11130395 11130391 -4 (-0.00%) 0.20%
handshake_tickets_ring_1.3_ecdsap256_chacha_client 31156524 31156532 8 (0.00%) 0.20%
handshake_tickets_ring_1.3_ecdsap256_aes_client 31227795 31227803 8 (0.00%) 0.20%
handshake_session_id_ring_1.3_rsa_chacha_server 31985020 31985012 -8 (-0.00%) 0.20%
handshake_session_id_ring_1.3_rsa_aes_server 32106670 32106662 -8 (-0.00%) 0.20%
handshake_tickets_ring_1.3_ecdsap256_chacha_server 32451622 32451614 -8 (-0.00%) 0.20%
handshake_tickets_ring_1.3_ecdsap256_aes_server 32553577 32553569 -8 (-0.00%) 0.20%
handshake_session_id_ring_1.2_rsa_aes_server 4299946 4299946 0 (0.00%) 0.20%

Wall-time

⚠️ Missing benchmarks

The following benchmark scenarios are present in the candidate but not in the baseline:

  • handshake_tickets_1.3_no_crypto

  • transfer_no_resume_1.3_no_crypto

  • handshake_session_id_1.3_no_crypto

  • handshake_no_resume_1.2_no_crypto

  • handshake_session_id_1.2_no_crypto

  • transfer_no_resume_1.2_no_crypto

  • handshake_tickets_1.2_no_crypto

  • handshake_no_resume_1.3_no_crypto

Significant differences

⚠️ There are significant wall-time differences

Click to expand
Scenario Baseline Candidate Diff Threshold
handshake_no_resume_aws_lc_rs_1.2_rsa_aes 1.05 ms 1.09 ms ⚠️ 0.04 ms (3.42%) 1.75%
handshake_no_resume_aws_lc_rs_1.3_rsa_aes 1.23 ms 1.27 ms ⚠️ 0.04 ms (3.18%) 2.41%
handshake_no_resume_aws_lc_rs_1.3_rsa_chacha 1.22 ms 1.26 ms ⚠️ 0.04 ms (2.97%) 2.94%

Other differences

Click to expand
Scenario Baseline Candidate Diff Threshold
handshake_session_id_aws_lc_rs_1.2_rsa_aes 1.61 ms 1.65 ms 0.04 ms (2.30%) 3.52%
handshake_tickets_aws_lc_rs_1.2_rsa_aes 1.78 ms 1.81 ms 0.03 ms (1.97%) 2.66%
transfer_no_resume_ring_1.2_rsa_aes 5.90 ms 5.84 ms -0.06 ms (-0.97%) 5.43%
handshake_no_resume_ring_1.3_ecdsap256_aes 471.99 µs 467.87 µs -4.12 µs (-0.87%) 3.74%
handshake_no_resume_ring_1.3_ecdsap256_chacha 468.56 µs 465.07 µs -3.49 µs (-0.74%) 3.61%
transfer_no_resume_aws_lc_rs_1.3_ecdsap384_aes 5.38 ms 5.34 ms -0.04 ms (-0.72%) 6.43%
handshake_no_resume_aws_lc_rs_1.3_ecdsap256_aes 593.75 µs 589.88 µs -3.87 µs (-0.65%) 3.91%
handshake_no_resume_ring_1.2_rsa_aes 953.76 µs 947.93 µs -5.84 µs (-0.61%) 1.58%
transfer_no_resume_aws_lc_rs_1.3_ecdsap256_aes 4.69 ms 4.66 ms -0.03 ms (-0.60%) 7.54%
transfer_no_resume_ring_1.3_ecdsap256_aes 5.46 ms 5.43 ms -0.03 ms (-0.47%) 6.37%
transfer_no_resume_ring_1.3_rsa_aes 5.94 ms 5.92 ms -0.03 ms (-0.47%) 5.73%
handshake_no_resume_ring_1.3_rsa_chacha 958.38 µs 954.07 µs -4.30 µs (-0.45%) 1.78%
handshake_no_resume_aws_lc_rs_1.3_ecdsap256_chacha 591.32 µs 588.93 µs -2.39 µs (-0.40%) 4.26%
handshake_no_resume_ring_1.3_rsa_aes 957.70 µs 954.15 µs -3.54 µs (-0.37%) 1.56%
handshake_session_id_aws_lc_rs_1.3_rsa_chacha 9.21 ms 9.24 ms 0.03 ms (0.34%) 2.85%
transfer_no_resume_aws_lc_rs_1.3_rsa_aes 5.32 ms 5.34 ms 0.02 ms (0.32%) 6.03%
transfer_no_resume_ring_1.3_ecdsap384_aes 8.60 ms 8.57 ms -0.02 ms (-0.28%) 4.05%
handshake_session_id_aws_lc_rs_1.3_ecdsap384_aes 9.26 ms 9.23 ms -0.03 ms (-0.28%) 3.39%
handshake_session_id_aws_lc_rs_1.3_rsa_aes 9.24 ms 9.27 ms 0.03 ms (0.27%) 3.10%
handshake_tickets_aws_lc_rs_1.3_ecdsap256_aes 8.77 ms 8.75 ms -0.02 ms (-0.27%) 3.30%
handshake_session_id_aws_lc_rs_1.3_ecdsap256_aes 8.59 ms 8.57 ms -0.02 ms (-0.26%) 3.57%
handshake_tickets_aws_lc_rs_1.3_rsa_aes 9.41 ms 9.44 ms 0.02 ms (0.24%) 3.01%
handshake_no_resume_aws_lc_rs_1.3_ecdsap384_aes 1.27 ms 1.27 ms -0.00 ms (-0.24%) 2.03%
handshake_tickets_aws_lc_rs_1.3_rsa_chacha 9.40 ms 9.42 ms 0.02 ms (0.24%) 2.50%
transfer_no_resume_aws_lc_rs_1.3_ecdsap256_chacha 13.16 ms 13.13 ms -0.03 ms (-0.23%) 2.55%
transfer_no_resume_ring_1.3_rsa_chacha 13.51 ms 13.48 ms -0.03 ms (-0.22%) 2.50%
handshake_no_resume_aws_lc_rs_1.3_ecdsap384_chacha 1.26 ms 1.26 ms -0.00 ms (-0.20%) 1.76%
handshake_tickets_aws_lc_rs_1.3_ecdsap384_aes 9.46 ms 9.44 ms -0.02 ms (-0.20%) 2.98%
transfer_no_resume_ring_1.3_ecdsap256_chacha 13.02 ms 12.99 ms -0.03 ms (-0.20%) 2.55%
transfer_no_resume_aws_lc_rs_1.3_ecdsap384_chacha 13.84 ms 13.81 ms -0.03 ms (-0.19%) 2.63%
handshake_session_id_aws_lc_rs_1.3_ecdsap384_chacha 9.22 ms 9.20 ms -0.02 ms (-0.18%) 2.94%
handshake_session_id_ring_1.3_ecdsap384_chacha 8.67 ms 8.69 ms 0.02 ms (0.18%) 1.00%
handshake_session_id_ring_1.3_rsa_chacha 6.03 ms 6.04 ms 0.01 ms (0.16%) 1.60%
handshake_tickets_ring_1.2_rsa_aes 1.60 ms 1.60 ms 0.00 ms (0.16%) 1.17%
transfer_no_resume_ring_1.3_ecdsap384_chacha 16.16 ms 16.14 ms -0.02 ms (-0.14%) 2.06%
handshake_session_id_aws_lc_rs_1.3_ecdsap256_chacha 8.55 ms 8.54 ms -0.01 ms (-0.12%) 3.37%
handshake_session_id_ring_1.3_ecdsap384_aes 8.70 ms 8.71 ms 0.01 ms (0.12%) 1.00%
handshake_tickets_ring_1.3_ecdsap384_aes 8.77 ms 8.78 ms 0.01 ms (0.10%) 1.00%
handshake_tickets_aws_lc_rs_1.3_ecdsap256_chacha 8.74 ms 8.73 ms -0.01 ms (-0.10%) 3.12%
handshake_tickets_aws_lc_rs_1.3_ecdsap384_chacha 9.42 ms 9.41 ms -0.01 ms (-0.10%) 2.82%
handshake_tickets_ring_1.3_ecdsap256_aes 5.64 ms 5.64 ms -0.00 ms (-0.07%) 1.37%
handshake_session_id_ring_1.2_rsa_aes 1.51 ms 1.51 ms -0.00 ms (-0.07%) 1.05%
handshake_tickets_ring_1.3_rsa_aes 6.13 ms 6.13 ms 0.00 ms (0.06%) 1.00%
handshake_tickets_ring_1.3_ecdsap256_chacha 5.60 ms 5.60 ms -0.00 ms (-0.06%) 1.58%
handshake_session_id_ring_1.3_ecdsap256_chacha 5.54 ms 5.54 ms 0.00 ms (0.05%) 1.74%
transfer_no_resume_aws_lc_rs_1.3_rsa_chacha 13.80 ms 13.81 ms 0.01 ms (0.05%) 2.35%
handshake_no_resume_ring_1.3_ecdsap384_aes 3.61 ms 3.61 ms -0.00 ms (-0.04%) 1.00%
handshake_session_id_ring_1.3_rsa_aes 6.07 ms 6.07 ms 0.00 ms (0.03%) 1.42%
transfer_no_resume_aws_lc_rs_1.2_rsa_aes 5.12 ms 5.13 ms 0.00 ms (0.02%) 6.13%
handshake_no_resume_ring_1.3_ecdsap384_chacha 3.61 ms 3.61 ms 0.00 ms (0.02%) 1.00%
handshake_session_id_ring_1.3_ecdsap256_aes 5.57 ms 5.57 ms 0.00 ms (0.01%) 1.37%
handshake_tickets_ring_1.3_ecdsap384_chacha 8.74 ms 8.74 ms -0.00 ms (-0.01%) 1.00%
handshake_tickets_ring_1.3_rsa_chacha 6.09 ms 6.09 ms -0.00 ms (-0.01%) 1.31%

Additional information

Historical results

Checkout details:

@codecov Codecov
Copy link
codecov bot commented Jun 10, 2025 •
edited
Loading

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 95.40%. Comparing base (96d6656) to head (abb9bdd).
Report is 3 commits behind head on main.

Additional details and impacted files 
@@           Coverage Diff           @@
##             main    #2483   +/-   ##
=======================================
  Coverage   95.40%   95.40%           
=======================================
  Files          97       97           
  Lines       21874    21874           
=======================================
  Hits        20868    20868           
  Misses       1006     1006

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@ctz ctz mentioned this pull request Jun 10, 2025
Closed
@ctz ctz requested a review from djc June 11, 2025 14:11
djc
djc approved these changes Jun 12, 2025
View reviewed changes
Copy link
Member
@djc djc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good idea!

I guess there's no easy way to patch out/prevent the rejection sampling?

@ctz ctz force-pushed the jbp-no-crypto-bench branch from 01abb03 to abb9bdd Compare June 12, 2025 13:36
@ctz
Copy link
Member Author
ctz commented Jun 12, 2025
8000

I guess there's no easy way to patch out/prevent the rejection sampling?

I don't think so; not without making the crypto significantly different to what we actually want to test (and rustls-fuzzing-provider is the extreme end of that road).

@ctz ctz added this pull request to the merge queue Jun 12, 2025
Merged via the queue into main with commit b9a530a Jun 12, 2025
34 checks passed
@ctz ctz deleted the jbp-no-crypto-bench branch June 12, 2025 13:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@djc djc djc approved these changes

@cpu cpu cpu approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@ctz @djc @cpu
0