10000 tests: linearize new test code helper by djc · Pull Request #2225 · rustls/rustls · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

tests: linearize new test code helper #2225

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 1 commit into from
Nov 21, 2024
Merged

tests: linearize new test code helper #2225

merged 1 commit into from
Nov 21, 2024

Conversation

djc
Copy link
Member
@djc djc commented Nov 21, 2024

@djc djc requested review from cpu and ctz November 21, 2024 14:51
@rustls-benchmarking Rustls Benchmarking
Copy link
rustls-benchmarking bot commented Nov 21, 2024 •
edited
Loading

Benchmark results

Instruction counts

Significant differences

There are no significant instruction count differences

Other differences

Click to expand
Scenario Baseline Candidate Diff Threshold
handshake_no_resume_aws_lc_rs_1.3_rsa_aes_server 10701483 10721817 20334 (0.19%) 1.15%
handshake_no_resume_aws_lc_rs_1.3_rsa_chacha_server 10748525 10728999 -19526 (-0.18%) 0.84%
handshake_no_resume_aws_lc_rs_1.3_ecdsap384_chacha_client 8591795 8576749 -15046 (-0.18%) 0.78%
handshake_no_resume_ring_1.3_ecdsap256_chacha_client 3624595 3620431 -4164 (-0.11%) 0.49%
handshake_no_resume_aws_lc_rs_1.3_ecdsap256_chacha_client 3080671 3083108 2437 (0.08%) 0.25%
handshake_no_resume_aws_lc_rs_1.3_ecdsap384_aes_client 8581525 8587392 5867 (0.07%) 1.15%
handshake_no_resume_ring_1.3_ecdsap256_aes_client 3622979 3620954 -2025 (-0.06%) 0.47%
handshake_no_resume_aws_lc_rs_1.2_rsa_aes_server 10414522 10417287 2765 (0.03%) 1.81%
handshake_no_resume_aws_lc_rs_1.3_ecdsap256_aes_server 1174481 1174395 -86 (-0.01%) 0.20%
handshake_no_resume_aws_lc_rs_1.3_ecdsap384_chacha_server 2076810 2076913 103 (0.00%) 0.20%
transfer_no_resume_aws_lc_rs_1.3_ecdsap256_aes_client 58253660 58251183 -2477 (-0.00%) 0.20%
handshake_tickets_aws_lc_rs_1.3_rsa_chacha_client 28199564 28199909 345 (0.00%) 0.20%
handshake_session_id_aws_lc_rs_1.3_ecdsap256_aes_server 28967823 28967479 -344 (-0.00%) 0.20%
handshake_no_resume_aws_lc_rs_1.3_ecdsap384_aes_server 2073786 2073764 -22 (-0.00%) 0.20%
handshake_no_resume_aws_lc_rs_1.3_ecdsap256_aes_client 3080387 3080357 -30 (-0.00%) 0.22%
handshake_tickets_aws_lc_rs_1.3_ecdsap256_aes_server 30427120 30427372 252 (0.00%) 0.20%
handshake_tickets_aws_lc_rs_1.3_ecdsap384_aes_client 28222872 28222642 -230 (-0.00%) 0.20%
handshake_session_id_aws_lc_rs_1.3_ecdsap256_chacha_client 27801758 27801540 -218 (-0.00%) 0.20%
handshake_no_resume_aws_lc_rs_1.3_rsa_aes_client 1925726 1925741 15 (0.00%) 0.20%
handshake_no_resume_aws_lc_rs_1.3_ecdsap256_chacha_server 1176774 1176782 8 (0.00%) 0.20%
handshake_tickets_aws_lc_rs_1.3_rsa_aes_server 30424333 30424164 -169 (-0.00%) 0.20%
handshake_session_id_aws_lc_rs_1.3_ecdsap384_aes_server 28967729 28967574 -155 (-0.00%) 0.20%
handshake_session_id_aws_lc_rs_1.3_rsa_aes_client 27865513 27865657 144 (0.00%) 0.20%
handshake_session_id_aws_lc_rs_1.3_ecdsap256_aes_client 27861286 27861147 -139 (-0.00%) 0.20%
handshake_session_id_aws_lc_rs_1.3_rsa_chacha_server 28886519 28886379 -140 (-0.00%) 0.20%
handshake_tickets_aws_lc_rs_1.3_ecdsap384_chacha_client 28192872 28192759 -113 (-0.00%) 0.20%
handshake_no_resume_aws_lc_rs_1.3_rsa_chacha_client 1932461 1932468 7 (0.00%) 0.20%
handshake_session_id_aws_lc_rs_1.3_ecdsap256_chacha_server 28889026 28889130 104 (0.00%) 0.20%
handshake_session_id_aws_lc_rs_1.3_ecdsap384_aes_client 27858648 27858553 -95 (-0.00%) 0.20%
handshake_session_id_aws_lc_rs_1.3_rsa_aes_server 28964770 28964868 98 (0.00%) 0.20%
handshake_tickets_aws_lc_rs_1.3_rsa_chacha_server 30381480 30381578 98 (0.00%) 0.20%
handshake_tickets_aws_lc_rs_1.3_ecdsap384_aes_server 30427561 30427467 -94 (-0.00%) 0.20%
handshake_tickets_aws_lc_rs_1.3_ecdsap256_aes_client 28225730 28225644 -86 (-0.00%) 0.20%
handshake_tickets_aws_lc_rs_1.3_ecdsap256_chacha_server 30384497 30384578 81 (0.00%) 0.20%
handshake_tickets_aws_lc_rs_1.3_ecdsap384_chacha_server 30384815 30384754 -61 (-0.00%) 0.20%
handshake_tickets_aws_lc_rs_1.3_ecdsap256_chacha_client 28196027 28195972 -55 (-0.00%) 0.20%
handshake_session_id_aws_lc_rs_1.3_ecdsap384_chacha_server 28889278 28889228 -50 (-0.00%) 0.20%
handshake_no_resume_ring_1.3_ecdsap384_chacha_server 7566460 7566470 10 (0.00%) 0.20%
handshake_tickets_aws_lc_rs_1.3_rsa_aes_client 28229623 28229660 37 (0.00%) 0.20%
handshake_session_id_aws_lc_rs_1.3_ecdsap384_chacha_client 27799247 27799280 33 (0.00%) 0.20%
handshake_no_resume_ring_1.3_ecdsap384_chacha_client 35182545 35182570 25 (0.00%) 0.20%
handshake_no_resume_ring_1.3_ecdsap384_aes_client 35180653 35180640 -13 (-0.00%) 0.20%
handshake_session_id_aws_lc_rs_1.3_rsa_chacha_client 27806064 27806073 9 (0.00%) 0.20%
handshake_no_resume_ring_1.3_ecdsap384_aes_server 7564314 7564316 2 (0.00%) 0.20%
transfer_no_resume_ring_1.3_rsa_aes_server 46487102 46487090 -12 (-0.00%) 0.20%
transfer_no_resume_aws_lc_rs_1.3_ecdsap384_aes_server 46456943 46456933 -10 (-0.00%) 0.20%
transfer_no_resume_ring_1.3_ecdsap256_aes_server 46467802 46467812 10 (0.00%) 0.20%
transfer_no_resume_ring_1.2_rsa_aes_server 46389596 46389587 -9 (-0.00%) 0.20%
transfer_no_resume_ring_1.2_rsa_aes_client 58235169 58235179 10 (0.00%) 0.20%
transfer_no_resume_ring_1.3_ecdsap384_aes_server 46470684 46470691 7 (0.00%) 0.20%
transfer_no_resume_aws_lc_rs_1.3_rsa_aes_client 58255497 58255491 -6 (-0.00%) 0.20%
transfer_no_resume_ring_1.3_ecdsap384_chacha_server 80540441 80540448 7 (0.00%) 0.20%
transfer_no_resume_aws_lc_rs_1.3_rsa_chacha_server 80661856 80661850 -6 (-0.00%) 0.20%
transfer_no_resume_aws_lc_rs_1.3_rsa_aes_server 46477531 46477528 -3 (-0.00%) 0.20%
transfer_no_resume_aws_lc_rs_1.3_rsa_chacha_client 92720773 92720778 5 (0.00%) 0.20%
transfer_no_resume_ring_1.3_ecdsap256_aes_client 58341601 58341604 3 (0.00%) 0.20%
transfer_no_resume_ring_1.3_ecdsap384_aes_client 58348750 58348747 -3 (-0.00%) 0.20%
transfer_no_resume_ring_1.3_rsa_chacha_server 80554931 80554935 4 (0.00%) 0.20%
transfer_no_resume_aws_lc_rs_1.3_ecdsap384_chacha_server 80641262 80641258 -4 (-0.00%) 0.20%
transfer_no_resume_ring_1.3_ecdsap384_chacha_client 92682826 92682830 4 (0.00%) 0.20%
transfer_no_resume_aws_lc_rs_1.3_ecdsap384_chacha_client 92716779 92716775 -4 (-0.00%) 0.20%
transfer_no_resume_aws_lc_rs_1.2_rsa_aes_client 58191643 58191645 2 (0.00%) 0.20%
transfer_no_resume_ring_1.3_ecdsap256_chacha_client 92673760 92673763 3 (0.00%) 0.20%
transfer_no_resume_ring_1.3_rsa_chacha_client 92684668 92684671 3 (0.00%) 0.20%
transfer_no_resume_ring_1.3_ecdsap256_chacha_server 80535650 80535648 -2 (-0.00%) 0.20%
transfer_no_resume_aws_lc_rs_1.2_rsa_aes_server 46423831 46423832 1 (0.00%) 0.20%
transfer_no_resume_aws_lc_rs_1.3_ecdsap256_aes_server 46464558 46464559 1 (0.00%) 0.20%
transfer_no_resume_aws_lc_rs_1.3_ecdsap384_aes_client 58253411 58253412 1 (0.00%) 0.20%
handshake_session_id_ring_1.3_ecdsap256_aes_server 41603721 41603721 0 (0.00%) 0.20%
handshake_session_id_ring_1.2_rsa_aes_client 4236939 4236939 0 (0.00%) 0.20%
handshake_tickets_aws_lc_rs_1.2_rsa_aes_client 4214929 4214929 0 (0.00%) 0.20%
handshake_no_resume_ring_1.2_rsa_aes_client 2563572 2563572 0 (0.00%) 0.20%
transfer_no_resume_aws_lc_rs_1.3_ecdsap256_chacha_server 80648884 80648884 0 (0.00%) 0.20%
handshake_session_id_ring_1.3_ecdsap384_chacha_server 41484849 41484849 0 (0.00%) 0.20%
handshake_session_id_ring_1.3_rsa_chacha_client 40202193 40202193 0 (0.00%) 0.20%
handshake_tickets_ring_1.2_rsa_aes_client 4497603 4497603 0 (0.00%) 0.20%
handshake_tickets_ring_1.2_rsa_aes_server 4695330 4695330 0 (0.00%) 0.20%
handshake_tickets_ring_1.3_rsa_chacha_server 41963018 41963018 0 (0.00%) 0.20%
handshake_session_id_ring_1.3_ecdsap256_chacha_server 41484771 41484771 0 (0.00%) 0.20%
handshake_tickets_ring_1.3_ecdsap256_aes_client 40590226 40590226 0 (0.00%) 0.20%
handshake_session_id_ring_1.3_ecdsap256_chacha_client 40198148 40198148 0 (0.00%) 0.20%
handshake_tickets_ring_1.3_ecdsap384_aes_client 40586975 40586975 0 (0.00%) 0.20%
handshake_tickets_ring_1.3_ecdsap384_aes_server 42065138 42065138 0 (0.00%) 0.20%
transfer_no_resume_aws_lc_rs_1.3_ecdsap256_chacha_client 92714544 92714544 0 (0.00%) 0.20%
handshake_tickets_ring_1.3_rsa_aes_client 40594220 40594220 0 (0.00%) 0.20%
handshake_tickets_ring_1.3_ecdsap256_chacha_server 41965918 41965918 0 (0.00%) 0.20%
handshake_session_id_ring_1.3_ecdsap256_aes_client 40292528 40292528 0 (0.00%) 0.20%
handshake_no_resume_ring_1.2_rsa_aes_server 11292386 11292386 0 (0.00%) 0.20%
handshake_no_resume_ring_1.3_ecdsap256_chacha_server 1613362 1613362 0 (0.00%) 1.07%
handshake_no_resume_ring_1.3_ecdsap256_aes_server 1611985 1611985 0 (0.00%) 1.08%
handshake_tickets_aws_lc_rs_1.2_rsa_aes_server 5020364 5020364 0 (0.00%) 0.20%
handshake_no_resume_ring_1.3_rsa_aes_server 11424773 11424773 0 (0.00%) 0.20%
handshake_no_resume_ring_1.3_rsa_chacha_client 2661760 2661760 0 (0.00%) 0.20%
handshake_no_resume_aws_lc_rs_1.2_rsa_aes_client 1717624 1717624 0 (0.00%) 0.20%
handshake_session_id_ring_1.2_rsa_aes_server 4230790 4230790 0 (0.00%) 0.20%
handshake_session_id_ring_1.3_ecdsap384_aes_client 40289595 40289595 0 (0.00%) 0.20%
handshake_session_id_ring_1.3_rsa_aes_server 41601128 41601128 0 (0.00%) 0.20%
handshake_tickets_ring_1.3_rsa_chacha_client 40519970 40519970 0 (0.00%) 0.20%
handshake_tickets_ring_1.3_ecdsap256_chacha_client 40515976 40515976 0 (0.00%) 0.20%
handshake_session_id_ring_1.3_ecdsap384_chacha_client 40195207 40195207 0 (0.00%) 0.20%
handshake_session_id_aws_lc_rs_1.2_rsa_aes_client 3872695 3872695 0 (0.00%) 0.20%
handshake_session_id_aws_lc_rs_1.2_rsa_aes_server 3870183 3870183 0 (0.00%) 0.20%
handshake_no_resume_ring_1.3_rsa_chacha_server 11430729 11430729 0 (0.00%) 0.20%
transfer_no_resume_ring_1.3_rsa_aes_client 58352502 58352502 0 (0.00%) 0.20%
handshake_tickets_ring_1.3_rsa_aes_server 42062048 42062048 0 (0.00%) 0.20%
handshake_tickets_ring_1.3_ecdsap384_chacha_server 41966108 41966108 0 (0.00%) 0.20%
handshake_tickets_ring_1.3_ecdsap384_chacha_client 40512717 40512717 0 (0.00%) 0.20%
handshake_tickets_ring_1.3_ecdsap256_aes_server 42064948 42064948 0 (0.00%) 0.20%
handshake_session_id_ring_1.3_rsa_aes_client 40296573 40296573 0 (0.00%) 0.20%
handshake_session_id_ring_1.3_rsa_chacha_server 41482178 41482178 0 (0.00%) 0.20%
handshake_session_id_ring_1.3_ecdsap384_aes_server 41603799 41603799 0 (0.00%) 0.20%
handshake_no_resume_ring_1.3_rsa_aes_client 2655895 2655895 0 (0.00%) 0.20%

Wall-time

Significant differences

There are no significant wall-time differences

Other differences

Click to expand
Scenario Baseline Candidate Diff Threshold
transfer_no_resume_aws_lc_rs_1.3_ecdsap256_aes 4.51 ms 4.43 ms -0.08 ms (-1.83%) 4.16%
transfer_no_resume_aws_lc_rs_1.3_ecdsap384_aes 5.23 ms 5.15 ms -0.08 ms (-1.62%) 3.63%
transfer_no_resume_aws_lc_rs_1.2_rsa_aes 5.10 ms 5.02 ms -0.08 ms (-1.58%) 4.46%
transfer_no_resume_aws_lc_rs_1.3_rsa_aes 5.16 ms 5.08 ms -0.08 ms (-1.56%) 3.70%
transfer_no_resume_ring_1.3_ecdsap256_aes 6.37 ms 6.29 ms -0.08 ms (-1.32%) 2.73%
handshake_no_resume_aws_lc_rs_1.3_ecdsap256_aes 459.96 µs 454.18 µs -5.78 µs (-1.26%) 3.12%
handshake_no_resume_aws_lc_rs_1.3_ecdsap256_chacha 458.42 µs 453.16 µs -5.25 µs (-1.15%) 2.38%
transfer_no_resume_ring_1.2_rsa_aes 6.79 ms 6.72 ms -0.07 ms (-1.08%) 2.85%
transfer_no_resume_ring_1.3_rsa_aes 6.86 ms 6.79 ms -0.07 ms (-1.08%) 3.14%
handshake_no_resume_ring_1.3_ecdsap256_aes 507.04 µs 502.52 µs -4.52 µs (-0.89%) 2.15%
transfer_no_resume_ring_1.3_ecdsap384_aes 9.52 ms 9.45 ms -0.08 ms (-0.81%) 1.63%
handshake_no_resume_ring_1.3_ecdsap256_chacha 503.62 µs 499.71 µs -3.91 µs (-0.78%) 2.24%
transfer_no_resume_aws_lc_rs_1.3_ecdsap256_chacha 12.98 ms 12.90 ms -0.08 ms (-0.64%) 1.46%
transfer_no_resume_aws_lc_rs_1.3_ecdsap384_chacha 13.71 ms 13.62 ms -0.08 ms (-0.60%) 1.43%
transfer_no_resume_ring_1.3_ecdsap256_chacha 13.01 ms 12.93 ms -0.08 ms (-0.59%) 1.23%
handshake_tickets_aws_lc_rs_1.3_ecdsap256_aes 4.48 ms 4.46 ms -0.02 ms (-0.55%) 1.42%
transfer_no_resume_ring_1.3_rsa_chacha 13.50 ms 13.43 ms -0.07 ms (-0.53%) 1.41%
transfer_no_resume_aws_lc_rs_1.3_rsa_chacha 13.63 ms 13.57 ms -0.07 ms (-0.50%) 1.75%
handshake_tickets_aws_lc_rs_1.3_rsa_aes 5.14 ms 5.12 ms -0.02 ms (-0.46%) 1.19%
transfer_no_resume_ring_1.3_ecdsap384_chacha 16.17 ms 16.10 ms -0.07 ms (-0.46%) 1.00%
handshake_tickets_aws_lc_rs_1.3_ecdsap384_aes 5.20 ms 5.18 ms -0.02 ms (-0.45%) 1.18%
handshake_no_resume_aws_lc_rs_1.2_rsa_aes 1.08 ms 1.07 ms -0.00 ms (-0.42%) 2.34%
handshake_session_id_aws_lc_rs_1.3_ecdsap256_aes 4.28 ms 4.26 ms -0.02 ms (-0.40%) 1.29%
handshake_tickets_aws_lc_rs_1.2_rsa_aes 1.79 ms 1.78 ms -0.01 ms (-0.40%) 2.17%
handshake_tickets_aws_lc_rs_1.3_rsa_chacha 5.14 ms 5.12 ms -0.02 ms (-0.40%) 1.55%
handshake_no_resume_ring_1.3_rsa_chacha 998.22 µs 994.59 µs -3.63 µs (-0.36%) 1.06%
handshake_no_resume_aws_lc_rs_1.3_ecdsap384_aes 1.18 ms 1.17 ms -0.00 ms (-0.36%) 1.29%
handshake_session_id_aws_lc_rs_1.3_rsa_aes 4.95 ms 4.93 ms -0.02 ms (-0.35%) 1.41%
handshake_session_id_aws_lc_rs_1.3_ecdsap384_aes 5.00 ms 4.98 ms -0.02 ms (-0.33%) 1.64%
handshake_tickets_aws_lc_rs_1.3_ecdsap384_chacha 5.17 ms 5.16 ms -0.02 ms (-0.32%) 1.31%
handshake_session_id_aws_lc_rs_1.3_rsa_chacha 4.92 ms 4.91 ms -0.02 ms (-0.31%) 1.91%
handshake_tickets_aws_lc_rs_1.3_ecdsap256_chacha 4.46 ms 4.45 ms -0.01 ms (-0.31%) 1.47%
handshake_no_resume_ring_1.2_rsa_aes 992.81 µs 990.04 µs -2.77 µs (-0.28%) 1.16%
handshake_session_id_aws_lc_rs_1.3_ecdsap384_chacha 4.97 ms 4.95 ms -0.01 ms (-0.28%) 1.32%
handshake_no_resume_aws_lc_rs_1.3_rsa_aes 1.10 ms 1.10 ms -0.00 ms (-0.28%) 2.90%
handshake_no_resume_aws_lc_rs_1.3_ecdsap384_chacha 1.17 ms 1.17 ms -0.00 ms (-0.26%) 1.14%
handshake_no_resume_ring_1.3_rsa_aes 997.54 µs 995.0 B5F1 5 µs -2.49 µs (-0.25%) 1.03%
handshake_session_id_aws_lc_rs_1.2_rsa_aes 1.62 ms 1.61 ms -0.00 ms (-0.23%) 1.62%
handshake_no_resume_aws_lc_rs_1.3_rsa_chacha 1.11 ms 1.11 ms 0.00 ms (0.20%) 4.09%
handshake_session_id_ring_1.2_rsa_aes 1.52 ms 1.52 ms -0.00 ms (-0.17%) 1.00%
handshake_session_id_aws_lc_rs_1.3_ecdsap256_chacha 4.26 ms 4.25 ms -0.01 ms (-0.16%) 1.56%
handshake_tickets_ring_1.2_rsa_aes 1.61 ms 1.61 ms -0.00 ms (-0.13%) 1.86%
handshake_no_resume_ring_1.3_ecdsap384_aes 3.66 ms 3.66 ms -0.00 ms (-0.10%) 1.00%
handshake_tickets_ring_1.3_rsa_chacha 6.96 ms 6.95 ms -0.01 ms (-0.09%) 1.00%
handshake_tickets_ring_1.3_ecdsap256_chacha 6.47 ms 6.46 ms -0.01 ms (-0.09%) 1.00%
handshake_tickets_ring_1.3_rsa_aes 7.02 ms 7.01 ms -0.00 ms (-0.07%) 1.00%
handshake_session_id_ring_1.3_rsa_aes 6.95 ms 6.95 ms -0.00 ms (-0.07%) 1.00%
handshake_session_id_ring_1.3_rsa_chacha 6.89 ms 6.88 ms -0.00 ms (-0.06%) 1.00%
handshake_tickets_ring_1.3_ecdsap256_aes 6.52 ms 6.52 ms -0.00 ms (-0.06%) 1.15%
handshake_tickets_ring_1.3_ecdsap384_chacha 9.61 ms 9.60 ms -0.01 ms (-0.05%) 1.00%
handshake_session_id_ring_1.3_ecdsap384_chacha 9.54 ms 9.53 ms -0.00 ms (-0.04%) 1.00%
handshake_no_resume_ring_1.3_ecdsap384_chacha 3.66 ms 3.66 ms -0.00 ms (-0.04%) 1.00%
handshake_tickets_ring_1.3_ecdsap384_aes 9.67 ms 9.66 ms -0.00 ms (-0.04%) 1.00%
handshake_session_id_ring_1.3_ecdsap384_aes 9.60 ms 9.59 ms -0.00 ms (-0.03%) 1.00%
handshake_session_id_ring_1.3_ecdsap256_chacha 6.40 ms 6.40 ms -0.00 ms (-0.01%) 1.00%
handshake_session_id_ring_1.3_ecdsap256_aes 6.45 ms 6.45 ms -0.00 ms (-0.00%) 1.08%

Additional information

Historical results

Checkout details:

@djc djc added this pull request to the merge queue Nov 21, 2024
Merged via the queue into main with commit 7d74de2 Nov 21, 2024
58 checks passed
@djc djc deleted the linearize branch November 21, 2024 15:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@cpu cpu cpu approved these changes

@ctz ctz ctz approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@djc @cpu @ctz
0