8000 unstable feature to support targets with no atomic pointers - REJECTED DRAFT PROPOSAL by brody4hire · Pull Request #2200 · rustls/rustls · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

unstable feature to support targets with no atomic pointers - REJECTED DRAFT PROPOSAL #2200

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 13 commits into from

Conversation

brody4hire
Copy link
Contributor
@brody4hire brody4hire commented Nov 8, 2024

STATUS: see TODO items below

Proposal to resolve issue: #2068

This is a proposal to support targets w/o atomic ptr by using portable-atomic Arc & critical-section, as optional features, with help from a recently published contribution: taiki-e/portable-atomic#195

This solution requires use of Rust nightly in order for the required coercion to work with the recent update from taiki-e/portable-atomic#195 ... looks like the stabilization is blocked pending this: rust-lang/rust#18598

Optional features added to rustls/Cargo.toml:

  • portable-atomic-arc - enable use of Arc from portable-atomic-util
  • critical-section - enable critical-section feature on once-cell & portable-atomic-util crates & include portable-atomic-arc feature - this is needed to support build on targets w/o atomic ptr

CI TESTING:

  • this adds a new job to try cross build with critical-section on multiple targets including non-atomic ptr target
  • Some additional CI testing with nightly Rust

COMPARISON TO ALTERNATIVE SOLUTIONS:

  • use alias to Rc for target(s) with no atomic ptr - already proposed in PR ALT DRAFT RFC: add option to use alloc::rc::Rc for no-std targets w/o built-in atomic ops - INITIAL HACK #2088 - seems to imply some API mangling & use of an ugly macro to enable or disable use of Send & Sync traits; possibly harder to document & understand the updated API
  • add & use casting macros to get this working with Rust stable - I have already done this in my work area, seems to imply some worse-looking API mangling, with more documentation burden & possibly harder to understand the updated API

STATEMENT OF IMPACT

This unstable critical-section feature would require using Rust nightly along with the unstable 8000 --cfg portable_atomic_unstable_coerce_unsized build option in order for the build to succeed with this feature.

The one form of impact I can think of is on users building rustls library with --all-features flag. I do NOT think this should affect anyone using rustls as a dependency for a higher-level library or application.

Alternative could be to use an unstable cfg option, like I had to do in taiki-e/portable-atomic#195, but this seems to be inconsistent with the other rustls build options.


XXX / TODO ITEMS:

  • resolve remaining XXX / TODO item(s) in .github/workflows/build.yml
  • resolve remaining documentation TODO item(s) doc items in rustls/src (rustls/src/lib.rs & rustls/src/crypto/mod.rs)
  • I noticed that https://docs.rs/rustls has references into documentation for Rust nightly, thinking these should point into documentation for Rust stable instead (moving this item out of PR top-level sync::Arc alias to support fork for targets with no atomic ptr #2285)
  • check for any other remaining remaining XXX / TODO item(s) in this PR

/cc @bjoernQ @taiki-e

Copy link
codecov bot commented Nov 8, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 94.82%. Comparing base (de8e612) to head (eb923b1).
Report is 26 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #2200      +/-   ##
==========================================
- Coverage   94.82%   94.82%   -0.01%     
==========================================
  Files         104      104              
  Lines       24100    24172      +72     
==========================================
+ Hits        22853    22920      +67     
- Misses       1247     1252       +5     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@brody4hire brody4hire changed the title DRAFT RFC: support targets w/o atomic ptr using portable-atomic Arc - WITH XXX / TODO ITEMS DO NOT MERGE DRAFT: add options to support targets with no atomic pointers... with XXX / TODO ITEMS DO NOT MERGE Nov 25, 2024
@brody4hire brody4hire marked this pull request as ready for review November 25, 2024 16:16
@brody4hire brody4hire force-pushed the portable-atomic-arc-option branch from 8f4a748 to cb94a72 Compare November 25, 2024 16:18
@brody4hire brody4hire marked this pull request as draft November 25, 2024 16:57
@brody4hire brody4hire changed the title DRAFT: add options to support targets with no atomic pointers... with XXX / TODO ITEMS DO NOT MERGE DRAFT: add option to support targets with no atomic pointers... with XXX / TODO ITEMS DO NOT MERGE Nov 27, 2024
@brody4hire brody4hire force-pushed the portable-atomic-arc-option branch from b22e503 to aaad952 Compare November 27, 2024 17:11
@brody4hire brody4hire changed the title DRAFT: add option to support targets with no atomic pointers... with XXX / TODO ITEMS DO NOT MERGE unstable feature to support targets with no atomic pointers - DRAFT DO NOT MERGE YET Dec 13, 2024
@djc
Copy link
Member
djc commented Dec 16, 2024

I think it's unlikely we will want to merge this in something close to its current form. rustls is a small project with a small group of maintainers and we don't necessarily have in-house expertise on embedded platforms and their needs. We also have API compatibility constraints that we want to hold up for the benefit of the (much larger) ecosystem of folks deploying rustls to more general purpose platforms (that is, where std::sync::Arc is available, at least..).

I haven't reviewed this in detail since I don't think it's close to being reviewable, in the sense of having (a) a high-level description of the constraints that made rustls a tricky fit for your platform and how you overcame these challenges and (b) a clean commit history with commits that incrementally change rustls to fit your needs.

Instead of this approach, I think a better solution might be to incrementally modularize rustls more such that you can build on top of the logic that rustls provides without the rustls project having to maintain a bunch of workarounds for the limits of your platform (as in, could we provide an API that doesn't rely on Arc at all?).

@ctz
Copy link
Member
ctz commented Dec 16, 2024

Yes, I agree.

I'm also pretty concerned about incorporating types from another crate in our public API (even behind a feature gate); I feel that would make cutting a rustls 1.0 even harder.

I would perhaps be willing to entertain something like:

diff --git a/rustls/src/lib.rs b/rustls/src/lib.rs
index d8562932..dc428236 100644
--- a/rustls/src/lib.rs
+++ b/rustls/src/lib.rs
@@ -674,6 +674,11 @@ pub mod lock;
 /// Polyfills for features that are not yet stabilized or available with current MSRV.
 pub(crate) mod polyfill;
 
+mod sync {
+    // This crate refers to Arc only via this point.
+    pub(crate) use alloc::sync::Arc;
+}
+
 #[cfg(any(feature = "std", feature = "hashbrown"))]
 mod hash_map {
     #[cfg(feature = "std")]

That would make such a fork much easier to maintain, as the difference would be limited to one place (assuming the replacement for Arc was truly interchangeable.)

@brody4hire
Copy link
Contributor Author

Any pointers how we could start improving the structure of this project?

From a quick look through this project I did already start having some ideas on reducing the amount of dependency on Arc & providing an Arc-less API. Definitely not an overnight project but I will see how much further I can get with these ideas.

I personally may not mind that mod sync idea, though I suspect some others may be disappointerd ... I was really hoping we could find a way to do this without the forking.

@bjoernQ
Copy link
bjoernQ commented Dec 17, 2024

Long-term seeing Rustls being more modular (and offering Arc-free APIs) would be a good thing.

I was also hoping for not going the fork-route but if that fork can be easily maintained that's better than nothing

@djc
Copy link
Member
djc commented Dec 17, 2024

Any pointers how we could start improving the structure of this project?

From a quick look through this project I did already start having some ideas on reducing the amount of dependency on Arc & providing an Arc-less API. Definitely not an overnight project but I will see how much further I can get with these ideas.

In my mind we mostly use Arc to avoid cloning (allocations) and to wrap around trait objects. Both of these cases are kind of accidental complexity in that they are not really necessary for the code to function. It might be possible to have a ClientConfig trait and use generics to abstract over the use of Arc<ClientConfig> (for example, I think we could use abstract types in the handshake state types).

However, this is just spitballing and we'd have to carefully control how to allocate the extra complexity, so whoever takes this on needs to walk a fine line.

@brody4hire
Copy link
Contributor Author
brody4hire commented Dec 17, 2024
+mod sync {
+    // This crate refers to Arc only via this point.
+    pub(crate) use alloc::sync::Arc;
+}

If I would continue with this kind of idea, to help with the possible forking, should I continue in this PR or raise a brand new PR to do this?

In my mind we mostly use Arc to avoid cloning (allocations) and to wrap around trait objects. Both of these cases are kind of accidental complexity in that they are not really necessary for the code to function.

After some trial-and-error in my personal workspace, I am now thinking of an idea to use config builders to build immutable config objects, that could be used from multiple threads, hopefully without all of the atomic locking requirements that come with Arc. I may need some time to consider this further ... I would really love to work on this, cannot guarantee near-term prioirity for this kind of idea.


P.S. I am now working on PR #2285 to propose over-writable Arc alias, hope to have this ready for review soon.

While I do think it would be ideal to offer an option to use once_cell with critical-section feature enabled, I would expect the improvement in terms of performance & resource usage to be negligible.

…, if enabled by `critical-section` feature (with XXX / TODO ITEMS)
@cpu
Copy link
Member
cpu commented Jan 25, 2025

I think #2285 is our preferred way to support targets w/o atomic pointers. Let's close this PR for now since it has conflicts with main and doesn't seem likely to be merge-able in the near future. Thanks!

@cpu cpu closed this Jan 25, 2025
@brody4hire brody4hire changed the title unstable feature to support targets with no atomic pointers - DRAFT DO NOT MERGE YET unstable feature to support targets with no atomic pointers - REJECTED DRAFT PROPOSAL Jan 31, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants
0