8000 Fix crash with `pip==25.1`, which changed caching for find_all_candidates by gkreitz · Pull Request #2178 · jazzband/pip-tools · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

Fix crash with pip==25.1, which changed caching for find_all_candidates #2178

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 2 commits into
base: main
Choose a base branch
from

Conversation

gkreitz
Copy link
@gkreitz gkreitz commented Apr 29, 2025

In pip==25.1, the implementation of caching for find_all_candidates changed, and it no longer exposes any method in its public interface to clear its cache. This causes pip-compile --generate-hashes to crash with an AttributeError.

In this PR, I replace the cache clearing with simply constructing a new object. I'm not sufficiently familiar with pip/pip-tools internals to know if that's a bad idea (it's unclear to me how much state lies in the finder object, beyond the cache), but I felt it was potentially cleaner than the alternative approach (catch the AttributeError and do finder._all_candidates.clear()) suggested in #2176.

Fixes #2176

Contributor checklist
  • Included tests for the changes.
  • PR title is short, clear, and ready to be included in the user-facing changelog.
Maintainer checklist
  • Verified one of these labels is present: backwards incompatible, feature, enhancement, deprecation, bug, dependency, docs or skip-changelog as they determine changelog listing.
  • Assign the PR to an existing or new milestone for the target version (following Semantic Versioning).

Copy link
Member
@sirosen sirosen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've just been reading through PackagerFinder in pip and I think this fix will work cleanly. I've hit the CI run button and let's see if it works cleanly -- obviously you don't need to add any new tests, since this will hopefully fix already-broken tests! 😄

@sirosen
Copy link
Member
sirosen commented May 20, 2025

I'm going to look into what's going on with all of these failing builds as soon as I have time. I'm pretty sure they have nothing to do with this PR, and are instead caused by some non-reproducible elements in the build envs.

We'll potentially merge this even with CI red; like I said, I'm pretty sure it's right and we can back it out if it turns out to be an issue after we unwind the other CI woes.

@AndydeCleyre
Copy link
Contributor
AndydeCleyre commented May 20, 2025

I think we need to remove the keyword option mix_stderr here in conftest.py:

cli_runner = CliRunner(mix_stderr=False)

AFAIU that option has done nothing for a while and is no longer part of Click.

EDIT:

From Click's 8.2.0 changelog:

Keep stdout and stderr streams independent in CliRunner. Always collect stderr output and never raise an exception. Add a new output stream to simulate what the user sees in its terminal. Removes the mix_stderr parameter in CliRunner. #2522 #2523

@sirosen
Copy link
Member
sirosen commented May 21, 2025

Yes, we will need to handle click 8.2+ changes. Since pip-tools still has reason to support Python 3.8, I think the appropriate short-term solution is to pin to click>=8.1,<8.2. We can potentially follow the guidance in https://click.palletsprojects.com/en/stable/support-multiple-versions/ , but right now pip-tools can be broken unless you pin it's dependencies1 -- so the priority is to get it fixed before we start making any additional changes.

Handling will depend on how exactly things go as I wade through the space of unconstrained packages in the tests.
Based on what I was most recently looking at in CI, I don't think click is the only or even the primary issue. But pinning it does seem to be helping my local testing exploration, so this might be the right way to start tackling things.

Footnotes

  1. There is some inherent humor in this, seeing as pip-tools is primarily useful for making things -- including CI -- highly reproducible...

@AndydeCleyre
Copy link
Contributor

I'd like to clear the way for fixing #2176 (thanks!) and #2131, so have put up #2183.

@sirosen sirosen force-pushed the 2176_fix_crash_with_pip_25_1 branch from a94047f to 4c46b71 Compare June 23, 2025 16:40
@sirosen sirosen added bug Something is not working pip Related to pip labels Jun 23, 2025
@sirosen sirosen added this to the 7.4.2 milestone Jun 23, 2025
@sirosen sirosen mentioned this pull request Jun 24, 2025
4 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something is not working pip Related to pip
Projects
None yet
Development

Successfully merging this pull request may close these issues.

pip-compile errors when generating hashes with AttributeError 'cache_clear' with pip v25.1
3 participants
0