8000 fix(Locker): don't store transport-options.ssl within the lock-file by rtm-ctrlz · Pull Request #12019 · composer/composer · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

fix(Locker): don't store transport-options.ssl within the lock-file #12019

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 5 commits into from
Jul 10, 2024

Conversation

rtm-ctrlz
Copy link
Contributor
@rtm-ctrlz rtm-ctrlz commented Jun 24, 2024

Description

Don't store transport-options.ssl within the lock-file otherwise the installation or updates will (in most cases) fail for repositories that use client TLS certificates, because during installation transport-options from composer.lock will be used instead of repository settings (certificates) from composer configuration file.


Example

Minimal configuration for case "private registry with client-certificates" (~/.composer/config.json)

{
    "repositories": {
        "private.registry": {
            "type": "composer",
            "url": "https://composer.registry.example.com",
            "options": {
                "ssl": {
                    "local_cert": "/home/user/.composer/certs.d/private.cert",
                    "local_key": "/home/user/.composer/certs.d/private.key"
                }
            }
        }
    }
}

Dummy project (~/project/composer.json)

{
    "name": "exmaple/dummy",
    "type": "project",
    "require": {
        "private-repo/package": "^1.0"
    }
}

Dummy project lock (~/project/compose.lock) (with current composer)

{
    "content-hash": "...",
    "packages": [
        {
            "name": "private-repo/package",
            "version": "1.0.0",
            "transport-options": {
                "ssl": {
                    "local_cert": "/home/user/.composer/certs.d/private.cert",
                    "local_key": "/home/user/.composer/certs.d/private.key"
                }
            }
        }
    ]
}

In this case installation/update will fail if the environment performing such task will have different configuration (~/.composer/config.json), because path to certificate will be different and that will lead to cURL error message ("wrong certificate").

Some uses cases when this will could happen:

  • storing composer.log within VCS and 2+ developers with different composer configurations
  • CI installations

Fixed lock-file

With this change lock-file (~/project/compose.lock) will look like this:

{
    "content-hash": "...",
    "packages": [
        {
            "name": "private-repo/package",
            "version": "1.0.0"
        }
    ]
}

So composer will use TLS transport-options from configuration and installation will succeed.

Is it breaking change?

I suppose this change as non-breaking, because it may will break in only one case - composer.lock file is somehow in sync with tls certificates in all places calling install/update and at the same time composer configuration still contains repository configuration, but is out of sync with certificates path; for me this approach is looking quite strange for me because it is "forming install environment around lock-file".

Is it te 8000 sted? - Yes

  • local tests with different TLS-configurations and other transport options
  • within my company CI (patched version)

@@ -431,6 +431,15 @@ private function lockPackages(array $packages): array

$spec = $this->dumper->dump($package);
unset($spec['version_normalized']);
// remove transport-options.ssl from lock file
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The comment states whats obvious from the code but does not explain 'why'

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Initially there was a detailed comment but It took several lines to have the answer.

But I agree that the current comment is meaningless, so I see some options:

  1. remove this comment and let other developers try to guess the reason
  2. put the full description there, so comment will have the answer
  3. add link to this PR so the answer could be found

@@ -431,6 +431,14 @@ private function lockPackages(array $packages): array

$spec = $this->dumper->dump($package);
unset($spec['version_normalized']);
// remove `transport-options.ssl` from lock file to prevent storing
// local-filesystem repo config paths in the lock file as that makes it less portable
if (isset($spec['transport-options']['ssl'])) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note: you don't need to check with isset if transport-options is set first, then ssl, you can do both at once, isset($random->deep['garbage']['not']->existing); works fine :) https://3v4l.org/UAaJW

@Seldaek
Copy link
Member
Seldaek commented Jul 10, 2024

Thanks yes that probably makes sense.

@Seldaek Seldaek merged commit 03bbfdd into composer:main Jul 10, 2024
19 of 20 checks passed
@Seldaek Seldaek added this to the 2.7 milestone Jul 10, 2024
@Seldaek Seldaek added the Bug label Jul 10, 2024
@naderman
Copy link
Member

It's entirely unclear to me how this is supposed to work. Once a package is in the lock file, you can't tell which repository it originally came from during a composer install from lock, so you can't tell what TLS options to apply.

All the settings necessary to download and install a package must be in the lock file, which of course includes TLS options. If you need to update TLS options in the lock file, this should be possible by running "composer update --lock" without modifying any versions.

Think this needs to be reverted. I don't understand the premise in this bug either. What's wrong with having the lock file point at those files? It's a very unusual set up to define repositories in a config file rather than the composer.json can you explain the use case there a bit more and what the problem even is?

@Seldaek
Copy link
Member
Seldaek commented Sep 13, 2024

@rtm-ctrlz A few more questions:

  • Do you just run composer update all the time? Or do you actually use the lock file?
  • Did this patch make composer install break for you as the lock file is missing the cert auth info now?
  • If you only run update, maybe you'd be better off without a lock file, by setting composer config lock false you can disable lock file creation.

Please note if we do not get any feedback here we will have to revert this from the next Composer release.

@rtm-ctrlz
Copy link
Contributor Author
rtm-ctrlz commented Sep 13, 2024

@Seldaek

  • Do you just run composer update all the time? Or do you actually use the lock file?

    we actully use lock file

  • Did this patch make composer install break for you as the lock file is missing the cert auth info now?

    no, because path to certificate is provided by ~/.composer/config.json

  • If you only run update, maybe you'd be better off without a lock file, by setting composer config lock false you can disable lock file creation.

    we need lock file, so we can't disable it

The goal of this change is not to store path of private certificates within composer.lock.

Our setup looks like this:

  • private package registry with TLS personal certificate authorisation
  • several developers, each has own location of the certificate, here is an example of ~/.composer/config.json:
    "repositories": {
        "registry.out.company.tld": {
            "type": "composer",
            "url": "https://registry.out.company.tld",
            "options": {
                "ssl": {
                    "local_cert": "/home/user/.config/certs.d/composer.cert"
                }
            }
        }
    }
    
    we use ~/.composer/config.json to specify the registry to keep it in one place, because we have number of private packages and in this case we need to specify it only one time, not for "any package that uses any of private packages", and it is more maintainable;
    storing registry configuration within composer.json will force to "hardcode" path to private certificate for all (developers and even CI)
  • code and composer.lock are stored with git
  • applications are "build" within CI, so we need to use composer.lock

Without this change we have several issues:

  • unnesesery changes in composer.lock made by any developer that need to install (even not update) the dependencies after using composer update --lock and having a need to merge these changes over and over again
  • CI job "build application" doesn't have access to git for private packages, but has own private certificate and proper ~/.composer/config.json, but composer update --lock fails in this case because it works like this:
    • tries to get package from registry (using certificate path from composer.lock) - in our case it will fail because path from lock points to "path on developers machine" and path from ~/.composer/config.json isn't used
    • tries to get package from git - in our case it will fail because there is no access to git for this job
    • as the result composer fails to install dependencies (wrong certificate path, no access to git)

My point is that there no need to store tls-options with composer.lock while they are specified with ~/.composer/config.json.

@stof
Copy link
Contributor
stof commented Sep 13, 2024

Does downloading dist archives from your custom repository requires to provide the certificate or only reading the metadata ?

@rtm-ctrlz
Copy link
Contributor Author

@stof both, in a nut shell there is NGinx with certificate verification serving static files generated by composer/satis

@stof
Copy link
Contributor
stof commented Sep 13, 2024

@rtm-ctrlz I suggest you to confirm whether this is actually true when performing an installation from lock file (without a cache of the downloaded archive). With this patch applied, there is no way for composer to associate the locked package to a TLS certificate it would add to send (as it is not associated to your custom repository in ~/.composer/config.json)

@rtm-ctrlz
Copy link
Contributor Author

@stof having specified local_cert within lock file will force compose to use specified file as certificate for requests to registry:

  • registry requires client certificate
  • composer install (curl) can't find certificate specified by lock (because different environment ex. "developer vs CI")
  • composer failing to fetch package

I'm not sure on how can I confirm this behaviour.

@stof
Copy link
Contributor
stof commented Sep 13, 2024

@rtm-ctrlz your setup of having the repository specified outside the project is quite specific though. Wanting to have a reusable lock file based on different repository definitions looks weird.

And this PR still does not result in Composer passing the TLS certificate from the global config but passing no TLS certificate.

I'm not sure on how can I confirm this behaviour.

Try downloading the archive with curl without providing a certificate, maybe.

@rtm-ctrlz
Copy link
Contributor Author
rtm-ctrlz commented Sep 13, 2024

@stof Indeed there is a lack of our configuration and private registry worked without requiring certificate, thanks for pointing.

With our private registry got fixed (now it requires client certificate, as intended) I'm facing that certificate is used by composer only for getting metadata, but not the dist-files, so clean install (no composer.lock, no vendor directory, empty composer cache) fails.
I made a test with pre-PR version:

  • empty package (compose init) with single dependency from private registry
  • clean install works, path to certificate is stored in lock file
  • (no vendor, no composer cache, but having lock file with path to certificate) after changing certificate path (and updating repositories) running composer update --lock or composer install with both pre-PR and current version are failing to install dependency.

Moving repositories configuration between composer.json and ~/.composer/config.json do not change anything at all.

@rtm-ctrlz
Copy link
Contributor Author

setup of having the repository specified outside the project is quite specific though.

I'd prefer to see it at "global composer configuration, for developer/CI" - most our work is done with private packages and I can't see the reason to "specify repository in composer.json for every package and in case of change update them all" instead of "setup workspace (composer) with repository configuration in one place and use for every package we develop".

Wanting to have a reusable lock file based on different repository definitions looks weird.

Indeed, we are trying to achieve (without success for now) configuration when composer is using path to certificate from composer's configuration, but not from lock file, because in our case (as I described above) using certificate from lock will lead to failures of installation (ex. two developers stores their certificates within home directory, but have different usernames, so the path to certificate will also be different).

@stof
Copy link
Contributor
stof comm 8000 ented Sep 13, 2024

I'd prefer to see it at "global composer configuration, for developer/CI" - most our work is done with private packages and I can't see the reason to "specify repository in composer.json for every package and in case of change update them all" instead of "setup workspace (composer) with repository configuration in one place and use for every package we develop".

This still expect that all developers have an equivalent repository defined in their global config instead of making the project self-contained.

I think your use case should be achieved by adding a new authentication type to Composer (authenticating with a client certificate), so that auth.json (which is not committed) can associate a client certificate to a domain (which is the basis for the Composer authentication system). This would allow each dev to have their own configuration for the client certificate.

@rtm-ctrlz
Copy link
Contributor Author
rtm-ctrlz commented Sep 13, 2024

This still expect that all developers have an equivalent repository defined in their global config instead of making the project self-contained.

Agree, but having less developers than packages makes updating developers configs easer than updating composer.json for every package.

I'll have a look towards implementing such authentication type for auth.json, thanks.

@Seldaek
Copy link
Member
Seldaek commented Sep 16, 2024

Ok in the meantime I will revert this as the patch is clearly broken.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants
0