8000 'download' does not respect 'scope' as expected CL Arg · Issue #7719 · rucio/rucio · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

'download' does not respect 'scope' as expected CL Arg #7719

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
voetberg opened this issue May 12, 2025 · 0 comments
Open

'download' does not respect 'scope' as expected CL Arg #7719

voetberg opened this issue May 12, 2025 · 0 comments
Labels
bug

Comments

@voetberg
Copy link
Contributor

Description

When using rucio download --dir {scope} --scope {scope} {file} an unknown database error is raised.

This is because "scope" is not parsed as a scope, producing the request data: [{"dids": [{"scope": "file", "name": "file"}], ... instead of the expected [{"dids": [{"scope": "scope "name": "file"}]

This seems to be somewhat intentional behavior (the DID is incorrectly supplied, missing the scope), but the kwargs do not make sense in that sense. If a scope is supplied, it should be used for the DID even if 'filter' is not used.

The argument descriptions for reference

Usage: rucio download [OPTIONS] [DIDS]...

  Download DID(s) (in the form of scope:name) to a local dir

Options:
  --dir TEXT                      The directory to store the downloaded file.
  --allow-tape                    Also consider tape endpoints as source of
                                  the download.
  --rses, --rse-exp TEXT          RSE Expression to specify allowed sources
  --impl TEXT                     Transfer protocol implementation to use
                                  (e.g: xrootd, gfal.NoRename, webdav,
                                  ssh.Rsync, rclone).
  --protocol TEXT                 Force the protocol to use.
  --nrandom INTEGER               Download N random files from the DID.
  --ndownloader INTEGER           Choose the number of parallel processes for
                                  download.
  --no-subdir                     Don't create a subdirectory for the scope of
                                  the files.
  --pfn TEXT                      Specify the exact PFN for the download.
  --no-resolve-archives           If set archives will not be considered for
                                  download.
  --ignore-checksum               Don't validate checksum for downloaded
                                  files.
  --check-local-with-filesize-only
                                  Don't use checksum verification for already
                                  downloaded files, use filesize instead.
  --transfer-timeout FLOAT        Transfer timeout (in seconds). Default:
                                  computed dynamically from --transfer-speed-
                                  timeout. If set to any value >= 0,
                                  --transfer-speed-timeout is ignored.
  --transfer-speed-timeout FLOAT  Minimum allowed average transfer speed (in
                                  KBps). Default: 500. Used to dynamically
                                  compute the timeout if --transfer-timeout
                                  not set. Is not supported for --pfn.
  --aria BOOLEAN                  Use aria2c utility if possible.
                                  (EXPERIMENTAL)
  --filter TEXT                   Filter files by key-value pairs like
                                  guid=2e2232aafac8324db452070304f8d745.
  --scope TEXT                    Scope if you are using the filter option and
                                  no full DID.
  --metalink TEXT                 Path to a metalink file.
  --no-show-download-exceptions   Does not raise NoFilesDownloaded,
                                  NotAllFilesDownloaded or incorrect number of
                                  output queue files Exception.
  --replica-selection TEXT        Select the best replica using a replica
                                  sorting algorithm provided by replica sorter
                                  (e.g., random, geoip).
  -h, --help                      Show this message and exit.

Steps to reproduce

In a testing setup - run

 rse, _ = rse_factory.make_posix_rse()
 scope = mock_scope.external
 tmp_file = did_factory.upload_test_file(rse_name=rse, scope=scope)['name']

cmd = f'rucio download --dir {tmp_dir} --scope {scope} {tmp_file}'

and see the file fails to upload.

Compare to the scope-less version, which works.

rucio download --dir {tmp_dir} {scope}:{tmp_file}

Rucio Version

No response

Additional Information

Full verbose log:

25-05-12 17:02:01,244 DEBUG   baseclient.py   HTTP header:  X-Rucio-Auth-Token: [hidden]
2025-05-12 17:02:01,244 DEBUG   baseclient.py   HTTP header:  X-Rucio-VO: def
2025-05-12 17:02:01,244 DEBUG   baseclient.py   HTTP header:  Connection: Keep-Alive
2025-05-12 17:02:01,244 DEBUG   baseclient.py   HTTP header:  User-Agent: rucio-clients/37.0.0
2025-05-12 17:02:01,244 DEBUG   baseclient.py   HTTP header:  X-Rucio-Script: rucio::-v
2025-05-12 17:02:01,244 DEBUG   baseclient.py   HTTP header:  X-Rucio-Account: root
2025-05-12 17:02:01,255 DEBUG   baseclient.py   HTTP Response: 200 OK
2025-05-12 17:02:01,255 DEBUG   downloadclient.py       schemes: None
2025-05-12 17:02:01,255 DEBUG   downloadclient.py       rse_expression: None
2025-05-12 17:02:01,255 DEBUG   downloadclient.py       impl: None
2025-05-12 17:02:01,255 DEBUG   downloadclient.py       num DIDs for list_replicas call: 1
2025-05-12 17:02:01,256 DEBUG   baseclient.py   HTTP request: POST https://rucio:443/replicas/list
2025-05-12 17:02:01,256 DEBUG   baseclient.py   HTTP header:  X-Rucio-Auth-Token: [hidden]
2025-05-12 17:02:01,256 DEBUG   baseclient.py   HTTP header:  X-Rucio-VO: def
2025-05-12 17:02:01,256 DEBUG   baseclient.py   HTTP header:  Connection: Keep-Alive
2025-05-12 17:02:01,256 DEBUG   baseclient.py   HTTP header:  User-Agent: rucio-clients/37.0.0
2025-05-12 17:02:01,256 DEBUG   baseclient.py   HTTP header:  X-Rucio-Script: rucio::-v
2025-05-12 17:02:01,256 DEBUG   baseclient.py   HTTP header:  X-Rucio-Account: root
2025-05-12 17:02:01,256 DEBUG   baseclient.py   HTTP header:  Accept: application/metalink4+xml
2025-05-12 17:02:01,256 DEBUG   baseclient.py   Request data (length=349): [{"dids": [{"scope": "file_c725e46d3fd342edb3bd8329bc6063b5", "name": "file_c725e46d3fd342edb3bd8329bc6063b5"}], "doma ... ts": true}]
2025-05-12 17:02:01,314 DEBUG   baseclient.py   HTTP Response: 500 INTERNAL SERVER ERROR
2025-05-12 17:02:01,314 DEBUG   baseclient.py   Response text (length=106): [{"ExceptionClass": "DatabaseException", "ExceptionMessage": "An unknown Database Exception has occurred."}]
2025-05-12 17:02:01,315 ERROR   Database exception.
Details: An unknown Database Exception has occurred.
@voetberg voetberg added the bug label May 12, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@voetberg
0