.command.run permission denied in 3.16.1, 3.17.0, 3.18.0 #1554

givtrah · 2025-05-13T20:11:27Z

Description of the bug

New versions of rnaseq pipeline fails when running the test pipeline under docker (and singularity, conda was not tested). 3.16.1, 3.17.0 and 3.18.0 is affected. 3.16.0 and 3.14.0 tested fine.

Command used and terminal output

run nf-core/rnaseq -r 3.16.0 -profile test,docker --outdir testout

Works

run nf-core/rnaseq -r 3.16.1 -profile test,docker --outdir testout

not working (3.17.0 and 3.18.0 gives the same error):

-[nf-core/rnaseq] Pipeline completed with errors-
WARN: Directive `process.shell` cannot contain new-line characters - offending value: [bash

set -e # Exit if a tool returns a non-zero status/exit code
set -u # Treat unset variables and parameters as an error
set -o pipefail # Returns the status of the last command to exit with a non-zero status or zero if all successfully execute
]
ERROR ~ Error executing process > 'NFCORE_RNASEQ:PREPARE_GENOME:GUNZIP_ADDITIONAL_FASTA (gfp.fa.gz)'

Caused by:
  Process `NFCORE_RNASEQ:PREPARE_GENOME:GUNZIP_ADDITIONAL_FASTA (gfp.fa.gz)` terminated with an error exit status (126)


Command executed:

  # Not calling gunzip itself because it creates files
  # with the original group ownership rather than the
  # default one for that user / the work directory
  gzip \
      -cd \
       \
      gfp.fa.gz \
      > gfp.fa
  
  cat <<-END_VERSIONS > versions.yml
  "NFCORE_RNASEQ:PREPARE_GENOME:GUNZIP_ADDITIONAL_FASTA":
      gunzip: $(echo $(gunzip --version 2>&1) | sed 's/^.*(gzip) //; s/ Copyright.*$//')
  END_VERSIONS

Command exit status:
  126

Command output:
  (empty)

Command error:
  /bin/bash: line 10: .command.run: Permission denied

Work dir:
  /home/givtrah/test/work/f5/a89cb7f2a3d6a658b5cdc768a35afe

Container:
  quay.io/nf-core/ubuntu:22.04

Tip: when you have fixed the problem you can continue the execution adding the option `-resume` to the run command line

 -- Check '.nextflow.log' file for details
ERROR ~ Pipeline failed. Please refer to troubleshooting docs: https://nf-co.re/docs/usage/troubleshooting

 -- Check '.nextflow.log' file for details

Relevant files

No response

System information

No response

lisch7 · 2025-05-14T01:52:39Z

Same Issue. I thought it was an internet problem, but it looks like someone else is having the same problem I am!

givtrah · 2025-05-14T05:39:57Z

I spent two days trying to figure out where my permission problem was, all to no avail until I recalled last time I ran it just fine using a previous version and... voila (also, great fun first troubleshooting that sample names for some reason can no longer be numeric when the error said something about a space in the filename.. sigh).

Of course it could still be specific to my setup, so good to hear I'm not alone in having this problem.

lisch7 · 2025-05-14T11:20:55Z

I spent two days trying to figure out where my permission problem was, all to no avail until I recalled last time I ran it just fine using a previous version and... voila (also, great fun first troubleshooting that sample names for some reason can no longer be numeric when the error said something about a space in the filename.. sigh).

Of course it could still be specific to my setup, so good to hear I'm not alone in having this problem.

Our experience is very similar, I have no problem at all if I specify version 3.16.0. Also, I have installed other piplines such as fetchings, sarek, differntialabundance,all without problems. But it's worth mentioning that I've installed 3.18.0 successfully before, but because of a slight problem with docker, I deleted all the containers, but it didn't work when I pulled it again

fbartusch · 2025-05-14T11:46:37Z

A user on our HPC-cluster has the same problem. I also thought our setup was the problem until I found this issue.

sebastiantonn · 2025-05-14T12:44:29Z

Same issue.
3.17.0 gives error like above, 3.16.0 seems to work.
Did not test other versions.

MatthiasZepper · 2025-05-14T16:07:27Z

It is the first time that I hear about all those issues, but the

Directive process.shell cannot contain new-line characters - offending value: [bash

part of your error message piqued my interest. Searching for issues in the nf-core organization quickly brought up this template fix by @ewels and an associated Nextflow issue.

Admittedly, I do not know what I can recommend to you to act on the problem. Since a change in the pipeline template was required, once probably has to consider these pipeline versions incompatible with certain Nextflow versions?

But maybe just providing a custom config with

// Set bash options
process.shell = [
    "bash",
    "-C",         // No clobber - prevent output redirection from overwriting files.
    "-e",         // Exit if a tool returns a non-zero status/exit code
    "-u",         // Treat unset variables and parameters as an error
    "-o",         // Returns the status of the last command to exit..
    "pipefail"    //   ..with a non-zero status or zero if all successfully execute
]

to nextflow run suffices. Not tested, though...

I think, this warrants a blog post from the nf-core maintainers once there is a good solution in place.

ewels · 2025-05-15T07:30:53Z

It looks like the dev branch of this pipeline already has the fix:

rnaseq/nextflow.config

Lines 307 to 315 in 15c160f

    
           // Set bash options 
        
           process.shell = [ 
        
               "bash", 
        
               "-C",         // No clobber - prevent output redirection from overwriting files. 
        
               "-e",         // Exit if a tool returns a non-zero status/exit code 
        
               "-u",         // Treat unset variables and parameters as an error 
        
               "-o",         // Returns the status of the last command to exit.. 
        
               "pipefail"    //   ..with a non-zero status or zero if all successfully execute 
        
           ]

So if you run the pipeline with -r dev it'll probably work. But we should prioritise a release ASAP.

This particular error only shows up when several specific criteria are met:

Running with Nextflow 24.12.0-edge or later (so likely coming up now because of the 25.04 stable release)
Pipeline has nf-core template update >= 3.0.0 and < 3.2.0

Fixes are:

Use an older version of Nextflow
Use an older version of the pipeline
Release a new version of the pipeline with the included fix

An explanation with more detail of what's going on can be found here: nf-core/tools#3416 (comment)

ewels · 2025-05-15T09:43:29Z

oh, and can also overwrite the above syntax with a custom config too - probably the easiest fix!

Untested, but:

nf_config_fix.config

 // Set bash options 
 process.shell = [ 
     "bash", 
     "-C",         // No clobber - prevent output redirection from overwriting files. 
     "-e",         // Exit if a tool returns a non-zero status/exit code 
     "-u",         // Treat unset variables and parameters as an error 
     "-o",         // Returns the status of the last command to exit.. 
     "pipefail"    //   ..with a non-zero status or zero if all successfully execute 
 ]

nextflow run nf-core/rnaseq -c nf_config_fix.config ...

givtrah · 2025-05-16T09:54:04Z

Thanks for all the suggestions and for getting back to me about this bug.

I can confirm that running the dev branch using -r dev works for both docker and singularity containers.

For now I just used 3.16.0.

arinwongprommoon · 2025-05-19T12:11:33Z

@ewels Just commenting to say thank you for suggesting a custom config (in #1554 (comment)).

I came across a very similar issue with a different Nextflow-based pipeline (avantonder/assembleBAC#28), and appending my current config with your text seems to have gotten me past the error. I was using Nextflow 24.10.5.

Hopefully this is helpful, at least in letting people know how widespread the issue is.

ewels · 2025-05-19T15:47:23Z

Yup, looks like that pipeline was originally based on the nf-core template. So even though it's not an nf-core pipeline, it's the same situation.

nttg8100 · 2025-05-20T14:54:50Z

I think the unhappy test cases for this feature is missing when the process.shell is provided via config. Hope to see new test cases to cover this on new release.
https://github.com/nextflow-io/nextflow/blob/9e675c6aba28f5b2a76217b1d97e4ef5da4e6f81/modules/nextflow/src/main/groovy/nextflow/processor/TaskConfig.groovy#L392C4-L406C6
https://github.com/nextflow-io/nextflow/blob/9e675c6aba28f5b2a76217b1d97e4ef5da4e6f81/modules/nextflow/src/test/groovy/nextflow/processor/TaskConfigTest.groovy#L35C5-L53C6

MatthiasZepper · 2025-05-21T14:38:33Z

I think the unhappy test cases for this feature is missing when the process.shell is provided via config. Hope to see new test cases to cover this on new release. https://github.com/nextflow-io/nextflow/blob/9e675c6aba28f5b2a76217b1d97e4ef5da4e6f81/modules/nextflow/src/main/groovy/nextflow/processor/TaskConfig.groovy#L392C4-L406C6 https://github.com/nextflow-io/nextflow/blob/9e675c6aba28f5b2a76217b1d97e4ef5da4e6f81/modules/nextflow/src/test/groovy/nextflow/processor/TaskConfigTest.groovy#L35C5-L53C6

I suggest forking the Nextflow repository and opening a pull request with the test cases that you have in mind. I am sure that this is the best way to get them in or at least find out, if they have deliberately been omitted.

nttg8100 · 2025-05-22T16:21:04Z

I fix the test cases, rewrite validate shell function to raise error @MatthiasZepper @ewels
nextflow-io/nextflow#6114

gibcus · 2025-05-23T14:50:08Z

I had the same issue. Thanks for the version suggestion!

givtrah added the bug Something isn't working label May 13, 2025

arinwongprommoon mentioned this issue May 19, 2025

Shovill fails with exit status 126 and raising .command.run: Permission denied avantonder/assembleBAC#28

Open

bk1n mentioned this issue May 21, 2025

No module named bioconda::fq #1555

Open

JosuaCarl mentioned this issue May 22, 2025

process.shell cannot contain new-line characters nf-core/demo#37

Open

nttg8100 mentioned this issue May 22, 2025

fix test cases for validating shell nextflow-io/nextflow#6114

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

.command.run permission denied in 3.16.1, 3.17.0, 3.18.0 #1554

.command.run permission denied in 3.16.1, 3.17.0, 3.18.0 #1554

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

.command.run permission denied in 3.16.1, 3.17.0, 3.18.0 #1554

.command.run permission denied in 3.16.1, 3.17.0, 3.18.0 #1554

Comments

Uh oh!

Description of the bug

Command used and terminal output

Relevant files

System information

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!