8000 Nf-test for merged nf-core template v3.3.1 by brovolia · Pull Request #246 · nf-core/bacass · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

Nf-test for merged nf-core template v3.3.1 #246

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 18 commits into from
Jul 2, 2025
Merged

Conversation

brovolia
Copy link

I resolved merging conflicts and tested the updated pipeline. The test completed successfully on our HPC cluster with Singularity support.

@brovolia brovolia requested a review from d4straub June 23, 2025 07:26
@d4straub d4straub mentioned this pull request Jun 23, 2025
Copy link
Collaborator
@d4straub d4straub left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So I have a few suggestions for changes. Those can be directly applied to this PR if you do not disagree with any. Let me know if you want to do it or I should add those.

@d4straub
Copy link
Collaborator

Alright, now the failing tests have meaningful output:

Copy link
Collaborator
@d4straub d4straub left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Your progress looks good to me!
However, there might be still many md5sums that are not stable. So I think .nftignore needs a bunch of more entries. Such as files with .checkpoint, files in busco/busco_downloads/. But you will get there.

@brovolia
Copy link
Author

Thanks for the hint! I already added the line
tests/default.nf.test.snap:.md5,.
to .nftignore to ignore lines with md5sums in the default.nf.test.snap.
However, there might still be some unstable outputs. You mentioned files like .checkpoint or those under busco/busco_downloads/. Could you let me know where I can check for these files or how I can systematically identify such outputs to extend the .nftignore list?

@d4straub
Copy link
Collaborator

Could you let me know where I can check for these files or how I can systematically identify such outputs to extend the .nftignore list?

Check out the tests that are running here automatically, e.g.
Run nf-test / docker | 24.04.2 | 1/1 (pull_request)
follow the link and scroll down, there you will see a list of files that had differing md5sums. I would put all that files into .nftignore and re-generate the .nf.test.snap file. Two or even three iterations might be needed, but that way all unstable files should be ignored eventually.

@d4straub
Copy link
Collaborator

Al 10000 most there, only 2 md5sums left that differ, looks good!

@brovolia
Copy link
Author

I saw in the dev ampliseq branch that you created many snapshots for different test profiles. Is it necessary for bacass as well?

@d4straub
Copy link
Collaborator

Hi, ultimately it would be good to add one .nf.test and .nf.test.snap file pair per test config. However, I think its fine to split template update + one test and the remaining tests into separate PRs. So I'd say this PR should be merged now (I'll review, and @Daniel-VM as an active maintainer should maybe also have the chance to look at it) and a separate PR cn then contain the remaining tests.

Copy link
Collaborator
@d4straub d4straub left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Template update and test look good to me.

I am however concerned about the addition of the parameter discard_trimmed_pass because it appear to me it should never be true anyway. @Daniel-VM what do you think?

Comment on lines -55 to +60
false, // we don't use discard_trimmed_pass at the moment
val_discard_trimmed_pass,
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking at the descripton of that param discard_trimmed_pass in subworkflows/nf-core/fastq_trim_fastp_fastqc/meta.yml, I'd say this param may never be true in this pipeline. Therefore I am not sure this change here is appropriate. Maybe this should have been kept here as false in any case?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You are right, this param should be set to "false"

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But I have already set them as false in nextflow.config

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, so the pipeline works as expected with this default setting. However, if anyone for any reason sets this to true, then the pipeline will fail, as far as I get it. Did you test that?
If the pipeline indeed cannot handle when --discard_trimmed_pass is used, as we suspect, then it makes no sense to allow this option.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tested it with following command
nextflow run path/to/bacass/main.nf -profile test,singularity --discard_trimmed_pass true --outdir results
it worked well.
As I understood, without this option it will be difficult to pass nf-core linting, as it requested from me the updated fastq_trim_fastp_fastqc subworkflow. What I can propose is to hide this option with "hidden": true in nextflow_schema.json

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, that is unexpected to me. Then I would prefer a proper description what the parameter does in the nextflow_schema.json, if thats unclear, then hide it.

@Daniel-VM
Copy link
Contributor

Hii!! Looks good.

Hi, ultimately it would be good to add one .nf.test and .nf.test.snap file pair per test config. However, I think its fine to split template update + one test and the remaining tests into separate PRs. So I'd say this PR should be merged now (I'll review, and @Daniel-VM as an active maintainer should maybe also have the chance to look at it) and a separate PR cn then contain the remaining tests.

Yes, totally agree. The remaining test can go in a separate PR.

Copy link
Collaborator
@d4straub d4straub left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok I think thats fine here for merging. Thanks a lot for your effort!

@Daniel-VM
Copy link
Contributor

Thanks for the amazing work, @brovolia . Could you go ahead and merge this PR (green button "merge pull request")?

Many thanks

@brovolia brovolia merged commit 5e58da7 into nf-core:dev Jul 2, 2025
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants
0