8000 JP-3829: Add flat handling to clean_flicker_noise by melanieclarke · Pull Request #9064 · spacetelescope/jwst · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

JP-3829: Add flat handling to clean_flicker_noise #9064

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 13 commits into from
Jan 23, 2025

Conversation

melanieclarke
Copy link
Collaborator
@melanieclarke melanieclarke commented Jan 9, 2025

Resolves JP-3829

Add an option to the clean_flicker_noise step to divide by a flat image before fitting background/noise, and restore it after removing the noise.

Tasks

  • request a review from someone specific, to avoid making the maintainers review every PR
  • add a build milestone, i.e. Build 11.3 (use the latest build if not sure)
  • Does this PR change user-facing code / API? (if not, label with no-changelog-entry-needed)
    • write news fragment(s) in changes/: echo "changed something" > changes/<PR#>.<changetype>.rst (see below for change types)
    • update or add relevant tests
    • update relevant docstrings and / or docs/ page
    • start a regression test and include a link to the running job (click here for instructions)
      • Do truth files need to be updated ("okified")?
        • after the reviewer has approved these changes, run okify_regtests to update the truth files
  • if a JIRA ticket exists, make sure it is resolved properly
news fragment change types...
  • changes/<PR#>.general.rst: infrastructure or miscellaneous change
  • changes/<PR#>.docs.rst
  • changes/<PR#>.stpipe.rst
  • changes/<PR#>.datamodels.rst
  • changes/<PR#>.scripts.rst
  • changes/<PR#>.fits_generator.rst
  • changes/<PR#>.set_telescope_pointing.rst
  • changes/<PR#>.pipeline.rst

stage 1

  • changes/<PR#>.group_scale.rst
  • changes/<PR#>.dq_init.rst
  • changes/<PR#>.emicorr.rst
  • changes/<PR#>.saturation.rst
  • changes/<PR#>.ipc.rst
  • changes/<PR#>.firstframe.rst
  • changes/<PR#>.lastframe.rst
  • changes/<PR#>.reset.rst
  • changes/<PR#>.superbias.rst
  • changes/<PR#>.refpix.rst
  • changes/<PR#>.linearity.rst
  • changes/<PR#>.rscd.rst
  • changes/<PR#>.persistence.rst
  • changes/<PR#>.dark_current.rst
  • changes/<PR#>.charge_migration.rst
  • changes/<PR#>.jump.rst
  • changes/<PR#>.clean_flicker_noise.rst
  • changes/<PR#>.ramp_fitting.rst
  • changes/<PR#>.gain_scale.rst

stage 2

  • changes/<PR#>.assign_wcs.rst
  • changes/<PR#>.badpix_selfcal.rst
  • changes/<PR#>.msaflagopen.rst
  • changes/<PR#>.nsclean.rst
  • changes/<PR#>.imprint.rst
  • changes/<PR#>.background.rst
  • changes/<PR#>.extract_2d.rst
  • changes/<PR#>.master_background.rst
  • changes/<PR#>.wavecorr.rst
  • changes/<PR#>.srctype.rst
  • changes/<PR#>.straylight.rst
  • changes/<PR#>.wfss_contam.rst
  • changes/<PR#>.flatfield.rst
  • changes/<PR#>.fringe.rst
  • changes/<PR#>.pathloss.rst
  • changes/<PR#>.barshadow.rst
  • changes/<PR#>.photom.rst
  • changes/<PR#>.pixel_replace.rst
  • changes/<PR#>.resample_spec.rst
  • changes/<PR#>.residual_fringe.rst
  • changes/<PR#>.cube_build.rst
  • changes/<PR#>.extract_1d.rst
  • changes/<PR#>.resample.rst

stage 3

  • changes/<PR#>.assign_mtwcs.rst
  • changes/<PR#>.mrs_imatch.rst
  • changes/<PR#>.tweakreg.rst
  • changes/<PR#>.skymatch.rst
  • changes/<PR#>.exp_to_source.rst
  • changes/<PR#>.outlier_detection.rst
  • changes/<PR#>.tso_photometry.rst
  • changes/<PR#>.stack_refs.rst
  • changes/<PR#>.align_refs.rst
  • changes/<PR#>.klip.rst
  • changes/<PR#>.spectral_leak.rst
  • changes/<PR#>.source_catalog.rst
  • changes/<PR#>.combine_1d.rst
  • changes/<PR#>.ami.rst

other

  • changes/<PR#>.wfs_combine.rst
  • changes/<PR#>.white_light.rst
  • changes/<PR#>.cube_skymatch.rst
  • changes/<PR#>.engdb_tools.rst
  • changes/<PR#>.guider_cds.rst

@melanieclarke
Copy link
Collaborator Author
melanieclarke commented Jan 9, 2025

Regtests here:
https://github.com/spacetelescope/RegressionTests/actions/runs/12699898959

No changes in regtests, as expected. The new optional parameter is covered by unit tests.

Copy link
codecov bot commented Jan 9, 2025

Codecov Report

Attention: Patch coverage is 97.82609% with 1 line in your changes missing coverage. Please review.

Project coverage is 78.23%. Comparing base (18359b6) to head (cbba455).
Report is 907 commits behind head on main.

Files with missing lines Patch % Lines
jwst/nsclean/nsclean_step.py 0.00% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #9064      +/-   ##
==========================================
+ Coverage   78.21%   78.23%   +0.01%     
==========================================
  Files         505      505              
  Lines       46204    46249      +45     
==========================================
+ Hits        36139    36183      +44     
- Misses      10065    10066       +1     
Flag Coverage Δ *Carryforward flag
nightly 77.79% <ø> (ø) Carriedforward from 4d59f89

*This pull request uses carry forward flags. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@melanieclarke melanieclarke marked this pull request as ready for review January 10, 2025 19:01
@melanieclarke melanieclarke requested review from a team as code owners January 10, 2025 19:01
Copy link
Collaborator
@emolter emolter left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm probably not the right person to approve this PR, but still reviewed it in the spirit of using review as a way to learn the code. My two comments are more like questions about how things work than suggestions for changes.

@melanieclarke melanieclarke force-pushed the jp-3829 branch 2 times, most recently from b9c71f5 to cc261a2 Compare January 13, 2025 16:50
Copy link
Contributo 8000 r
@tapastro tapastro left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, with one minor request:

The reference file documentation has a pair of tables that map reftypes to steps and vice-versa. Could the flat <-> clean_flicker_noise connection be added?
https://jwst-pipeline.readthedocs.io/en/latest/jwst/references_general/references_general.html#reference-file-types

@melanieclarke
Copy link
Collaborator Author

The reference file documentation has a pair of tables that map reftypes to steps and vice-versa. Could the flat <-> clean_flicker_noise connection be added?

Thanks for pointing that out, I didn't know that table existed. I'll get it added.

@tapastro
Copy link
Contributor

Thanks for pointing that out, I didn't know that table existed. I'll get it added.

I think it's fallen a bit out of date, but it's a bookmark I visit on occasion and appreciate!

@tapastro tapastro self-requested a review January 23, 2025 21:27
B471
Copy link
Contributor
@tapastro tapastro left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

@melanieclarke melanieclarke merged commit 760c113 into spacetelescope:main Jan 23, 2025
31 checks passed
@melanieclarke melanieclarke deleted the jp-3829 branch January 23, 2025 21:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants
0