8000 JP-3727: changed psfalign product from quadmodel to cubemodel by emolter · Pull Request #8747 · spacetelescope/jwst · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

JP-3727: changed psfalign product from quadmodel to cubemodel #8747

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Sep 5, 2024

Conversation

emolter
Copy link
Collaborator
@emolter emolter commented Sep 4, 2024

Resolves JP-3727

Closes #8746

A previous pull request implemented a change that simplified the pipeline's coronagraphic data processing such that the alignment of the PSF to the science data is only done on the first science integration, then assumed to be the same for all subsequent integrations. Prior to this change, the shifted PSF was stored in a 4-dimensional data structure (QuadModel of shape n_sci_integrations, n_psf_integrations, n_rows, n_cols), which made sense when the shift was different on each science integration. Now, though, this model is identical over the zeroth axis and therefore redundant with itself. Furthermore, this model uses a lot of memory (see JP-3724), and is likely a (the?) cause of the out-of-memory failures reported on multiple datasets in JWSTDMS-921 (see comments on that ticket for additional datasets). This PR stores the shifted PSF as a CubeModel instead.

Checklist for PR authors (skip items if you don't have permissions or they are not applicable)

  • added entry in CHANGES.rst within the relevant release section
  • updated or added relevant tests
  • updated relevant documentation
  • added relevant milestone
  • added relevant label(s)
  • ran regression tests, post a link to the Jenkins job 8000 below.
    How to run regression tests on a PR
  • All comments are resolved
  • Make sure the JIRA ticket is resolved properly

@emolter
Copy link
Collaborator Author
emolter commented Sep 4, 2024

@emolter emolter added this to the Build 11.1 milestone Sep 4, 2024
@emolter emolter removed the testing label Sep 4, 2024
@emolter emolter changed the title changed psfalign product from quadmodel to cubemodel JP-3727: changed psfalign product from quadmodel to cubemodel Sep 4, 2024
Copy link
codecov bot commented Sep 4, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 60.63%. Comparing base (acb0c15) to head (8381a26).

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #8747      +/-   ##
==========================================
+ Coverage   60.59%   60.63%   +0.04%     
==========================================
  Files         372      372              
  Lines       38401    38372      -29     
==========================================
- Hits        23269    23267       -2     
+ Misses      15132    15105      -27     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@emolter
Copy link
Collaborator Author
emolter commented Sep 4, 2024

Regression tests are all accounted for: 18 of the 22 failures match the ones on the nightly runs here. The other four (two each for NIRCam and MIRI) are all directly testing the psfalign product, which with this PR changed from a QuadModel to a CubeModel, so these failures are expected.. The _i2d and other products output by coron3 are all identical.

@emolter emolter marked this pull request as ready for review September 4, 2024 17:46
@emolter emolter requested a review from a team as a code owner September 4, 2024 17:46
Copy link
Collaborator
@melanieclarke melanieclarke left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks great! Thanks for the analysis and for putting it together.

Copy link
Collaborator
@melanieclarke melanieclarke left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, pending concurrence from any of the INS coronography folks.

@mperrin
Copy link
Contributor
mperrin commented Sep 4, 2024

I am going to defer or hand off doing a review for this one, as I'm not an INS coronagraphy folks anymore. I would suggest that instead @aggle should review and concur for MIRI, and @juliengirard for NIRCam.

Copy link
Contributor
@bsunnquist bsunnquist left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me - the psf-subtracted images from the PID-1386 test case look reasonable as well.

@aggle
Copy link
Contributor
aggle commented Sep 4, 2024

Looks good to me as well!

@melanieclarke
Copy link
Collaborator

Okay, sounds like we have agreement. I'll go ahead and merge. Thanks all!

@melanieclarke melanieclarke merged commit ac1473d into spacetelescope:master Sep 5, 2024
29 checks passed
@emolter emolter deleted the JP-3727 branch September 5, 2024 14:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

All frames of the psfalign QuadModel are identical
5 participants
0