8000 replace check_memory_allocation in resample by braingram · Pull Request #8324 · spacetelescope/jwst · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

replace check_memory_allocation in resample # 8000 8324

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Mar 4, 2024

Conversation

braingram
Copy link
Collaborator
@braingram braingram commented Mar 1, 2024

This PR repaces check_memory_allocation in resample.

The replaced function allocates a large array, then measures the size of the array to determine if it's safe to allocate an array of that size. See spacetelescope/stdatamodels#272 for more details.

The replacement instead allocates a small (1x1) array to get the expected dtype then computes the expected size of the upcoming array allocation.

Checklist for maintainers

  • added entry in CHANGES.rst within the relevant release section
  • updated or added relevant tests
  • updated relevant documentation
  • added relevant milestone
  • added relevant label(s)
  • ran regression tests, post a link to the Jenkins job below.
    How to run regression tests on a PR
  • Make sure the JIRA ticket is resolved properly


allowed_memory = kwargs['allowed_memory']
if allowed_memory is None:
allowed_memory = os.environ.get('DMODEL_ALLOWED_MEMORY', allowed_memory)
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note that this is the only use of this variable in the code.

There is one test that uses it:

monkeypatch.setenv('DMODEL_ALLOWED_MEMORY', "0.9")

Perhaps this should be deprecated/removed and it's usage replaced? Is there a way to provide this parameter to calwebb_image3 without using an environment variable?

If it is kept, renaming it might be considered.

@braingram braingram marked this pull request as ready for review March 1, 2024 19:48
@braingram braingram requested a review from a team as a code owner March 1, 2024 19:48
@braingram
Copy link
Collaborator Author

Regression tests show 1 unrelated error:
https://plwishmaster.stsci.edu:8081/blue/organizations/jenkins/RT%2FJWST-Developers-Pull-Requests/detail/JWST-Developers-Pull-Requests/1255/tests
[stable-deps] test_duplicate_names – jwst.associations.tests.test_level3_duplicate failed due to a missing log message (likely due to stpipe logging issues).

@nden
Copy link
Collaborator
nden commented Mar 3, 2024

This looks good.
Can the code be written as a datamodel independent function? Then it can go in stcal and used by Roman and JWST.

@hbushouse hbushouse added this to the Build 10.2 milestone Mar 4, 2024
Copy link
Collaborator
@hbushouse hbushouse left a comment

Choose a reason for hiding this comment

The reason will be displayed t 8000 o describe this comment to others. Learn more.

LGTM

@braingram
Copy link
Collaborator Author

This looks good. Can the code be written as a datamodel independent function? Then it can go in stcal and used by Roman and JWST.

Great idea! I will take a look and open an stcal issue. What about something like the following?

@nden
Copy link
Collaborator
nden commented Mar 4, 2024

@braingram The plan sounds good.

@nden nden merged commit 7c4f1e8 into spacetelescope:master Mar 4, 2024
@braingram braingram deleted the allowed_memory branch March 4, 2024 7077 20:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants
0