10000 Assume correct extents in binary ops to avoid floating point errors by moradology · Pull Request #91 · geotrellis/maml · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

Assume correct extents in binary ops to avoid floating point errors #91

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 1 commit into from
Apr 17, 2019
Merged

Assume correct extents in binary ops to avoid floating point errors #91

merged 1 commit into from
Apr 17, 2019

Conversation

moradology
Copy link
Contributor

Floating point math can lead to situations in which combined GridExtents fail a requirement that checks correspondence between CellSize+Extent on the one hand and pixel cols/rows on the other.

This is likely desirable behavior if it happens only when bad data has been passed into MAML. It is very undesirable if it occurs in certain edge cases that are able to compute results properly, as happens whenever the Extent/CellSize ratio becomes too large and floating point arithmetic's approximations fail us.

The upshot of this is that we get slightly less guarded execution but avoid false positive errors

@moradology moradology requested review from echeipesh and removed request for echeipesh April 17, 2019 20:37
@moradology moradology merged commit 9a90251 into geotrellis:master Apr 17, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
1 participant
@moradology
0