Improve how copies are handled #514
Open
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This simplifies things into an allocation with optional init, user init sequentially on host, and copy to final memory space. Previously with separate allocData and scopedMoveData there was an extra copy to device and copy back to host with allocations and deallocations.
This avoids a host device race condition that could occur if the memory space used was host.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Improve copies
This does a number of things to improve how copies are handled.
Add synchronization to copyData to ensure those copies complete. This fixes a potential issue around host->device and device->device copies with cuda and hip.
Use the programming model memcpy implementation instead of copyData in some sections of the code. This fixes a potential issue where a sequential memcpy implementation was used for host to host copies with cuda and hip that could lead to a host-device race condition.
Make a combined scopedMoveData and allocData API called allocDataForInit, along with variants that initialize memory. This generally reduces the number of allocations and copies used by allocating directly into the memory space used during initialization. This simplifies usage in setup routines, but makes allocate and copy patterns elsewhere more explicit.