|
CUB
|
A simple caching allocator for device memory allocations.
active_stream. Once freed, the allocation becomes available immediately for reuse within the active_stream with which it was associated with during allocation, and it becomes available for reuse within other streams when all prior work submitted to active_stream has completed.bin_growth provided during construction. Unused device allocations within a larger bin cache are not reused for allocation requests that categorize to smaller bin sizes.bin_growth ^ min_bin) are rounded up to (bin_growth ^ min_bin).bin_growth ^ max_bin) are not rounded up to the nearest bin and are simply freed when they are deallocated instead of being returned to a bin-cache.max_cached_bytes, allocations for that device are simply freed when they are deallocated instead of being returned to their bin-cache.bin_growth = 8min_bin = 3max_bin = 7max_cached_bytes = 6MB - 1BDefinition at line 104 of file util_allocator.cuh.
Public Methods | |
| CachingDeviceAllocator (unsigned int bin_growth, unsigned int min_bin, unsigned int max_bin, size_t max_cached_bytes, bool skip_cleanup=false) | |
| Constructor. More... | |
| CachingDeviceAllocator (bool skip_cleanup=false) | |
| Default constructor. More... | |
| cudaError_t | SetMaxCachedBytes (size_t max_cached_bytes) |
| Sets the limit on the number bytes this allocator is allowed to cache per device. | |
| cudaError_t | DeviceAllocate (int device, void **d_ptr, size_t bytes, cudaStream_t active_stream=0) |
| Provides a suitable allocation of device memory for the given size on the specified device. More... | |
| cudaError_t | DeviceAllocate (void **d_ptr, size_t bytes, cudaStream_t active_stream=0) |
| Provides a suitable allocation of device memory for the given size on the current device. More... | |
| cudaError_t | DeviceFree (int device, void *d_ptr) |
| Frees a live allocation of device memory on the specified device, returning it to the allocator. More... | |
| cudaError_t | DeviceFree (void *d_ptr) |
| Frees a live allocation of device memory on the current device, returning it to the allocator. More... | |
| cudaError_t | FreeAllCached () |
| Frees all cached device allocations on all devices. | |
| virtual | ~CachingDeviceAllocator () |
| Destructor. | |
|
inline |
Constructor.
| bin_growth | Geometric growth factor for bin-sizes |
| min_bin | Minimum bin |
| max_bin | Maximum bin |
| max_cached_bytes | Maximum aggregate cached bytes per device |
| skip_cleanup | Whether or not to skip a call to FreeAllCached() when the destructor is called. (Useful for preventing warnings when the allocator is declared at file/static/global scope: by the time the destructor is called on program exit, the CUDA runtime may have already shut down and freed all allocations.) |
Definition at line 259 of file util_allocator.cuh.
|
inline |
Default constructor.
Configured with:
bin_growth = 8min_bin = 3max_bin = 7max_cached_bytes = (bin_growth ^ max_bin) * 3) - 1 = 6,291,455 byteswhich delineates five bin-sizes: 512B, 4KB, 32KB, 256KB, and 2MB and sets a maximum of 6,291,455 cached bytes per device
| skip_cleanup | Whether or not to skip a call to FreeAllCached() when the destructor is called. (Useful for preventing warnings when the allocator is declared at file/static/global scope: by the time the destructor is called on program exit, the CUDA runtime may have already shut down and freed all allocations.) |
Definition at line 294 of file util_allocator.cuh.
|
inline |
Provides a suitable allocation of device memory for the given size on the specified device.
Once freed, the allocation becomes available immediately for reuse within the active_stream with which it was associated with during allocation, and it becomes available for reuse within other streams when all prior work submitted to active_stream has completed.
| [in] | device | Device on which to place the allocation |
| [out] | d_ptr | Reference to pointer to the allocation |
| [in] | bytes | Minimum number of bytes for the allocation |
| [in] | active_stream | The stream to be associated with this allocation |
Definition at line 347 of file util_allocator.cuh.
|
inline |
Provides a suitable allocation of device memory for the given size on the current device.
Once freed, the allocation becomes available immediately for reuse within the active_stream with which it was associated with during allocation, and it becomes available for reuse within other streams when all prior work submitted to active_stream has completed.
| [out] | d_ptr | Reference to pointer to the allocation |
| [in] | bytes | Minimum number of bytes for the allocation |
| [in] | active_stream | The stream to be associated with this allocation |
Definition at line 485 of file util_allocator.cuh.
|
inline |
Frees a live allocation of device memory on the specified device, returning it to the allocator.
Once freed, the allocation becomes available immediately for reuse within the active_stream with which it was associated with during allocation, and it becomes available for reuse within other streams when all prior work submitted to active_stream has completed.
Definition at line 506 of file util_allocator.cuh.
|
inline |
Frees a live allocation of device memory on the current device, returning it to the allocator.
Once freed, the allocation becomes available immediately for reuse within the active_stream with which it was associated with during allocation, and it becomes available for reuse within other streams when all prior work submitted to active_stream has completed.
Definition at line 604 of file util_allocator.cuh.
1.8.4