Feature/add freeOnDevice() method

[XL64]

Add freeOnDevice() method to allow freeing memory from GPU before the object destruction.
This method is useful to remove temporary working data from GPU space, freeing GPU memory.
Add logger using callback for free/allocate when activated.

[corbett5]

This should be in Array not ArrayView.

[sframba]

Hi @corbett5 , I think this should be available from ArrayView, for two reasons:

• All the device memory manipulations (move, registerTouch, ...) are located in ArrayView. They do not modify the underlying array data, only its availability on device;
• from the application side, sometime we only have an ArrayView, but we still want to be able to free memory on device. This is our use case in GEOS: we use the (read-only for us) maps, but we wish to free them from GPU after use

Could we instead keep this in ArrayView, but overload the move method to allow (optionally) freeing the device memory when moving data to host?
What do you think?

[corbett5]

free can change the array in ways that move and registerTouch cannot. Specifically if the array only exists in the memory space we are freeing it from. And as a counter point all the explicitly allocating functions live in Array, and it makes sense to me that the deallocating functions should live there too.

I also think that adding free to ArrayView would add more ways you could shoot yourself in the foot, since the parent Array would have to do some kind of "update" to know that the data was free'd.

My suggestion would be to re-think how you're planning on doing it in GEOS. Either pass an Array to the functions where you want to free the data on the GPU or just free the data after calling the functions.

TLDR: I think your suggestion makes more sense if the functionality was "free the array on device but only if it also has a copy on host". But that seems super specific and I would much rather have general "free in the provided memory space".

[sframba]

Thanks for the feedback, I'll try to move it to Array as suggested then. I'll see if we can use in this way from GEOS. Thanks !

[corbett5]

Sorry that this will require some changes in GEOS, but if you have any trouble there feel free to let me know.

[rrsettgast]

@corbett5 I think a freeInSpace(memorySpace) is what they need here. Perhaps where such a function should live is a deeper design question. The allocation of the device memory space (or any space other than the original allocation) via a RAJA capture is done using an ArrayView. Thus an ArrayView has provided the interface to perform the allocation.

Should the ArrayView be allowed to provide an interface function to deallocate the allocation it triggered? If not, then the user has to get a reference to the actual Array, create an ArrayView, capture the ArrayView, then deallocate using the Array. I think we might shield access to the underlying Array in many cases so this could be problematic. (i.e. we get an ArrayView from the repo, not an Array)

Alternatively, we can just make a hard copy of a host array on device, and delete that copy after use, but that seems like the wrong way to go.

[wrtobin]

Per discussions in the meeting today, we're thinking instead of labeling this functionality as a free and moving into Array, it should instead be thought of as a reduction of the valid set of memory spaces the ArrayView can be moved to / accessed from, and any potential free resulting from that space reduction is a byproduct of the reduction. Additionally, one should not be allowed to remove the default memory space, and one should not be able to dynamically add back a removed memory space, instead requiring the user to create/copy into a new array with the correct memory spaces.

[corbett5]

Alternatively, we can just make a hard copy of a host array on device, and delete that copy after use, but that seems like the wrong way to go.

@rrsettgast if you know for a fact that the existing array does not exist on device and you just want to move it there, do some computation and then free the device memory using this pattern of copying to a new device-only array seems fairly natural to me. I forget what the specific use case is here, but if you're working in a CPU only environment then this pattern would be more wasteful (allocate a copy on the CPU, then free it) than just accessing the existing array and a no-op freeOnDevice.

Additionally, one should not be allowed to remove the default memory space, and one should not be able to dynamically add back a removed memory space, instead requiring the user to create/copy into a new array with the correct memory spaces.

@wrtobin I'm not sure what you mean by "one should not be able to dynamically add back a removed memory space". Do you mean that if I free a view on device that operation forbids the view from being automatically moved to device next time it is captured?

Whatever the case may be, y'all have put much more thought into this than I have so I'm fine doing whatever you think works best.

[corbett5]

I definitely like the added message here.

[corbett5]

Great, we definitely need this functionality! My thought though is that this new functionality should be accommodated by adding a space parameter to (or overload of) bufferManipulation::free.

LvArray/src/bufferManipulation.hpp

Line 188 in 145bb1c

void free( BUFFER & buf, std::ptrdiff_t const size )

Then Array should have a method free( MemorySpace space ) or something like that.

[rrsettgast]

@wrtobin Are you ok with this in its current form?

[wrtobin]

@corbett5 Randy and I discussed that particular point and IIRC we decided it wasn't a necessary restriction. My concern was primarily preventing users from erroneously de- and re-allocating on device and losing performance as a result.