Add IPCHandle class for inter-process communication

[Devdoot57]

This pull request adds a new IPCHandle class to facilitate inter-process communication (IPC) for sharing Cupy arrays between processes (#8029). The IPCHandle class provides methods for creating IPC handles and managing shared memory, allowing for efficient data transfer between processes.
The .get_ipc_handle() method returns an instance of IPCHandle, which can be passed to other processes (e.g. via multiprocessing). On the destination process, use the .get() method to create a new ndarray object that shares the allocation from the original process.

# On source process
import cupy
arr = cupy.ones((3,3))
arr_ipc = arr.get_ipc_handle()

# On destination process
arr = arr_ipc.get()

[kmaehashi]

Thanks @Devdoot57!

[kmaehashi]

Is it possilbe to handle ndarray-subclassing? The source might be an instance of user-defined subclass of cupy.ndarray.

[Devdoot57]

Ok, so you want get() to return an instance of whatever class (ndarray or its subclass) was used while creating the handle?

[kmaehashi]

I was thinking of that, but on second thought, I think it's difficult to support it without modifying pickling behavior in the base class (ndarray). Maybe adding a warning in get_ipc_handle() if it is a subclass instance would be sufficient for now 🙂

[Devdoot57]

Ok, so for now one can still get a handle for a subclass, but open() will always return an instance of ndarray. I'll push the requested changes

[kmaehashi]

Let's name it IPCMemoryHandle to distinguish it from IPC event handles.

[kmaehashi]

Suggested change

	handle = ipcGetMemHandle(self.data.ptr)
	return IPCHandle(self.shape, self.size, self.dtype, self.strides, handle)
	return IPCMemoryHandle(self)

I think it's better to retain the reference to self in the handle instance. Otherwise a = cupy.arange(10).get_ipc_handle() won't work as the backing ndarray will be freed immediately.

https://docs.nvidia.com/cuda/archive/9.0/cuda-runtime-api/group__CUDART__DEVICE.html#group__CUDART__DEVICE_1g01050a29fefde385b1042081ada4cde9

Calling cudaFree on an exported memory region before calling cudaIpcCloseMemHandle in the importing context will result in undefined behavior.

It is also better to document that users are responsible for keeping the IPC instance alive in the source process for the lifetime of the corresponding array in the destination process.

[kmaehashi]

😄

Suggested change

	handle for sharing Cupy arrays between processes.
	handle for sharing CuPy arrays between processes.

[kmaehashi]

Suggested change

	if self.arr_ptr != -1:
	ipcCloseMemHandle(self.arr_ptr)
	self.close()

[kmaehashi]

This check might be inaccurate, because it is not guaranteed that the process that created an IPC handle is the same one as the fork parent of the current process.

Suggested change

	if get_start_method() == "fork":
	raise RuntimeError(
	"The handle cannot be spawned on a forked process.")

[Devdoot57]

Got it. But, should I add an additional check because it throws cudaErrorInitializationError anyway?

[kmaehashi]

Sounds nice to show hint message for users in that case!

[kmaehashi]

How about putting this file as cupy/cuda/_ipc_handle.py? I think this one is not a part of core.

[kmaehashi]

I think def open(self) was better, considering close() is provided as an opposing concept 😅