Stream Management

Stream Management#

HIP Runtime API Reference: Stream Management

Collaboration diagram for Stream Management:

Typedefs
typedef void(*	hipStreamCallback_t) (hipStream_t stream, hipError_t status, void *userData)

Functions
hipError_t	hipStreamCreate (hipStream_t *stream)
	Creates an asynchronous stream.

hipError_t	hipStreamCreateWithFlags (hipStream_t *stream, unsigned int flags)
	Creates an asynchronous stream with flag.

hipError_t	hipStreamCreateWithPriority (hipStream_t *stream, unsigned int flags, int priority)
	Creates an asynchronous stream with the specified priority.

hipError_t	hipDeviceGetStreamPriorityRange (int leastPriority, int greatestPriority)
	Returns numerical values that correspond to the least and greatest stream priority.

hipError_t	hipStreamDestroy (hipStream_t stream)
	Destroys the specified stream.

hipError_t	hipStreamQuery (hipStream_t stream)
	Returns hipSuccess if all of the operations in the specified `stream` have completed, or hipErrorNotReady if not.

hipError_t	hipStreamSynchronize (hipStream_t stream)
	Waits for all commands in the stream to complete.

hipError_t	hipStreamWaitEvent (hipStream_t stream, hipEvent_t event, unsigned int flags)
	Makes the specified compute stream wait for the specified event.

hipError_t	hipStreamGetFlags (hipStream_t stream, unsigned int *flags)
	Returns flags associated with this stream.

hipError_t	hipStreamGetId (hipStream_t stream, unsigned long long *streamId)
	Queries the Id of a stream.

hipError_t	hipStreamGetPriority (hipStream_t stream, int *priority)
	Queries the priority of a stream.

hipError_t	hipStreamGetDevice (hipStream_t stream, hipDevice_t *device)
	Gets the device associated with the stream.

hipError_t	hipExtStreamCreateWithCUMask (hipStream_t stream, uint32_t cuMaskSize, const uint32_t cuMask)
	Creates an asynchronous stream with the specified CU mask.

hipError_t	hipExtStreamGetCUMask (hipStream_t stream, uint32_t cuMaskSize, uint32_t *cuMask)
	Gets CU mask associated with an asynchronous stream.

hipError_t	hipStreamAddCallback (hipStream_t stream, hipStreamCallback_t callback, void *userData, unsigned int flags)
	Adds a callback to be called on the host after all currently enqueued items in the stream have completed. For each hipStreamAddCallback call, a callback will be executed exactly once. The callback will block later work in the stream until it is finished.

hipError_t	hipStreamSetAttribute (hipStream_t stream, hipStreamAttrID attr, const hipStreamAttrValue *value)
	Sets stream attribute. Updated attribute is applied to work submitted to the stream.

hipError_t	hipStreamGetAttribute (hipStream_t stream, hipStreamAttrID attr, hipStreamAttrValue *value_out)
	queries stream attribute.

Detailed Description

This section describes the stream management functions of HIP runtime API. The following Stream APIs are not (yet) supported in HIP:

hipStreamAttachMemAsync is a nop
hipDeviceGetStreamPriorityRange returns hipSuccess

Typedef Documentation

◆ hipStreamCallback_t

typedef void(* hipStreamCallback_t) (hipStream_t stream, hipError_t status, void *userData)

Stream CallBack struct

Function Documentation

◆ hipDeviceGetStreamPriorityRange()

hipError_t hipDeviceGetStreamPriorityRange	(	int *	leastPriority,
		int *	greatestPriority
	)

Returns numerical values that correspond to the least and greatest stream priority.

Parameters

[in,out]	leastPriority	Pointer in which a value corresponding to least priority is returned.
[in,out]	greatestPriority	Pointer in which a value corresponding to greatest priority is returned.

Returns: hipSuccess

Returns in *leastPriority and *greatestPriority the numerical values that correspond to the least and greatest stream priority respectively. Stream priorities follow a convention where lower numbers imply greater priorities. The range of meaningful stream priorities is given by [*leastPriority,*greatestPriority]. If the user attempts to create a stream with a priority value that is outside the meaningful range as specified by this API, the priority is automatically clamped to within the valid range.

Warning: This API is under development on AMD GPUs and simply returns hipSuccess.

◆ hipExtStreamCreateWithCUMask()

hipError_t hipExtStreamCreateWithCUMask	(	hipStream_t *	stream,
		uint32_t	cuMaskSize,
		const uint32_t *	cuMask
	)

Creates an asynchronous stream with the specified CU mask.

Parameters

[in,out]	stream	Pointer to new stream
[in]	cuMaskSize	Size of CU mask bit array passed in.
[in]	cuMask	Bit-vector representing the CU mask. Each active bit represents using one CU. The first 32 bits represent the first 32 CUs, and so on. If its size is greater than physical CU number (i.e., multiProcessorCount member of hipDeviceProp_t), the extra elements are ignored. It is user's responsibility to make sure the input is meaningful.

Returns: hipSuccess, hipErrorInvalidHandle, hipErrorInvalidValue

Creates a new asynchronous stream with the specified CU mask. stream returns an opaque handle that can be used to reference the newly created stream in subsequent hipStream* commands. The stream is allocated on the heap and will remain allocated even if the handle goes out-of-scope. To release the memory used by the stream, application must call hipStreamDestroy.

See also: hipStreamCreate, hipStreamSynchronize, hipStreamWaitEvent, hipStreamDestroy

◆ hipExtStreamGetCUMask()

hipError_t hipExtStreamGetCUMask	(	hipStream_t	stream,
		uint32_t	cuMaskSize,
		uint32_t *	cuMask
	)

Gets CU mask associated with an asynchronous stream.

Parameters

[in]	stream	Stream to be queried
[in]	cuMaskSize	Number of the block of memories (uint32_t *) allocated by user
[out]	cuMask	Pointer to a pre-allocated block of memories (uint32_t *) in which the stream's CU mask is returned. The CU mask is returned in a chunck of 32 bits where each active bit represents one active CU.

Returns: hipSuccess, hipErrorInvalidHandle, hipErrorInvalidValue

See also: hipStreamCreate, hipStreamSynchronize, hipStreamWaitEvent, hipStreamDestroy

◆ hipStreamAddCallback()

hipError_t hipStreamAddCallback	(	hipStream_t	stream,
		hipStreamCallback_t	callback,
		void *	userData,
		unsigned int	flags
	)

Adds a callback to be called on the host after all currently enqueued items in the stream have completed. For each hipStreamAddCallback call, a callback will be executed exactly once. The callback will block later work in the stream until it is finished.

Parameters

[in]	stream	- Stream to add callback to
[in]	callback	- The function to call once preceding stream operations are complete
[in]	userData	- User specified data to be passed to the callback function
[in]	flags	- Reserved for future use, must be 0

Returns: hipSuccess, hipErrorInvalidHandle, hipErrorNotSupported

See also: hipStreamCreate, hipStreamCreateWithFlags, hipStreamQuery, hipStreamSynchronize, hipStreamWaitEvent, hipStreamDestroy, hipStreamCreateWithPriority

◆ hipStreamCreate()

hipError_t hipStreamCreate ( hipStream_t * stream )

Creates an asynchronous stream.

Parameters

[in,out] stream Valid pointer to hipStream_t. This function writes the memory with the newly created stream.

Returns: hipSuccess, hipErrorInvalidValue

Creates a new asynchronous stream with its associated current device. The stream returns an opaque handle that can be used to reference the newly created stream in subsequent hipStream* commands. The stream is allocated on the heap and will remain allocated even if the handle goes out-of-scope. To release the memory used by the stream, the application must call hipStreamDestroy.

Returns: hipSuccess, hipErrorInvalidValue

See also: hipStreamCreateWithFlags, hipStreamCreateWithPriority, hipStreamSynchronize, hipStreamWaitEvent, hipStreamDestroy

◆ hipStreamCreateWithFlags()

hipError_t hipStreamCreateWithFlags	(	hipStream_t *	stream,
		unsigned int	flags
	)

Creates an asynchronous stream with flag.

Parameters

[in,out]	stream	Pointer to new stream
[in]	flags	Parameters to control stream creation

Returns: hipSuccess, hipErrorInvalidValue

Creates a new asynchronous stream with its associated current device. stream returns an opaque handle that can be used to reference the newly created stream in subsequent hipStream* commands. The stream is allocated on the heap and will remain allocated even if the handle goes out-of-scope. To release the memory used by the stream, application must call hipStreamDestroy.

The flags parameter controls behavior of the stream. The valid values are hipStreamDefault and hipStreamNonBlocking.

See also: hipStreamCreate, hipStreamCreateWithPriority, hipStreamSynchronize, hipStreamWaitEvent, hipStreamDestroy.

◆ hipStreamCreateWithPriority()

hipError_t hipStreamCreateWithPriority	(	hipStream_t *	stream,
		unsigned int	flags,
		int	priority
	)

Creates an asynchronous stream with the specified priority.

Parameters

[in,out]	stream	Pointer to new stream
[in]	flags	Parameters to control stream creation
[in]	priority	Priority of the stream. Lower numbers represent higher priorities.

Returns: hipSuccess, hipErrorInvalidValue

Creates a new asynchronous stream with the specified priority, with its associated current device. stream returns an opaque handle that can be used to reference the newly created stream in subsequent hipStream* commands. The stream is allocated on the heap and will remain allocated even if the handle goes out-of-scope. To release the memory used by the stream, application must call hipStreamDestroy.

The flags parameter controls behavior of the stream. The valid values are hipStreamDefault and hipStreamNonBlocking.

See also: hipStreamCreate, hipStreamSynchronize, hipStreamWaitEvent, hipStreamDestroy

◆ hipStreamDestroy()

hipError_t hipStreamDestroy ( hipStream_t stream )

Destroys the specified stream.

Parameters

[in] stream Stream identifier

Returns: hipSuccess hipErrorInvalidHandle

Destroys the specified stream.

If commands are still executing on the specified stream, some may complete execution before the queue is deleted.

The queue may be destroyed while some commands are still inflight, or may wait for all commands queued to the stream before destroying it.

See also: hipStreamCreate, hipStreamCreateWithFlags, hipStreamCreateWithPriority, hipStreamQuery, hipStreamWaitEvent, hipStreamSynchronize

◆ hipStreamGetAttribute()

hipError_t hipStreamGetAttribute	(	hipStream_t	stream,
		hipStreamAttrID	attr,
		hipStreamAttrValue *	value_out
	)

queries stream attribute.

Parameters

[in]	stream	- Stream to geet attributes from
[in]	attr	- Attribute ID for the attribute to query
[out]	value	- Attribute value output

Returns: hipSuccess, hipErrorInvalidValue, hipErrorInvalidResourceHandle

◆ hipStreamGetDevice()

hipError_t hipStreamGetDevice	(	hipStream_t	stream,
		hipDevice_t *	device
	)

Gets the device associated with the stream.

Parameters

[in]	stream	Stream to be queried
[out]	device	Device associated with the stream

Returns: hipSuccess, hipErrorInvalidValue, hipErrorContextIsDestroyed, hipErrorInvalidHandle, hipErrorNotInitialized, hipErrorDeinitialized, hipErrorInvalidContext

See also: hipStreamCreate, hipStreamDestroy, hipDeviceGetStreamPriorityRange

◆ hipStreamGetFlags()

hipError_t hipStreamGetFlags	(	hipStream_t	stream,
		unsigned int *	flags
	)

Returns flags associated with this stream.

Parameters

[in]	stream	Stream to be queried
[in,out]	flags	Pointer to an unsigned integer in which the stream's flags are returned

Returns: hipSuccess, hipErrorInvalidValue, hipErrorInvalidHandle.

See also: hipStreamCreateWithFlags

◆ hipStreamGetId()

hipError_t hipStreamGetId	(	hipStream_t	stream,
		unsigned long long *	streamId
	)

Queries the Id of a stream.

Parameters

[in]	stream	Stream to be queried
[in,out]	flags	Pointer to an unsigned long long in which the stream's id is returned

Returns: hipSuccess, hipErrorInvalidValue, hipErrorInvalidHandle.

See also: hipStreamCreateWithFlags, hipStreamGetFlags, hipStreamCreateWithPriority, hipStreamGetPriority

◆ hipStreamGetPriority()

hipError_t hipStreamGetPriority	(	hipStream_t	stream,
		int *	priority
	)

Queries the priority of a stream.

Parameters

[in]	stream	Stream to be queried
[in,out]	priority	Pointer to an unsigned integer in which the stream's priority is returned

Returns: hipSuccess, hipErrorInvalidValue, hipErrorInvalidHandle.

See also: hipStreamCreateWithPriority

◆ hipStreamQuery()

hipError_t hipStreamQuery ( hipStream_t stream )

Returns hipSuccess if all of the operations in the specified stream have completed, or hipErrorNotReady if not.

Parameters

[in] stream Stream to query

Returns: hipSuccess, hipErrorNotReady, hipErrorInvalidHandle

This is thread-safe and returns a snapshot of the current state of the queue. However, if other host threads are sending work to the stream, the status may change immediately after the function is called. It is typically used for debug.

See also: hipStreamCreate, hipStreamCreateWithFlags, hipStreamCreateWithPriority, hipStreamWaitEvent, hipStreamSynchronize, hipStreamDestroy

◆ hipStreamSetAttribute()

hipError_t hipStreamSetAttribute	(	hipStream_t	stream,
		hipStreamAttrID	attr,
		const hipStreamAttrValue *	value
	)

Sets stream attribute. Updated attribute is applied to work submitted to the stream.

Parameters

[in]	stream	- Stream to set attributes to
[in]	attr	- Attribute ID for the attribute to set
[in]	value	- Attribute value for the attribute to set

Returns: hipSuccess, hipErrorInvalidValue, hipErrorInvalidResourceHandle

◆ hipStreamSynchronize()

hipError_t hipStreamSynchronize ( hipStream_t stream )

Waits for all commands in the stream to complete.

Parameters

[in] stream Stream identifier.

Returns: hipSuccess, hipErrorInvalidHandle

This command is host-synchronous : the host will block until all operations on the specified stream with its associated device are completed. On multiple device systems, the stream is associated with its device, no need to call hipSetDevice before this API.

This command follows standard null-stream semantics. Specifying the null stream will cause the command to wait for other streams on the same device to complete all pending operations.

This command honors the hipDeviceScheduleBlockingSync flag, which controls whether the wait is active or blocking.

See also: hipStreamCreate, hipStreamCreateWithFlags, hipStreamCreateWithPriority, hipStreamWaitEvent, hipStreamDestroy

◆ hipStreamWaitEvent()

hipError_t hipStreamWaitEvent	(	hipStream_t	stream,
		hipEvent_t	event,
		unsigned int	flags
	)

Makes the specified compute stream wait for the specified event.

Parameters

[in]	stream	Stream to make wait
[in]	event	Event to wait on
[in]	flags	Parameters to control the operation [must be 0]

Returns: hipSuccess, hipErrorInvalidHandle

This function inserts a wait operation into the specified stream. All future work submitted to stream will wait until event reports completion before beginning execution.

This function only waits for commands in the current stream to complete. Notably, this function does not implicitly wait for commands in the default stream to complete, even if the specified stream is created with hipStreamNonBlocking = 0.

See also: hipStreamCreate, hipStreamCreateWithFlags, hipStreamCreateWithPriority, hipStreamSynchronize, hipStreamDestroy

Stream Management

Stream Management#

Typedefs

Functions

Detailed Description

Typedef Documentation

◆ hipStreamCallback_t

Function Documentation

◆ hipDeviceGetStreamPriorityRange()

◆ hipExtStreamCreateWithCUMask()

◆ hipExtStreamGetCUMask()

◆ hipStreamAddCallback()

◆ hipStreamCreate()

◆ hipStreamCreateWithFlags()

◆ hipStreamCreateWithPriority()

◆ hipStreamDestroy()

◆ hipStreamGetAttribute()

◆ hipStreamGetDevice()

◆ hipStreamGetFlags()

◆ hipStreamGetId()

◆ hipStreamGetPriority()

◆ hipStreamQuery()

◆ hipStreamSetAttribute()

◆ hipStreamSynchronize()

◆ hipStreamWaitEvent()