Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unexpected VUID-vkCmdPipelineBarrier2-srcStageMask-03849 #7928

Open
MennoVink opened this issue Apr 30, 2024 · 1 comment
Open

Unexpected VUID-vkCmdPipelineBarrier2-srcStageMask-03849 #7928

MennoVink opened this issue Apr 30, 2024 · 1 comment
Assignees
Labels
Synchronization Synchronization Validation Object Issue

Comments

@MennoVink
Copy link

Environment:

  • OS: Windows
  • GPU and driver version: NVIDIA GeForce RTX 3070 (driver version: 2311913472)
  • SDK or header version if building from repo: SDK 1.3.280.0 headers 1.3.275.0
  • Options enabled (synchronization, best practices, etc.):
    Vulkan 1.2
    VK_KHR_synchronization2 + feature
    other most likely irrelevant stuff

Describe the Issue

I'm moving my graphics framework over to VK_KHR_synchronization2. This extension contains the following message leading to me updating all my acquire/release barriers to contain identical dependency information:
Where two synchronization commands need to be matched up (queue transfer operations, events), the dependency information specified in each place must now match completely for consistency.

This is problematic because the the VU that's hitting checks if the srcStageMask is valid in the command buffer that will be submitted to dstQueueFamilyIndex. This is the barrier in question:
image

Queue Family 0 is the graphics QF and QF 5 is the dedicated transfer QF.
The barrier goes from eColorAttachmentOutput/eColorAttachmentWrite on the graphics queue to eTransfer/eMemoryRead on the dedicated transfer queue. eColorAttachmentOutput is not valid on the dedicated transfer queue, so this VU is hitting.

Expected behavior

Tell me if i'm misreading the spec, but here's the information i've found that suggests i should be able to (and since sync2 in fact must) make the cross QF release/aquire barriers match up.

https://registry.khronos.org/vulkan/specs/1.3-extensions/man/html/VK_KHR_synchronization2.html
Where two synchronization commands need to be matched up (queue transfer operations, events), the dependency information specified in each place must now match completely for consistency.

https://registry.khronos.org/vulkan/specs/1.3-extensions/man/html/VkImageMemoryBarrier2KHR.html
The first synchronization scope and access scope described by this structure include only operations and memory accesses specified by srcStageMask and srcAccessMask.

When executed on a queue in the family identified by dstQueueFamilyIndex, this barrier defines a queue family acquire operation for the specified image subresource range, and the first synchronization, the first synchronization scope does not apply to this operation.

https://registry.khronos.org/vulkan/specs/1.2-extensions/html/vkspec.html#synchronization-queue-transfers-acquire
srcAccessMask is ignored for such a barrier, such that no availability operation is executed - the value of this mask does not affect the validity of the barrier.

Valid Usage ID

[ VUID-vkCmdPipelineBarrier2-srcStageMask-03849 ] Object 0: handle = 0x242069c2b00, type = VK_OBJECT_TYPE_COMMAND_BUFFER; | MessageID = 0xa7eeda8f | vkCmdPipelineBarrier2KHR(): pDependencyInfo->pImageMemoryBarriers[0].srcStageMask (VK_PIPELINE_STAGE_COLOR_ATTACHMENT_OUTPUT_BIT) is not compatible with the queue family properties (VK_QUEUE_TRANSFER_BIT|VK_QUEUE_SPARSE_BINDING_BIT|VK_QUEUE_OPTICAL_FLOW_BIT_NV) of this command buffer. The Vulkan spec states: The srcStageMask member of any element of the pMemoryBarriers, pBufferMemoryBarriers, or pImageMemoryBarriers members of pDependencyInfo must only include pipeline stages valid for the queue family that was used to create the command pool that commandBuffer was allocated from (https://vulkan.lunarg.com/doc/view/1.3.280.0/windows/1.3-extensions/vkspec.html#VUID-vkCmdPipelineBarrier2-srcStageMask-03849)

Other
This issue seems to have been reported and resolved previously here #2590. That issue is quite slim on information though so it might have been something else entirely.

@spencer-lunarg spencer-lunarg added the Synchronization Synchronization Validation Object Issue label Apr 30, 2024
@artem-lunarg artem-lunarg self-assigned this May 1, 2024
@artem-lunarg
Copy link
Contributor

The recent version of the specification added the following parts:

The release operation happens-after the availability operation. dstStageMask is also ignored for such a barrier as defined by buffer memory ownership transfer and image memory ownership transfer.

Since a release and acquire operation does not synchronize with second and first scopes respectively, the VK_PIPELINE_STAGE_ALL_COMMANDS_BIT stage must be used to wait for a release operation to complete. Typically, a release and acquire pair is performed by a VkSemaphore signal and wait in their respective queues. Signaling a semaphore with vkQueueSubmit waits for VK_PIPELINE_STAGE_ALL_COMMANDS_BIT. With vkQueueSubmit2, stageMask for the signal semaphore must be VK_PIPELINE_STAGE_ALL_COMMANDS_BIT. Similarly, for the acquire operation, waiting for a semaphore must use VK_PIPELINE_STAGE_ALL_COMMANDS_BIT to make sure the acquire operation is synchronized.

The first quote explicitly says that dstStageMask is ignored for the release barrier.

Since a release and acquire operation does not synchronize with second and first scopes respectively - I read this part that
srcStageMask is also ignored for the acquire barrier.

One quick fix can be to set dstStageMask for the release barrier and srcStageMask for the acquire barrier to STAGE_NONE (because they are ignored). And then according to the recommendation from the second quote to use ALL_COMMANDS for the semaphore signal/wait in QueueSubmit.

I still need to check VUID-vkCmdPipelineBarrier2-srcStageMask-03849, I tend to think it should not trigger an error when arbitrary values are used for the parameters that should be ignored.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Synchronization Synchronization Validation Object Issue
Projects
None yet
Development

No branches or pull requests

3 participants