Switching rendering back-ends within the same window does not always work correctly

[johannesschaeufele]

Describe the bug
When switching back-ends with the same existing window, switching from DX11/DX12 to anything other than DX11/12 or switching from Vulkan to OpenGL results in unexpected behavior, with frames rendered after the switch not appearing in the window. Performing the same switches with newly created windows instead results in expected behavior and performing switches between back-ends other than those mentioned within the same window also produces expected behavior.

To Reproduce

1. Apply the following patch:

--- a/src/bgfx.cpp
+++ b/src/bgfx.cpp
@@ -2674,6 +2674,8 @@ namespace bgfx
 
 	RendererContextI* rendererCreate(const Init& _init)
 	{
+		static RendererType::Enum prevRenderer = RendererType::Noop;
+
 		int32_t scores[RendererType::Count];
 		uint32_t numScores = 0;
 
@@ -2748,6 +2750,31 @@ namespace bgfx
 					score += RendererType::Direct3D11 == renderer ? 10 : 0;
 				}
 
+				switch (prevRenderer)
+				{
+				default:
+					if (ii == RendererType::Vulkan || ii == RendererType::Direct3D11 || ii == RendererType::Direct3D12)
+					{
+						score += 1000;
+					}
+					break;
+
+				case RendererType::Vulkan:
+					if (ii == RendererType::OpenGL)
+					{
+						score += 1000;
+					}
+					break;
+
+				case RendererType::Direct3D11:
+				case RendererType::Direct3D12:
+					if (ii != RendererType::Direct3D11 && ii != RendererType::Direct3D12)
+					{
+						score += 1000;
+					}
+					break;
+				}
+
 				scores[numScores++] = (score<<8) | uint8_t(renderer);
 			}
 		}
@@ -2761,6 +2788,7 @@ namespace bgfx
 			renderCtx = s_rendererCreator[renderer].createFn(_init);
 			if (NULL != renderCtx)
 			{
+				prevRenderer = renderer;
 				break;
 			}

2. Build and run the amalgamated examples (example-glue)
3. Switch to a different example
4. Example that was switched to does not appear in the window

Expected behavior
Examples run and render as they would without the patch

Additional context
Tested on Windows, various versions from Windows 7 to Windows 11, different NVIDIA consumer graphics cards

[bkaradzic]

Please create repro without modifying bgfx.

[johannesschaeufele]

Please create repro without modifying bgfx.

Patch:

diff --git a/examples/00-helloworld/helloworld.cpp b/examples/00-helloworld/helloworld.cpp
index 884a9ab0a..45f41741a 100644
--- a/examples/00-helloworld/helloworld.cpp
+++ b/examples/00-helloworld/helloworld.cpp
@@ -37,6 +37,7 @@ public:
 		init.resolution.width  = m_width;
 		init.resolution.height = m_height;
 		init.resolution.reset  = m_reset;
+		init.type = bgfx::RendererType::Direct3D11;
 		bgfx::init(init);
 
 		// Enable debug text.
diff --git a/examples/01-cubes/cubes.cpp b/examples/01-cubes/cubes.cpp
index 4f8084001..1dcd1a5e5 100644
--- a/examples/01-cubes/cubes.cpp
+++ b/examples/01-cubes/cubes.cpp
@@ -152,6 +152,7 @@ public:
 		init.resolution.width  = m_width;
 		init.resolution.height = m_height;
 		init.resolution.reset  = m_reset;
+		init.type = bgfx::RendererType::Vulkan;
 		bgfx::init(init);
 
 		// Enable debug text.
diff --git a/examples/02-metaballs/metaballs.cpp b/examples/02-metaballs/metaballs.cpp
index 95ca1dbe0..702a92c2d 100644
--- a/examples/02-metaballs/metaballs.cpp
+++ b/examples/02-metaballs/metaballs.cpp
@@ -511,6 +511,7 @@ public:
 		init.resolution.width  = m_width;
 		init.resolution.height = m_height;
 		init.resolution.reset  = m_reset;
+		init.type = bgfx::RendererType::OpenGL;
 		bgfx::init(init);
 
 		// Enable debug text.

Repro: Cycle through examples in order with example-glue

[bkaradzic]

I reproduced this on Windows, on Linux everything works fine.

What's your GPU/drivers?

[johannesschaeufele]

What's your GPU/drivers?

Most recently reproduced this with a 2080 Ti and driver version 531.29, but have reproduced this with other NVIDIA consumer GPUs and driver versions, so I don't think this issue is specific to the GPU or driver version.

[bkaradzic]

so I don't think this issue is specific to the GPU or driver version.

It's not, I just wanted to get extra data point...

[bkaradzic]

I investigated more, and it's definitely DXGI that causes this issue. This worked a few years ago but since then I rewrote parts of DXGI and changed flip model. I tried multiple things to recover HWND to get back in original state, but haven't find fix yet...

But here is someone with the same/similar issue:
https://stackoverflow.com/questions/20647359/resetting-window-after-using-directx-11

[johannesschaeufele]

There's also the similar Vulkan issue, which would only manifest itself in the previous repro after the DX issue is fixed - not sure if that should be separately tracked, as your finding indicates that the underlying cause may not be directly related.

diff --git a/examples/00-helloworld/helloworld.cpp b/examples/00-helloworld/helloworld.cpp
index 884a9ab0a..e5a717798 100644
--- a/examples/00-helloworld/helloworld.cpp
+++ b/examples/00-helloworld/helloworld.cpp
@@ -37,6 +37,7 @@ public:
 		init.resolution.width  = m_width;
 		init.resolution.height = m_height;
 		init.resolution.reset  = m_reset;
+		init.type = bgfx::RendererType::Vulkan;
 		bgfx::init(init);
 
 		// Enable debug text.
diff --git a/examples/01-cubes/cubes.cpp b/examples/01-cubes/cubes.cpp
index 4f8084001..1ec84f1cc 100644
--- a/examples/01-cubes/cubes.cpp
+++ b/examples/01-cubes/cubes.cpp
@@ -152,6 +152,7 @@ public:
 		init.resolution.width  = m_width;
 		init.resolution.height = m_height;
 		init.resolution.reset  = m_reset;
+		init.type = bgfx::RendererType::OpenGL;
 		bgfx::init(init);