Modified rendering to use target source size instead of parent source size to account for scaling applied by filters.

[Bemjo]

When using the spout filter applied to a source, it doesn't take into account the transformed size from any filters such as scaling/aspect ratio or cropping. Using these filters on a source in addition to the spout filter renders the image incorrectly on a spout capture source. The capture source is the un-transformed size of the original source before any scaling or cropping would be applied, and is either shrunk with excess space filled in with black, or zoomed in such that the full image of the original source with the spout filter cannot be fully seen.

The original source with scaling/aspect ratio or cropping, and a spout filter is on the left, and the spout 2 capture source is on the right with the properties window shown

This changes the rendering code to use the target source width and height the filter is applied to, instead of the original parent size.

[campbellwmorgan]

Thanks a lot for this. We'll get our team to check it out and hopefully merge soon. Thanks a lot :)

[campbellwmorgan]

@Bemjo Thanks a lot for both bringing this to our attention and have a crack at it. I've been running your branch locally and for some reason I continue to a render target in the spout viewer that is the dimensions of the original video, not the resize and cropped version with your code. Is there perhaps a step that I'm missing or perhaps that you may have forgot to push?

When I step through it in the debugger, parent and target appear to have the same dimensions and neither of them are cropped, but they also are not the final output size. (My output render target is 1920x1080 - size of the original video, the debug value is 730x410 but the cropped video should be square. This suggests that perhaps the scaling is being picked up, but the crop not. The Crop and Scale filters are both before the Spout 2 filter.

[campbellwmorgan]

Apologies, the SpoutReceiver.exe shows 16:9 even if the output isn't. I see that with the Scaling and Crop vertically above the Spout filter this does work correctly.

However when below they do not.

I'm not really sure what the expected behaviour is in this case. It seems that all effect filters are applied to the render target irrespective of the order, however some orders communicate the new dimensions when others don't.

@Bemjo can you think of a solution for this? Does this matter?

[Bemjo]

@campbellwmorgan Just to clarify, have you verified that the changes in this pull request do work? Or is something still not working correctly? I have not tested it with the spout receiver program, only with the spout filter, and a spout capture source inside of OBS.

As for expected behaviour for applying scaling or cropping after the filter, I don't have any. I don't believe there's a way to know specifically where in the filter chain you are to account for this to propose a solution, and it doesn't really make a lot of sense to me to use it in that way.

[campbellwmorgan]

@campbellwmorgan Just to clarify, have you verified that the changes in this pull request do work? Or is something still not working correctly? I have not tested it with the spout receiver program, only with the spout filter, and a spout capture source inside of OBS.

Yes I've tested your branch locally. I realise I've been unclear. Here's my summary:

1. For the specific case you've outlined, your code works: it does now correctly composite the filtered output on top of the scene
2. In some circumstances (outlined in my last comment), the render texture that is shared via Spout is resized correctly, in others it is not.
3. In the circumstance when it is not resized correctly, it does feel broken because other effects are applied and the main image is scaled, but the background not.

My question is really whether point 3. is something you had thought about?

I feel that this PR is still usefully mergable for people as it does not impair current users and may help some, but it would be nice to figure out point 3 whether that is by working out a way to resize the output render target to the correct resolution or perhaps by documenting this side effect better

[Bemjo]

A very quick and dirty test has led me to believe your point about 3, where applying a scaling or cropping filter after the spout filter causes the display to be incorrect, is caused by the threaded offscreen rendering not having necessarily correct information about the current source state.

Removing the offscreen rendering, and moving everything into the main render callback, and dealing only with the target (not the parent) causes the spout filter to behave in the following ways:

1. If the spout filter is applied last, it behaves exactly as this fix does, it will present an exact copy of the final transformed source.
2. If the spout filter is applied first, before any other transforming filters, it will present an exact copy of the original untransformed source.
3. If the spout filter is applied somewhere in the middle, it will present a copy of it's transformed state at it's point in the filter chain. ex/ If the original source is 1920x1080, and the filter chain is scaling (down to 1280x720) -> Spout Filter -> Scaling (up to 1920x1080) --- The spout filter will present a 1280x720 render of the source, as it lies in the filter chain.

I do not know the specific reasons for using the threaded offscreen render in this plugin, and removing that and using only the rendering callback is no quick and simple task (there are gamma and color space issues to contend with) if it were to be removed.

As of right now I'm not sure if there is a nice solution to this problem that does not involve more extensive modifications, as my attempts to simply cache the target dimensions inside the main render callback are not giving me valid values, though this may be an avenue to explore more.

[campbellwmorgan]

@Bemjo I've had time to do another test with a colleague and we think there are no downsides to merging and releasing this in its current form. How do you feel about merging this as-is and then moving these other issues to a new ticket which we could attempt at a future point?

[Bemjo]

@campbellwmorgan I have no issues with this