Add support for callable shaders in gfx

[skallweitNV]

Breaking change:

The ABI around gfx shader table creation is changed. Users of gfx will need to rebuild from source after upgrading.

There are no breaking changes in Slang.

[skallweitNV]

Falcor tests are failing because this change breaks binary compatibility. @csyonghe what is the best way to deal with this? Ignore the failing test and update Falcor ASAP?

[csyonghe]

Yeah, looks like we have to ignore this and update Falcor on the CO servers later.

The alternative is to add a Desc2 and createSjaderTable2 function.

[skallweitNV]

yeah, I don't think that change is worth splitting the struct/function

[skallweitNV]

One thing that might be worth considering is adding a Desc2 that uses a separate list of names and then just uses indices into that list for referencing them. This would allow for more performant creation of the shader binding table, because with the current version every entry needs to do a hash table lookup with a string key. I'm imagining something like this:

struct Desc2
{
    // List of entry point and hit group names referenced in the program.
    GfxCount nameCount;
    const char** names;

    GfxCount rayGenShaderCount;
    const GfxIndex* rayGenShaderEntryPointNames;
    const ShaderRecordOverwrite* rayGenShaderRecordOverwrites;

    GfxCount missShaderCount;
    const GfxIndex* missShaderEntryPointNames;
    const ShaderRecordOverwrite* missShaderRecordOverwrites;

    GfxCount hitGroupCount;
    const GfxIndex* hitGroupsNames;
    const ShaderRecordOverwrite* hitGroupRecordOverwrites;

    GfxCount callableShaderCount;
    const GfxIndex* callableShaderEntryPointNames;
    const ShaderRecordOverwrite* callableShaderRecordOverwrites;

    IShaderProgram* program;
};

This would allow to lookup the names once, and then just use indices, making construction of large tables more performant.

[csyonghe]

That sounds like a good idea. Would you implement this in this PR so we keep binary compabitity and also provide a more efficient interface?

[skallweitNV]

I think I would add the changes to Desc as is in this PR. And then add Desc2 in a separate PR, this way both APIs have the ability to add callable shaders.

[csyonghe]

If we can just do that in Desc2 and keep Desc unchanged for now, that will avoid breaking Falcor test, and once Desc2 is there we can update Falcor on our CI servers. There are a lot of work going on here that require a functional Falcor test, and I am little worried that things will slip through if we are not having Falcor test for a while.

[skallweitNV]

@csyonghe can we merge this now that falcor tests are decoupled from gfx changes?

[csyonghe]

Yes, let's do it.