Wrap a mtlx document into a nodedef (through API or graphEditor)

[MatthieuBEAUD]

Hi,
I'm pretty new to the MaterialX API, but I can't seem to find a way to wrap a mtlx into a NodeDef:

• I have created a materialX eye shader with some input and outputs, and I would need to use Houdini's mtlx2hda.py to convert it into a HDA. For this to work, I need the materialX shader to be defined into a nodeDef.
• I tried the following:
• load my materialX into a first document (doc_in)
• create a second document (doc_out)
• create a NodeDef in doc_out
• copy the content from doc_in into the nodeDef

    doc_in = mx.createDocument()
    mx.readFromXmlFile(doc_in, "my_material.mtlx")

    doc_out = mx.createDocument()
    node_def = mtlx_doc_out.addNodeDef(name="node_def_name", node="my_custom_node")
    node_def.copyContentFrom(doc_in)

    mx.writeToXmlFile(doc_out, "my_material_def.mtlx")

In the result the NodeDef disappeared:

<?xml version="1.0"?>
<materialx version="1.38" xmlns:xi="http://www.w3.org/2001/XInclude">
  <xi:include href="my_materialx.mtlx" />
</materialx>

I thought I should get:

<?xml version="1.0"?>
<materialx version="1.38">
  <nodedef name="node_def_name" node="my_custom_node">
    <xi:include href="my_materialx.mtlx" />
    (OR the actual content of my_materialx.mtlx)
  </nodedef>
</materialx>

I also tried having all the nodes of my_material into a NodeGraph, and then using Document.addNodeDefFromGraph but it only set the outputs in the NodeDef, but not the inputs.

How could one wrap the content of a materialX into a NodeDef ?

Could it be possibly done into the graphEditor ?

Thank you

[jstone-lucasfilm]

This is a good question, @MatthieuBEAUD, and I'm CC'ing @kwokcb who has done some initial work on adding NodeDef publication methods to the MaterialX API.

One related pull request may be found here:
#1403

[kwokcb]

Hi @MatthieuBEAUD ,

This line is a bit "strange" in approach

node_def.copyContentFrom(doc_in)

as you're trying to copy the entire document into a single element.

It will embed the document inside but it's has a source reference (adds a sourceURI reference). Thus when writing it's only including the reference as an "xinclude" by default. In this case ou can set the following XmlWriteOptions to expand the include.

/// If true, elements with source file markings will be written as
/// XIncludes rather than explicit data.  Defaults to true.
bool writeXIncludeEnable;

BTW< It's an interesting way to populate a definition from an external whole document source which from what I see should not have any side-effects but is not a well tested scenario.

I'm not sure if this fits your workflow but perhaps taking a "libraries" approach is useful ? For example loadLibrary() loads in a external definition references in using importLibrary(). The "dummy" nodedef it seems is just a placeholder which isn't really needed if you just intend to import an entire document.

Hope this helps, and is an workflow that would interesting to get more clarification on.

[MatthieuBEAUD]

Hi @kwokcb,

Thank you for the explainations.
So it is expected that doing:

node_def.copyContentFrom(doc_in)

would remove the nodeDef from the materialX and replace it with the <xinclude> (instead of adding the <xinclude> inside the NodeDef) since we're trying to copy the content from a document into a single element, so it would default to copying the content into the parent document of the node_def, right ?

I'll try with the libraries. But would it need to import a NodeDef or I could import any materialX graph ?

The workflow would be for the creation of an eye shader with MaterialX. It would be as follows:

• Create a MaterialX through the GraphEditor. (and could be edited later if necessary).
• We then would need to use this MaterialX in different DCCs (maya, houdini...). It would not only be an simple import/reference but we'd also need to expose some of the shader parameters for animation (for example, in an eye shader, we would need to animate/keyframe the size of the pupil, so some inputs of the materialX would need to be exposed one way or another. We wouldn't want to export one materialX per keyframe).
• In Houdini, it is possible to create a HDA (Houdini Digital Asset: a custom node that can be used in the houdini graphs) from a MaterialX using one of their python script mtlx2hda.py. This would allow to have this Materialx as a custom node with some exposed parameters within Houdini and keyframe those. But this script can only convert a MaterialX NodeDef to a HDA. So I need a way to convert a MaterialX without NodeDef to one with a NodeDef. And I was looking for ways to do it through the MaterialX API, instead of manipulating the XML directly (which is dirty and dangerous).
• For maya, I was planning on using the MaterialX ShaderGen, to get the glsl code, and then wrap it manually into the maya format .ogsfx to expose some of the Uniforms of the glsl in the maya UI.

So basically, my main concern was to make a materialX graph into a nodeDef.

I don't know if there are any plans for MaterialX to have exposable / dynamic parameters (like the inputs of a shader) ?

Thank you again for your time !

[kwokcb]

Hi @MatthieuBEAUD ,

It's great to have this context information.

In a nutshell this sounds like dynamic nodedef creation.

1. If you assume the premise that a document can contain N nodegraphs then there is an API to create a nodedef on the fly from a nodegraph. It basically copies the nodegraph and modifies it to be an implementation graph ("functional graph"), creates a nodedef and then sets up the implementation relationship. I have some fixes for it which I should really finish off the PR for :(.

It's not hard to do this with just Python API as well as I wrote up a utility here which will patch the current API call as needed and also has some logic for content copying. Since the code is not checked in yet. you need to assume use_1_38_8 is false.

2. If you assume your whole document is a definition, then you'd need top level inputs and output nodes as needed. From that you can create a nodegraph and place these elements as children of the nodegraph. Then as before create the definition.

For Maya I'm not sure from your comments if you know there OGSFX shader generator which will produce ogsfx for the viewport rendering. Thus I don't think you have to go this route. I haven't checked the current state of the MaterialX Stack support but custom definitions should be supported. I'm unsure if they can dynamically reload at runtime. Best to ping folks directly there.

Anyways the key elements are:

1. Load 1 or more nodegraphs from a document, or wrap a document in a nodegraph
2. Copy the graphs to create the implementation graphs.
3. Create a nodedef per graph. Using the inputs and outputs from that graph to expose the definition interfaces.
4. Update any dependent definition libraries such as the USD SDR if you need that ( I guess that's what's being done with mtlx2hda.

For dynamic parameter exposure, on a nodegraph it's already there by adding input nodes to a graph. For a definition they could be possible but definitions are meant to be statically created once for a definition. In your case you can dynamically change the inputs to expose and rebuild the definition as needed.

[kwokcb]

BTW, I wrote a tutorial for definition creation (from graphs or source) here. Maybe overkill but has all the "gory" details (especially for the latter) :).

[MatthieuBEAUD]

Thank you so much for all the documentation and help @kwokcb ! It'll help a lot !

[kwokcb]

No problem. If you need something more interactive and not on them, the ASWF Slack channels have a lot of great folks.