Ambiguity of nodes with omitted inputs

[ld-kerley]

  <mix name="myMix" type="color3">
    <input name="fg" type="color3" nodename="upstreamA" />
    <input name="bg" type="color3" nodename="upstreamB" />
  </mix>

This node could resolve to one of two different node definitions in the standard library.

  <nodedef name="ND_mix_color3" node="mix" nodegroup="compositing">
    <input name="fg" type="color3" value="0.0, 0.0, 0.0" />
    <input name="bg" type="color3" value="0.0, 0.0, 0.0" />
    <input name="mix" type="float" value="0.0" uisoftmin="0.0" uisoftmax="1.0" />
    <output name="out" type="color3" defaultinput="bg" />
  </nodedef>
  <nodedef name="ND_mix_color3_color3" node="mix" nodegroup="compositing">
    <input name="fg" type="color3" value="0.0, 0.0, 0.0" />
    <input name="bg" type="color3" value="0.0, 0.0, 0.0" />
    <input name="mix" type="color3" value="0.0, 0.0, 0.0" uisoftmin="0,0,0" uisoftmax="1,1,1" />
    <output name="out" type="color3" defaultinput="bg" />
  </nodedef>

For the purposes of the document itself, this ambiguity is actually unimportant, because the implementations of both node definitions will behave the same if there is no mix parameter provided. But this becomes an issue for MaterialX integrations that resolve to a node definition.

I first discovered this while debugging an issue when using the UsdMtlx plugin. The myMix node above resolves to ND_mix_color3_color3 in UsdMtlx, but I happened to have wanted ND_mix_color3 because I had other USD layers that were going to author an edit to the mix input on the generated shader node.

After discovering this I decided to look elsewhere, closer to home. MaterialXGraphEditor will actually select the opposite node definition, resolving to ND_mix_color3 to UsdMtlx. This inconsistency is at the very least annoying. In fact in MaterialXGraphEditor the ambiguity manifests itself a little worse. If I decide to create a mix_color3_color3 node from the node menu, then I correctly see a color3 typed input for the mix parameter, but If I do not edit anything on the node, save the document and reload it, the my mix node reverts to a mix_color3 with a float typed input for the mix node, which is not what I created.

Note:
All of this ambiguity would be resolved with the addition of an <input> definition on the node to constrain the type of the mix input. The node below is no longer ambiguous.

  <mix name="myMix" type="color3">
    <input name="fg" type="color3" nodename="upstreamA" />
    <input name="bg" type="color3" nodename="upstreamB" />
    <input name="mix" type="float" />
  </mix>

I don't have any concrete proposals for how to resolve this, but I do have some ideas for discussion.

We could:

• Codify a concrete ordering of node definitions, for a given node family, so that there can at least be some level of consistency between integrations.
• Improve document validation to inform the user if there are nodes in the document that are ambiguous as to their node definition? - This perhaps could be too noisy, or report "errors" that aren't in fact errors?
• Update the document save functionality, such that if it detects it's writing an ambiguous node, then it will add any necessary <input> or <output> elements to concretely constrain the node.

[ld-kerley]

I was experimenting a little with Houdini 20, exporting MaterialX files, and it appears that SideFX get around this problem by always writing all of the <input> tags on each node - which leads to a larger than necessary mtlx file, but removes the ambiguity.

Tagging @crydalch incase he has any insight here.

[crydalch]

@ld-kerley that's just how we tend to author things in Houdini, as it helps reduce future updates from changing things unexpectedly.

[kwokcb]

Another way to remove the ambiguity is to write out the nodedef=<> attribute on the node which I believe is similar to what is done for USD with info:id but this doesn't solve the issue.

BTW, I remember that this issue occurred with other ambiguous signatures and the result was to change the node category so it not both mix so it's mix_float vs mix_color or something like that.

[jstone-lucasfilm]

@kwokcb In this case, the ability to overload generic nodes such as mix and add is an important feature that we want to preserve in MaterialX, so that graph editors can support dynamic resolution of nodes based on the typed data to which an artist connects them.

As @ld-kerley notes, though, it seems valuable for all MaterialX tools to resolve to identical nodes when typed inputs have been omitted from a graph, and this seems like an area where we can be more precise in the MaterialX API and documentation. Would it make sense to impose a rule that the first matching node definition in the data libraries must be used when multiple matches are present? We could then validate that the MaterialX API and example applications always provide this behavior, making any required changes to the codebase to ensure this.

[dbsmythe]

I think that arbitrarily choosing the "first" matching node definition could still not be deterministic enough- what if a site an another definition of a node in its own local site_defs.mtlx file and that happened to be loaded before stdlib_defs.mtlx in some situations? Instead, I would propose that it should match the "simplest" matching definition, however we want to define that. E.g. perhaps choosing the matching definition with fewer parameters, or one using using a float rather than a color/vector for an unconnected input. Or perhaps we could add a new attribute to nodedefs indicating a matching order or preference, where if two or more nodedefs matched, the one with the lower matching order value would win. Obviously we'd need to put a LOT more thought into this, but what do folks think of this as a general idea?

[kwokcb]

It is quite possible to have order change as it does depend on definition load order which can include target order and namespace order.

There are signatures with the exact same number of input parameters (as in this example). Also you could have the same matching "score". Als in this example, the definitions each have 3 inputs -- 1 which the differentiating input (float vs color3), but the node instance does not override the differentiating input.

There is already a mechanism to return "first" and an "exact" match option + logic which scans all inputs when trying to determine what definition matches.

My feeling is to try to avoid complex "matching" rules to be followed if there is ambiguity by either returning all matches and let the integration handle it properly, or just fail. I'm leaning towards the latter.

As noted in Slack, I'd still like to propose that if there are 2 or more nodefefs with differentiating inputs that must be specified that there is some meta-data to mark them as such. Then if it's missing then a failure condition can be easily found, both at creation time and at validation or definition matching time.
e.g. adding a required tag.

 <nodedef name="ND_mix_color3" node="mix" nodegroup="compositing">
    <input name="fg" type="color3" value="0.0, 0.0, 0.0" />
    <input name="bg" type="color3" value="0.0, 0.0, 0.0" />
    <input name="mix" type="float" value="0.0" uisoftmin="0.0" uisoftmax="1.0" required="true"/>
    <output name="out" type="color3" defaultinput="bg" />
  </nodedef>
  <nodedef name="ND_mix_color3_color3" node="mix" nodegroup="compositing">
    <input name="fg" type="color3" value="0.0, 0.0, 0.0" />
    <input name="bg" type="color3" value="0.0, 0.0, 0.0" />
    <input name="mix" type="color3" value="0.0, 0.0, 0.0" uisoftmin="0,0,0" uisoftmax="1,1,1" required="true"/>
    <output name="out" type="color3" defaultinput="bg" />
  </nodedef>

[dbsmythe]

This is exactly the situation I was thinking about when suggesting "use the simpler of the two matching definitions if both signatures match": in this example, ND_mix_color3 is "simpler" than ND_mix_color3_color3" because the one input ("mix") that has different types for each signature is "float" in the first, and "color3" in the second, and "float" is simpler than "color3". This wouldn't work if the only differing input was e.g. a "color3" vs a "vector3" but I don't recall that actually being the case. And we could make this "which nodedef is simpler" more explicit by adding an attribute (name TBD) where the first could have a value of "10" and the second a value of "20" (leaving room before, in the middle and after for other nodedefs to slide into the matching scale for this node wherever they need to. (Side note: in a <mix> node, I can't imagine a real scenario where the mix input would have neither a provided value or a connection, except in the situation where you were in progress wiring up a nodegraph. So there wouldn't likely be an ambiguity in a real scene.)

[jstone-lucasfilm]

@kwokcb Just to provide additional motivation for the development of rigorous matching rules, it's worth noting that marking inputs as "required" would not be sufficient to resolve this issue, as we'd still need a rigorous set of matching rules to handle documents that omit the input anyway (e.g. legacy documents). While we can make sure documents in this state emit validation warnings, we still need to generate valid shading code in every language from these documents, and rigorous precedence rules will still be required.

@dbsmythe I'm open to any simple set of rules that fixes the order in which matching definitions should be considered, though it would be ideal if our existing set of MaterialX attributes is sufficient to resolve all cases, as opposed to requiring new attributes to create order.

Giving precedence to the simplest data type (e.g. float > color3 > color4) is one valid option, though selecting the first matching node might be easier for MaterialX integrations to support, and it would give pipeline developers more control over whether their custom nodes are intended to take precedence over the standard set. Just to give a real-world example, some pipeline developers will intentionally replace shading models such as standard_surface with their own custom interfaces, and in this case they will likely take control over the library import order to ensure that their version takes precedence over the standard one.

[kwokcb]

About allowing invalid documents still generate code. I’ve always found this to be ambiguous since some validation failures can still allow generation to continue but others do not. In this case it still feels wrong to make a calculated guess as to what is the correct definition to use. At the time of usage there should be no ambiguity. This still allows for a node to be created without a unique signature but it’s very possible to guess one signature and then have code which sets an input be nonambiguous afterwards. In this the wrong definition is created and the subsequent input setting will fail.

[jstone-lucasfilm]

In order to keep this discussion focused, it may help to provide a concrete example to discuss, and I'll choose this one from the standard_surface_brick_procedural.mtlx example in our repository:

MaterialX/resources/Materials/Examples/StandardSurface/standard_surface_brick_procedural.mtlx

Line 54 in 5f9fee7

<clamp name="node_clamp_0" type="color3">

This clamp node is technically ambiguous, as its low and high inputs have not been stated, so they could either be 0.0 and 1.0, or they could be 0.0, 0.0, 0.0 and 1.0, 1.0, 1.0, depending on which nodedef is selected at runtime.

Following the design of the MaterialX standard libraries, though, it's important to note that this selection has no effect on rendered visuals, as these two interpretations generate identical results. This is an intentional design choice rather than an accident, and my recommendation would be to lean into this design principle, making it clear in our specification that nodes with multiple overlapping signatures are expected to generate identical visuals.

We can still provide a recommended order in which nodedefs are matched to nodes, as this will be valuable for the USD use case that originally motivated this thread, but we don't need to make ambiguous nodes such as the example above invalid in documents.

[jstone-lucasfilm]

I wanted to highlight #1753, which addresses an inconsistency in our NodeDef lookups across compilers, without changing the expectation of declaration order in the current codebase.

I'll leave this GitHub Issue open, so that we can discuss what order for NodeDef lookups would be ideal, and the recent fix should give us a more consistent baseline to build upon.

[jstone-lucasfilm]

Let's go ahead and close out this GitHub Issue, as our latest codebase resolves the earlier inconsistency in NodeDef lookups across platforms, and we can continue the discussion about whether improvements to our rules for NodeDef lookups are needed in the future.

Thanks for bringing this up, @ld-kerley!

[ld-kerley]

If it's the position of the project that node definitions should be selected by taking the first match - then maybe we should add some language as such to the specification? Currently an implementor doesn't have any guidance as to what to do if there is an ambiguous node resolution.

[jstone-lucasfilm]

@ld-kerley That sounds wise to me, and this new language should be valuable in the context of future normative specifications for MaterialX and USD, which need to remain independent of the codebases for our projects.