Batch Tools for channel packing and pipeline flexibility

[Teck Lee Tan]

I've been trying to figure out if what I'd like to do is possible with Substance and/or the batch tools.

As we're currently at the prototyping phase of a new project and don't have anything locked down wrt what texture maps we're going to need/use and how to pack them, I'd like to put together a relatively flexible pipeline to allow us to change things down the line in a way that's transparent to the artists.

Currently what I'm envisioning is for the artists to author their substances with all major outputs defined (everything from albedo and normal map, to metallic, emissive, height, etc). We would then take those outputs and export what we need, channel packed as needed (eg. Metallic, Roughness, Emissive packed to a single RGB mask texture).

Now while I could do this via photoshop scripts (or if I were working in Unity, take advantage of the assetpostprocessor), I'm trying to figure out if it's possible to keep the entire pipeline within Substance/batch tools.

I've built what's essentially a routing graph, which takes the abovementioned inputs, and has outputs for the final maps (channel packed). I was hoping to be able to use batch tools to basically take the authored substances, hook up their outputs to named inputs in the routing graph, and then export the resulting textures. I don't seem to be able to do this without a lot of intermediate batch steps, though! (cook to sbsar, export textures, mutate specialise and hook textures to inputs, cook resulting specialised graph, render to final textures).

Is what I'm describing doable in SD/batch tools? Or am I barking up the wrong tree and should just revert to photoshop scripts? Is it at all possible for batch tools to take an SBS (not archive) directly, and hook its outputs to another SBS graph's inputs, and directly export the resulting textures?

[Vincent Gault]

Looping @Gaetan Lassagne who may have a better answer

[mark.sneddon]

Yeah there is a couple of ways to do this.

Firstly you can probably just put an instance of your intermediate sbs into each artist sbs at the end of a graph, then whenever you change your source intermediate sbs you can batch re-export all of the artist sbs. I'm not sure how well that would deal with drastic changes to the intermediate sbs (such as adding/removing a lot of stuff at the same time)

You can also patch the intermediate sbs node directly into the artist.sbs files - but that requires a lot of hacking the xml format of the .sbs open and stitching individual nodes together, and I wouldn't recommend it.

[Teck Lee Tan]

I'm actually becoming quite comfortable patching the sbs files (I'm currently doing some post-save validation and health checks at the moment, parsing via beautifulsoup). That said, you're right that that's potentially quite brittle. I'm not currently planning to have the router in the artist sbs, though, purely because the provided shaders won't work with the channel-packed textures. It occurs to me I could provide custom shaders, though...

My current workflow simply involves our custom texture export script triggering the batch tools to export all outputs in the correct folders with the correct naming, and subsequently using imagemagick to channel pack as needed. This works well enough as I can also use the same functionality to batch through some/all of our substances if necessary.

Cheers for the suggestions, though!

Moving forward, I might put together something akin to the Substance Center app I saw on your site.

[edit]
I'm realising that the imagemagick route may be the more flexible one, since while modifying the way we pack channels might be trivial with a router nodegraph, adding outputs will bring us back to patching the sbs to add output nodes and hook them up.
Still hoping for a proper API!
[/edit]

[mark.sneddon]

I guess you could also use sbsmutator for this functionality. You would use mutator to create a specialized "packer" sbs for each of your artist SBS. You hook up the outputted textures from the artist SBS as inputs to the packer sbs, and render it, which outputs the packed textures into the same location. You could then safely delete the packer sbs. This seems like it would be a fast process, and you could trivially change the packed format of your textures by modifying your intermediate sbs and then running this process on your artist sbs library.