Custom shader problems and limitations

[tobias.czyzewski]

Hi,

some time ago I worked on a custom shader for Substance Designer. After finishing it I wanted to support Substance Painter with the shader, too, but I realized that the shader has to be set up in a different way for Painter and that it's not possible to create a custom vertex shader there. Here's a link to a previous discussion on this forum: https://forum.allegorithmic.com/index.php/topic,16318.0.html.

However, I figured out that there might be a way to implement the shader the way I want even if I have to use the default vertex shader for painter. I was able to copy some pieces of code into the custom fragment shader for Substance Painter (I work with 2.6.1) but some things had to be changed according to the shader documentation for Substance Painter. It looks like it's not possible to declare a specific version of OpenGL nor to integrate custom extensions because the custom shader code seems to get integrated into other, hardcoded GLSL code. And this is my problem right now:

Code: [Select]

vec3 get_environment_colour_UPDATED(in vec3 direction , in float lod)
{
return _linear(textureCubeLod(s_environment_map, texcoordEnvSwizzle(direction), lod).rgb);
}

vec3 cube_ambient(in vec3 N)
{
  return _linear(textureCube(s_ambient, texcoordEnvSwizzle(N)).rgb);
}
In these few lines of code I use a function (textureCubeLod) which was removed in the OpenGL version used in Substance Painter by default. In addition I use a function (textureCube) which requires an extension but as mentioned before I'm unable to integrate custom extensions (Substance Painter generates errors if I do so). So here're the errors I get when I work without versions and extensions:

Code: [Select]

[GenericMaterial] FS compile log:
[GenericMaterial] ------------------
[GenericMaterial] File index:
[GenericMaterial] 0: shader-skeleton.frag.glsl
[GenericMaterial] 1: shader-common.frag.glsl
[GenericMaterial] 2: lib-utils.glsl
[GenericMaterial] 3: pbr-total-war.fs.glsl
[GenericMaterial]
[GenericMaterial] pbr-total-war.fs.glsl:208 | 3(208) : error C7616: global function textureCubeLod is removed after version 140
[GenericMaterial] pbr-total-war.fs.glsl:213 | 3(213) : error C7531: global function textureCube requires "#extension GL_NV_shadow_samplers_cube : enable" before use
So my question is: is there a way to work around these problems or is it impossible to do so because I would have to change things which are hardcoded?

Thanks for any help in advance!

[Jeremie Noguer]

Currently this is not possible due to our recent switch to Core Profile which made these functions deprecated. We need to take a look in our shader to see if we can simply activate flags to allow these textureCube functions or if it requires more changes.

[tobias.czyzewski]

Okay, thanks for the answer! Is it realistic to expect an update in the near future which adds these mentioned features?

I was able to work around the use of the textureCube functions because it would just have implemented an additional feature of the shader. However, unfortunately a new problem appeared which I want to illustrate with some pictures:


This picture shows how my custom shader looks like in Substance Designer.


This picture shows how my custom shader looks like in Substance Painter. Since Painter requires different shader files I adjusted the shader of Designer to get it to work in Painter but in the end the calculations are exactly the same.

So I'm not exactly sure what the problem is. But I figured out that there might be a default setting of Painter which sets the darkest rendered color:


Here I only applied a completely back diffuse / base color texture on the model.

But it isn't rendered completely black obviously. However, the shader code should render the model in uniform black in this case and it works in Substance Designer. So it looks like the color is overridden by a default color set in Substance Painter. Am I right with the assumption that there is a value which generates this behaviour? If yes, how can I turn this off?

Any help / advice is appreciated!

[tobias.czyzewski]

A short update but still the same problem:

it looks like it isn't necessary related to Substance Painter settings. But it looks like there gets something messed up with the color calculations:


This is how the diffuse color should look like and how the actual texture looks like.


This is how the diffuse gets rendered if I only activate the diffuse channel.

I'm not sure if this problem directly relates to an issue with Painter anymore but maybe someone of you has an idea what the problem might be anyway?

[tobias.czyzewski]

Okay, let's do a third post in a row. I finally figured out the problem! Since I took most of the code from my shader for Substance Designer I realized that I was manually converting the color spaces for the textures in the code. However, Substance Painter sets the color space automatically (if selected) and because I didn't remove the manual color space calculations everything got messed up. So I fixed the calculations and everything works fine now.