QML changes in 2019.3

[Matthew Vitalone]

I had a pretty extensive custom QML UI built for a custom shader that runs correctly in 2019.2.  When I upgraded to 2019.3 it  is failing with some unhelpful warnings in the log.  I don't see any QML changes listed in the release notes for 2019.3.  Were there changes made that would explain why things stopped working?

[philippe hermite]

Our QML library was modified to follow the new guidelines from Qt. The scripting documentation was updated to reflect these changes. It should have no impact on your plugins, and the use of our QML library is fully compatible between 2019.2.x and 2019.3.x versions.
The new version of Qt introduces new warnings and errors. Most error message should be clear enough. However, if you need help fixing these warnings, feel free to reach out to me.

[Matthew Vitalone]

Okay I've narrowed down the different results I get in 2019.2 versus 2019.3 and it's very weird.  On 2019.2 my custom shader and custom qml ui both work correctly.  On 2019.3 my shader loads without error but my custom UI fails to load any of the parameters.
I reverted my shader back to an older version and in 2019.3 the custom UI started working again.  Then I started merging in data from the newest version of the shader piece by piece and reloading to see when the UI would break.  Once I get to a file length of 2525 lines of code in the shader, I get this error in the log "[Plugin - customUI/cloud_chamb...] Cannot read property 'description' of null" and the parameters don't load into the custom UI. 
If I remove one line so the length is 2524, it works correctly.  I'm testing with the extra lines just being simple comments "\\" to make sure it's not related to the code itself, and if I add one more past 2524, all parameters fail to load into a custom UI.

[Froyok]

Do you think you could share files that reproduce the issue ?

[Matthew Vitalone]

Yes.  Is there a private link I can send them to?  I can't share them publicly.

[Froyok]

I answered you by PM.

[Froyok]

So I gave a try at your shader + qml UI and even if I remove one or few lines I still get the same error message:

Code: [Select]

[Plugin] Custom UI located at 'file:///E:/DATA/Documents/Allegorithmic/Substance Painter/shelf/shaders/custom-ui/material-layering/custom-ui_cc.qml' used for shader parameters display
[Plugin - customUI/shelf/shade...] Cannot read property 'description' of null
Not sure why you don't get it all the time in your case.

As for inlining as I suggested by PM, I meant converting this:

Code: [Select]

vec3 flattenNormal(vec3 normal, float flatness)
{
    return normalize(mix(normal, vec3(0,0,1), flatness));
}
To this:

Code: [Select]

vec3 flattenNormal(vec3 normal, float flatness){ return normalize(mix(normal, vec3(0,0,1), flatness)); }As an example.

You will save a few line returns (if your issues is really from there). We don't support custom includes for now with our shader system. Could be related to a GPU drivers bug as well, since it is the one compiling the shader. What are your current hardware specifications ?

[Matthew Vitalone]

Hi Fabrice,
I narrowed down the cause somewhat.  It's not the length of the file that is making it fail, it's the time it takes for alg.shaders.materials() to return.  Below I've added a new version of displayShaderParameters() that will make the custom ui qml I sent you load.
The main difference I see between 2019.2 and 2019.3 is that this load time has gotten much longer.  There also seems to be an issue when you load the custom ui qml a second time.  I put a timer on how long alg.shaders.materials() takes to return and print it to the log.  On a scene with just a sphere loaded, my custom material, and no extra data added it takes 3.5 seconds to load initially.  if I then reload that file, it takes 62.3 seconds to return.  I tried to compare to 2019.2 but it doesn't load the version of the custom ui that waits for alg.shaders.materials() to return.  I know it is much faster since I didn't need to do any of these Promise/Resolve tricks to get it to load.  We are using NVIDIA graphics cards over here (RTX 2080, Intel Xeon 3.6GHZ, and 128GB ram)

function displayShaderParameters(shaderId) {
    try {
      // Retrieve the list of each shader parameters with its full description for each one

      // get Materials.  We do this via Promise as it can take a long time to return
      var materials = {};
      let materialsPromise = new Promise(resolve => resolve(alg.shaders.materials(shaderId)));
      currentTime = Date.now();
      materialsPromise.then(
        function(result){
            materials = result;
            var timeElapsed = Date.now() - currentTime;
            alg.log.info("alg.shaders.materials() returned in: " + timeElapsed/1000.0 + " seconds.")

           var parameters = alg.shaders.parameters(shaderId);
           var model = {};
          for (var i in materials) {
              model[materials.description.label] = {'material': materials, 'parameters': []};
          }

          // For each parameter, connect it to a QML component inside the
          // common parameters group or a material parameters group
          for (var i in parameters) {
              var parameter = parameters;
              if (parameter in materials) continue;
              var group = "group" in parameter.description? parameter.description.group : "";
              if (group in model) {
                model[group].parameters.push(parameter);
              }
              else {
                commonParameters.connectParameter(parameter);
              }
            }

          // Order parameters by the order of declaration in the shader
          for(var i in model) {
              model.parameters.sort(function(a,b) { return a.indexInShader-b.indexInShader })
          }
          materialsModel = Object.keys(model).map(function(v) { return model[v] })
              .sort(function(a,b) { return a.material.indexInShader-b.material.indexInShader });
          displayedMaterial = materialsModel[0].material.description.label;
        },
        error => alg.log.error(error)
      );
    }
    catch(e) {
      alg.log.error(e.message);
    }
  }

[Matthew Vitalone]

Just pinging this again.  Because the issue is not file length, inlining code doesn't get us up and running.  We're still unable to use 2019.3 currently.

[Froyok]

Sorry for the delay. We looked a bit into it and it seems related to some QML change my by Qt which we upgraded in Substance Painter 2019.3. It seems related to sync issues between some functions and we are still investigating.
It looks like your shader is so big it takes a lot of time to compute and QML tries to finish/update before the shader is available. This is why you get your error.

A workaround we found is to replace the call to some of our API functions by copies that you put in your own script:

Code: [Select]

  function filterObject(object, condition) {
    var outputObject = {};
    for (var i in object) {
      if (condition(object[i])) {
        outputObject[i] = object[i];
      }
    }
    return outputObject;
  }
  function displayShaderParameters(shaderId) {
    try {
      // Retrieve the list of each shader parameters with its full description for each one
      var parameters = alg.shaders.parameters(shaderId);
      var materials = filterObject(parameters, function(object) {
        var description = object.description;
        return 'usage' in description.properties && description.properties.usage === 'MATERIAL';
        }
      );
That should do the trick.

[Matthew Vitalone]

So far I seem to be getting similar results with your workaround as I was with my promise/then workaround.  It does succeed in loading, but it just takes MUCH longer than it did in 2019.2.  If I open the same file in both 2019.2 and 2019.3, switching between two texture sets (which causes the shader custom UI to re-generate) can go from about 2-3 seconds to update to 400 seconds to update.

[Froyok]

Yes indeed, it takes a lot more time than before. We don't know yet why, we need to investigate further to find the root cause. It's likely related to our update to Qt 5.12 with Substance Painter 2019.3, but we don't know for sure. Unfortunately we had to jump on other topics so this problem is currently in our backlog.

[Matthew Vitalone]

Tested out Painter 2020 and this issue still remains.  I was hoping additional updates to QML or your internal functions might have resolved this but that doesn't appear to be the case.  This is still a huge problem for us that prevents us from upgrading beyond Painter 2019.2.

[Matthew Vitalone]

Actually I tried the old workaround and it does work on Painter 2020.  Load times seem close to what they were in 2019.2.  Still slower than I was hoping for, but much better than what I was seeing in 2019.3.  It fails to return the shader parameters, without the workaround, which it didn't on 2019.2.

[xinyuanHu]

I have the same problem in development as this thread，My UI panel not works after adding an extra shader parameter.The log file shows the "[Plugin - customUI/project0/QS...] Cannot read property 'description' of null" error,I did a lot of debugging，Then it narrowed the question down further to two causes：
1.AlgWidgets version，When errors such as reporting occur, you need to modify the algWidget version of QML from 2.0 to 1.0：

Code: [Select]

import AlgWidgets 1.0，ps：AlgWidgets.Style version is not necessarily modified, but I still think the 1.0 Style is more aesthetically pleasing
2.Logical dependency of widgets and shader Parameter，If you have a lot of parameters, and the logic of drawing the UI needs to be judged by the parameters,You need to decouple the drawing UI logic from the shader parameters，like qml's visible，parameters connect etc.

In my case they did work, so good luck to another guys！