Substance Plugin Update issue making graphs to recalculate

[RenderPub]

Hey,

After updating recent substance plugin, many of my Unreal projects do have substance graphs, so the problem now is for each material it will recalculate these graphs, and well it makes unreal crash!after a while. if we don't go through this process none of my previous substance graphs or materials work out! its a waste! Being spent so much time, end result is this? its really a problem. Is there a solution for this or please let us know before updates are pushed and changes that are required. Because it just don't work out if the calculations take a long time and since there will be lot materials, its very difficult to maintain project. Is there any command line process so that we can run on background without opening the unreal?

[NotYetVisible]

Same here, the materials are unuseable, but we can't recalculate the graphs because the GPU runs ot of VRAM (11GB). Is there any possibilty to get the previous version of the plugin or some kind of a fix? Running the process partial doesn't work either, because it restarts the process after the last crash next time...

[Cpt.Sloth1992]

+1, I can´t open most of my levels as it always crashes during the rebuilding of the Graphs. Only happens after the latest plugin update.

[juangauthier]

Got the same problem! works fine with the previous.
Any way to reverse to the previous substance plugin in Unreal??

[-kg-]

Hey,

After updating recent substance plugin, many of my Unreal projects do have substance graphs, so the problem now is for each material it will recalculate these graphs, and well it makes unreal crash!after a while. if we don't go through this process none of my previous substance graphs or materials work out! its a waste! Being spent so much time, end result is this? its really a problem. Is there a solution for this or please let us know before updates are pushed and changes that are required. Because it just don't work out if the calculations take a long time and since there will be lot materials, its very difficult to maintain project. Is there any command line process so that we can run on background without opening the unreal?

Hey guys @juangauthier , @Cpt.Sloth1992 , @NotYetVisible , @RenderPub ,

We are currently tracking this issue. How many Substance files are in your projects? Are they large scale projects? Are they 'heavy' Substance files?

I'm currently attempting to repro this issue on my end, but any info or even repro projects / Substance files would be useful.

Sorry for the inconvenience in the meantime!

[Cpt.Sloth1992]

I fixed it on my end by going to the folder of each substance factory inside UE4 and manually unchecked and checked each map channel in the substance instances. I noticed that every map was generated 3 times for some reason so I deleted the
 dublicates manually before checking the map channel again. After doing this to every Substance File, the plugin still wanted to rebuild the graphs when I opened any level, but it did not run out of memory anymore.

I don´t know why there were 3 copies of each map, it was not this way before the upgrade as far as I can tell and it was not the number of rebuild attempts I started.

Overall I would say that I have 20-30 Substance Files in my project, most of which are full materials with ~6 outputs.
It is also a long-time project that started on UE 4.15 or before. I only got 4Gb Vram, so if the memory is not freed up after one graph is rebuild that might easily fill the memory, especially if each map is generated 3 times.

One more thing: the first of the three maps (by name suffix) was always corrupted and just showed (x,x,x,0) Values.

[-kg-]

I fixed it on my end by going to the folder of each substance factory inside UE4 and manually unchecked and checked each map channel in the substance instances. I noticed that every map was generated 3 times for some reason so I deleted the
 dublicates manually before checking the map channel again. After doing this to every Substance File, the plugin still wanted to rebuild the graphs when I opened any level, but it did not run out of memory anymore.

I don´t know why there were 3 copies of each map, it was not this way before the upgrade as far as I can tell and it was not the number of rebuild attempts I started.

Overall I would say that I have 20-30 Substance Files in my project, most of which are full materials with ~6 outputs.
It is also a long-time project that started on UE 4.15 or before. I only got 4Gb Vram, so if the memory is not freed up after one graph is rebuild that might easily fill the memory, especially if each map is generated 3 times.

One more thing: the first of the three maps (by name suffix) was always corrupted and just showed (x,x,x,0) Values.

@Cpt.Sloth1992 ,

Thanks for the info, this was very helpful. I'm glad you were able to find a workaround for now.

[GeeksGoneBad]

This is happening to us as well - any resolution to this other than the work around? we have way too many materials loaded to hand edit each one

[segev@missingworldsmedia.com]

Hey,

After updating recent substance plugin, many of my Unreal projects do have substance graphs, so the problem now is for each material it will recalculate these graphs, and well it makes unreal crash!after a while. if we don't go through this process none of my previous substance graphs or materials work out! its a waste! Being spent so much time, end result is this? its really a problem. Is there a solution for this or please let us know before updates are pushed and changes that are required. Because it just don't work out if the calculations take a long time and since there will be lot materials, its very difficult to maintain project. Is there any command line process so that we can run on background without opening the unreal?

Hey guys @juangauthier , @Cpt.Sloth1992 , @NotYetVisible , @RenderPub ,

We are currently tracking this issue. How many Substance files are in your projects? Are they large scale projects? Are they 'heavy' Substance files?

I'm currently attempting to repro this issue on my end, but any info or even repro projects / Substance files would be useful.

Sorry for the inconvenience in the meantime!

Well, I can provide that, but it's about 50 gigs. How can I get it to you? (I'm working with GeeksGoneBad here.)

[-kg-]

Hey,

After updating recent substance plugin, many of my Unreal projects do have substance graphs, so the problem now is for each material it will recalculate these graphs, and well it makes unreal crash!after a while. if we don't go through this process none of my previous substance graphs or materials work out! its a waste! Being spent so much time, end result is this? its really a problem. Is there a solution for this or please let us know before updates are pushed and changes that are required. Because it just don't work out if the calculations take a long time and since there will be lot materials, its very difficult to maintain project. Is there any command line process so that we can run on background without opening the unreal?

Hey guys @juangauthier , @Cpt.Sloth1992 , @NotYetVisible , @RenderPub ,

We are currently tracking this issue. How many Substance files are in your projects? Are they large scale projects? Are they 'heavy' Substance files?

I'm currently attempting to repro this issue on my end, but any info or even repro projects / Substance files would be useful.

Sorry for the inconvenience in the meantime!

Well, I can provide that, but it's about 50 gigs. How can I get it to you? (I'm working with GeeksGoneBad here.)

@segev@missingworldsmedia.com ,

Can you provide a few Substances that are in the project (pre-update), so that I can attempt to reproduce the issue on my end? What version of UE4 are you updating from?

In trying to reproduce this, I went from 4.23 - 4.24 with over 100 Substances and was able to update fine, I think it might be related to something with the Substance files themselves.  Are these Substance files hand-created or are they from Source?

[segev@missingworldsmedia.com]

Wes McDermott just reached out to me from a request I put in before finding this thread - he gave me an @adobe.com email to send this to. I have done so, and sent him a sample pre-update material.
Can you reach him? No sense getting it going down two paths. I've notified him of this thread's existence.
I am 85% sure they're from Source.

I should also note that the crash happens with only one referenced material in the project - I kinda deleted all the rest as a test. (it was a copy of the project, I just did some destructive testing on it - removing all the materials let it open fine, putting any one back in caused the project to enter the failure mode.)

[Wes McDermott]

Wes McDermott just reached out to me from a request I put in before finding this thread - he gave me an @adobe.com email to send this to. I have done so, and sent him a sample pre-update material.
Can you reach him? No sense getting it going down two paths. I've notified him of this thread's existence.
I am 85% sure they're from Source.

I should also note that the crash happens with only one referenced material in the project - I kinda deleted all the rest as a test. (it was a copy of the project, I just did some destructive testing on it - removing all the materials let it open fine, putting any one back in caused the project to enter the failure mode.)

Hi,

I received your email and was able to get the zip file you sent. I have sent this to our QA team.

Cheers,
Wes

[NotYetVisible]

Hi,

the issue appeared after We upgraded from the Substance Plugin Version 4.24.0.1 to the current version. An upgrade from Unreal 4.23 to 4.24 didn't cause any problems, so it might be related to the plugin itself. We fixed it by downgrading the plugin to 4.24.0.1 from another PC, where we did not install the current version. But I fear the moment, when we upgrade to 4.25...

[Wes McDermott]

Hi,

the issue appeared after We upgraded from the Substance Plugin Version 4.24.0.1 to the current version. An upgrade from Unreal 4.23 to 4.24 didn't cause any problems, so it might be related to the plugin itself. We fixed it by downgrading the plugin to 4.24.0.1 from another PC, where we did not install the current version. But I fear the moment, when we upgrade to 4.25...

Hi,

Are you experiencing the same issue as posted above where you are getting a video memory error? We are looking into this problem to see what the cause but the team is needing to find a way to reproduce this. The Substances in your project, are they coming from Substance Source?

I have more information from the dev team.

We are now using UE4's serialization for anything new that is added to the plugin such as utexture2d and material instances. However, we are still needing to do manual serialization of data stored for the Substance object imported. This could cause issues if data was stored differently between versions. We have not been able to completely remove the manual serialization, as it would break some projects across all versions of our plugin. 

The way its working with the 4.24.0.3 update.
Assets created in 4.16 or before will be missing data, these can persist if the project has been updated through versions. The missing data can cause issues with the new 4.24.0.3 plugin since we have to update the objects to be compatible with utexture2d and material instancing changes.

Can you please let us know the following?

1. The version number of UE4 the substance materials were first added to the project. Was it 4.16?
2. How much vram does your GPU have?
3. How big are the substance materials in your project? Do they have embedded or nodes set to (absolute) generate high resolutions such 4K?


Cheers,
Wes

[segev@missingworldsmedia.com]

Okay. So I've fixed most problems by the following technique:
I move one folder at a time to the project
I open unreal
It complains about substance graphs
I say yes
it does a lot of things, then crashes
I go into the folder, find everything that's been modified.
Delete the old files, rename the file_name1.uasset to file_name.uasset.
Re-open unreal
and it doesn't substance graph on those files again

But! we're now getting the following message:


 /Game/Materials/Textures/Substances/Metal/Aged_Scratched_Metal_MAT : Failed import for SubstanceTexture2D /Game/Materials/Textures/Substances/Metal/Aged_Scratched_Metal_INST_roughness.Aged_Scratched_Metal_INST_roughness Referenced by export MaterialExpressionTextureSampleParameter2D /Game/Materials/Textures/Substances/Metal/Aged_Scratched_Metal_MAT.Aged_Scratched_Metal_MAT:MaterialExpressionTextureSampleParameter2D_2

I mean a _lot_ of them, 900+

Our Substances are older than 4.16, yeah.

How do we fix this?