Help to optimize memory

[Gilles Boulard 0]

Hello,

substance api is not really helpful : not up to date (api refers to 2.2 when plugin is 2.5), and almost no info on public functions, not even a comment in the code above functions.

we are using substance to generate lots of random different textures from like maybe 50+ SubstanceGraph.
we would like to have a random texture each time we launch the app or on demand during a loading for example. but we don't need to make the txture chage after it, so apparently we don't need to keep the SubstanceGraph.

Currently the "Total System Memory Usage" is sky rocketting (each substance graph is generating 3 1024² textures : albedo, normal, metallic).

What would be the script to keep only the material and textures once the generation has been done. Would it be enough to solve our memory problem ?

we've tried the Substance.Free(string assetPath) after we copied the material, but textures seems to be destroyed.
maybe keep a ref on textures with GetGeneratedTextures ?

any help aprreciated

[Gilles Boulard 0]

still struggling through memory problems....

if i used the already generated default material and textures in the archive, memory is totally fine.
as soon as i will use a QueueForRender, Render(A)Sync memory is going up drastically, like 1Go memory per substance...

where does all that memory comes from ? is it only the generated textures? deos the graph have some sort of garbage data that i can clean up  ?

[Gilles Boulard 0]

Ok, so i idg a bit more in the problem

this is apparently coming from the QueueForRender() +RenderAsync()

i tried to copy 1000 times the 3 same 512² textures, and got like 2Go more memory.
then i launch 50 times the same QueueForRender() +RenderAsync() (with a random seed) WITHOUT storing the resulting textures and the memory got up 2Go at each generation!

note: when exiting playmode in unity, the memory stays the same in edit mode, it needs to get "played" again to reset back to 0.

So :
-  what garbage does he process keep in memory ?
- can it come from our archive ? what could it be ?
- and finally : how can i clean all this garbage once the texture has been generated ?

[-kg-]

Ok, so i idg a bit more in the problem

this is apparently coming from the QueueForRender() +RenderAsync()

i tried to copy 1000 times the 3 same 512² textures, and got like 2Go more memory.
then i launch 50 times the same QueueForRender() +RenderAsync() (with a random seed) WITHOUT storing the resulting textures and the memory got up 2Go at each generation!

note: when exiting playmode in unity, the memory stays the same in edit mode, it needs to get "played" again to reset back to 0.

So :
-  what garbage does he process keep in memory ?
- can it come from our archive ? what could it be ?
- and finally : how can i clean all this garbage once the texture has been generated ?

@Gilles Boulard 0 ,

can you provide a unitypackage or small project that exhibits this issue? .sbs files would also be useful as well (if you created these Substance files) to see if there is any optimization to also be done there.

[Gilles Boulard 0]

I Will try to build a sample app (and ask if i can send those fileto my hierarchy ^^)

further investigations :
i can generate endlessly on 1 archive and it will be just fine.
as soon as i add different SubstanceGraph, it takes like 2Go in memory per graph.
I wouldn't mind if this was temporary, but this memory is never freed.

so the problem could be summarize in one question :
what is the way to completely clear the memory that is used by an archive once textures have been generated ?

[Gilles Boulard 0]

sample send to @keston (mp/mail)

[-kg-]

sample send to @keston (mp/mail)

@Gilles Boulard 0 ,

Thank you! I got it, will take a look. 

[Gilles Boulard 0]

fyi @keston (and others)

we did manage to pass-through the problem and avoid memory stacking at runtime :

- put substances in assetbundles (so it can be reloaded later on)
- load assetbundles wisth substances
- render textures with randomseed
- CopyTextures
- once a substance archive is no longer used, call

Code: [Select]

Substance.Game.Substance.Unload(graph.substance.assetpath)
to free memory , warning don't reuse any graph from taht archive anymore
- do not store any ref to the substancegraph, only keep the copied textures and materials

-if you need to regenerate textures later on :
- reload the assetbundle to restore the substance archive
-redo render+copy process

still, it would be nice to be able to reload the graph without having to reload the entire assets.
and, when loading the unity project for the first time, we still have to do it by batch of 4 or 5 substance files and closing unity in between to free memory. so a patch is still welcomed (but we are not stuck anymore )

[-kg-]

fyi @keston (and others)

we did manage to pass-through the problem and avoid memory stacking at runtime :

- put substances in assetbundles (so it can be reloaded later on)
- load assetbundles wisth substances
- render textures with randomseed
- CopyTextures
- once a substance archive is no longer used, call

Code: [Select]

Substance.Game.Substance.Unload(graph.substance.assetpath)
to free memory , warning don't reuse any graph from taht archive anymore
- do not store any ref to the substancegraph, only keep the copied textures and materials

-if you need to regenerate textures later on :
- reload the assetbundle to restore the substance archive
-redo render+copy process

still, it would be nice to be able to reload the graph without having to reload the entire assets.
and, when loading the unity project for the first time, we still have to do it by batch of 4 or 5 substance files and closing unity in between to free memory. so a patch is still welcomed (but we are not stuck anymore )

Great info, thanks @Gilles Boulard 0 !