Author Topic: how to mask a mask  (Read 570 times)

Hello guys. A beginner question here:
How can I paint a mask to limit an already existing bitmap mask, WITHOUT affecting the other layers in the mask stack? I know I can load multiple bitmaps into one stack to use for masking, and I know I can add, say a paint layer, set it to multiply and have that as a driver to reveal the masks below. But, how can I get the paint layer to affect only one specific bitmap mask in the stack, and not the other ones below? It seems like a waist of recourses to have to pull in the rgb map several times to create different masks for each instance of the rgb. A much better approach would be to be able to resolve everything with just masks on one single rgb map, but I seem to fail at getting this to work...
Does anyone know how to do this?

Not sure if I totally got what you want to do, but maybe doing a group and adding a mask to that group would help.
Inside the group could be an other layers or groups with masks.

Hello. Perhaps the attached jpg makes my issue clearer? I hope so. Does anyone know the answer to this?

@stger: What I would like is to be able to just use the masking stack as is. I cannot create groups on this stack as far as I can see. Or am I wrong?


If I understand the question you would just put your layers in a Folder and apply a mask to the folder itself, then you can keep doing this Folder->Folder->Folder to keep masking. This is what stger was suggesting.
I teach people how to use Substance Painter. :)

@CAGameDEV: Thanks for the reply. Does this not just do the same thing that I was describing as a problem in the jpg - namely that every mask applied to , in this case, every new folder in the stack will affect all other masks downwards? I would like it to affect only one mask in the stack, as if being linked to this specific mask only. Sorry if these are stupid questions, but I am still not getting it.


I might have missed something when I first looked at it. What you're looking to do is easy to accomplish assuming I got it this time.

You want to take image 1, add a blue with a mask and pattern, then edit the mask to mask out the pattern part but retain the blue?

I teach people how to use Substance Painter. :)

Not quite. If you look at the jpg I attached earlier it is a bit more complicated than this. Imagine that instead of painting an edit to the mask that you have shown in your image, you apply, say, a generator (or a tri-planar bitmap) as the edit. This will apply the masking edit across the entire object. Now, in a second step, limit the applied generator effect to ONLY the side of the cube you did the paint mask on (using a painted mask). And do this WITHOUT AFFECTING the first mask (the pattern). This is what I don't understand how to do.
Is this more clear?

 :o :o :o

Maybe the way you're describing this is a bit hard to follow especially since what you had shown visually didn't give me that impression. In your final image it just looks like you're blotting out some of the fill layer in the mask to be honest...

So here is the mask fill as per the prior example. Now I'm applying a dirt generator in the second image, but in the 3rd it only affects to forehead part without masking out the layer underneath.

Is this what you're asking about?

If not I have zero idea how to interpret this any other way based on your example as this is exactly that, I use one fill for the entire head, a second generator for the dirt effect, and like you said in your post it only affects a part of the mesh without affecting the first mask (the pattern under remains).
I teach people how to use Substance Painter. :)

That looks more like it, yes. As I said, I might be asking stupid questions. I am a compositor that uses Nuke daily, and in a node based system making masks such as what I want is trivially easy, but the working process is very different from a layer stack method, hence my confusion (especially since I am new to this program).

Can you please show me the layer stack and how it is set up in your example, as well as the results on the head?


You can't apply a second fill effect to the mask, and then mask that fill effect using a paint effect in that same effect stack.

What you need to do is apply the first fill effect to the mask of a fill layer, then put that fill effect into a group, apply the second fill effect to the mask of the group, and then create a paint effect in the effect stack of that group to paint away the mask.

If thats not what you want, I'm lost.
Last Edit: January 27, 2021, 12:50:59 pm

Yea, this is 100% what I did above, and talked about prior so I'm lost to what else this could be???  :o :o :o
I teach people how to use Substance Painter. :)

Hello again. Finally had some time to revisit the issue again. I think that my brain ist starting to get it now. Thank you for your answers! One little thing still: when I look at my stack and the stack in your picture they look a little different. To me it looks as if you have applied fills straight onto your group, whereas I have had to first create a mask on the group and then a fill on that mask. When I try to create a fill on the group itselft the fill option is greyed out. Only possible on the mask. Is this just a UI difference, or a difference in how the UI looks between versions? Or perhaps some settings in your preferences. I've attached an image to show how my stack looks like.
Thanks again :)