Author Topic: AO bake error in crevices  (Read 1148 times)

I'm having trouble getting my Ambient Occlusion to bake properly. Many crevices fail to shadow the innermost corner.
I'm wondering what settings I should use to fix this, if it can be done.
Here is an image of a demo scene I made to test materials, showing off the AO with default baker settings.
Last Edit: January 01, 2019, 02:08:37 am

Try unchecking the Ignore Backface option in the AO settings.

I tested that, but it didn't make a difference.

I figured out how to display the AO directly, and it makes the problem a lot more obvious.
I updated the OP.
Last Edit: January 01, 2019, 12:13:43 am

Are you baking off a High Poly, or as your using Low as High?

I am using a high poly mesh.
Also, the result is the same whether the AO is between separate geometry or just a crevice in a single piece of geometry. You can see the effect in the image; the larger rock mass is a single fused mesh without any intersecting geometry, but the smaller rocks and platform are all separate. The issue exists across all of them.

I also have been running into this issue - but I haven´t been able to solve it. First I thought it has to do with the scaling of the geometry, but no luck so far

I played with the values in the baker. the only value, which seemed to have an effect on the bright line in the crevice is the max occluder distance - but I´m not sure if lowering this value is what we want.
If I change the min occluder distance from its default value of 0,00001 to 0,001, the bright line becomes even more noticeable.

So maybe this is about the balance between min and max occluder distance? To me it seems like a weakness of the substance baker (same issues in designer) - at least compared with the very high quality of the other bakes at default settings

sorry for the small screenshots, but I had to zoom in quite a bit.

here are a few images:

Last Edit: January 02, 2019, 12:57:18 pm

Depending on your high poly mesh, these bright lines are normal. On the rocks in OP's post, if these 2 tall rocks are part of the same object and have a smooth transition between them, then the middle of the crevice faces outwards and has no other surface occluding the light it receives, resulting in that bright line.
Hard to say without seeing the high poly mesh.

makes sense... in my tests it was just a v-shaped geometry with a tiny smooth transition (I admit it)

but now I removed the transition (essentially using the low as highpoly) - and now I don´t have that bright line anymore

never thought of it like this, thanks for the input - I think I just understood the fundamental difference between a cavity map and an ao map :o

It's there when the rocks are separated pieces of geometry as well. You can see the same effect in the image between the rocks and the platform, as well as the ring on top of the platform. It was also there when the rocks were originally just bashed together. I only merged them to see if it would fix it.
The lowpoly is being generated automatically in Houdini, so the separation/connection of geometry is and was uniform between the high and the low.

sorry for having hijacked your thread - I believe the cavity/ao bake explanations do not apply to the intersecting geometry-bake

If it helps to figure out what's happening, I did test Concavity by using Mask Builder in Designer without an AO input, and the issue was not there with that.
I tried baking it in Designer as well. It took an eternity, because I assume it doesn't use the GPU, but the result had the same problem.
Finally I tried baking the AO in Marmoset Toolbag, and the issue was NOT there. At least not with the merged geometry. The crevices between the rocks and the platform still had the same issue... That aside, the result was still significantly better, and I'm hoping I won't have to do my bakes in a separate piece of software if I can help it.
Last Edit: January 05, 2019, 12:00:51 am

Could you share the mesh and normal map by any chance?

The site won't let me. If I try attaching them here the site flips over to trying to start a new topic after I hit Post. PMs don't seem to allow attachments at all.

Finally I tried baking the AO in Marmoset Toolbag, and the issue was NOT there. At least not with the merged geometry. The crevices between the rocks and the platform still had the same issue... That aside, the result was still significantly better, and I'm hoping I won't have to do my bakes in a separate piece of software if I can help it.

I am wondering, how Marmoset is dealing with the "bright line" issue - because even if (like Jeremie explained), the bright line for merged geometry with a smooth transition discussed in this thread is expected from a technical point of view, the marmoset bake without those bright lines is looking so much better.

I found a number of threads about the intersecting geometry causing undesired white areas, and all of them are pointing to backface culling vs double sided bakes as a solution. Not sure, why this doesn´t make a difference in your tests.

https://polycount.com/discussion/173259/ao-light-leaks-where-meshes-intersect-in-substance-painter
https://polycount.com/discussion/139339/ao-bake-problem-intersection
https://forum.allegorithmic.com/index.php/topic,13050.msg56706/highlight,white+ambient+occlusion.html#msg56706



Last Edit: January 05, 2019, 01:00:59 pm