I am seeing this same problem. I don't usually have this problem when there is just a few B2M materials in a scene, but as I continue to work on a Scene I get this problem more and more.
The worst part for me is that when I load up the scene, things look fine, but after I hit play and it generates the maps for runntime, I end up only having like 5 materials, and these 5 materials are applied to everything in my scene.
When I hit stop, all the materials are messed up.
Sometimes I have to manually go through and drag the original texture back up to the B2M material to get it to update to what it should be, but sometimes this doesn't do anything. On those times where this doesn't do anything I will attempt to Reimport that B2M stack, which will either cause Unity to freeze, or to crash Unity.