Well you can clearly see that the normal map is sharp and the height map is blurry, but to make it visible, here is a corrected height map. This is what it should look like coming out of substance painter.

Im painting the material only on the color and height channel and not the normal channel. In substance the normal channel is completely blank, yet on export its dumping information into the normal channel, even though it should be blank. So the height information is basically getting split across the 2 images when it should be in a single one. So the sharp details if you will are going to the normal and the softer details are going to the height. But this should not be happening if the normal map in painter is blank and normal export is disabled. So what I end up with is a blurry height map that's missing a lot of details.
Now I was able to correct this height map by joining it with the normal map in photoshop, which works fine for this since its just a tileable tile. But if I where to paint the character, and then export it I would not be able to combine the 2 images since the normal channel contains data from the original Zbrush normal map. So the question is, why is it putting data into the normal channel when its disabled? and how can I stop it from doing this so I get all the height data into the height map? I shouldn't have to correct height maps in post before using them on a model.
Here are a few quick and dirty renders to show the difference in detail between the blurry and sharp height map. The map is simply tiled across the entire arm. There is clearly a difference between the 2. Also a skin ref image.
For some reason the board image host isnt accepting the render so I had to upload them to an external host. Need to click on the images to see the full size, can only see the difference at full size.
Blurry map

Sharp Map

Blurry Map

Sharp Map

Skin for Reference

If I do use the normal channel and render with both images in blender the result is the same as the sharp image, so clearly the sharp height map is what should be coming out when the normal channel is not used. The reason I am not using the normal channel is because my normal map was created by zbrush and substance cant handle a 16K map and should not contain any skin detail, just model detail. So since that map is already using the normal channel, I need all the skin data to go into the height map.