A rant about Microsoft's wonderful HLSL compiler.

Posted by spike1 on May 4, 2015, 11:41 a.m.

Ok, I have seriously got a bone to pick with the Unity team. I've been developing a shader generator quite a while now, and while Unity 5's new lighting and GI system kinda slowed me down, they were an improvement, fair enough. But they also decided to do a wonderful switch in regards to their shader compiler. Previously it compiled NVidia's CG shader language, which is pretty good. The new compiler uses HLSL, which looks about the same (It's just a bit more strict, for example previously I could put float3(0.2), and it would equal float3(0.2,0.2,0.2). Not anymore, but meh), however the compiler is genuinely broken. I'll give an example.

I was working on a coherent noise generator (Like perlin noise but blockier), that get's fed a coordinate and outputs a value. When working, the noise looks like this:

The way it works is pretty simple:

1. Receive coordinate (UV), and feed it into the function NoiseB2D(float2 P)

2. Find the cell that the coordinates are within, then calculate a random number for each bordering cell corner.

3. Interpolate between each cell point based on the relative position that the coordinate holds.

Here's the code:

float NoiseB2D(float2 P)
{
	float SS = Unique2D(P);//Always returns the same number for the same point.
	float SE = Unique2D(P+float2(1,0));
	float ES = Unique2D(P+float2(0,1));
	float EE = Unique2D(P+float2(1,1));
	float xx = Lerp2D(frac(P),SS,SE,ES,EE);
	return xx;
}

But for some reason, the noise was appearing strange, points would randomly cutoff and flicker as I moved the camera.

This is the kicker. I looked through the generated assembly code, and found that it referenced a number that was in another part of the code, 0.99999. It decided they were close enough, and managed to bundle them together in a few "mads". So instead of being +float2(1,0), it was +float2(0.99999,0), which thew the entire calculation out. I ended up having to change it to 1.0000001 just to fix this silly, destructive compiler bug. Looking through assembly code isn't fun.

EDIT; Just to clarify, the 0.9999 wasn't the entire problem, as you probably may have guessed it would just output different numbers. However, due to the precision of the floats on certain GPUs, it would cause the flickering as sometimes it would just add closer to 1, and other times not.

And this isn't the only case of the compiler doing destructive optimizations. In my perlin noise function, the frac function ((Returns x-floor(x)) or (In goes 16.45 , and out comes 0.45)) was being removed when the number going into it was a constant, fair enough. Except it didn't do the correct operation. I fed 1.9 into it and in the assembler it output 78.4! wtf!

This stuff is absolutely mind boggling, and also incredibly difficult to predict. The number of if statements I've had to add to Shader Sandwich has been enormous to try and predict these little compiler quirks.

Ok, that was cathartic, back to work XD.

Comments

spike1 9 years, 6 months ago

Fixed the noise image.

Phoebii 9 years, 6 months ago

I have no idea what are you talking about.

XD

Jani_Nykanen 9 years, 6 months ago

Yep, Microsoft's products suck. That's why I use OpenGL and GLSL (although I'm looking for the day when they are replaced by Vulkan).

Cpsgames 9 years, 6 months ago

Quote:
Microsoft's products suck

Visual Studio is fantastic. I don't see how it "sucks," unless people consider anything by Microsoft to be instantly bad.

Jani_Nykanen 9 years, 6 months ago

Quote:
I don't see how it "sucks," unless people consider anything by Microsoft to be instantly bad.
Yep, that's how I live: Microsoft automatically sucks, open-source projects rule the world! I close my ears from arguments that say even Linux can crash and have some serious security flaws.

But you are right, not all Microsoft's product suck, especially newer ones are pretty good (e.g. Windows 10 is still in beta, and it beats all the previous Windows versions).

Cpsgames 9 years, 6 months ago

I wasn't implying that, I just focused on your blanket statement of Microsoft's products. Either way, this is off topic so I'll just fly on out of here.

spike1 9 years, 6 months ago

@Zuurix

Same here XD.

Nopykon 9 years, 6 months ago

Not a bad blog. I'm into co-noise. ;P Last year I tried to create a game with only generated

assets for the spring competition, using diamond-square, perlin and some sample-maps. I also tried to make it tiny in exe size but I never finished. :/ How will you use this stuff?

P.S. DX owned GL last decade and GLSL is just as bothersome if not more than HLSL. I hope Vulkan brings order.

spike1 9 years, 6 months ago

Thanks, and yeah, coherent noise is pretty cool :D. It's part of an editor asset for Unity I'm making, which helps make shaders in a visual editor (Sorry for the cheap plug XD).

For example, in this image all the raindrops and puddles are drawn using perlin noise.

Or at the start of this video(Please mute it, it sucks XD) (Sorry, no image :( ) you can see a procedural wood texture I made :D.

Of course you can do even more stuff, like procedural terrain, grass motion, texture splatting, etc, so it's pretty useful. I hadn't seen the diamond-square one till now, I might add it in a future version of SS. Normally I used perlin for procedural terrain, but it really is too slow to use for the number of octaves you normally need so diamond-square could be useful :D.