**Since I wrote this post I've written a new technical paper called "Surface Gradient Based Bump Mapping Framework" which does a better and more complete job describing the following.**

**All my papers can be found at https://mmikkelsen3d.blogspot.com/p/3d-graphics-papers.html**

So I wrote a small demo of the no tangents bump mapping technique from my paper and Andy Davies has worked with me by supplying the Gothic window model and textures.

There is both a binary and source code available. However, it is a Direct3D11 sample so for those with older cards......get an upgrade!

When running the Gothic window you can toggle between height map and derivative map on the M key.

For the model on the right there are no options since it's doing triplanar bump mapping.

In other words it's generating texture coordinates from three planar projections and then mixing based on the normal which would be significantly more cumbersome to achieve using conventional tangent space based normal mapping since no one likes to store three sets of tangent spaces.

I would also like to say thank you to Rune Stubbe of Square Enix for pointing out that triplanar is a good application for my method!

Finally, the triplanar example on the right is using three different textures (one for each plane). There is a derivative map, a height map and a procedural function. For anyone who's still not getting this. There is no texture unwrap in the triplanar case! :)

I'd also like to point out that the shader on the Gothic window is using an auto-generated bump scale to match xNormal so this sample is a good reference for that as well. The triplanar is a good reference for seeing how you can mix different kinds of derivatives. And this includes scenarios where these are obtained from different texture spaces.

That's it for this time.

Very nice work :-) I've been watching your blog for a while and do hope to test out your technique at some point. I develop a voxel terrain library (http://www.volumesoffun.com/wordpress/?page_id=8), and the terrain can be editied at runtime which basically means no precomputed UVs or tangent spaces. My users have been using triplanar texturing for a while and it seems like your technique could be a nice addition.

ReplyDeleteDavid

Once again, nice work!

ReplyDeleteI discuss derivative map with a character artist which tell me how to tweak the derivative map once generated by xnormal ?

Do you have any advice for him ? Actually, the artist generate normal map with xnormal then tweak it under crazybump or even photoshop. What could be a good workflow for derivative map ?

And as a side question, is the technic supporting skinned character (But I am sure it should :) ) ?

Thank you for sharing your great work.

To you David just wanted to say thank for the feedback and if you ever get your hands on some screen shots using this with your stuff I'd love for you to show them :) Thanks.

ReplyDeleteAs for you Sebastien thank you to you too.

I do have several things to say actually. First of all tweaking a derivative map is no harder then tweaking a normal map from an artists point of view. About the same. If he wants a more familiar look he can do ctrl+i in photoshop on all 3 channels and it'll look a lot like a normal map though in reality blue channel is completely unused. Also yes the method definitely works with skinning.

If the artist is looking for ways that are more intuitive to interface with his bump map then another approach which I am personally a huge fan of is working on floating point height maps during the authoring stage. You can export these from zbrush (or multi-res bake in Blender). Other options are of course painting bump maps using Blender where you can see the lit result interactively:

http://cgcookie.com/blender/2011/10/18/using-the-texture-paint-layer-add-on/

http://vimeo.com/21186170

On the code/tools side you could simply convert to a derivative map at the last minute but still allow the artist to work on a floating point height map until the end.

If not then using ctrl+i should still let him get away with quite a lot of how he's used to tweaking normal maps.

Actually, tweaking derivative maps is probably slightly easier because the blue channel has no impact on results. you can mix freely between multiple derivative maps. If you want to reduce intensity then simply mix the derivative map with another map that's (128,128,128)

ReplyDeleteSorry here I am posting again :) I just wanted to say also as you see in the demo in many cases you can even get very reasonable results even by using a BC4 height map instead of a BC5 derivative map. The BC5 derivative map is same size as a normal map but a BC4 height map is half the size. So this is another motivation for height maps. The height map that's in the demo is one that I reversed from the derivative map using my own commandline tool but of course if the artist actually works with height maps this would not be needed.

ReplyDeleteIn the reduce intensity case I meant (128,128,0) of course unless you did a ctrl+i on it in which case it becomes (128,128,255) that you want to mix with.

ReplyDeleteThank you for all the advices.

ReplyDelete> you can mix freely between multiple derivative maps

by mix, you mean "lerp" ? Is there any operation to do in postprocess after the mix, like normalize for normal map ?

Cheers

>Is there any operation to do in postprocess after the mix, like normalize for normal map

ReplyDeleteYes I mean like lerp. And no there's nothing you have to do after mixing/lerping derivatives to normalize. You can mix as you please with impunity. Once you're done you just use the final derivative.

You can mix these height derivatives as long as they are given w.r.t. the same domain. So for instance mixing in photoshop works. You can do the same thing in a shader but here you can even mix when they are not given w.r.t. the same domain space as you see in the triplanar bump shader. All 3 height derivatives are obtained and are then obtained w.r.t. screen-space and then are finally mixed here. The final height derivative is then just inserted into the usual normal perturbation code. Hope that clears it up. This is a strong advantage to this technique that it's so extremely flexible when it comes to combining derivative signals.

I think this is pretty inspiring work. It is seldom I read a paper that gives mean entirely new perspective on a topic I thought I already knew. That it also leads to a practical solution is even better :)

ReplyDeleteI have experimented with derivative maps at work, and it works as advertised, with the small caveat that you now have another magic parameter (the maximum derivative) to tweak. I was a little worried that the new distribution of representable directions would require more precision, but it seems, with a bit of tweaking, you can get reasonable results even with dxt5 (green+alpha) encoding.

Not having to worry about generating tangent space vectors, which is especially nice for dynamic objects, makes this feel like a much more elegant solution. The linearity of the maps is a particularly nice added bonus, especially for us, as we do a lot of normal map blending.

That the derivative map of the blend of two heightmaps is the same as the blend of their derivative maps gives it all a nice physical interpretation - it just feels natural, whereas blending normal maps does not! You are left with either blending the normals directly (yuck) or essentially transforming them to derivatives (divide by z), blending and then transforming them back (yuck^2). This gets perticularly ackward if the normals are not from the same tangent space. With derivative maps this is all simple and elegant.

Implementing this I also noticed what seems to be a small optimization opportunity. It seems the following expressions in the transformation from derivatives to normal can be simplified slightly:

float3 vGrad = sign(fDet) * ( dHdxy.x * R1 + dHdxy.y * R2 );

return normalize( abs(fDet)*surf_norm - vGrad);

As we end up with normalizing the result, we can freely multiply (abs(fDet)*surf_norm - vGrad) with any scalar, as long as it's positive. If we multiply the expression with abs(fDet), use the identities abs(fDet)^2=fDet^2 and sign(fDet)abs(fDet)=fDet and rearrange the terms we end up with:

float3 vGrad = ( dHdxy.x * R1 + dHdxy.y * R2 );

return normalize( fDet*(fDet*surf_norm - vGrad));

Although it is the same number of muls and adds, we now got rid of the abs and the sign. abs is essentially free, but sign is not. This of course depends on the platform, but I can't really see it being worse. In terms of d3d9 shader assembly it seems to be a saving of 3 instructions.

But I guess in many cases you know that fDet is always of the same sign (when dot(face_normal, vertex_normal) is always of the same sign). In this case, this is not really relevant, and you can omit the outermost multiplication with fDet as well.

Keep doing cool stuff! :)

Thanks for the kind words :)

ReplyDeleteAnd that's a neat trick you're suggesting I'll have to give it a shot here and see if I get a reduction in instruction count as well.

As for magic scales you are correct in the sense that we now have a bump scale we can adjust on the run-time part. Of course in regards to the offline part artists have always had a scale to adjust when converting height maps to normal maps so in regards to range utilization nothing has changed in this regard.

As for derivative maps baked using xNormal the demo shows how to compute the inverse scale of what xNormal used to produce the baked normal map. The most recent version of the demo has a third example http://mmikkelsen3d.blogspot.com/2012/01/how-to-do-more-generic-derivative-map.html which shows how to maintain scale invariance even when using UV scales and offsets.

Scale invariance is of course a feature we are used to when using traditional normal mapping. Anyway, I agree that it's both a freedom and an added complexity to the whole thing.

You are probably familiar with all of this but I thought I'd point it out just in case.

Amazing info. Great job. I really appreciate your work. It is very helpful for 3d graphic. Keep it up.

ReplyDeleteHave you every looked into doing tessellation on the Xbox360 in XNA using Npatches?

ReplyDeleteBasically the technique uses Npatches which seem to be allowed in XNA:

http://msdn.microsoft.com/en-us/library/microsoft.xna.framework.graphics.vertexelementmethod(v=xnagamestudio.31).aspx

LookUp - This constant is supported only by the programmable pipeline on N-patch data, if N-patches are enabled.

http://msdn.microsoft.com/en-us/library/microsoft.xna.framework.graphics.vertexelementusage(v=xnagamestudio.40).aspx

TessellateFactor

I've been looking for folks with more experience than me to help figure it out. I've found an old ATI sample project that had tessellation and npatches working as an example on DX10. I am now working on porting it to XNA and need some help. I purchased a ati radeon 3000x card which is basically what the xbox has in it. The ati sample works on the computer and tessellation looks great.

Mr. Mikkelsen, could you please re-upload the source code? I've been meaning to implement your technique, but for some reason I cannot get it to work correctly and I've accidentally delete your source zip file :(

ReplyDeleteHi Oguz,

ReplyDeleteI apologize for the server being down but until it's back up

you can use the following for source:

https://dl.dropbox.com/u/55891920/bump_demo/bumpdemo_src.zip

and if you just want the binary you can use

https://dl.dropbox.com/u/55891920/bump_demo/bumpdemo_bin.zip

Hey,

DeleteAny chance we can get the source code?

drop box links seem broken.

May be github?

Thank you very much! Hopefully I can find out what I'm doing wrong :)

ReplyDelete