用四元数来表示切线空间的法线映射——我遇到的问题

Inspired by crytek's presentation on using quaternions to store tangent space in quaternions for smaller vertices, I came to the logical conclusion that if you can use quaternions to store tangent space, then you could also lerp quaternions between vertices and use them to rotate normals directly. This would eliminate the need to re-orthogonalize your tangent space vectors, or reconstruct one of them, and it would cut out a per-fragment matrix-vector multiplication, replacing it all with a single quaternion-vector multiplication.

受到crytek关于使用四元数在较小顶点的四元数中存储切空间的演示的启发，我得出了逻辑结论:如果可以使用四元数来存储切空间，那么也可以在顶点之间插入四元数，并使用它们直接旋转法线。这将消除对切空间向量重新正交化的需要，或者重构它们中的一个，它将切割出每个片段的矩阵向量乘法，用一个四元数向量乘法来替换它。

I tried to implement it in my OpenGL app, using my home-made quaternion class, and I'm having some issues. I know that my quaternion can be constructed from a matrix, multiply the quaternion by a vector, and get the same result as multiplying the matrix with the vector - I've done so successfully on the cpu side. However, once I start working with them in GLSL, everything tends to go haywire.

我试着在我的OpenGL应用中实现它，用我自制的四元数类，我有一些问题。我知道我的四元数可以用一个矩阵来构造，用一个向量乘以四元数，得到的结果和用向量乘以矩阵一样——我在cpu方面做得很成功。然而，一旦我开始在GLSL中与他们合作，一切都会变得一团糟。

It is very interesting to note that I can in fact discern the pattern of the normal map, so I think I'm on the right track. Unfortunately, it seems that my colors go haywire.

很有趣的是，我能分辨出法线图的模式，所以我认为我的方向是正确的。不幸的是，我的颜色似乎乱了套。

This is the quaternion math that I use in glsl:

这是我在glsl中使用的四元数:

vec4 multQuat(vec4 q1, vec4 q2)
{
    return vec4(
        (q1.w * q2.y) + (q1.y * q2.w) + (q1.x * q2.z) - (q1.z * q2.x),
        (q1.w * q2.z) + (q1.z * q2.w) + (q1.y * q2.x) - (q1.x * q2.y),
        (q1.w * q2.w) - (q1.x * q2.x) - (q1.y * q2.y) - (q1.z * q2.z),
        (q1.w * q2.x) + (q1.x * q2.w) + (q1.z * q2.y) - (q1.y * q2.z)
        );
}

vec3 rotateVector(vec4 quat, vec3 vec)
{
    return vec + 2.0 * cross(quat.xyz, cross(quat.xyz, vec) + (quat.w * vec));
}

This is how it's passed from the vertex shader:

这是它从顶点着色器经过的方法:

vQtangent = multQuat(inQtangent, quatView);

Where quatView is a quaternion made from the view matrix. This might be my issue, because the the code that generates this quaternion assumes that the matrix is orthogonal.

其中四元数是由视图矩阵构成的四元数。这可能是我的问题，因为生成这个四元数的代码假设矩阵是正交的。

Finally, we calculate the bumped normal in the fragment shader:

最后，我们计算了碎片着色器中的凹凸法线:

vec3 calcBumpedNormal(void)
{
    vec4 qtangent = normalize(vQtangent);
    vec3 normal = texture2D(texNormal, vTexCoord).xyz;
    normal = (normal * 2) - 1;
    return normalize(rotateVector(qtangent, normal));
};

Here's how I calculate a quaternion from 3 vec3's (How I get the quaternion from the tbn vectors):

下面是我如何从3向量3中计算四元数(我如何从tbn向量中得到四元数):

inline static quat fromMat3(const vec3& col0, const vec3& col1, const vec3& col2)
{
    /* warning - this only works when the matrix is orthogonal and special orthogonal */

    float w = sqrtf(1.0f + col0.x + col1.y + col2.z) / 2.0f;

    return quat(
        (col1.z - col2.y) / (4.0f * w),
        (col2.x - col0.z) / (4.0f * w),
        (col0.y - col1.x) / (4.0f * w),
        w);
}

And here's how I calclate the quaternion from a mat4 (how I get the quatView from the view matix):

下面是我如何从mat4中计算四元数(如何从matix视图中获取四元数):

inline static quat fromMat4(const mat4& mat)
{
    /* warning - this only works when the matrix is orthogonal and special orthogonal */

    float w = sqrtf(1.0f + mat.m[0][0] + mat.m[1][1] + mat.m[2][2]) / 2.0f;

    return quat(
        (mat.m[1][2] - mat.m[2][1]) / (4.0f * w),
        (mat.m[2][0] - mat.m[0][2]) / (4.0f * w),
        (mat.m[0][1] - mat.m[1][0]) / (4.0f * w),
        w);
}

I am aware that neither work with non-orthogonal matrices.

我知道这两种方法都不适用于非正交矩阵。

However, only the x and y of the normal are stored in the normal buffer, I reconstruct z in the light pass fragment shader using the sqrt trick. Because these normals are meant to be in view-space, the z component is always positive.

然而，只有正常的x和y存储在正常的缓冲区中，我使用sqrt技巧在光线通过的碎片着色器中重构z。因为这些法线应该在视图空间中，所以z分量总是正的。

Unfortunately, my results are incorrect, and I don't know where to look. I can discern the pattern of the normal map, so something has to be right.

不幸的是，我的结果是错误的，我不知道去哪里找。我能辨别出正常地图的模式，所以一定要正确。

If anybody would let me know where my problem might be, or if they have experience doing this themselves, any advice is greatly appreciated.

如果有人能让我知道我的问题在哪里，或者如果他们自己有这样做的经验，任何建议都是非常感激的。

1 个解决方案

#1

Does your code work fine if you use the per-vertex quaternion only in the vertex shader (by transforming the light & camera vectors into the tangential space)? If it breaks only when you try to rotate the normal in the pixel shader, then your problem is quaternion interpolation (if not, then I've just wasted 20mins).

如果你只在顶点着色器中使用每个顶点四元数(通过将光和相机矢量转换成切向空间)，你的代码是否能正常工作?如果只有当你在像素着色器中旋转法线时它才会断裂，那么你的问题就是四元数插值(如果不是，那我就浪费了20分钟)。

Problem

Quaternions are not in 1:1 relation with the ortho-normal matrices of selected handedness (I assume your handedness is fine, but you should verify that). If you multiply each of the quaternion components by -1, you'll get the same transformation.

四元数不是与所选的手性的标准正交矩阵成1:1的关系(我假设你的手性是好的，但你应该验证一下)。如果你把每个四元数的分量乘以-1，你会得到相同的变换。

Now, your fromMat3 always produces a quaternion with positive W component. Imagine how interpolation goes along an edge between (0.99,0,0,0.1) and (-0.99,0,0,0.1). The X component will travel all the way through its axis, causing all sorts of shading issues for you.

现在，你的fromMat3总是产生一个W分量为正的四元数。想象一下插值是如何沿着(0.99,0,0,0,0。1)和(-0.99,0,0,0,0。1)之间的一条边进行的。X分量会沿着它的轴运动，给你带来各种各样的阴影问题。

Solution

You've got to make sure that any quaternion interpolation (QI) is happening between quaternions belonging to the same hemisphere, i.e. dot(q1,q2) > 0. It's easy to see how this check fails for the example quaternions I mentioned, and how it works fine if you multiply the second quaternion by -1.

你必须确保任何四元数插值(QI)都发生在属于同一半球的四元数之间，即点(q1,q2) > 0。对于我提到的例子四元数，很容易看出这个检查是如何失败的，如果把第二个四元数乘以-1，它是如何工作的。

The tricky part is that ensuring QI-correctness may require splitting edges and adding new vertices, so it is best to be done on the exporter side, not during the model loading. Have a look at the KRI mesh exporter code for the reference.

需要注意的是，确保齐齐正确性可能需要分割边和添加新顶点，所以最好在导出端完成，而不是在模型加载期间。请查看KRI网格导出代码以供参考。

Conclusion

I don't recommend you going there for practical reasons, unless you are very persistent. Instead, you can just happily use quaternions in the vertex shader. If you ever get a hand of GPU Pro 3 book, you can find my article on quaternions there, explaining the very same problem (and the solution) in detail.

我不建议你出于实际原因去那里，除非你非常坚持。相反，你可以在顶点着色器中使用四元数。如果你有一本GPU Pro 3的书，你可以在那里找到我关于四元数的文章，详细解释同样的问题(和解决方案)。