Say I have an image which I have obtained after applying a homography transformation H to some original image. The original image is not shown. The result of the homography H applied to the original image is this image:
假设我有一个图像是在对原始图像进行H的同形变换后得到的。原始图像未显示。H对原始图像的归一化结果是:
I want to rotate this image by 30 degrees about a suitable axis (possibly where a camera would be located, if there was one) to get this image:
我想把这幅图像绕一个合适的轴旋转30度(如果有摄像机的话，可能是在哪里)来得到这幅图像:
How can I apply this rotation transformation using python if I don't know the camera parameters? I can only specify the degrees by which I want to rotate the image and the approximate axis about which I want to rotate. Also, how can I deduce the homography H' between the original image (before homography) and the final rotated image using H and the rotation transformation?
如果我不知道相机的参数，我怎么用python来应用这个旋转变换呢?我只能指定旋转图像的角度和我想旋转的近似轴。另外，如何利用H和旋转变换推导出原始图像(在原始图像之前)和最终旋转图像之间的H' ?
2 个解决方案
#1
5
An interesting problem. To help explain my solution I'm going to define a few symbols:
一个有趣的问题。为了解释我的解决方案，我将定义一些符号:
- I1: the original image.
- I1:原始图像。
- I2: the image you have after transforming I1 by H.
- I2:对I1进行H变换后得到的图像。
- I3: the image you have by transforming I2 by the 3D camera rotation R (which you set yourself).
- I3:你通过3D摄像机旋转R(你自己设置)来变换I2的图像。
- The unknown camera intrinsic matrix K that corresponds to I2.
- 未知的摄像机固有矩阵K对应于I2。
Because your camera is rotating and not translating, you can synthesize virtual views for any rotation matrix R by warping your images with a corresponding homography matrix. Therefore you don't need to try to reconstruct the scene in 3D in order to synthesize these views.
因为你的相机是旋转的，而不是翻译的，你可以通过用相应的同调矩阵扭曲你的图像来合成任何旋转矩阵R的虚拟视图。因此，你不需要尝试重建3D场景来合成这些视图。
For now I'm going to assume we have an estimate of K and give the equation for the homography from I1 to I3. This answers the last part of your question. Finally I'll give a nice way to estimate K. Then you have all you need.
现在我假设我们有一个关于K的估计值并给出了从I1到I3的同像方程。这就回答了你问题的最后一部分。最后，我将给出一个估计k的好方法，这样你就得到了所有你需要的。
Let p=(px,py) be a 2D point in I1. We define this point in homogeneous coordinates with the vector p=(px,py,1). Similarly let the point q=(qx,qy,1) be the position of point p in I3. The homography matrix H' that transforms p to q is given by H' = K R inv(K) H. For any R that you specify, you would use this to compute H' then you can warp I1 to synthesise the new view using e.g. OpenCV's warpPerspective function.
设p=(px,py)是I1中的一个二维点。我们用向量p=(px,py,1)在齐次坐标下定义这个点。同样，令点q=(qx,qy,1)为点p在I3中的位置。变换p到q的同形矩阵H'由H' = K R inv(K) H给出对于任何你指定的R，你可以用它来计算H'然后你可以扭曲I1来合成新的视图使用OpenCV的warpPerspective函数。
Derivation. we first apply H to get the point into I2. Next we transform the point into its position in 3D caméra coordinates by inv(K). We then apply the rotation R and finally project back onto the image with K. If you're unsure about applying projective transforms like this then I highly recommend reading in depth with Hartley and Zisserman's book Multiple View Geometry.
推导。我们先用H来表示I2。接下来，我们将这个点在inv(K)的3D摄像机坐标中转换为它的位置。然后我们应用旋转R，最后用k投射到图像上，如果你不确定应用像这样的投影变换，那么我强烈推荐你阅读哈特利和泽斯曼的《多视角几何》。
Computing K. For this I propose a cunning strategy using the Statue of Liberty. Specifically, notice that she is standing on a platform, which I am going to assume is square. This is the killer trick! Now we are going to do a rough camera calibration using the square. I'm going to assume there is no lens distortion and K has simplified form with K = [f,0,cx;0,f,cy;0,0,1]. This means the aspect ratio is 1 (usually roughly the case for digital cameras) and the principal point is at the centre of the image: cx=w/2 and cy=h/2 where w and h are the width and height of the image respectively. Trying to estimate lens distortion and a more complex K matrix would be very hard. Lens distortion doesn't seem significant because the edges of the wood are all roughly straight in the images, so it can be ignored.
为此，我提出了一个使用*女神像的巧妙策略。特别要注意的是，她站在一个平台上，我假设它是正方形的。这是致命的诡计!现在我们要用正方形做一个粗略的相机校准。我假设没有透镜畸变K有简化形式K = [f,0,cx;0,f,cy;0,0,1]这意味着长宽比为1(通常是数码相机的大致情况)，主点位于图像的中心:cx=w/2, cy=h/2，其中w和h分别为图像的宽度和高度。试图估计透镜的失真和一个更复杂的K矩阵是非常困难的。镜头的失真并不明显，因为木头的边缘在图像中都大致是直的，所以可以忽略。
So now we are going to compute f. This will be done using plane-based camera calibration. The famous reference for this is Zhang: A Flexible New Technique for Camera Calibration, located at https://www.microsoft.com/en-us/research/publication/a-flexible-new-technique-for-camera-calibration/
现在我们要计算f，这将通过基于平面的摄像机校准来完成。著名的例子是Zhang:一种灵活的相机校准新技术，位于https://www.microsoft.com/en-us/research/publication/a- Flexible - New Technique -for Camera - Calibration /
The way this would work is first to click on the 4 corners of the statue plane's four visible corners in I2 (see attached image). Let's call these p1 p2 p3 and p4, starting bottom left and going round clockwise. You can then use OpenCV's camera calibration methods to get the estimate of K from these 4 corner points. Importantly the reason why we can do this is because we know the platform is square. For a deeper insight in plane-based calibration I recommend reading Zhang's paper. If you are experiencing difficulty I could do it myself in a couple of minutes and send over the K matrix.
这样做的方法是首先点击在I2(见附图)的雕像平面四个可见角落的四个角。我们称这些为p1 p2 p3和p4，从左下角开始，顺时针旋转。然后你可以使用OpenCV的相机校准方法从这4个角点得到K的估计值。重要的是，我们之所以能够做到这一点，是因为我们知道平台是正方形的。为了更深入地了解基于行星的校准，我推荐阅读张的论文。如果你遇到困难，我可以在几分钟内自己做，然后把K矩阵发送出去。
As a final point, a slight variation of this approach is to calibrate using your original image (assuming you still have it). The reason for this is that H could distort I2 so that its aspect ratio is not close to 1 and principal point not near the image centre. If you calibrate using your original image (let's call the matrix K1) then you would use K = H K1.
最后一点，这种方法的一个微小变化是使用原始图像进行校准(假设您仍然拥有原始图像)。原因是H可以扭曲I2，使其长宽比不接近1，主点不靠近图像中心。如果你用原始图像(我们叫矩阵K1)进行校准，那么你就会用K = H K1。
#2
5
To apply the homography I would recommend using OpenCV, more specifically the warpPerspective function https://docs.opencv.org/3.0-beta/modules/imgproc/doc/geometric_transformations.html#warpperspective
为了应用homography，我建议使用OpenCV，尤其是warpPerspective函数https://docs.opencv.org/3.0-beta/modules/imgproc/doc/geometric_transformations.html# warpPerspective
Because we are talking about a pure rotation, no camera translation, you can indeed produce the image that corresponds to this rotation by just using a homography. But to find the homography parameters as a function of the axis direction and rotation angle, you'll need to know the camera intrinsic parameters, mainly the focal length.
因为我们讨论的是纯旋转，没有摄像机平移，你确实可以产生与这个旋转相对应的图像通过使用同调法。但是，要想找到作为轴向和旋转角度的函数，你需要知道相机的固有参数，主要是焦距。
If you had the camera model you could work out the equations, but another way to obtain the homography matrix is just calculating what would be the destination point coordinates after the transform, and then use the findHomography function. Or you can find the matching points, and then you calculate the homography.
如果你有相机模型，你可以算出方程，但是另一种获得同像矩阵的方法是计算变换后的目标点坐标，然后使用findHomography函数。或者你可以找到匹配的点，然后你就可以计算出全像。
If you don't have the camera model and rotation parameters, or matching points in both images, there is nothing you can do, you need any of these to find out the homography. You can try to guess the camera model perhaps. What exactly is the information you have?
如果你没有相机模型和旋转参数，或者两个图像中的匹配点，你什么都做不了，你需要它们中的任何一个来找出同构。你可以试着猜一下相机的型号。你有什么确切的信息?
#1
5
An interesting problem. To help explain my solution I'm going to define a few symbols:
一个有趣的问题。为了解释我的解决方案，我将定义一些符号:
- I1: the original image.
- I1:原始图像。
- I2: the image you have after transforming I1 by H.
- I2:对I1进行H变换后得到的图像。
- I3: the image you have by transforming I2 by the 3D camera rotation R (which you set yourself).
- I3:你通过3D摄像机旋转R(你自己设置)来变换I2的图像。
- The unknown camera intrinsic matrix K that corresponds to I2.
- 未知的摄像机固有矩阵K对应于I2。
Because your camera is rotating and not translating, you can synthesize virtual views for any rotation matrix R by warping your images with a corresponding homography matrix. Therefore you don't need to try to reconstruct the scene in 3D in order to synthesize these views.
因为你的相机是旋转的，而不是翻译的，你可以通过用相应的同调矩阵扭曲你的图像来合成任何旋转矩阵R的虚拟视图。因此，你不需要尝试重建3D场景来合成这些视图。
For now I'm going to assume we have an estimate of K and give the equation for the homography from I1 to I3. This answers the last part of your question. Finally I'll give a nice way to estimate K. Then you have all you need.
现在我假设我们有一个关于K的估计值并给出了从I1到I3的同像方程。这就回答了你问题的最后一部分。最后，我将给出一个估计k的好方法，这样你就得到了所有你需要的。
Let p=(px,py) be a 2D point in I1. We define this point in homogeneous coordinates with the vector p=(px,py,1). Similarly let the point q=(qx,qy,1) be the position of point p in I3. The homography matrix H' that transforms p to q is given by H' = K R inv(K) H. For any R that you specify, you would use this to compute H' then you can warp I1 to synthesise the new view using e.g. OpenCV's warpPerspective function.
设p=(px,py)是I1中的一个二维点。我们用向量p=(px,py,1)在齐次坐标下定义这个点。同样，令点q=(qx,qy,1)为点p在I3中的位置。变换p到q的同形矩阵H'由H' = K R inv(K) H给出对于任何你指定的R，你可以用它来计算H'然后你可以扭曲I1来合成新的视图使用OpenCV的warpPerspective函数。
Derivation. we first apply H to get the point into I2. Next we transform the point into its position in 3D caméra coordinates by inv(K). We then apply the rotation R and finally project back onto the image with K. If you're unsure about applying projective transforms like this then I highly recommend reading in depth with Hartley and Zisserman's book Multiple View Geometry.
推导。我们先用H来表示I2。接下来，我们将这个点在inv(K)的3D摄像机坐标中转换为它的位置。然后我们应用旋转R，最后用k投射到图像上，如果你不确定应用像这样的投影变换，那么我强烈推荐你阅读哈特利和泽斯曼的《多视角几何》。
Computing K. For this I propose a cunning strategy using the Statue of Liberty. Specifically, notice that she is standing on a platform, which I am going to assume is square. This is the killer trick! Now we are going to do a rough camera calibration using the square. I'm going to assume there is no lens distortion and K has simplified form with K = [f,0,cx;0,f,cy;0,0,1]. This means the aspect ratio is 1 (usually roughly the case for digital cameras) and the principal point is at the centre of the image: cx=w/2 and cy=h/2 where w and h are the width and height of the image respectively. Trying to estimate lens distortion and a more complex K matrix would be very hard. Lens distortion doesn't seem significant because the edges of the wood are all roughly straight in the images, so it can be ignored.
为此，我提出了一个使用*女神像的巧妙策略。特别要注意的是，她站在一个平台上，我假设它是正方形的。这是致命的诡计!现在我们要用正方形做一个粗略的相机校准。我假设没有透镜畸变K有简化形式K = [f,0,cx;0,f,cy;0,0,1]这意味着长宽比为1(通常是数码相机的大致情况)，主点位于图像的中心:cx=w/2, cy=h/2，其中w和h分别为图像的宽度和高度。试图估计透镜的失真和一个更复杂的K矩阵是非常困难的。镜头的失真并不明显，因为木头的边缘在图像中都大致是直的，所以可以忽略。
So now we are going to compute f. This will be done using plane-based camera calibration. The famous reference for this is Zhang: A Flexible New Technique for Camera Calibration, located at https://www.microsoft.com/en-us/research/publication/a-flexible-new-technique-for-camera-calibration/
现在我们要计算f，这将通过基于平面的摄像机校准来完成。著名的例子是Zhang:一种灵活的相机校准新技术，位于https://www.microsoft.com/en-us/research/publication/a- Flexible - New Technique -for Camera - Calibration /
The way this would work is first to click on the 4 corners of the statue plane's four visible corners in I2 (see attached image). Let's call these p1 p2 p3 and p4, starting bottom left and going round clockwise. You can then use OpenCV's camera calibration methods to get the estimate of K from these 4 corner points. Importantly the reason why we can do this is because we know the platform is square. For a deeper insight in plane-based calibration I recommend reading Zhang's paper. If you are experiencing difficulty I could do it myself in a couple of minutes and send over the K matrix.
这样做的方法是首先点击在I2(见附图)的雕像平面四个可见角落的四个角。我们称这些为p1 p2 p3和p4，从左下角开始，顺时针旋转。然后你可以使用OpenCV的相机校准方法从这4个角点得到K的估计值。重要的是，我们之所以能够做到这一点，是因为我们知道平台是正方形的。为了更深入地了解基于行星的校准，我推荐阅读张的论文。如果你遇到困难，我可以在几分钟内自己做，然后把K矩阵发送出去。
As a final point, a slight variation of this approach is to calibrate using your original image (assuming you still have it). The reason for this is that H could distort I2 so that its aspect ratio is not close to 1 and principal point not near the image centre. If you calibrate using your original image (let's call the matrix K1) then you would use K = H K1.
最后一点，这种方法的一个微小变化是使用原始图像进行校准(假设您仍然拥有原始图像)。原因是H可以扭曲I2，使其长宽比不接近1，主点不靠近图像中心。如果你用原始图像(我们叫矩阵K1)进行校准，那么你就会用K = H K1。
#2
5
To apply the homography I would recommend using OpenCV, more specifically the warpPerspective function https://docs.opencv.org/3.0-beta/modules/imgproc/doc/geometric_transformations.html#warpperspective
为了应用homography，我建议使用OpenCV，尤其是warpPerspective函数https://docs.opencv.org/3.0-beta/modules/imgproc/doc/geometric_transformations.html# warpPerspective
Because we are talking about a pure rotation, no camera translation, you can indeed produce the image that corresponds to this rotation by just using a homography. But to find the homography parameters as a function of the axis direction and rotation angle, you'll need to know the camera intrinsic parameters, mainly the focal length.
因为我们讨论的是纯旋转，没有摄像机平移，你确实可以产生与这个旋转相对应的图像通过使用同调法。但是，要想找到作为轴向和旋转角度的函数，你需要知道相机的固有参数，主要是焦距。
If you had the camera model you could work out the equations, but another way to obtain the homography matrix is just calculating what would be the destination point coordinates after the transform, and then use the findHomography function. Or you can find the matching points, and then you calculate the homography.
如果你有相机模型，你可以算出方程，但是另一种获得同像矩阵的方法是计算变换后的目标点坐标，然后使用findHomography函数。或者你可以找到匹配的点，然后你就可以计算出全像。
If you don't have the camera model and rotation parameters, or matching points in both images, there is nothing you can do, you need any of these to find out the homography. You can try to guess the camera model perhaps. What exactly is the information you have?
如果你没有相机模型和旋转参数，或者两个图像中的匹配点，你什么都做不了，你需要它们中的任何一个来找出同构。你可以试着猜一下相机的型号。你有什么确切的信息?