如何使用带有OpenCV和Python的掩码(或透明)在图像中找到模板?

Let us assume we are looking for this template:

让我们假设我们正在寻找这个模板:

Stop http://oi48.tinypic.com/2u7q1l4.jpg

停止http://oi48.tinypic.com/2u7q1l4.jpg

The corners of our template are transparent, so the background will vary, like so:

我们的模板的每个角落都是透明的，所以背景会是不同的，比如:

Stop on moon http://i49.tinypic.com/ziw3mc.png

停止在月球http://i49.tinypic.com/ziw3mc.png

Stop on everest http://i45.tinypic.com/2unwxhu.png

站在珠穆朗玛峰http://i45.tinypic.com/2unwxhu.png

Stop on leaves http://i48.tinypic.com/t06v7k.png

叶子站http://i48.tinypic.com/t06v7k.png

Assuming we could use the following mask with our template:

假设我们可以在模板中使用以下掩码:

Stop http://oi48.tinypic.com/2u7q1l4.jpg Stop mask http://i49.tinypic.com/ogclfd.png

停止:http://i49.tinypic.com/ogclfd.png

It would be very easy to find it.

很容易找到它。

What I have tried:

我已经尝试:

I have tried matchTemplate but it doesn't support masks (as far as I know), and using the alpha channel (transparency) in the template does not achieve this, as it compares the alpha channels instead of ignoring those pixels.

我尝试过matchTemplate，但它不支持遮罩(据我所知)，并且在模板中使用alpha通道(透明)也不能实现这一点，因为它比较alpha通道而不是忽略这些像素。

I have also looked into "region of interest", which I thought would be the solution, but with it you can only specify a rectangular area. I'm not even sure if it works on the template or not.

我还研究了“感兴趣的区域”，我认为这是解决方案，但是用它你只能指定一个矩形区域。我甚至不确定它是否适用于模板。

I'm sure this is possible to do by writing my own algorithm, but I was hoping this is possible via. standard OpenCV to avoid reinventing the wheel. Not to mention, it would most likely be more optimised than my own.

我相信这可以通过我自己的算法来实现，但是我希望这可以通过。避免重新发明*的标准OpenCV。更不用说，它很可能比我自己的更乐观。

So, how could I do something like this with OpenCV + Python?

那么，我怎么能用OpenCV + Python做这样的事情呢?

5 个解决方案

#1

This could be achieved using only matchTemplate function, but a little workaround is needed.

这可以通过使用matchTemplate函数来实现，但是需要一些变通方法。

Lets analyse the default metrics(CV_TM_SQDIFF_NORMED). According to matchTemplate documentation the default metrics looks like this

让我们分析一下默认指标(CV_TM_SQDIFF_NORMED)。根据matchTemplate文档，默认指标如下所示

R(x, y) = sum (I(x+x', y+y') - T(x', y'))^2

R(x,y)=(我总和(x + x,y + y)- T(x,y))^ 2

Where I is image matrix, T is template, R is result matrix. Summation is done over template coordinates x' and y',

其中I为图像矩阵，T为模板，R为结果矩阵。对模板坐标x'和y'求和，

So, lets alter this metrics by inserting weight matrix W, which has the same dimensions as T.

因此，我们通过插入权重矩阵W来改变这个指标，它的维度与T相同。

Q(x, y) = sum W(x', y')*(I(x+x', y+y') - T(x', y'))^2

Q(x,y)=和W(x,y)*(我(x + x,y + y)- T(x,y))^ 2

In this case, by setting W(x', y') = 0 you can actually make pixel be ignored. So, how to make such metrics? With simple math:

在本例中，通过设置W(x'， y') = 0，您实际上可以忽略像素。那么，如何制定这样的指标呢?用简单的数学:

Q(x, y) = sum W(x', y')*(I(x+x', y+y') - T(x', y'))^2
        = sum W(x', y')*(I(x+x', y+y')^2 - 2*I(x+x', y+y')*T(x', y') + T(x', y')^2)
        = sum {W(x', y')*I(x+x', y+y')^2} - sum{W(x', y')*2*I(x+x', y+y')*T(x', y')} + sum{W(x', y')*T(x', y')^2)}

So, we divided Q metrics into tree separate sums. And all those sums could be calculated with matchTemplate function (using CV_TM_CCORR method). Namely

因此，我们将Q指标分成了树形的单独的和。所有这些总和可以用matchTemplate函数(使用CV_TM_CCORR方法)计算。即

sum {W(x', y')*I(x+x', y+y')^2} = matchTemplate(I^2, W, method=2)
sum{W(x', y')*2*I(x+x', y+y')*T(x', y')} = matchTemplate(I, 2*W*T, method=2)
sum{W(x', y')*T(x', y')^2)} = matchTemplate(T^2, W, method=2) = sum(W*T^2)

The last element is a constant, so, for minimisation it does not have any effect. On the other hand, it still might me useful to see if our template have perfect match (if Q is approaching to zero). Nonetheless, for last element we actually do not need matchTemplate function, since it could be calculated directly.

最后一个元素是常数，因此，对于极小化，它没有任何影响。另一方面，如果我们的模板有完美的匹配(如果Q接近于0)，我可能仍然会很有用。尽管如此，对于最后一个元素，我们实际上不需要matchTemplate函数，因为它可以直接计算。

The final pseudocode looks like this:

最后的伪代码是这样的:

result = matchTemplate(I^2, W, method=2) - matchTemplate(I, 2*W*T, method=2) + as.scalar(sum(W*T^2))

Does it really do exactly as defined? Mathematically yes. Practically, there is some small rounding error, because matchTemplate function works on 32-bit floating-point, but I believe it is not a big problem.

它是否真的像定义的那样?数学上是的。实际上，有一些小的舍入误差，因为matchTemplate函数在32位浮点上工作，但是我认为这不是一个大问题。

Please note, that you can extent analysis and have weighted equivalents for any metrics offered by matchTemplate.

请注意，您可以对matchTemplate提供的任何度量进行范围分析并拥有加权等价物。

This actually worked for me. I am sorry I don't give actual code. I am working in R, so I don't have the code in Python. But idea is quite straightforward.

这对我很有效。对不起，我没有给出实际的代码。我在R工作，所以我没有Python中的代码。但是想法很简单。

I hope this will help.

我希望这能有所帮助。

#2

One answer to your question is convolution. Use the template as kernel and filter the image.

你的问题的一个答案是卷积。使用模板作为内核并过滤图像。

The destination Mat will have dense bright areas where your template might be. You'll have to cluster the results (e.g. Mean-shift).

目标席将有密集的明亮区域，您的模板可能在那里。你必须把结果集中在一起(例如，均值漂移)。

In that way, you'll have a very simplistic implementation of the Generalized Hough Transform or a Template-based convolution matching.

这样，您将得到一个非常简单的广义霍夫变换的实现或基于模板的卷积匹配。

#3

What worked for me the one time I needed this was to fill the "mask" areas with white noise. Then it gets effectively washed out of the correlation when looking for matches. Otherwise I got, as I presume you did, false matches on the masked areas.

对我来说，我唯一需要的就是用白噪音填充“蒙版”区域。然后它在寻找匹配时被有效地从相关性中洗去。否则，我猜你是在蒙面地区做假火柴。

#4

ImageMagick has logic for finding subimages in other images and it works quite well.

ImageMagick有在其他图像中找到子图像的逻辑，而且效果很好。

compare -verbose -dissimilarity-threshold 0.1 -subimage-search subimage bigimage

I've used it to find and blur watermarks off some products. Don't ask.

我用它来查找和模糊一些产品上的水印。不要问。

(Sometimes you have to do what you have to do..)

(有时你必须做你必须做的事。)

#5

Imagemagick 7.0.3.9 now has a masked compare capability so that you can limit the template matching region. See http://www.imagemagick.org/discourse-server/viewtopic.php?f=4&t=31053

Imagemagick 7.0.3.9现在有了屏蔽比较功能，这样就可以限制模板匹配区域。参见http://www.imagemagick.org/discourse-server/viewtopic.php?f=4&t=31053

Also, I see that OpenCV 3.0 now has masked template matching. See http://docs.opencv.org/3.0.0/df/dfb/group__imgproc__object.html#ga586ebfb0a7fb604b35a23d85391329be

另外，我还看到OpenCV 3.0现在有了屏蔽模板匹配。看到http://docs.opencv.org/3.0.0/df/dfb/group__imgproc__object.html ga586ebfb0a7fb604b35a23d85391329be

However, it is only for method == CV_TM_SQDIFF and method == CV_TM_CCORR_NORMED. see python opencv matchTemplate is mask feature implemented?

但是，它只适用于method = CV_TM_SQDIFF和method = CV_TM_CCORR_NORMED。见python opencv matchTemplate掩码特性实现了吗?

#1