Computer Vision Project 2 – Harris Corner Detector

姓名: 王兴路
学号: 3140102282
指导老师: 宋明黎
2016-12-16 19:30:22 星期五

Content:

Computer Vision Project 2 – Harris Corner Detector
ReadMe
Run Demo
Develop Platform
FileList
Functionality
Implementation
Pipeline
Second Moment Matrix M
Adapative Non Maxima Suppression
Experiments
Implemetation
Demo & Experiments
Median or Guassian
k
aperture_size
Reference

ReadMe

Run Demo

This program implements Harris Corner Detector via Adaptive Nonmaxia Suppression.

Click Demo.bat to see Demo, the middle resule will be generated in current directory.
Run it though cmd/bash following usage below:
- Usage 1:


prj2.exe inputImgName k=(0.04) aperture_size(=3)
       guassian=(true)
       Ada_NMS=(true) R_thre_left=(5000) nms_left=(500) c_robust=(0.9)

- Usage 2:


prj2.exe inputImgName k=(0.04) aperture_size(=3)
       guassian=(true)
       Ada_NMS=(false) left=(500)

inputImgName is required. All other params are alternetive, the default is showed in Usage 1. For example, the following has been tested.


prj2.exe roof.jpg
prj2.exe roof.jpg 0.04 5

Develop Platform

OS system: Win10
VS 2013 + OpenCV 2.4.12

FileList


.
├── 3140102282_WangXinglu_Prj2.pdf
├── Demo
│   ├── *.jpg (sample input img)
│   ├── Demo.bat (click to run)
│   ├── *.dll (dependence)
│   ├── prj2.exe 
│   └── SampleOut
└── Src
    ├── prj2
    │   ├── harris.cpp
    │   ├── harris.h
    │   ├── main.cpp
    │   └── prj2.vcxproj
    ├── prj2.sdf
    ├── prj2.sln
    └── prj2.v12.suo

Functionality

求解 R response: $R = |M|-k*Tr(M)$
输出中间结果 $ \lambda_{max}$, $\lambda{min}$
输出JET Pseudo Img可视化, 对R图进行$Rectify$+$log$操作，符合人类视觉的感受范围。
NMS部分实现Adapative NMS和一般的NMS两种算法。
命令行解析参数，方便调参。包括k, aperture_size, 是否采用guassian核, 是否采用Ada_NMS, 以及对应的NMS参数。并且都有默认参数。

Implementation

Pipeline



st=>start: Start
read=>operation: Parse Argc
cal=>operation: calculate 
R responese map 
& middle result
nms=>operation: nms
vis=>operation: visualze and imwrite
ee=>end

st->read->cal->nms->vis->ee

Second Moment Matrix M

课件上的公式没有将标量和矩阵分清

$$M = \sum\limits_{x,y} {\left\{ {G(x,y)\left[ {\begin{array}{*{20}{c}}
{{I_x}^2}&{{I_x}{I_y}}\\
{{I_x}{I_y}}&{{I_y}^2}
\end{array}} \right]} \right\}} $$

上面式子中$M$事实上是标量，正确的标量形式写法应当是：

$$M(i,j) = \sum\limits_{x,y} {\left\{ {G(x,y)\left[ {\begin{array}{*{20}{c}}
{{I_x}^2(i + x,j + y)}&{{I_x}{I_y}(i + x,j + y)}\\
{{I_x}{I_y}(i + x,j + y)}&{{I_y}^2(i + x,j + y)}
\end{array}} \right]} \right\}} $$

使用$Correlation$算符$\otimes$，以矩阵形式简写如下：

$$M_{N \times N \times 2 \times 2} = f\left( {\left[ {\begin{array}{*{20}{c}}
{{M_{11}}}&{{M_{12}}}\\
{{M_{21}}}&{{M_{22}}}
\end{array}} \right]} \right) = f\left( {\left[ {\begin{array}{*{20}{c}}
{G \otimes \left( {{I_x}^2} \right)}&{G \otimes \left( {{I_x}{I_y}} \right)}\\
{G \otimes \left( {{I_x}{I_y}} \right)}&{G \otimes \left( {{I_y}^2} \right)}
\end{array}} \right]} \right)$$

其中$f$是Reshape函数，实现维度变换。因为高维矩阵$M$在每一个像素点上都是一个$2 \times 2$的二阶动量矩阵。计算得$M_{ij}$后，需要将对应位置的值拼接起来形成二阶动量矩阵。

$$f:\left( {\begin{array}{*{20}{c}}
{{R_{N \times N}}}&{{R_{N \times N}}}\\
{{R_{N \times N}}}&{{R_{N \times N}}}
\end{array}} \right) \to {R_{N \times N \times 2 \times 2}}$$

Adapative Non Maxima Suppression

我们找到的Harris Corner点数量较多，且容易出现聚簇情况，需要对Harris Corner点的数量和位置分布进行处理，这就用到了Adaptive Non-Maximal Suppression。

该算法在这篇文章中提出[Multi-Image Matching using Multi-Scale Oriented Patches, Brown et al., CVPR 2005]

使用ANMS需要完成两个任务：

从图像中抽取指定数量的interest points
使interest points均匀分布在图像中。

算法原理如下:

设共有$N$个特征点,

$\mathbf{x}_i$代表特征点$i$的坐标
$v_i$代表特征点$i$的R response值。

对于特征$i$,定义抑制半径

$$
r_i = \min_{j \in I_i} \| \mathbf{x}_i - \mathbf{x}_j \|_2
$$

其中$I_i$是满足如下公式的所有特征点的集合

$$
I_i = \{ j \in \{1,\dots, n\} \mid v_i < c_\textrm{robust} v_j \}.
$$

对于每个点计算$r_i$，降序排序，取Top $NMS_left$ 个点，即为所求的需要保留的特征点。

从直观上解释，先不看$c_\textrm{robust}$, 如果一个不厉害的点很近的地方就有一个很厉害的点，那么这个点就应该排名靠后。所以按照距离排序。

[========]

Experiments

可以看到：

Ada_NMS在自然场景照片中获得了很好的效，这张图没有一个点会重复。
Ada_NMS在规则格子图上效果相比NMS稍稍好一些。但是25个点不能恰好分布到25个格点。如果具体计算会发现，这是因为Ada_NMS只能够给临近的$4 \times 4$划出两个等级$1$和$\sqrt{2}$。

\	global	local
NMS
Ada_NMS
NMS
Ada_NMS

Implemetation


    vector<Point_float_pair> Point_R;
//    minMaxLoc(R, &R_min, &R_max);
    for (int row = 0; row < R.rows; row++)
    {
        for (int col = 0; col < R.cols; col++)
        {
            float R_tmp = R.at<float>(row, col);
            if (R_tmp < 0 || R_tmp <1000)
                continue;
            Point_R.push_back(make_pair(Point(row, col), R_tmp));
        }
    }
    R_thre_left =  min(R_thre_left, (int)Point_R.size());
    partial_sort(Point_R.begin(), Point_R.begin()+R_thre_left,Point_R.end(), Point_float_sort());
    Point_R.resize(R_thre_left);
    vector<Point_float_pair> Point_dist;
    for (int i = 0; i < Point_R.size(); ++i){
        float dist = m_infty;
        Point now_point = Point_R.at(i).first;
        for (int j = 0; j < Point_R.size(); ++j)
        {
            Point counter_point = Point_R.at(j).first;
            if (i != j && Point_R.at(i).second <c_robust * Point_R.at(j).second){ //0.01==>all 1 ; 10 ==> all inf
                float dist_tmp = norm(now_point - counter_point);
                if (dist_tmp < dist)
                    dist = dist_tmp;
            }
        }
        Point_dist.push_back(make_pair(now_point, dist));
    }
     nms_left = min(nms_left, (int)Point_dist.size());
    partial_sort(Point_dist.begin(), Point_dist.begin() + nms_left, Point_dist.end(), Point_float_sort());
    Point_dist.resize(nms_left);
    Mat R_nms(R.size(), CV_32F, Scalar::all(0));
    for (int i = 0; i <Point_dist.size(); ++i){
        Point now_point = Point_dist.at(i).first;
        int row = now_point.x;
        int col = now_point.y;
        R_nms.at<float>(row, col) = 1;
        circle(img_harris, Point(col, row), 4, Scalar(rng.uniform(0, 255), rng.uniform(0, 255), rng.uniform(0, 255)));
    }

Demo & Experiments

其中$R_{psu}$的具体操作是：

将所有负响应置为R图中除去负数的最小值
$log$操作
使用pseudo map 展示

可以看到$R_{psu}$中深蓝色的，即为原本为负数的平坦区域

$\lambda_{max}$	$\lambda_{min}$	$R_{psu}$

$R$	$R_{NMS}$	$img_{harris}$

$\lambda_{max}$	$\lambda_{min}$	$R_{psu}$

$R$	$R_{NMS}$	$img_{harris}$

Median or Guassian

从测试图片看没有明显差别(Gassian略好)：

Median	Guassian

k

k增大，检测到的点数变少，效果变差。（不稳定，最符合Corner特性的点没有被保留）

0.04	0.20	0.24	0.28

aperture_size

k增大，检测到的点数变少，效果变好。(稳定，响应大的点保留，最符合Corner特性的点保留)

3	9

Reference

https://stackedit.io/editor#fn:footnote

https://www.zybuluo.com/AntLog/note/63228

https://www.learnopencv.com/applycolormap-for-pseudocoloring-in-opencv-c-python/

http://*.com/questions/23680073/finding-local-maxima-in-an-image

http://docs.opencv.org/2.4/modules/core/doc/operations_on_arrays.html#min

From WizNote

秒客网

cv_prj2