ISP PIPLINE (二) LensShading Correct

时间:2021-06-30 06:12:27

what is the LSC?

lens shading 分为:Y-shading , color shading。

在讲LSC之前,我们先来理解一个重要的术语--CRA(Chief ray angle)。

CRA:分为lens cra , sensor cra两种。

1.1什么是lens CRA?

定义:最大像高处的主光线与光轴的倾角。备注:最大像高定义为 像素响应降低为零度角像素响应的80%的像素点。

因为Lens CRA 的存在,所以就出现了FOV (filed of view) 所谓的视角的概念。视角大小=2*CRA。

下图为lens CRA示意图。

ISP PIPLINE (二) LensShading Correct

1.2 什么是sensor的CRA

               sensor的CRA 说白了是sensor与之配对的micro lens的CRA.

圈1 表示lens折射过来的光,圈2表示micro lens再次折射的光。可以看出,sensor的光电转换二极管接收到的光并不是                 lens直接折射过来的,而是经过microlens再次折射过来的。所以你明白了什么呢?! 这也就是为什么图像边缘会变暗                   的原因,光电转换二极管并不会全部接收到光能。这是LensShading中Y-shading的主要原因。

ISP PIPLINE (二) LensShading Correct

一般情况下,当lens_cra < micro_cra的时候,在sensor传感器的边缘只会出现Y-shading。

ISP PIPLINE (二) LensShading Correct

倘若,lens_cra > micro_cra, sensor传感器可能会因为无法有效捕捉某频率的光,就会出现偏色的情况,即Color-                          shading。比如microlens 对R ,G,B的折射率不同,在R_pixel上接收了很小的能量,而在B_pixel上接受了相对较多的能                  量。

ISP PIPLINE (二) LensShading Correct

如何处理或者叫减轻shading带来的图像不均问题呢?

              1.lens cra要小于micro cra,这样既可以减缓y-shading,又可以大大避免color-shading.

              2.因为镜头是圆的,画面的对角线是最接近镜头成像直径的,所以sensor有效区域最大尺寸是对角线尺寸

3.硬件矫正

ISP PIPLINE (二) LensShading CorrectISP PIPLINE (二) LensShading Correct

4.软件矫正(下面会细讲)

how to LSC with software?

一般模组厂会根据矫正工具将图像划分为17*13(qualcomm),15*15(MTK),33*25(Samsung)等区间,计算每一块的四通道平均值,然后对每个通道与中心块的四通道平均值做除法,得到每个点的增益倍数,整个lut存储在otp中,手机端就可以根据lut进行插值计算每个pixel的增益。

一般用双线性差值或者多项式拟合的方法,确定每个pixel的增益。

ISP PIPLINE (二) LensShading Correct

原始图片的rgb通道(节选自网络):

ISP PIPLINE (二) LensShading Correct

y-shading 增益之后,基本与中心数值在同一个平面(节选自网络):

ISP PIPLINE (二) LensShading Correct

color-shading 增益之后,各区域的r/g,b/g均值相同(节选自网络):

ISP PIPLINE (二) LensShading Correct

上面三张图片为了让大家理解,节选自网络。本人也模拟了shading矫正过程,但由于colorshading不严重所以矫正效果不明显。下面是我的矫正过程:

1.原始图:

ISP PIPLINE (二) LensShading Correct

2.三通道三维图

ISP PIPLINE (二) LensShading Correct

3.(17*13block)r,g,b平均

ISP PIPLINE (二) LensShading Correct

4.colorgain

ISP PIPLINE (二) LensShading Correct

5、lensgain=ygain+colorgain

ISP PIPLINE (二) LensShading Correct

6. 还原后三通道值

ISP PIPLINE (二) LensShading Correct

7.还原后图片

ISP PIPLINE (二) LensShading Correct

附上我的代码:

clc
clear all;
close all;
BLOCK_H = 17
BLOCK_V = 13
img = imread('1.bmp');
r= medfilt2(img(:,:,1));%中值滤波抑制边缘凸起噪声
g= medfilt2(img(:,:,2));
b= medfilt2(img(:,:,3));
figure(1)
[rows,cols,channels] = size(img);
width = 1:cols; height = 1:rows; [X1,Y1] = meshgrid(width,height);
mesh(X1,Y1,im2double(r)); hold on; mesh(X1,Y1,im2double(g)); hold on; mesh(X1,Y1,im2double(b));

subSampleRGB=ones(BLOCK_V+1,BLOCK_H+1,3);%6*6*3 gainlut

strideX=floor(cols/BLOCK_H)
strideY=floor(rows/BLOCK_V)
for i=1:BLOCK_H+1%cols
    for j=1:BLOCK_V+1%rows
        for k=1:3%channels
            if i==1
                if j==1%left_top
                   subSampleRGB(j,i,k)=mean2( img(1:floor(strideY*0.5),                                1:floor(strideX*0.5),k) );
                elseif j==BLOCK_V+1 %left_bottom
                   subSampleRGB(j,i,k)=mean2( img(floor((BLOCK_V-0.5)*strideY):BLOCK_V*strideY,        1:floor(strideX*0.5),k) );
                else %left_middle
                   subSampleRGB(j,i,k)=mean2( img(floor((j-1-0.5)*strideY):floor((j-1+0.5)*strideY),   1:floor(strideX*0.5),k) );
                end
            elseif i==BLOCK_H+1
                if j==1%right_top
                   subSampleRGB(j,i,k)=mean2( img(1:floor(strideY*0.5),                                floor(strideX*(BLOCK_H-0.5)):BLOCK_H*strideX,k) );
                elseif j==BLOCK_V+1 %right_bottom
                   subSampleRGB(j,i,k)=mean2( img(floor((BLOCK_V-0.5)*strideY):BLOCK_V*strideY,        floor(strideX*(BLOCK_H-0.5)):BLOCK_H*strideX,k) );
                else %right_middle
                   subSampleRGB(j,i,k)=mean2( img(floor((j-1-0.5)*strideY):floor((j-1+0.5)*strideY),        floor(strideX*(BLOCK_H-0.5)):BLOCK_H*strideX,k) );
                end
            elseif j==1%top without corner
                if i~=1&&i~=BLOCK_H+1
                    subSampleRGB(j,i,k)=mean2( img(1:floor(0.5*strideY),                               floor(strideX*(i-1-0.5)):floor(strideX*(i-1+0.5)),k) );
                end
            elseif j==BLOCK_V+1%bottom without corner
                if i~=1&&i~=BLOCK_H+1
                    subSampleRGB(j,i,k)=mean2( img(floor((BLOCK_V-0.5)*strideY):BLOCK_V*strideY,       floor(strideX*(i-1-0.5)):floor(strideX*(i-1+0.5)),k));
                end
            else
                if i~=1&&i~=BLOCK_H+1&&j~=1&&j~=BLOCK_V+1 %center area
                    subSampleRGB(j,i,k)=mean2( img(floor((j-1-0.5)*strideY):floor((j-1+0.5)*strideY),  floor(strideX*(i-1-0.5)):floor(strideX*(i-1+0.5)),k));
                end
            end
        end
    end
end

figure(2)
[A,B] = meshgrid(1:BLOCK_H+1,1:BLOCK_V+1);
mesh(A,B,subSampleRGB(:,:,1));
hold on
mesh(A,B,subSampleRGB(:,:,2));
hold on
mesh(A,B,subSampleRGB(:,:,3));

%Gainlut luma gain (Y-shading gain)
gainlut=zeros(BLOCK_V+1,BLOCK_H+1,3);% gainlut
for i = 1:BLOCK_H+1
    for j =1:BLOCK_V+1
        for k = 1:3
           gainlut(j,i,k)=mean2( subSampleRGB(floor(BLOCK_V/2+1) : floor(BLOCK_V/2+1)+1,  floor(BLOCK_H/2+1) : floor(BLOCK_H/2+1)+1,k)) / subSampleRGB(j,i,k);
        end
    end
end

%Gainlut chroma gain (colorshading gain)
gainlut_c = zeros(BLOCK_V+1,BLOCK_H+1,2);
center_RoverG = mean2( subSampleRGB(floor(BLOCK_V/2+1) : floor(BLOCK_V/2+1)+1,  floor(BLOCK_H/2+1) : floor(BLOCK_H/2+1)+1,1)) / mean2( subSampleRGB(floor(BLOCK_V/2+1) : floor(BLOCK_V/2+1)+1,  floor(BLOCK_H/2+1) : floor(BLOCK_H/2+1)+1,2));
center_BoverG = mean2( subSampleRGB(floor(BLOCK_V/2+1) : floor(BLOCK_V/2+1)+1,  floor(BLOCK_H/2+1) : floor(BLOCK_H/2+1)+1,3)) / mean2( subSampleRGB(floor(BLOCK_V/2+1) : floor(BLOCK_V/2+1)+1,  floor(BLOCK_H/2+1) : floor(BLOCK_H/2+1)+1,2));
for i = 1:BLOCK_H+1
    for j =1:BLOCK_V+1
           RoverG = subSampleRGB(j,i,1)/subSampleRGB(j,i,2);
           BoverG = subSampleRGB(j,i,3)/subSampleRGB(j,i,2);
           gainlut_c(j,i,1) = center_RoverG/RoverG-1;
           gainlut_c(j,i,2) = center_BoverG/BoverG-1;
    end
end

%chroma+luma gain
gainlut_full = zeros(BLOCK_V+1,BLOCK_H+1,2);
for i = 1:BLOCK_H+1
    for j =1:BLOCK_V+1
           gainlut_full(:,:,1) = gainlut(:,:,1) + gainlut_c(j,i,1);
           gainlut_full(:,:,2) = gainlut(:,:,2) + gainlut_c(j,i,2);
    end
end

%(BLOCK_H+1)*(BLOCK_V+1) colorgain
figure(3)
[C,D] = meshgrid(1:BLOCK_H+1,1:BLOCK_V+1);
mesh(C,D,gainlut_c(:,:,1))
hold on
mesh(C,D,gainlut_c(:,:,2))

figure(4)
mesh(C,D,gainlut_full(:,:,1))
hold on
mesh(C,D,gainlut_full(:,:,2))
hold on
mesh(C,D,gainlut(:,:,2))

%双线性插值FullGainLut
x = 1:strideX:cols+1;
x(end) = cols;
y = 1:strideY:rows+1;
y(end) = rows;

xitp = 1:cols;
yitp = 1:rows;
[Xitp,Yitp]=meshgrid(xitp,yitp);
rgain=interp2(x,y,gainlut(:,:,1),Xitp,Yitp);
ggain=interp2(x,y,gainlut(:,:,2),Xitp,Yitp);
bgain=interp2(x,y,gainlut(:,:,3),Xitp,Yitp);

%shading r,b,g gain
figure(5)
r=im2uint8( (im2double(r).*rgain) );
mesh(X1,Y1,im2double(r));
b=im2uint8( (im2double(b).*bgain) );
hold on
mesh(X1,Y1,im2double(b));
g=im2uint8( (im2double(g).*ggain) );
hold on
mesh(X1,Y1,im2double(g));

%shading corrected img
figure(6)
lscgain = cat(3,rgain,ggain,bgain);%merge三通道gain
imgDoubleType = im2double(img);
imgDoubleType = imgDoubleType.*lscgain;
imgUint = im2uint8(imgDoubleType);
imshow(imgUint)