【转载请注明出处】http://www.cnblogs.com/mashiqi
(居然有朋友说内容不接地气，那么我就再加一段嘛，请喜欢读笑话的同学直接看第二段)假设这里有一组向量$\left\{ x_i \right\}_{i=1}^n$和一个待投影的向量$u$。假设$u$和每个$x_i$的内积都为正数，也就是说$u$和每个$x_i$的夹角都小于90度。那么当我们把$u$投影到$\left\{ x_i \right\}_{i=1}^n$上时，理所应当地每个$x_i$的系数$\beta_i$也都应该大于零：$$u = x_1\beta_1+\cdots+x_n\beta_n,\beta_i\geq0$$不知道读者们的空间直觉怎么样，反正我最开始就是这么天真的认为的。最近看了Efron的“Least Angle Regression”后，才明白原来不是这样的，自己以前too young了。有些时候系数会变成负的。
(大家好我是第二段)举个栗子，设$$x_1=(0.1793,0.1216,0.9762)^T$$$$x_2=(-0.5947,0.5150,0.6173)^T$$$$x_3=(-0.2013,0.1782,-0.9632)^T$$并令$$u=(3.7952,6.9738,-0.5415)^T$$那么我们可以计算出$$(u,x_1)=(u,x_2)=(u,x_3)=1 > 0$$也就是说$x_1,x_2,x_3$与$u$的夹角都小于90度。但是通过解矩阵方程$X\beta=u$我们可以得到$$\beta = (36.5949,-6.6537,33.3891)^T$$也就是说：$$u = 36.5949x_1 -6.6537x_2 + 33.3891x_3$$看呐！$x_2$君的系数居然是负的！$x_1$和$x_3$一样，$x_2$君也是同样思念着$u$君的，还思念的同样深(内积都是$1$ = =)。可是$x_2$君为了集体的利益甘愿牺牲自己成全整个$\left\{ x_i \right\}_{i=1}^3$集合，这是多么伟大的奉献精神，此处应有掌声，啪啪啪，啪啪啪。。。
下面贴一个小MATLAB代码，自己去体会吧！
%{
% This small matlab program show you a unexpected result in
% high-dimensional geometry: for any set of n-dimensional vectors
% {x_1,...,x_n}, if these vectors are indepentent, then you can always find
% an equiangular vector u, so that the inner product (x_i,u)= for all i.
% BUT, if we project u into {x_i}, some coefficients may be 'negative'!
%}
%%
clear;
close all;
%% You can change the following two variable's value
dimension = 3;
vectors = 3;
%% In case of unpredicted problems, dont change the following code.
if vectors > dimension
disp('Please set vectors <= dimension.');
end
X = randn(dimension,vectors); % every column is a vector
X = X./repmat(sqrt(sum(X.^2,1)),dimension,1); % standardize
if rank(X) ~= size(X,2)
disp('These vectors are not independent. Run again.');
end
w = (X'*X)\ones(vectors,);
u = X*w;
w % the coefficient
u % the equiangular vector
X'*u % the correlation value
if dimension == 3 % '' is the upper bound of dimensions of humans where we can draw.
quiver3(0,0,0,X(1,1),X(2,1),X(3,1),'r'),hold on;
quiver3(0,0,0,X(1,2),X(2,2),X(3,2),'r'),hold on;
quiver3(0,0,0,X(1,3),X(2,3),X(3,3),'r'),hold on;
quiver3(0,0,0,u(1),u(2),u(3),'b'),hold off;
end
总结一句话：高维空间有危险，忽久留= =||