为numpy/scipy快速索引的dot产品。

I'm using numpy to do linear algebra. I want to do fast subset-indexed dot and other linear operations.

我用numpy来做线性代数。我要做的是快速的子集索引点和其他线性操作。

When dealing with big matrices, slicing solution like A[:,subset].dot(x[subset]) may be longer than doing the multiplication on the full matrix.

在处理大的矩阵时，像[:，子集].dot(x[子集])那样的切片解决方案可能比在全矩阵上做乘法的时间要长。

A = np.random.randn(1000,10000)
x = np.random.randn(10000,1)
subset = np.sort(np.random.randint(0,10000,500))

Timings show that sub-indexing can be faster when columns are in one block.

计时显示当列在一个块中时，子索引可以更快。

%timeit A.dot(x)
100 loops, best of 3: 4.19 ms per loop

%timeit A[:,subset].dot(x[subset])
100 loops, best of 3: 7.36 ms per loop

%timeit A[:,:500].dot(x[:500])
1000 loops, best of 3: 1.75 ms per loop

Still the acceleration is not what I would expect (20x faster!).

但加速度不是我所期望的(20x更快!)

Does anyone know an idea of a library/module that allow these kind of fast operation through numpy or scipy?

有谁知道一个库/模块的想法，允许这种快速操作通过numpy或scipy?

For now on I'm using cython to code a fast column-indexed dot product through the cblas library. But for more complex operation (pseudo-inverse, or subindexed least square solving) I'm not shure to reach good acceleration.

现在，我正在使用cython通过cblas库编写一个快速的列索引的dot产品。但对于更复杂的运算(伪逆，或子索引的最小平方解)，我不需要达到良好的加速度。

Thanks!

谢谢!

1 个解决方案

#1

Well, this is faster.

这是更快。

%timeit A.dot(x)
#4.67 ms

%%timeit
y = numpy.zeros_like(x)
y[subset]=x[subset]
d = A.dot(y)
#4.77ms

%timeit c = A[:,subset].dot(x[subset])
#7.21ms

And you have all(d-ravel(c)==0) == True.

你们都有(d-ravel(c)==0) == True。

Notice that how fast this is depends on the input. With subset = array([1,2,3]) you have that the time of my solution is pretty much the same, while the timing of the last solution is 46micro seconds.

请注意，这取决于输入的速度。对于子集=数组([1,2,3])，我的解决方案的时间几乎是相同的，而最后一个解决方案的时间是46微秒。

Basically this will be faster if the size ofsubset is not much smaller than the size of x

如果子集的大小不小于x的大小，这就会更快。

#1