在python中存储大对称稀疏矩阵的最有效方法

I was working on drafting/testing a technique I devised for solving differential equations for speed and efficiency.

我正在起草/测试我为解决速度和效率的微分方程而设计的技术。

It would require a storing, manipulating, resizing, and (at some point) probably diagonalizing very large sparse matrices. I would like to be able to have rows consisting of zeros and a few (say <5) ones, and add them a few at a time (on the order of the number of cpus being used.)

它需要存储,操纵,调整大小和(在某些时候)可能对齐非常大的稀疏矩阵。我希望能够有由零和一些(比如<5)组成的行,并一次添加几个(按照正在使用的cpu数量的顺序)。

I thought it would be useful to have gpu accelleration--so any suggestions as to the best way to take advange of that would be appreciated too (say pycuda, theano, etc.)

我认为让gpu加速是有用的 - 所以任何关于采取优势的最佳方式的建议也会受到赞赏(比如pycuda,theano等)

3 个解决方案

#1

You can use a dictionary and tuples to access the data:

您可以使用字典和元组来访问数据:

>>> size = (4,4)
>>> mat = {}
>>> mat[0,1] = 3
>>> mat[2,3] = 5
>>> for i in range(size[0]):
        for j in range(size[1]):
            print mat.get((i,j), 0) ,
        print

0 3 0 0
0 0 0 0
0 0 0 5
0 0 0 0

Of course you should make a class for that and add the methods you need:

当然你应该为它做一个类并添加你需要的方法:

class Sparse(dict):
    pass

BTW, You can also use scipy.sparse from the scipy lib

顺便说一句,你也可以使用scipy lib中的scipy.sparse

#2

The most efficient storage method for symmetric sparse matrices is probably sparse skyline format (this is what Intel MKL uses, for example). AFAIK scipy.sparse doesn't contain a sparse, symmetric matrix format. Pysparse, however, does. Using Pysparse, you can build the matrix incrementally using the link list format, then convert the matrix into sparse skyline format. Performance wise, I have usually found Pysparse to be superior to scipy with large sparse systems, and all the basic building blocks (matrix product, eigenvalue solver, direct solver, iterative solver) as present, although the range of routines is perhaps a little smaller than what is available in scipy.

对称稀疏矩阵的最有效存储方法可能是稀疏天际线格式(例如,这是英特尔MKL使用的)。 AFAIK scipy.sparse不包含稀疏的对称矩阵格式。然而,Pysparse确实如此。使用Pysparse,您可以使用链接列表格式逐步构建矩阵,然后将矩阵转换为稀疏天际线格式。性能方面,我经常发现Pysparse优于scipy大型稀疏系统,并且所有基本构建块(矩阵乘积,特征值求解器,直接求解器,迭代求解器)都存在,尽管例程范围可能稍微小一点比scipy中的可用。

#3

Use scipy.sparse.

#1