在Python中使用numpy / scipy进行binning的向量化方法

I am binning a 2d array (x by y) in Python into the bins of its x value (given in "bins"), using np.digitize:

我使用np.digitize将Python中的2d数组(x by y)分类到其x值的区间(在“bins”中给出):

elements_to_bins = digitize(vals, bins)

where "vals" is a 2d array, i.e.:

其中“vals”是一个二维数组,即:

 vals = array([[1, v1], [2, v2], ...]).

elements_to_bins just says what bin each element falls into. What I then want to do is get a list whose length is the number of bins in "bins", and each element returns the y-dimension of "vals" that falls into that bin. I do it this way right now:

elements_to_bins只是说每个元素落入哪个bin。我当时想要做的是得到一个列表,其长度是“bins”中的bin数,每个元素返回落入该bin的“vals”的y维度。我现在这样做:

points_by_bins = []
for curr_bin in range(min(elements_to_bins), max(elements_to_bins) + 1):
    curr_indx = where(elements_to_bins == curr_bin)[0]
    curr_bin_vals = vals[:, curr_indx]
    points_by_bins.append(curr_bin_vals)

is there a more elegant/simpler way to do this? All I need is a list of of lists of the y-values that fall into each bin.

有没有更优雅/更简单的方法来做到这一点?我只需要列出每个bin中的y值列表。

thanks.

3 个解决方案

#1

If I understand your question correctly:

如果我理解你的问题:

vals = array([[1, 10], [1, 11], [2, 20], [2, 21], [2, 22]])  # Example

(x, y) = vals.T  # Shortcut
bin_limits = range(min(x)+1, max(x)+2)  # Other limits could be chosen
points_by_bin = [ [] for _ in bin_limits ]  # Final result
for (bin_num, y_value) in zip(searchsorted(bin_limits, x, "right"), y):  # digitize() finds the correct bin number
    points_by_bin[bin_num].append(y_value)

print points_by_bin  # [[10, 11], [20, 21, 22]]

Numpy's fast array operation searchsorted() is used for maximum efficiency. Values are then added one by one (since the final result is not a rectangular array, Numpy cannot help much, for this). This solution should be faster than multiple where() calls in a loop, which force Numpy to re-read the same array many times.

Numpy的快速数组操作searchsorted()用于最大效率。然后逐个添加值(因为最终结果不是矩形数组,Numpy对此无能为力)。这个解决方案应该比循环中的多个where()调用更快,这迫使Numpy多次重新读取同一个数组。

#2

This will return a data structure analogous to IDL HISTOGRAM's Reverse_Indices:

这将返回类似于IDL HISTOGRAM的Reverse_Indices的数据结构:

ovec = np.argsort(vals)
ivec = np.searchsorted(vals, bin_limits, sorter=ovec)

Then the list of elements that fall into bin #i is

那么落入bin #i的元素列表就是

ovec[ ivec[i] : ivec[i+1] ]

(my quick timing tests say this is 5x faster than EOL's algorithm, since it doesn't bother creating different-sized lists)

(我的快速计时测试表明这比EOL的算法快5倍,因为它不会创建不同大小的列表)

#3

Are the bin keys just integers, no binning, as in your example ? Then you could just do this, without numpy:

bin键是否只是整数,没有binning,就像你的例子一样?然后你可以这样做,没有numpy:

from collections import defaultdict
bins = defaultdict(list)  # or [ [] ...] as in EOL

vals = [[1, 10], [1, 11], [2, 20], [2, 21], [2, 22]]  # nparray.tolist()
for nbin, val in vals:
    bins[nbin].append(val)

print "bins:", bins
# defaultdict(<type 'list'>, {1: [10, 11], 2: [20, 21, 22]})

#1