将pandas数据帧拆分为N个块

I'm currently trying to split a pandas dataframe into an unknown number of chunks containing each N rows.

我目前正在尝试将一个pandas数据帧拆分为包含每N行的未知数量的块。

I have tried using numpy.array_split() this funktion however splits the dataframe into N chunks containing an unknown number of rows.

我尝试过使用numpy.array_split()这个功能但是将数据帧拆分为包含未知行数的N个块。

Is there a clever way to split a python dataframe into multiple dataframes, each containing a specific number of rows from the parent dataframe

有没有一种聪明的方法可以将python数据帧拆分成多个数据帧,每个数据帧包含父数据帧中的特定行数

3 个解决方案

#1

You can try this:

你可以试试这个:

def rolling(df, window, step):
    count = 0
    df_length = len(df)
    while count < (df_length -window):
        yield count, df[count:window+count]
        count += step

Usage:

for offset, window in rolling(df, 100, 100):
    # |     |                      |     |
    # |     The current chunk.     |     How many rows to step at a time.
    # The current offset index.    How many rows in each chunk.
    # your code here
    pass

There is also this simpler idea:

还有一个更简单的想法:

def chunk(seq, size):
    return (seq[pos:pos + size] for pos in range(0, len(seq), size))

Usage:

for df_chunk in chunk(df, 100):
    #                     |
    #                     The chunk size
    # your code here

BTW. All this can be found on SO, with a search.

BTW。所有这些都可以在SO上找到,并进行搜索。

#2

You can calculate the number of splits from N:

您可以从N计算分割数:

splits = int(np.floor(len(df.index)/N))
chunks = np.split(df.iloc[:splits*N], splits)
chunks.append(df.iloc[splits*N:])

#3

calculate the index of splits :

计算分裂的索引:

size_of_chunks =  3
index_for_chunks = list(range(0, index.max(), size_of_chunks))
index_for_chunks.extend([index.max()+1])

use them to split the df :

用它们来拆分df:

dfs = {}
for i in range(len(index_for_chunks)-1):
    dfs[i] = df.iloc[index_for_chunks[i]:index_for_chunks[i+1]]