将3D数组的每个元素连续写入文件的最有效方法是什么?

时间:2021-10-12 20:12:55

I have a large 3D array that I want to write as a single column to a file. The data needs to vary first by Z (high to low) then by x (low to high) then by y (low to high). At the moment I am doing this by:

我有一个大型3D数组,我想将其作为单个列写入文件。数据首先需要通过Z(从高到低)然后通过x(从低到高)然后通过y(从低到高)变化。目前我这样做:

arr <<- array(0,dim=c(x,y,z)) # values aren't really 0
dataf <- data.frame(Px=rep(0,x*y*z))
ticker <- 0
  for (j in 1:y){
    for (i in 1:x){
      for (k in z:1){
        ticker <- ticker +1
        dataf[ticker,1] <- arr[i,j,k]
        }
      }
    }
write.table(dataf,file=filename,row.names=F,col.names=F)

This process is slow and seems to slow down as the iterations progress (seen with a progress bar). I'm sure theres a way to use adply but I can't get it to work. I switched the order of the z data like this:

这个过程很慢,并且随着迭代的进展似乎变慢(用进度条看)。我确定这是一种使用adply的方法,但我无法让它工作。我改变了z数据的顺序,如下所示:

    for (n in 1:z)
      arr_inv[,,n] <- arr[,,(z-n+1)]

Then try to write like this:

然后尝试这样写:

write.table(adply(arr,.margins=c(1,2,3))[4],file=filename,row.names=F,col.names=F)

Nothing I do seems to be fast, so I'm wondering if you know how I should approach this? Thanks

我所做的一切似乎都很快,所以我想知道你是否应该知道我应该如何做到这一点?谢谢

1 个解决方案

#1


1  

One (convoluted) way would be to split the array across the third dimension into list elements using lapply then use mapply to reshape the matrix and finally turn it into an output vector:

一种(复杂的)方法是使用lapply将数组在第三维上拆分为列表元素,然后使用mapply重新整形矩阵,最后将其转换为输出向量:

mat <- array( rep(1:9,each=3) , dim = c(3,3,3) )
mat
, , 1

     [,1] [,2] [,3]
[1,]    1    2    3
[2,]    1    2    3
[3,]    1    2    3

, , 2

     [,1] [,2] [,3]
[1,]    4    5    6
[2,]    4    5    6
[3,]    4    5    6

, , 3

     [,1] [,2] [,3]
[1,]    7    8    9
[2,]    7    8    9
[3,]    7    8    9


as.vector( t( mapply( c , lapply( 1:3 , function(x) as.vector( t(mat[,,x]) ) ) ) ) )
[1] 1 4 7 2 5 8 3 6 9 1 4 7 2 5 8 3 6 9 1 4 7 2 5 8 3 6 9

As anside, please include a reproducible example next time

作为一个方面,请在下次包含可重复的示例

#1


1  

One (convoluted) way would be to split the array across the third dimension into list elements using lapply then use mapply to reshape the matrix and finally turn it into an output vector:

一种(复杂的)方法是使用lapply将数组在第三维上拆分为列表元素,然后使用mapply重新整形矩阵,最后将其转换为输出向量:

mat <- array( rep(1:9,each=3) , dim = c(3,3,3) )
mat
, , 1

     [,1] [,2] [,3]
[1,]    1    2    3
[2,]    1    2    3
[3,]    1    2    3

, , 2

     [,1] [,2] [,3]
[1,]    4    5    6
[2,]    4    5    6
[3,]    4    5    6

, , 3

     [,1] [,2] [,3]
[1,]    7    8    9
[2,]    7    8    9
[3,]    7    8    9


as.vector( t( mapply( c , lapply( 1:3 , function(x) as.vector( t(mat[,,x]) ) ) ) ) )
[1] 1 4 7 2 5 8 3 6 9 1 4 7 2 5 8 3 6 9 1 4 7 2 5 8 3 6 9

As anside, please include a reproducible example next time

作为一个方面,请在下次包含可重复的示例