I am running some matrix algebra on a large data set. Each iteration of the outer most loop populates one row of two different vectors that are allocated to 64,797 rows. I am printing a counter to screen for the outer loop to check progress. This might not be ideal. R is still working, according to task manager and using a good bit of memory and processor. However, the R console is not responding and I can only read at the end that I am at least to row 31,000ish (there is scroll space, but I cannot scroll down to see the last number printed). I do not know if the program is "hung" (no longer iterating outer loop) and I am wasting my time waiting, or if I should stick it out. The machine has been running for a few days. Given the program's structure, I can END the process and restart from the last row populated. However, if I end the process, will I lose the previously assigned data in my vector I am populating? That would be bad, as I'd have to start all over. Here is the code below. The end goal are the vectors called: save.trace and save.trace2.
我在大型数据集上运行一些矩阵代数。最外层循环的每次迭代都会填充分配给64,797行的两个不同向量中的一行。我打印一个计数器到外部循环的屏幕来检查进度。这可能不太理想。根据任务经理的说法,R仍在工作,并使用了大量的内存和处理器。但是,R控制台没有响应,我只能在最后读到我至少排31,000ish(有滚动空间,但我无法向下滚动以查看最后打印的数字)。我不知道程序是否“挂起”(不再迭代外循环)而且我在浪费时间等待,或者我是否应该坚持下去。机器已经运行了几天。鉴于程序的结构,我可以结束该过程并从填充的最后一行重新启动。但是,如果我结束这个过程,我会在我填充的向量中丢失先前分配的数据吗?那会很糟糕,因为我必须从头开始。这是下面的代码。最终目标是名为save.trace和save.trace2的向量。
for (i in 1:nrow(coor.cal)){
print(i)
for (j in 1:nrow(coor.cal)){
dist<-( (coor.cal[i,1]-coor.cal[j,1])^2 + (coor.cal[i,2]-coor.cal[j,2])^2)^.5
#finding distances between observations
w[j]<-exp(-0.5*((dist/bw)^2))#computing weight matrix for observation i
if (dist>bw){w[j]<-0}
}
for (k in 1:27){
xv<-xmat[ ,k]
xtw[k, ]<-xv*w
}
xtwx<-xtw%*%xmat
xtwx.inv<-ginv(xtwx)
xtwx.inv.xtw<-xtwx.inv%*%xtw
xrow<-xmat[i, ]
temp<-xrow%*%xtwx.inv.xtw
save.trace[i]<-temp[i]
save.trace2[i]<-sum(temp*temp)
}
2 个解决方案
#1
1
Here's a better example.
这是一个更好的例子。
saved <- 0
for(i in 1:100)
{
saved <- i
Sys.sleep(0.1)
}
Run this code, and press escape sometime in the next 10 seconds (before the loop completes).
运行此代码,并在接下来的10秒内(循环完成之前)按某个时间退出。
Take a look at the value of saved
. It should be more than 0
, indicating that your progress has been stored.
看看保存的价值。它应该大于0,表示您的进度已存储。
#2
0
I did not have the memory to risk an experiment to answer my question. I just borrowed another machine, tried it, and indeed you CAN end a process and still retain previously stored information. I had not run into this problem before. I attempted to delete my question, but could not. I'll leave this in case it helps someone else.
我没有记忆冒险尝试回答我的问题。我刚刚借了另一台机器,试了一下,确实你可以结束一个过程并仍然保留以前存储的信息。我之前没遇到过这个问题。我试图删除我的问题,但不能。我会留下这个,以防它帮助别人。
#1
1
Here's a better example.
这是一个更好的例子。
saved <- 0
for(i in 1:100)
{
saved <- i
Sys.sleep(0.1)
}
Run this code, and press escape sometime in the next 10 seconds (before the loop completes).
运行此代码,并在接下来的10秒内(循环完成之前)按某个时间退出。
Take a look at the value of saved
. It should be more than 0
, indicating that your progress has been stored.
看看保存的价值。它应该大于0,表示您的进度已存储。
#2
0
I did not have the memory to risk an experiment to answer my question. I just borrowed another machine, tried it, and indeed you CAN end a process and still retain previously stored information. I had not run into this problem before. I attempted to delete my question, but could not. I'll leave this in case it helps someone else.
我没有记忆冒险尝试回答我的问题。我刚刚借了另一台机器,试了一下,确实你可以结束一个过程并仍然保留以前存储的信息。我之前没遇到过这个问题。我试图删除我的问题,但不能。我会留下这个,以防它帮助别人。