I'm doing some analysis that requires me to compute a distance matrix (Euclidean for now, likely Mahalanobis later) from a 7000x182 array that will result in a 7000x7000 matrix (assuming I take the upper part of the matrix, too).
我正在进行一些分析,要求我从一个7000x182阵列计算一个距离矩阵(现在是Euclidean,可能是Mahalanobis),这将产生一个7000x7000矩阵(假设我也采用了矩阵的上半部分)。
z <- as.matrix(dist(training.data[1:7000,], diag=TRUE, upper=TRUE))
The problem I'm running into is that I keep getting the "cannot allocate vector..." warning. I've read up enough to know that R can't allocate enough contiguous memory on my PC to perform the action. My PC has 3GB of RAM, and despite my best efforts or rm()'ing objects, using gc(), increasing my memory allocation to R through memory.limit(size=####), it doesn't work.
我遇到的问题是我不断收到“无法分配矢量......”的警告。我已经阅读了足够的知道R无法在我的PC上分配足够的连续内存来执行操作。我的PC有3GB的RAM,尽管我尽了最大的努力或rm()'对象,使用gc(),通过memory.limit(size = ####)增加我的内存分配,它不起作用。
I like what I've read about the bigmemory library; however, when I attempt to install it I get a warning that it's not available on R 2.15.2. Am I missing something here? Is there a way around this? I've read about some other R tools for managing big data, but none seem apropros to my task like bigmemory.
我喜欢我读过的关于bigmemory库的内容;但是,当我尝试安装它时,我得到一个警告,它在R 2.15.2上不可用。我在这里错过了什么吗?有没有解决的办法?我已经阅读了一些其他用于管理大数据的R工具,但似乎没有像bigmemory那样适应我的任务。
2 个解决方案
#1
1
I've worked with bigmemory on windows before, and it seems like the only R version that I can use it on is 2.15.3 (it should install easily with a standard call to install.packages). After the upgrade to R 3.0.x, it seems like bigmemory support for windows is "temporarily suspended" (see http://cran.r-project.org/web/packages/bigmemory/NEWS). I was also never able to get it to work on R 2.15.2 or earlier either; I always ran into errors like the ones you mentioned.
我之前在windows上使用过bigmemory,看起来我可以使用的唯一R版本是2.15.3(它应该可以通过标准调用install.packages轻松安装)。在升级到R 3.0.x之后,似乎对Windows的大量内存支持“暂时中止”(请参阅http://cran.r-project.org/web/packages/bigmemory/NEWS)。我也无法让它在R 2.15.2或更早版本上工作;我总是遇到你提到的错误。
My solution has been to have R 2.15.3 and the most recent version of R both installed on my windows machine. I use R 2.15.3 for bigmemory and switch back to R 3.1.0 when not using it. It's a pain, but bigmemory is worth it :)
我的解决方案是在我的Windows机器上安装R 2.15.3和最新版本的R。我将R 2.15.3用于bigmemory,并在不使用时切换回R 3.1.0。这是一个痛苦,但bigmemory是值得的:)
#2
0
For me this works:
对我来说这有效:
install.packages(c("BH","biglm"))
library(devtools)
devtools::install_github('kaneplusplus/bigmemory')
library(bigmemory)
PS you can also install library(Rdsm)
on Windows using
PS你也可以在Windows上安装库(Rdsm)
devtools::install_github('tomwenseleers/Rdsm')
library(Rdsm)
#1
1
I've worked with bigmemory on windows before, and it seems like the only R version that I can use it on is 2.15.3 (it should install easily with a standard call to install.packages). After the upgrade to R 3.0.x, it seems like bigmemory support for windows is "temporarily suspended" (see http://cran.r-project.org/web/packages/bigmemory/NEWS). I was also never able to get it to work on R 2.15.2 or earlier either; I always ran into errors like the ones you mentioned.
我之前在windows上使用过bigmemory,看起来我可以使用的唯一R版本是2.15.3(它应该可以通过标准调用install.packages轻松安装)。在升级到R 3.0.x之后,似乎对Windows的大量内存支持“暂时中止”(请参阅http://cran.r-project.org/web/packages/bigmemory/NEWS)。我也无法让它在R 2.15.2或更早版本上工作;我总是遇到你提到的错误。
My solution has been to have R 2.15.3 and the most recent version of R both installed on my windows machine. I use R 2.15.3 for bigmemory and switch back to R 3.1.0 when not using it. It's a pain, but bigmemory is worth it :)
我的解决方案是在我的Windows机器上安装R 2.15.3和最新版本的R。我将R 2.15.3用于bigmemory,并在不使用时切换回R 3.1.0。这是一个痛苦,但bigmemory是值得的:)
#2
0
For me this works:
对我来说这有效:
install.packages(c("BH","biglm"))
library(devtools)
devtools::install_github('kaneplusplus/bigmemory')
library(bigmemory)
PS you can also install library(Rdsm)
on Windows using
PS你也可以在Windows上安装库(Rdsm)
devtools::install_github('tomwenseleers/Rdsm')
library(Rdsm)