I have a main function in R which calls other files to run my program. I call the main file through a bat file(.exe). When I run it line-by-line it runs without a memory error, but when I call the bat file to run it, it halts and gives me the following error:
我在R中有一个主函数,它调用其他文件来运行我的程序。我通过bat文件(.exe)调用主文件。当我逐行运行它时,它不会出现内存错误,但是当我调用bat文件来运行它时,它会停止运行并给出如下错误:
Cannot allocate memory greater than 51 MB.
不能分配大于51 MB的内存。
How can I avoid this?
我怎样才能避免这种情况呢?
2 个解决方案
#1
1
Yes you should be using 64bit R, if you can.
是的,如果可以的话,你应该用64位R。
See this question, and this from the R docs.
看看这个问题,这是来自R文档的。
#2
2
Memory limitations in R such as this are a recurring nightmare for a lot of us.
像这样的R中的内存限制对于我们很多人来说是一个反复出现的噩梦。
Very often the problem is a limit imposed by your OS limits (which can usually be changed on a Bash or PowerShell command line), architecture (32 v. 64 bit), or the availability of contiguous free RAM, irregardless of overall available memory.
通常,问题是您的OS限制(通常可以在Bash或PowerShell命令行上更改)、体系结构(32 v 64位)或连续空闲RAM的可用性,而不考虑整个可用内存。
It's hard to say why something would not cause a memory issue when run line by line, but would hit the memory limit when run as a .bat.
很难解释为什么一些东西在逐行运行时不会引起内存问题,但在以.bat的形式运行时会达到内存限制。
What version of R are you running? Do you have both installed? Is 32-bit being called by Rscript
when you run your .bat file whereas you run a 64-bit version line by line? You can check the version of R that's being run with R.Version()
.
你在运行什么版本的R ?你们都装好了吗?当您运行.bat文件而逐行运行64位版本时,Rscript会调用32位吗?您可以查看正在运行的R版本的R版本()。
You can test this by running the command memory.limit()
in both your R IDE/terminal and in your .bat file (be sure to print
or save the result as an object in your .bat file). You might also do well to try setting memory.limit()
in your .bat file, as it may just have a smaller default, perhaps due to differences in your R Profile that's invoked in your IDE or terminal versus the .bat file.
您可以在您的IDE/终端和.bat文件中运行command memory.limit()命令来测试这一点(请确保将结果打印或保存为.bat文件中的对象)。您也可以尝试在.bat文件中设置memory.limit(),因为它的默认值可能更小,这可能是由于IDE或终端中调用的R配置文件与.bat文件之间的差异造成的。
If architecture isn't the cause of your memory error, then you have several more troubleshooting steps to try:
如果架构不是导致您的内存错误的原因,那么您还需要进行几个故障排除步骤:
- Check memory usage in both environments (in R directly and via your .bat process) using this:
sort( sapply(ls(),function(x){object.size(get(x))}))
- 使用这个:sort(ls()、函数(x){object.size(get(x))})检查两个环境中的内存使用情况(在R中直接通过.bat进程)
- Run the garbage collector explicitly in your scripts, that's the
gc()
command - 在脚本中显式地运行垃圾收集器,这是gc()命令
- Check all object sizes to make sure there are no unexpected results in your .bat process:
sort( sapply(ls(),function(x){format(object.size(get(x)), units = "Mb")}))
- 检查所有对象大小,确保.bat过程中没有意外结果:sort(sapply(ls(),函数(x){format(object.size(get(x), units = "Mb")}))
-
Try memory profiling:
试着内存分析:
Rprof(tf <- "rprof.log", memory.profiling=TRUE) Rprof(NULL) summaryRprof(tf)
-
While this is a RAM issue, for good measure you might want to check that the compute power available is both sufficient and not varying between these two ways of running your code:
parallel::detectCores()
虽然这是一个RAM问题,但是为了更好地衡量,您可能想要检查在这两种运行代码的方式之间,可用的计算能力是否充分且没有变化:parallel: detectcore ()
-
Examine your performance with Prof. Hadley Wikham's
lineprof
tool (warning: requiresdevtools
and doesn't work on lines of code which call the C programming language)使用Hadley Wikham教授的lineprof工具检查您的性能(警告:需要devtools,不支持调用C编程语言的代码行)
References While I'm pulling these snippets out of my own code, most of them originally came from other, related * posts, such as:
当我从我自己的代码中提取这些代码片段时,它们中的大多数都来自其他相关的*帖子,比如:
- Reaching memory allocation in R
- 到达R中的内存分配
- R Memory Allocation "Error: cannot allocate vector of size 75.1 Mb"
- R内存分配“错误:无法分配大小为75.1 Mb的向量”
- R memory limit warning vs "unable to allocate..."
- R内存限制警告vs“无法分配…”
- How to compute the size of the allocated memory for a general type
- 如何计算为一般类型分配的内存的大小
- R : Any other solution to "cannot allocate vector size n mb" in R?
- R:在R中还有其他“无法分配向量大小为n mb”的解决方案吗?
#1
1
Yes you should be using 64bit R, if you can.
是的,如果可以的话,你应该用64位R。
See this question, and this from the R docs.
看看这个问题,这是来自R文档的。
#2
2
Memory limitations in R such as this are a recurring nightmare for a lot of us.
像这样的R中的内存限制对于我们很多人来说是一个反复出现的噩梦。
Very often the problem is a limit imposed by your OS limits (which can usually be changed on a Bash or PowerShell command line), architecture (32 v. 64 bit), or the availability of contiguous free RAM, irregardless of overall available memory.
通常,问题是您的OS限制(通常可以在Bash或PowerShell命令行上更改)、体系结构(32 v 64位)或连续空闲RAM的可用性,而不考虑整个可用内存。
It's hard to say why something would not cause a memory issue when run line by line, but would hit the memory limit when run as a .bat.
很难解释为什么一些东西在逐行运行时不会引起内存问题,但在以.bat的形式运行时会达到内存限制。
What version of R are you running? Do you have both installed? Is 32-bit being called by Rscript
when you run your .bat file whereas you run a 64-bit version line by line? You can check the version of R that's being run with R.Version()
.
你在运行什么版本的R ?你们都装好了吗?当您运行.bat文件而逐行运行64位版本时,Rscript会调用32位吗?您可以查看正在运行的R版本的R版本()。
You can test this by running the command memory.limit()
in both your R IDE/terminal and in your .bat file (be sure to print
or save the result as an object in your .bat file). You might also do well to try setting memory.limit()
in your .bat file, as it may just have a smaller default, perhaps due to differences in your R Profile that's invoked in your IDE or terminal versus the .bat file.
您可以在您的IDE/终端和.bat文件中运行command memory.limit()命令来测试这一点(请确保将结果打印或保存为.bat文件中的对象)。您也可以尝试在.bat文件中设置memory.limit(),因为它的默认值可能更小,这可能是由于IDE或终端中调用的R配置文件与.bat文件之间的差异造成的。
If architecture isn't the cause of your memory error, then you have several more troubleshooting steps to try:
如果架构不是导致您的内存错误的原因,那么您还需要进行几个故障排除步骤:
- Check memory usage in both environments (in R directly and via your .bat process) using this:
sort( sapply(ls(),function(x){object.size(get(x))}))
- 使用这个:sort(ls()、函数(x){object.size(get(x))})检查两个环境中的内存使用情况(在R中直接通过.bat进程)
- Run the garbage collector explicitly in your scripts, that's the
gc()
command - 在脚本中显式地运行垃圾收集器,这是gc()命令
- Check all object sizes to make sure there are no unexpected results in your .bat process:
sort( sapply(ls(),function(x){format(object.size(get(x)), units = "Mb")}))
- 检查所有对象大小,确保.bat过程中没有意外结果:sort(sapply(ls(),函数(x){format(object.size(get(x), units = "Mb")}))
-
Try memory profiling:
试着内存分析:
Rprof(tf <- "rprof.log", memory.profiling=TRUE) Rprof(NULL) summaryRprof(tf)
-
While this is a RAM issue, for good measure you might want to check that the compute power available is both sufficient and not varying between these two ways of running your code:
parallel::detectCores()
虽然这是一个RAM问题,但是为了更好地衡量,您可能想要检查在这两种运行代码的方式之间,可用的计算能力是否充分且没有变化:parallel: detectcore ()
-
Examine your performance with Prof. Hadley Wikham's
lineprof
tool (warning: requiresdevtools
and doesn't work on lines of code which call the C programming language)使用Hadley Wikham教授的lineprof工具检查您的性能(警告:需要devtools,不支持调用C编程语言的代码行)
References While I'm pulling these snippets out of my own code, most of them originally came from other, related * posts, such as:
当我从我自己的代码中提取这些代码片段时,它们中的大多数都来自其他相关的*帖子,比如:
- Reaching memory allocation in R
- 到达R中的内存分配
- R Memory Allocation "Error: cannot allocate vector of size 75.1 Mb"
- R内存分配“错误:无法分配大小为75.1 Mb的向量”
- R memory limit warning vs "unable to allocate..."
- R内存限制警告vs“无法分配…”
- How to compute the size of the allocated memory for a general type
- 如何计算为一般类型分配的内存的大小
- R : Any other solution to "cannot allocate vector size n mb" in R?
- R:在R中还有其他“无法分配向量大小为n mb”的解决方案吗?