AWS上的R和data.table

时间:2021-05-27 23:23:30

This is a very strange error I am running into trying to install a specific R library on AWS EC2 instance (Amazon Linux AMI). A link in the AWS forums here, posted by someone else, actually highlights my issue well.

这是一个非常奇怪的错误,我试图在AWS EC2实例(Amazon Linux AMI)上安装特定的R库。此处AWS论坛中的链接由其他人发布,实际上很好地突出了我的问题。

The main / relevant part of the error message is:

错误消息的主要/相关部分是:

data.table.h:6:12: fatal error: omp.h: No such file or directory
#include <omp.h> 

I did some research (while I'm not sure), I think this is related to whether or not there is an OpenMP enabled compiler on the server. The data.table installation page on GitHub itself discusses this a bit here, but I'm not sure how to update or fix this on my EC2 instance.

我做了一些研究(虽然我不确定),我认为这与服务器上是否有支持OpenMP的编译器有关。 GitHub上的data.table安装页面在这里讨论了这一点,但我不知道如何在我的EC2实例上更新或修复它。

Any help with this is appreciated.

对此有任何帮助表示赞赏。

EDIT - this is a new problem, as i was able to successfully install data.table on a previous, similar EC2 instance less than a month ago.

编辑 - 这是一个新问题,因为我能够在不到一个月前在之前类似的EC2实例上成功安装data.table。

EDIT 2 - I got around this issue by taking a previous EC2 instance of mine with data.table already installed on it, and creating a custom AMI from it. By using this custom AMI when launching new instances, they already came with the data.table library installed. If I notice AWS resolve this issue on its own, I'll try to remember to come back and update this post!

编辑2 - 我解决了这个问题,我先前已经安装了data.table,并从中创建了一个自定义AMI。通过在启动新实例时使用此自定义AMI,它们已经安装了data.table库。如果我发现AWS自己解决了这个问题,我会尽量记得回来更新这篇文章!

3 个解决方案

#1


6  

The problem here is that data.table doesn't play nice with the default gcc compiler (gcc72-c++.x86_64 gets installed as a dependency of R-devel.x86_64). Point R to an older version by adding

这里的问题是data.table与默认的gcc编译器不兼容(gcc72-c ++。x86_64作为R-devel.x86_64的依赖项安装)。通过添加将R指向旧版本

CC=gcc64

in ~/.R/Makevars. If you start from a "clean" Amazon Linux AMI this file doesn't exist and you can just type

在〜/ .R / Makevars中。如果您从“干净的”Amazon Linux AMI开始,此文件不存在,您只需键入即可

mkdir ~/.R
echo "CC=gcc64" >> ~/.R/Makevars

#2


3  

For some reason setting CC=gcc64 in the ~/.R/Makevars did not work for me. R was still using default gcc to compile.

出于某种原因,在〜/ .R / Makevars中设置CC = gcc64对我来说不起作用。 R仍然使用默认的gcc进行编译。

However there is another option. You can edit the Makeconf file that R uses during compilation directly. If you're using Amazon Linux the file location is /usr/lib64/R/etc/Makeconf. Once you locate the file the trick is just the same, that is to change the CC = gcc to CC = gcc64. You also might want to make sure that gcc64 is installed by running sudo yum install gcc64.

但是还有另一种选择。您可以直接编辑R在编译期间使用的Makeconf文件。如果您使用的是Amazon Linux,则文件位置为/ usr / lib64 / R / etc / Makeconf。一旦找到文件,技巧就是一样,即将CC = gcc更改为CC = gcc64。您还可以通过运行sudo yum install gcc64来确保安装gcc64。

#3


0  

I have the following workaround.

我有以下解决方法。

1) Connect with ssh.

1)用ssh连接。

2) Download package source

2)下载包源

wget https://cran.r-project.org/src/contrib/data.table_1.10.4-3.tar.gz

3) Install package

3)安装包

R CMD INSTALL data.table_1.10.4-3.tar.gz

This will install the package in the local package directory, not necessarily the one of the RStudio user(s).

这将把软件包安装在本地软件包目录中,不一定是RStudio用户之一。

4) Use .libPaths() in R and RStudio to figure out where packages are stored.

4)在R和RStudio中使用.libPaths()来确定存储包的位置。

5) Copy package into the desired directory using sudo cp -R SOURCE DEST.

5)使用sudo cp -R SOURCE DEST将包复制到所需目录。

Remark: This is a workaround not a solution because you need to install each package manually this way, i.e. handling dependencies can be cumbersome, but at least data.table works now.

备注:这是一种解决方法而不是解决方案,因为您需要以这种方式手动安装每个包,即处理依赖关系可能很麻烦,但至少data.table现在可以工作。

#1


6  

The problem here is that data.table doesn't play nice with the default gcc compiler (gcc72-c++.x86_64 gets installed as a dependency of R-devel.x86_64). Point R to an older version by adding

这里的问题是data.table与默认的gcc编译器不兼容(gcc72-c ++。x86_64作为R-devel.x86_64的依赖项安装)。通过添加将R指向旧版本

CC=gcc64

in ~/.R/Makevars. If you start from a "clean" Amazon Linux AMI this file doesn't exist and you can just type

在〜/ .R / Makevars中。如果您从“干净的”Amazon Linux AMI开始,此文件不存在,您只需键入即可

mkdir ~/.R
echo "CC=gcc64" >> ~/.R/Makevars

#2


3  

For some reason setting CC=gcc64 in the ~/.R/Makevars did not work for me. R was still using default gcc to compile.

出于某种原因,在〜/ .R / Makevars中设置CC = gcc64对我来说不起作用。 R仍然使用默认的gcc进行编译。

However there is another option. You can edit the Makeconf file that R uses during compilation directly. If you're using Amazon Linux the file location is /usr/lib64/R/etc/Makeconf. Once you locate the file the trick is just the same, that is to change the CC = gcc to CC = gcc64. You also might want to make sure that gcc64 is installed by running sudo yum install gcc64.

但是还有另一种选择。您可以直接编辑R在编译期间使用的Makeconf文件。如果您使用的是Amazon Linux,则文件位置为/ usr / lib64 / R / etc / Makeconf。一旦找到文件,技巧就是一样,即将CC = gcc更改为CC = gcc64。您还可以通过运行sudo yum install gcc64来确保安装gcc64。

#3


0  

I have the following workaround.

我有以下解决方法。

1) Connect with ssh.

1)用ssh连接。

2) Download package source

2)下载包源

wget https://cran.r-project.org/src/contrib/data.table_1.10.4-3.tar.gz

3) Install package

3)安装包

R CMD INSTALL data.table_1.10.4-3.tar.gz

This will install the package in the local package directory, not necessarily the one of the RStudio user(s).

这将把软件包安装在本地软件包目录中,不一定是RStudio用户之一。

4) Use .libPaths() in R and RStudio to figure out where packages are stored.

4)在R和RStudio中使用.libPaths()来确定存储包的位置。

5) Copy package into the desired directory using sudo cp -R SOURCE DEST.

5)使用sudo cp -R SOURCE DEST将包复制到所需目录。

Remark: This is a workaround not a solution because you need to install each package manually this way, i.e. handling dependencies can be cumbersome, but at least data.table works now.

备注:这是一种解决方法而不是解决方案,因为您需要以这种方式手动安装每个包,即处理依赖关系可能很麻烦,但至少data.table现在可以工作。