I am trying to teach myself how to use the Snowfall package, and I have run into the following problem when I try a function that calls a second function (this is a simplified use case of what I ultimately want to implement).
我试图自学如何使用Snowfall包,当我尝试一个调用第二个函数的函数时遇到了以下问题(这是我最终想要实现的一个简化的用例)。
I currently have:
我现在:
library (snowfall)
f1 <- function(n) { return (n-1) }
f2 <- function(n) { return (f1(n)^2) }
# initialize cluster
sfInit (parallel=TRUE , cpus=4)
# parallel computing
result <- sfLapply(1:10, f2)
# stop cluster
sfStop ()
but I receive the error message:
但是我收到了错误信息:
Error in checkForRemoteErrors(val) :
4 nodes produced errors; first error: could not find function "f1"
However, if I then run lapply(1:10, f2) I receive the following output:
但是,如果我运行lapply(1:10, f2),我将收到以下输出:
lapply(1:10, f2)
[[1]]
[1] 0
[[2]]
[1] 1
[[3]]
[1] 4
[[4]]
[1] 9
[[5]]
[1] 16
[[6]]
[1] 25
[[7]]
[1] 36
[[8]]
[1] 49
[[9]]
[1] 64
[[10]]
[1] 81
I ultimately want to use snowfall to implement a parallelized search procedures for multidimensional minimization problems, so will definitely need to be able to call functions from the main parallelized function.
我最终想使用snowfall来实现多维最小化问题的并行搜索过程,因此肯定需要能够从主并行函数调用函数。
Can anyone help with this?
有人能帮忙吗?
2 个解决方案
#1
4
You need to export the f1
function to the workers using the sfExport
function between sfInit
and sfLapply
:
您需要使用sfInit和sfLapply之间的sfExport函数将f1函数导出给工人:
sfExport('f1')
This is the snowfall
equivalent to the snow
clusterExport
function.
这相当于降雪团出口功能。
To export multiple variables, you can either use multiple arguments or the list
argument:
要导出多个变量,可以使用多个参数或列表参数:
sfExport('f1', 'x')
sfExport(list=c('f1', 'x'))
To export all variables in your global environment, use sfExportAll
:
要导出全局环境中的所有变量,请使用sfExportAll:
sfExportAll()
#2
0
I do not know the specifics of the snowfall package, but I do know that when doing parallel computing in R, you need to assume that each core you pass information to is a brand-new R instance. Meaning, regardless of what you pass through your host prior to registering clusters, there is just the vanilla instance of R on your clones.
我不知道snowfall包的具体细节,但我知道在R中进行并行计算时,您需要假设传递信息的每个核都是一个全新的R实例。也就是说,无论您在注册集群之前通过主机传递什么,克隆上都只有R的实例。
It looks like there is a call sfExportAll() which will put all of your global variables onto the clone instances. I would try it out, but I can't run snowfall on my Windows machine.
看起来有一个调用sfExportAll(),它将把所有全局变量放在克隆实例上。我想试一试,但是我不能在我的Windows机器上运行降雪。
#1
4
You need to export the f1
function to the workers using the sfExport
function between sfInit
and sfLapply
:
您需要使用sfInit和sfLapply之间的sfExport函数将f1函数导出给工人:
sfExport('f1')
This is the snowfall
equivalent to the snow
clusterExport
function.
这相当于降雪团出口功能。
To export multiple variables, you can either use multiple arguments or the list
argument:
要导出多个变量,可以使用多个参数或列表参数:
sfExport('f1', 'x')
sfExport(list=c('f1', 'x'))
To export all variables in your global environment, use sfExportAll
:
要导出全局环境中的所有变量,请使用sfExportAll:
sfExportAll()
#2
0
I do not know the specifics of the snowfall package, but I do know that when doing parallel computing in R, you need to assume that each core you pass information to is a brand-new R instance. Meaning, regardless of what you pass through your host prior to registering clusters, there is just the vanilla instance of R on your clones.
我不知道snowfall包的具体细节,但我知道在R中进行并行计算时,您需要假设传递信息的每个核都是一个全新的R实例。也就是说,无论您在注册集群之前通过主机传递什么,克隆上都只有R的实例。
It looks like there is a call sfExportAll() which will put all of your global variables onto the clone instances. I would try it out, but I can't run snowfall on my Windows machine.
看起来有一个调用sfExportAll(),它将把所有全局变量放在克隆实例上。我想试一试,但是我不能在我的Windows机器上运行降雪。