如何配置包含SNOW Cluster的R代码

时间:2021-03-31 13:52:07

I have a nested loop that I'm using foreach, DoSNOW, and a SNOW socket cluster to solve for. How should I go about profiling the code to make sure I'm not doing something grossly inefficient.

我有一个嵌套循环,我正在使用foreach,DoSNOW和SNOW套接字集群来解决。我应该如何分析代码以确保我没有做一些非常低效的事情。

Also is there anyway to measure the data flows going between the master and nodes in a Snow cluster?

无论如何还要测量Snow集群中主节点和节点之间的数据流量吗?

Thanks,

谢谢,

James

詹姆士

1 个解决方案

#1


2  

That is an excellent question. From the top of my head, start with a comparison between

这是一个很好的问题。从头顶开始,比较一下

  • a serial solution (no snow),
  • 一系列解决方案(没有雪),
  • a serial solution with snow (to get an idea of overhead) and
  • 雪的连续解决方案(了解开销)和
  • a parallel solution maybe controlling N to see what type of increase you get.
  • 并行解决方案可能控制N以查看您获得的增加类型。

The never-released-on-CRAN version 0.3.4 of snow also has additional plotting commands that are useful for analysis. You can get it from this directory at Luke Tierney's site.

从未发布的CRAN 0.3.4版本的雪也有额外的绘图命令,可用于分析。你可以从Luke Tierney网站上的这个目录中获取它。

Real profiling, of course, is hard given the distributed nature.

当然,鉴于分布式特性,实际剖析很难。

#1


2  

That is an excellent question. From the top of my head, start with a comparison between

这是一个很好的问题。从头顶开始,比较一下

  • a serial solution (no snow),
  • 一系列解决方案(没有雪),
  • a serial solution with snow (to get an idea of overhead) and
  • 雪的连续解决方案(了解开销)和
  • a parallel solution maybe controlling N to see what type of increase you get.
  • 并行解决方案可能控制N以查看您获得的增加类型。

The never-released-on-CRAN version 0.3.4 of snow also has additional plotting commands that are useful for analysis. You can get it from this directory at Luke Tierney's site.

从未发布的CRAN 0.3.4版本的雪也有额外的绘图命令,可用于分析。你可以从Luke Tierney网站上的这个目录中获取它。

Real profiling, of course, is hard given the distributed nature.

当然,鉴于分布式特性,实际剖析很难。