I'm currently looking into Clojure and Incanter as an alternative to R. (Not that I dislike R, but it just interesting to try out new languages.) I like Incanter and find the syntax appealing, but vectorized operations are quite slow as compared e.g. to R or Python.
我目前正在研究Clojure和Incanter作为R的替代品。（不是我不喜欢R，但尝试使用新语言会很有趣。）我喜欢Incanter并且发现语法很吸引人，但是矢量化操作比较慢例如到R或Python。
As an example I wanted to get the first order difference of a vector using Incanter vector operations, Clojure map and R . Below is the code and timing for all versions. As you can see R is clearly faster.
作为一个例子，我想使用Incanter向量运算，Clojure map和R来获得向量的第一阶差分。以下是所有版本的代码和时间。如你所见，R显然更快。
Incanter and Clojure:
Incanter和Clojure：
(use '(incanter core stats))
(def x (doall (sample-normal 1e7)))
(time (def y (doall (minus (rest x) (butlast x)))))
"Elapsed time: 16481.337 msecs"
(time (def y (doall (map - (rest x) (butlast x)))))
"Elapsed time: 16457.850 msecs"
R:
R：
rdiff <- function(x){
n = length(x)
x[2:n] - x[1:(n-1)]}
x = rnorm(1e7)
system.time(rdiff(x))
user system elapsed
1.504 0.900 2.561
So I was wondering is there a way to speed up the vector operations in Incanter/Clojure? Also solutions involving the use of loops, Java arrays and/or libraries from Clojure are welcome.
所以我想知道有没有办法加速Incanter / Clojure中的矢量操作？还欢迎涉及使用来自Clojure的循环，Java数组和/或库的解决方案。
I have also posted this question to Incanter Google group with no responses so far.
我还向Incanter Google小组发布了此问题，目前尚无回复。
UPDATE: I have marked Jouni's answer as accepted, see below for my own answer where I have cleaned up his code a bit and added some benchmarks.
更新：我已将Jouni的答案标记为已接受，请参阅下面的我自己的答案，我已经清理了他的代码并添加了一些基准测试。
6 个解决方案
#1
13
Here's a Java arrays implementation that is on my system faster than your R code (YMMV). Note enabling the reflection warnings, which is essential when optimizing for performance, and the repeated type hint on y (the one on the def didn't seem to help for the aset) and casting everything to primitive double values (the dotimes makes sure that i is a primitive int).
这是我的系统上的Java数组实现比R代码（YMMV）更快。注意启用反射警告，这在优化性能时是必不可少的，并且y上的重复类型提示（def上的那个似乎对aset没有帮助）并将所有内容都转换为原始double值（dotimes确保我是一个原始的int）。
(set! *warn-on-reflection* true)
(use 'incanter.stats)
(def ^"[D" x (double-array (sample-normal 1e7)))
(time
(do
(def ^"[D" y (double-array (dec (count x))))
(dotimes [i (dec (count x))]
(aset ^"[D" y
i
(double (- (double (aget x (inc i)))
(double (aget x i))))))))
#2
20
My final solutions
After all the testing I found two slightly different ways to do the calculation with sufficient speed.
在所有测试之后，我发现两种略有不同的方法以足够的速度进行计算。
First I've used the function diff with different types of return values, below is the code returning a vector, but I have also timed a version returning a double-array (replace (vec y) with y) and Incanter.matrix (replace (vec y) with matrix y). This function is only based on java arrays. This is based on Jouni's code with some extra type hints removed.
首先我使用了不同类型返回值的函数diff，下面是返回向量的代码，但我还定时返回一个双数组的版本（用y替换（vec y））和Incanter.matrix（替换（vec y）矩阵y）。此函数仅基于java数组。这是基于Jouni的代码，删除了一些额外的类型提示。
Another approach is to do the calculations with Java arrays and store the values in a transient vector. As you see from the timings this is slightly faster than approach 1 if you wan't the function to return and array. This is implemented in function difft.
另一种方法是使用Java数组进行计算，并将值存储在瞬态向量中。从时间上看，如果你不想返回和数组的功能，这比接近1快一点。这是在函数difft中实现的。
So the choice really depends on what you wan't to do with the data. I guess a good option would be to overload the function so that it returns the same type that was used in the call. Actually passing a java array to diff instead of a vector makes ~1s faster.
所以选择真的取决于你不想对数据做什么。我想一个好的选择是重载函数，以便它返回与调用中使用的相同的类型。实际上将java数组传递给diff而不是向量会使~1s更快。
Timings for the different functions:
diff returning vector:
差异返回矢量：
(time (def y (diff x)))
"Elapsed time: 4733.259 msecs"
diff returning Incanter.matrix:
diff returns Incanter.matrix：
(time (def y (diff x)))
"Elapsed time: 2599.728 msecs"
diff returning double-array:
diff返回双数组：
(time (def y (diff x)))
"Elapsed time: 1638.548 msecs"
difft:
difft：
(time (def y (difft x)))
"Elapsed time: 3683.237 msecs"
The functions
(use 'incanter.stats)
(def x (vec (sample-normal 1e7)))
(defn diff [x]
(let [y (double-array (dec (count x)))
x (double-array x)]
(dotimes [i (dec (count x))]
(aset y i
(- (aget x (inc i))
(aget x i))))
(vec y)))
(defn difft [x]
(let [y (vector (range n))
y (transient y)
x (double-array x)]
(dotimes [i (dec (count x))]
(assoc! y i
(- (aget x (inc i))
(aget x i))))
(persistent! y)))
#3
3
Not specific to your example code, but since this turned into a discussion on Clojure performance, you might enjoy this link: Clojure is Fast
不是特定于您的示例代码，但由于这变成了对Clojure性能的讨论，您可能会喜欢这个链接：Clojure是快速的
#4
2
Bradford Cross's blog has a bunch of posts about this (he uses this stuff for the startup he works on link text. In general, using transients in inner loops, type hinting (via *warn-on-reflection*) etc are all good for speed increases. The Joy of Clojure has a great section on performance tuning, which you should read.
Bradford Cross的博客上有很多关于此的帖子（他将这些东西用于他在链接文本上工作的初创公司。一般来说，在内部循环中使用瞬态，类型提示（通过*警告反射*）等都有利于速度提升.Clojure的Joy有很多关于性能调整的内容，你应该阅读。
#5
1
Here's a solution with transients - appealing but slow.
这是一个带有瞬态的解决方案 - 吸引人而且速度慢。
(use 'incanter.stats)
(set! *warn-on-reflection* true)
(def x (doall (sample-normal 1e7)))
(time
(def y
(loop [xs x
xs+ (rest x)
result (transient [])]
(if (empty? xs+)
(persistent! result)
(recur (rest xs) (rest xs+)
(conj! result (- (double (first xs+))
(double (first xs)))))))))
#6
0
All the comments thus far are by people who don't seem to have much experience speeding up Clojure code. If you want Clojure code to perform identical to Java - the facilities are available to do so. It may make more sense however to defer to mature Java libraries like Colt or Parallel Colt for vector math. It may make sense to use Java arrays for the absolute highest performance iteration.
到目前为止，所有评论都是那些似乎没有太多经验加速Clojure代码的人。如果您希望Clojure代码与Java完全相同 - 可以使用这些工具。然而，按照Colt或Parallel Colt这样的成熟Java库进行矢量数学可能会更有意义。将Java数组用于绝对最高性能的迭代可能是有意义的。
@Shane's link is so full of outdated information to be hardly worth looking at. Also @Shane's comment that code is slower than by factor of 10 is simply inaccurate (and unsupported http://shootout.alioth.debian.org/u32q/compare.php?lang=clojure, and these benchmarks don't account for the kinds of optimization possible in 1.2.0 or 1.3.0-alpha1). With a little bit of work it's usually easy to get Clojure code w/in 4X-5X. Beyond that usually requires a deeper knowledge of Clojure's fast paths - something isn't widely disseminated as Clojure is a fairly young language.
@ Shane的链接充满了过时的信息，几乎不值得一看。另外@ Shane的评论说代码比10倍慢是完全不准确的（并且不支持http://shootout.alioth.debian.org/u32q/compare.php?lang=clojure，这些基准测试不能解释可以在1.2.0或1.3.0-alpha1中进行各种优化。通过一些工作，通常很容易在4X-5X中获得Clojure代码。除此之外，通常需要更深入地了解Clojure的快速路径 - 因为Clojure是一种相当年轻的语言，因此没有广泛传播。
Clojure is plenty fast. But learning how to make it fast is going to take a bit of work/research as Clojure discourages mutable operations and mutable datastructures.
Clojure很快。但是学习如何快速学习将需要一些工作/研究，因为Clojure不鼓励可变操作和可变数据结构。
#1
13
Here's a Java arrays implementation that is on my system faster than your R code (YMMV). Note enabling the reflection warnings, which is essential when optimizing for performance, and the repeated type hint on y (the one on the def didn't seem to help for the aset) and casting everything to primitive double values (the dotimes makes sure that i is a primitive int).
这是我的系统上的Java数组实现比R代码（YMMV）更快。注意启用反射警告，这在优化性能时是必不可少的，并且y上的重复类型提示（def上的那个似乎对aset没有帮助）并将所有内容都转换为原始double值（dotimes确保我是一个原始的int）。
(set! *warn-on-reflection* true)
(use 'incanter.stats)
(def ^"[D" x (double-array (sample-normal 1e7)))
(time
(do
(def ^"[D" y (double-array (dec (count x))))
(dotimes [i (dec (count x))]
(aset ^"[D" y
i
(double (- (double (aget x (inc i)))
(double (aget x i))))))))
#2
20
My final solutions
After all the testing I found two slightly different ways to do the calculation with sufficient speed.
在所有测试之后，我发现两种略有不同的方法以足够的速度进行计算。
First I've used the function diff with different types of return values, below is the code returning a vector, but I have also timed a version returning a double-array (replace (vec y) with y) and Incanter.matrix (replace (vec y) with matrix y). This function is only based on java arrays. This is based on Jouni's code with some extra type hints removed.
首先我使用了不同类型返回值的函数diff，下面是返回向量的代码，但我还定时返回一个双数组的版本（用y替换（vec y））和Incanter.matrix（替换（vec y）矩阵y）。此函数仅基于java数组。这是基于Jouni的代码，删除了一些额外的类型提示。
Another approach is to do the calculations with Java arrays and store the values in a transient vector. As you see from the timings this is slightly faster than approach 1 if you wan't the function to return and array. This is implemented in function difft.
另一种方法是使用Java数组进行计算，并将值存储在瞬态向量中。从时间上看，如果你不想返回和数组的功能，这比接近1快一点。这是在函数difft中实现的。
So the choice really depends on what you wan't to do with the data. I guess a good option would be to overload the function so that it returns the same type that was used in the call. Actually passing a java array to diff instead of a vector makes ~1s faster.
所以选择真的取决于你不想对数据做什么。我想一个好的选择是重载函数，以便它返回与调用中使用的相同的类型。实际上将java数组传递给diff而不是向量会使~1s更快。
Timings for the different functions:
diff returning vector:
差异返回矢量：
(time (def y (diff x)))
"Elapsed time: 4733.259 msecs"
diff returning Incanter.matrix:
diff returns Incanter.matrix：
(time (def y (diff x)))
"Elapsed time: 2599.728 msecs"
diff returning double-array:
diff返回双数组：
(time (def y (diff x)))
"Elapsed time: 1638.548 msecs"
difft:
difft：
(time (def y (difft x)))
"Elapsed time: 3683.237 msecs"
The functions
(use 'incanter.stats)
(def x (vec (sample-normal 1e7)))
(defn diff [x]
(let [y (double-array (dec (count x)))
x (double-array x)]
(dotimes [i (dec (count x))]
(aset y i
(- (aget x (inc i))
(aget x i))))
(vec y)))
(defn difft [x]
(let [y (vector (range n))
y (transient y)
x (double-array x)]
(dotimes [i (dec (count x))]
(assoc! y i
(- (aget x (inc i))
(aget x i))))
(persistent! y)))
#3
3
Not specific to your example code, but since this turned into a discussion on Clojure performance, you might enjoy this link: Clojure is Fast
不是特定于您的示例代码，但由于这变成了对Clojure性能的讨论，您可能会喜欢这个链接：Clojure是快速的
#4
2
Bradford Cross's blog has a bunch of posts about this (he uses this stuff for the startup he works on link text. In general, using transients in inner loops, type hinting (via *warn-on-reflection*) etc are all good for speed increases. The Joy of Clojure has a great section on performance tuning, which you should read.
Bradford Cross的博客上有很多关于此的帖子（他将这些东西用于他在链接文本上工作的初创公司。一般来说，在内部循环中使用瞬态，类型提示（通过*警告反射*）等都有利于速度提升.Clojure的Joy有很多关于性能调整的内容，你应该阅读。
#5
1
Here's a solution with transients - appealing but slow.
这是一个带有瞬态的解决方案 - 吸引人而且速度慢。
(use 'incanter.stats)
(set! *warn-on-reflection* true)
(def x (doall (sample-normal 1e7)))
(time
(def y
(loop [xs x
xs+ (rest x)
result (transient [])]
(if (empty? xs+)
(persistent! result)
(recur (rest xs) (rest xs+)
(conj! result (- (double (first xs+))
(double (first xs)))))))))
#6
0
All the comments thus far are by people who don't seem to have much experience speeding up Clojure code. If you want Clojure code to perform identical to Java - the facilities are available to do so. It may make more sense however to defer to mature Java libraries like Colt or Parallel Colt for vector math. It may make sense to use Java arrays for the absolute highest performance iteration.
到目前为止，所有评论都是那些似乎没有太多经验加速Clojure代码的人。如果您希望Clojure代码与Java完全相同 - 可以使用这些工具。然而，按照Colt或Parallel Colt这样的成熟Java库进行矢量数学可能会更有意义。将Java数组用于绝对最高性能的迭代可能是有意义的。
@Shane's link is so full of outdated information to be hardly worth looking at. Also @Shane's comment that code is slower than by factor of 10 is simply inaccurate (and unsupported http://shootout.alioth.debian.org/u32q/compare.php?lang=clojure, and these benchmarks don't account for the kinds of optimization possible in 1.2.0 or 1.3.0-alpha1). With a little bit of work it's usually easy to get Clojure code w/in 4X-5X. Beyond that usually requires a deeper knowledge of Clojure's fast paths - something isn't widely disseminated as Clojure is a fairly young language.
@ Shane的链接充满了过时的信息，几乎不值得一看。另外@ Shane的评论说代码比10倍慢是完全不准确的（并且不支持http://shootout.alioth.debian.org/u32q/compare.php?lang=clojure，这些基准测试不能解释可以在1.2.0或1.3.0-alpha1中进行各种优化。通过一些工作，通常很容易在4X-5X中获得Clojure代码。除此之外，通常需要更深入地了解Clojure的快速路径 - 因为Clojure是一种相当年轻的语言，因此没有广泛传播。
Clojure is plenty fast. But learning how to make it fast is going to take a bit of work/research as Clojure discourages mutable operations and mutable datastructures.
Clojure很快。但是学习如何快速学习将需要一些工作/研究，因为Clojure不鼓励可变操作和可变数据结构。