Scheme / Common Lisp / Clojure中的自动微分库

I've heard that one of McCarthy's original motivations for inventing Lisp was to write a system for automatic differentiation. Despite this, my Google searches haven't yielded any libraries/macros for doing this. Are there any Scheme/Common Lisp/Clojure libraries (macros) out there for taking a function F and returning a function dF/dx that calculates the derivative of F?

我听说麦卡锡发明Lisp的最初动机之一是编写一个自动微分系统。尽管如此，我的谷歌搜索并没有产生任何库/宏来实现这一点。有没有什么方案/通用的Lisp/Clojure库(macros)来获取一个函数F并返回一个函数dF/dx来计算F的导数?

I would want it to support F's with multiple arguments. The user would choose which of these is the x to differentiate with respect to. Ideally, the differentiator would work even for vector-valued F's and x's.

我希望它支持多参数的F。用户可以选择其中的哪个是x来求导。理想情况下，微分器对向量值F和x都适用。

EDIT: Several people have mentioned symbolic differentiation. The difference between symbolic differentiation and automatic differentiation is a subtle one, but it's summarized well in Wikipedia, and particularly in this picture. This distinction isn't as strong in lisp, where symbolic expressions can be turned into working programs as-is, but there remains a potential difficulty:

编辑:有几个人提到了符号分化。符号区分和自动区分之间的区别是很微妙的，但是在*上有很好的总结，特别是在这幅图中。这种区别在lisp中不太明显，在lisp中符号表达式可以被转换成工作程序，但是仍然存在一个潜在的困难:

Symbolic differentiation requires the expression being differentiated to be composed of operations with known derivatives. For example, someone mentioned SICP's example of a macro that churns through simple sexps like (+ y (* (x y))), and uses the chain rule, along with knowledge of how to differentiate + and *, to return a sexp that represents the derivative. I would need that to work with expressions like (* (foo x y) (bar x)), where foo and bar may in turn call other functions whose derivatives aren't known at differentiation time.

符号区分要求被区分的表达式由已知导数的运算组成。例如，有人提到了SICP的一个宏示例，它通过简单的sexps (+ y (* (* (x y)))来产生，并使用链式法则，以及如何区分+和*的知识来返回一个表示导数的sexp。我需要它与像(* (foo x y) (bar x))这样的表达式一起工作，在这里，foo和bar可以调用其他函数，它们的导数在微分时不知道。

This would be fine if there's a way to take an expression like (foo x y) and replace it with its function body, substituting any mention of the arguments with x and y in a hygenic way. Is there?

如果有一种方法可以将一个表达式(foo x y)替换为它的函数体，以一种很原始的方式将所有的参数替换为x和y，这就很好了。是吗?

Also, none of the above addresses complications that come about when differentiating vector-valued functions with respect to vector-valued arguments... which is what most autodifferentiation implementations are geared for.

此外，上述任何一种方法都不能解决在区分向量值函数与向量值参数时产生的复杂问题……这是大多数自动区分实现的目标。

7 个解决方案

#1

There are two other packages, both for automatic differentiation in Scheme. The second is based on the first, but reworked as a chicken egg. These support both forward and reverse mode.

还有另外两个包，都用于自动区分方案。第二个是基于第一个，但重新设计为一个鸡蛋。它们同时支持正向和反向模式。

https://github.com/qobi/R6RS-AD
https://github.com/qobi/R6RS-AD
https://github.com/abarbu/AD
https://github.com/abarbu/AD

#2

Alexey Radul writes:

Alexey拉多写道:

Well, there's the automatic differentiation system in Scmutils

有一个自动分化系统

http://groups.csail.mit.edu/mac/users/gjs/6946/linux-install.htm

(which coincidentally also does symbolic differentiation). I don't know of any other released implementations, though you might check http://autodiff.org/ .

(巧合的是，它也有符号区分)。我不知道还有其他发布的实现，不过您可以查看http://autodiff.org/。

There's also a nice explanation of how to implement it yourself in an appendix of Structure and Interpretation of Classical Mechanics

在古典力学的结构和解释的附录中也有一个很好的解释

http://mitpress.mit.edu/sicm/

as well as in the academic literature. Particularly forward mode is not that hard, though you do have to be careful to avoid perturbation confusion. You might consult the publications of Barak Pearlmutter and Jeffrey Mark Siskind, who are collaborating on a high-performance Lisp variant that incorporates AD and have been publishing on surrounding issues.

以及学术文献。特别向前模式不是那么困难，尽管你必须小心避免扰动混乱。您可以参考巴拉克•皮尔穆特(Barak Pearlmutter)和杰弗里•马克•西斯金德(Jeffrey Mark Siskind)的著作，他们正在合作开发一种高性能的Lisp变体，该变体集成了AD，并一直在发布相关问题。

http://scholar.google.com/scholar?q=Barak+Pearlmutter+and+Jeffrey+Mark+Siskind

http://scholar.google.com/scholar?q=Barak +工作+和+ Jeffrey + + Siskind标志

#3

If you are looking for a symbolic system, you could try maxima (or here). It runs on a number of Common-Lisp/OS platform combinations, but is more of a complete system than a library.

如果您正在寻找一个符号系统，您可以尝试maxima(或者在这里)。它运行在许多公共lisp /OS平台组合上，但是它更像是一个完整的系统而不是一个库。

Console output is OK, but it can produce quite nice looking output when coupled with texmacs.

控制台输出是可以的，但是当与texmacs结合时，它可以产生非常好看的输出。

Maxima 5.23.2 http://maxima.sourceforge.net
using Lisp GNU Common Lisp (GCL) GCL 2.6.8 (a.k.a. GCL)
Distributed under the GNU Public License. See the file COPYING.
Dedicated to the memory of William Schelter.
The function bug_report() provides bug reporting information.
(%i1) diff(sin(1/x),x);
                                        1
                                     cos(-)
                                         x
(%o1)                              - ------
                                        2
                                       x

EDIT

编辑

OK, looks like I misunderstood the question. A bit of googling suggests that there are some tools for this in SCMUTILS here, download here, user manual here (see p24 onwards).

看来我误解了这个问题。谷歌的一些搜索结果显示，在SCMUTILS中有一些工具，这里下载，用户手册(见p24)。

#4

It may be of interest that scmutlis has now been ported to Clojure. There is a lot more work to be done, but the code in the fist chapters of the SICM book seems to be running fine.

scmutlis现在已经被移植到Clojure中，这可能会引起人们的兴趣。还有很多工作要做，但是在SICM书的第一章中的代码似乎运行得很好。

Differentiation routines and operators also seem OK with what little testing I've done, and it is even free of some bugs that appear to have crept into later versions of scmutils.

区分例程和操作符对我所做的小测试似乎也没有什么问题，甚至没有出现在scmutils的后续版本中。

I think that scmutils covers OP's requirements re differentiation, since it will correctly handle derivatives of both known and unknown (literal) functions. This page gives the details needed to see how well it fits requirements: SICM - Derivatives - Notation

我认为scmutils涵盖了OP的需求重新区分，因为它将正确地处理已知和未知(文字)函数的导数。这一页提供了详细的细节，以了解它如何适合需求:SICM -导数-表示法

One of the advantages of running on the JVM, is that will run as a standalone if so required, no need even to install Clojure!

在JVM上运行的优点之一是，如果需要的话，它将作为独立的运行，甚至不需要安装Clojure!

It is very close to the original Scheme, minimal concessions made for Clojure syntax.

它非常接近原始方案，对Clojure语法做出了最小的让步。

You can see it here: https://github.com/littleredcomputer/sicmutils#sicmutils

您可以在这里看到:https://github.com/littleredcomputer/sicmutils#sicmutils

===

= = =

Addendum: Here is an example of automatic differentiation in the SicmUtils Clojure package. This is a common example circulating on various Internet sites, the code to be differentiated is

附录:这里有一个镰状芽孢杆菌的自动分化的例子。这是一个在各个网站上流传的常见例子，要区分的代码是

    function f(x)
      y = x;
      for i=1...100
        y = sin(x+y);
      return y

After Clojurifying it a bit we have

在对它进行了一些封禁之后

   > (defn inner [y] (fn[x] (sin (+ x y))))
   > (defn f100 [x] (nth (iterate (inner x) x) 100))
   ;; value of derivative at 6
   > ((D f100) 6)
    => 0.51603111348625
   ;; value of the 4th derivative at 1
   > (((expt D 4) f100) 1)
    => -1.7853200839806143

#5

Here is an implementation of AD in common lisp.

这是在common lisp中实现的AD。

#6

Worth checking out Deriva, which does Automatic Differentiation for both Clojure and Java:

值得一看的衍生品，它可以自动区分Clojure和Java:

https://github.com/lambder/Deriva
https://github.com/lambder/Deriva

You may also be interested in expresso, which is more about numerical expression manipulation but still has some differentiation features and could probably be adapted to most AD use cases:

您可能还对expresso感兴趣，它更多的是关于数字表达式操作，但是仍然有一些不同的特性，并且可能适合大多数广告用例:

https://github.com/clojure-numerics/expresso
https://github.com/clojure-numerics/expresso

#7

-1

Google for 'lisp symbolic differentiation' and you will find plenty examples, e.g.

谷歌用于“lisp符号区分”，你会发现很多例子。

http://mitpress.mit.edu/sicp/full-text/sicp/book/node39.html

#1