可以在典型项目中安全地使用-ffast-math吗?

时间:2021-04-16 09:10:48

While answering a question where I suggested -ffast-math, a comment pointed out that it is dangerous.

在回答我建议的问题时,一个评论指出它是危险的。

My personal feeling is that outside scientific calculations, it is OK. I also asume that serious financial applications use fixed point instead of floating point.

我个人的感觉是,在科学计算之外,没关系。我还认为严肃的财务应用程序使用固定点而不是浮点数。

Of course if you want to use it in your project the ultimate answer is to test it on your project and see how much it affects it. But I think a general answer can be given by people who tried and have experience with such optimizations:

当然,如果你想在你的项目中使用它,最终的答案是在你的项目上测试它,看看它对它有多大影响。但我认为,尝试并具有这种优化经验的人可以给出一般答案:

Can ffast-math be used safely on a normal project?

ffast-math可以在正常项目中安全使用吗?

Given that IEEE 754 floating point has rounding errors, the assumption is that you are already living with inexact calculations.

鉴于IEEE 754浮点具有舍入误差,假设您已经生活在不精确的计算中。


This answer was particular illuminating on the fact that -ffast-math does much more than reordering operations that would result in a slightly different result (does not check for NaN or zero, disables signed zero just to name a few), but I fail to see what the effects of these would ultimately be in a real code.

这个答案特别启发了这样一个事实:-ffast-math比重新排序操作更能导致结果略有不同(不检查NaN或零,禁用签名零只是为了说明一些),但我没有看看这些最终会在真实代码中产生什么影响。


I tried to think of typical uses of floating points, and this is what I came up with:

我试着想一下浮点的典型用法,这就是我提出的:

  • GUI (2D, 3D, physics engine, animations)
  • GUI(2D,3D,物理引擎,动画)
  • automation (e.g. car electronics)
  • 自动化(例如汽车电子)
  • robotics
  • 机器人
  • industrial measurements (e.g. voltage)
  • 工业测量(例如电压)

and school projects, but those don't really matter here.

和学校项目,但这些并不重要。

6 个解决方案

#1


9  

One of the especially dangerous things it does is imply -ffinite-math-only, which allows explicit NaN tests to pretend that no NaNs ever exist. That's bad news for any code that explicitly handles NaNs. It would try to test for NaN, but the test will lie through its teeth and claim that nothing is ever NaN, even when it is.

它所做的一件特别危险的事情就是暗示-ffinite-math-only,它允许明确的NaN测试假装没有NaN存在。对于任何明确处理NaN的代码来说,这都是坏消息。它会尝试测试NaN,但测试将通过它的牙齿,并声称没有任何东西NaN,即使它是。

This can have really obvious results, such as letting NaN bubble up to the user when previously they would have been filtered out at some point. That's bad of course, but probably you'll notice and fix it.

这可以产生非常明显的结果,例如让NaN冒泡到用户以前在某个时候过滤掉它们。这当然很糟糕,但可能你会注意到并修复它。

A more insidious problem arises when NaN checks were there for error checking, for something that really isn't supposed to ever be NaN. But perhaps through some bug, bad data, or through other effects of -ffast-math, it becomes NaN anyway. And now you're not checking for it, because by assumption nothing is ever NaN, so isnan is a synonym of false. Things will go wrong, spuriously and long after you've already shipped your software, and you will get an "impossible" error report - you did check for NaN, it's right there in the code, it cannot be failing! But it is, because someone someday added -ffast-math to the flags, maybe you even did it yourself, not knowing fully what it would do or having forgotten that you used a NaN check.

当NaN检查用于错误检查时,出现了一个更加隐蔽的问题,因为那些确实不应该是NaN的东西。但也许通过一些错误,糟糕的数据,或通过-ffast-math的其他效果,无论如何它都变成了NaN。现在你没有检查它,因为假设没有任何东西是NaN,所以isnan是false的同义词。在您已经发送软件后,事情会出现错误,虚假和很长时间,并且您将收到“不可能”的错误报告 - 您确实检查了NaN,它就在代码中,它不会失败!但事实是,因为某人有一天会在旗帜上添加-ffast-math,也许你自己也做了,不知道它会做什么或者忘记你使用过NaN检查。

So then we might ask, is that normal? That's getting quite subjective, but I would not say that checking for NaN is especially abnormal. Going fully circular and asserting that it isn't normal because -ffast-math breaks it is probably a bad idea.

那么我们可能会问,这是正常的吗?这是非常主观的,但我不会说检查NaN特别不正常。完全循环并断言它不正常,因为-ffast-math打破它可能是一个坏主意。

It does a lot of other scary things as well, as detailed in other answers.

它还做了许多其他可怕的事情,详见其他答案。

#2


8  

I wouldn't recommend to avoid using this option, but I remind one instance where unexpected floating-point behavior struck back.

我不建议避免使用此选项,但我提醒一个实例意外的浮点行为回击。

The code was saying like this innocent construct:

代码就像这个无辜的构造一样:

float X, XMin, Y;
if (X < XMin)
{
    Y= 1 / (XMin - X);
    XMin= X;
}

This was sometimes raising a division by zero error, because when the comparison was carried out, the full 80 bits representation (Intel FPU) was used, while later when the subtraction was performed, values were truncated to the 32 bits representation, possibly being equal.

这有时会增加除零误差,因为在进行比较时,使用了完整的80位表示(Intel FPU),而后来执行减法时,值被截断为32位表示,可能相等。

#3


7  

Yes, you can use -ffast-math on normal projects, for an appropriate definition of "normal projects." That includes probably 95% of all programs written.

是的,您可以在正常项目中使用-ffast-math,以获得“正常项目”的适当定义。这可能包括95%的所有程序。

But then again, 95% of all programs written would not benefit much from -ffast-math either, because they don't do enough floating point math for it to be important.

但话说回来,95%的所有程序编写都不会从-ffast-math中获益,因为它们没有做足够的浮点数学,因为它很重要。

#4


5  

Given that IEEE 754 floating point has rounding errors, the assumption is that you are already living with inexact calculations.

鉴于IEEE 754浮点具有舍入误差,假设您已经生活在不精确的计算中。

The question you should answer is not whether the program expects inexact computations (it had better expect them, or it will break with or without -ffast-math), but whether the program expects approximations to be exactly those predicted by IEEE 754, and special values that behave exactly as predicted by IEEE 754 as well; or whether the program is designed to work fine with the weaker hypothesis that each operation introduces a small unpredictable relative error.

你应该回答的问题不是程序是否需要不精确的计算(它更好地期望它们,或者它会在有或没有-ffast-math的情况下中断),但是程序是否期望近似值与IEEE 754预测的那些相近,并且特殊这些值的行为与IEEE 754的预测完全相同;或者该程序是否设计为与较弱的假设一起工作,即每个操作引入一个小的不可预测的相对误差。

Many algorithms do not make use of special values (infinities, NaN) and are designed to work well in a computation model in which each operation introduces a small nondeterministic relative error. These algorithms work well with -ffast-math, because they do not use the hypothesis that the error of each operation is exactly the error predicted by IEEE 754. The algorithms also work fine when the rounding mode is other than the default round-to-nearest: the error in the end may be larger (or smaller), but a FPU in round-upwards mode also implements the computation model that these algorithms expect, so they work more or less identically well in these conditions.

许多算法不使用特殊值(无穷大,NaN),并且被设计为在计算模型中很好地工作,其中每个操作引入小的非确定性相对误差。这些算法适用于-ffast-math,因为它们不使用每个操作的错误正好是IEEE 754预测的错误的假设。当舍入模式不是默认的舍入模式时,算法也可以正常工作。最近:最后的误差可能更大(或更小),但是向上舍入模式的FPU也实现了这些算法所期望的计算模型,因此它们在这些条件下或多或少地相同地工作。

Other algorithms (for instance Kahan summation, “double-double” libraries in which numbers are represented as the sum of two doubles) expect the rules to be respected to the letter, because they contain smart shortcuts based on subtle behaviors of IEEE 754 arithmetic. You can recognize these algorithms by the fact that they do not work when the rounding mode is other than expected either. I once asked a question about designing double-double operations that would work in all rounding modes (for library functions that may be pre-empted without a chance to restore the rounding mode): it is extra work, and these adapted implementations still do not work with -ffast-math.

其他算法(例如Kahan求和,“双倍”库,其中数字表示为两个双精度的总和)期望规则被尊重为字母,因为它们包含基于IEEE 754算法的微妙行为的智能快捷方式。您可以通过以下事实识别这些算法:当舍入模式不是预期时,它们不起作用。我曾经问过一个关于设计双重操作的问题,这些操作可以在所有舍入模式下工作(对于可能会被抢占而没有机会恢复舍入模式的库函数):这是额外的工作,而这些改编的实现仍然没有使用-ffast-math。

#5


5  

The short answer: No, you cannot safely use -ffast-math except on code designed to be used with it. There are all sorts of important constructs for which it generates completely wrong results. In particular, for arbitrarily large x, there are expressions with correct value x but which will evaluate to 0 with -ffast-math, or vice versa.

简短的回答:不,除了设计用于它的代码之外,你不能安全地使用-ffast-math。有各种各样的重要结构,它们会产生完全错误的结果。特别是,对于任意大的x,存在具有正确值x的表达式,但是使用-ffast-math将其评估为0,反之亦然。

As a more relaxed rule, if you're certain the code you're compiling was written by someone who doesn't actually understand floating point math, using -ffast-math probably wrong make the results any more wrong (vs. the programmer's intent) than they already were. Such a programmer will not be performing intentional rounding or other operations that badly break, probably won't be using nans and infinities, etc. The most likely negative consequence is having computations that already had precision problems blow up and get worse. I would argue that this kind of code is already bad enough that you should not be using it in production to begin with, with or without -ffast-math.

作为一个更宽松的规则,如果你确定你正在编译的代码是由一个实际上不了解浮点数学的人编写的,那么使用-ffast-math可能会导致结果错误(与程序员的意图相比) )比他们已经。这样的程序员不会进行故意舍入或其他严重破坏的操作,可能不会使用nans和infinities等。最可能的负面后果是计算已经有精确问题爆炸并变得更糟。我认为这种代码已经足够糟糕,你不应该在生产中使用它,无论有没有-ffast-math。

From personal experience, I've had enough spurious bug reports from users trying to use -ffast-math (or even who have it buried in their default CFLAGS, uhg!) that I'm strongly leaning towards putting the following fragment in any code with floating point math:

根据个人经验,我有足够的虚假错误报告来自尝试使用-ffast-math的用户(甚至是将其隐藏在默认CFLAGS中的人,呃!)我强烈倾向于在任何代码中放入以下片段浮点数学:

#ifdef __FAST_MATH__
#error "-ffast-math is broken, don't use it"
#endif

If you still want to use -ffast-math in production, you need to actually spend the effort (lots of code review hours) to determine if it's safe. Before doing that, you probably want to first measure whether there's any benefit that would be worth spending those hours, and the answer is likely no.

如果你仍然想在生产中使用-ffast-math,你需要花费精力(大量的代码审查时间)来确定它是否安全。在此之前,您可能想首先衡量是否有值得花费这些时间的任何好处,答案可能是否定的。

#6


-5  

Yes, they can be used safely, provided that you know what you are doing. This implies that you understand that they represent magnitudes, not exact values. This means:

是的,只要您知道自己在做什么,它们就可以安全使用。这意味着您了解它们代表的是数量级,而不是精确值。意即:

  1. You always do a sanity check on any external fp input.
  2. 您总是对任何外部fp输入进行健全性检查。
  3. You never divide by 0.
  4. 你永远不会除以0。
  5. You never check for equality, unless you know it is an integer with an absolute value below the max value of the mantissa.
  6. 你永远不会检查是否相等,除非你知道它是一个绝对值低于尾数最大值的整数。
  7. etc.
  8. 等等

In fact, I would argue the converse. Unless you are working in very specific applications where NaNs and denormals have meaning, or if you really need that tiny incremental bit of reproduceability, then -ffast-math should be on by default. That way, your unit tests have a better chance of flushing out errors. Basically, whenever you think fp calculations have either reproduceability or precision, even under ieee, you are wrong.

事实上,我认为反过来。除非你在非常特殊的应用程序中工作,其中NaNs和非正规数有意义,或者如果你真的需要那么微小的可再现性,那么-ffast-math应该默认启用。这样,您的单元测试有更好的机会来清除错误。基本上,每当你认为fp计算具有可重现性或精确度时,即使在ieee下,你也错了。

#1


9  

One of the especially dangerous things it does is imply -ffinite-math-only, which allows explicit NaN tests to pretend that no NaNs ever exist. That's bad news for any code that explicitly handles NaNs. It would try to test for NaN, but the test will lie through its teeth and claim that nothing is ever NaN, even when it is.

它所做的一件特别危险的事情就是暗示-ffinite-math-only,它允许明确的NaN测试假装没有NaN存在。对于任何明确处理NaN的代码来说,这都是坏消息。它会尝试测试NaN,但测试将通过它的牙齿,并声称没有任何东西NaN,即使它是。

This can have really obvious results, such as letting NaN bubble up to the user when previously they would have been filtered out at some point. That's bad of course, but probably you'll notice and fix it.

这可以产生非常明显的结果,例如让NaN冒泡到用户以前在某个时候过滤掉它们。这当然很糟糕,但可能你会注意到并修复它。

A more insidious problem arises when NaN checks were there for error checking, for something that really isn't supposed to ever be NaN. But perhaps through some bug, bad data, or through other effects of -ffast-math, it becomes NaN anyway. And now you're not checking for it, because by assumption nothing is ever NaN, so isnan is a synonym of false. Things will go wrong, spuriously and long after you've already shipped your software, and you will get an "impossible" error report - you did check for NaN, it's right there in the code, it cannot be failing! But it is, because someone someday added -ffast-math to the flags, maybe you even did it yourself, not knowing fully what it would do or having forgotten that you used a NaN check.

当NaN检查用于错误检查时,出现了一个更加隐蔽的问题,因为那些确实不应该是NaN的东西。但也许通过一些错误,糟糕的数据,或通过-ffast-math的其他效果,无论如何它都变成了NaN。现在你没有检查它,因为假设没有任何东西是NaN,所以isnan是false的同义词。在您已经发送软件后,事情会出现错误,虚假和很长时间,并且您将收到“不可能”的错误报告 - 您确实检查了NaN,它就在代码中,它不会失败!但事实是,因为某人有一天会在旗帜上添加-ffast-math,也许你自己也做了,不知道它会做什么或者忘记你使用过NaN检查。

So then we might ask, is that normal? That's getting quite subjective, but I would not say that checking for NaN is especially abnormal. Going fully circular and asserting that it isn't normal because -ffast-math breaks it is probably a bad idea.

那么我们可能会问,这是正常的吗?这是非常主观的,但我不会说检查NaN特别不正常。完全循环并断言它不正常,因为-ffast-math打破它可能是一个坏主意。

It does a lot of other scary things as well, as detailed in other answers.

它还做了许多其他可怕的事情,详见其他答案。

#2


8  

I wouldn't recommend to avoid using this option, but I remind one instance where unexpected floating-point behavior struck back.

我不建议避免使用此选项,但我提醒一个实例意外的浮点行为回击。

The code was saying like this innocent construct:

代码就像这个无辜的构造一样:

float X, XMin, Y;
if (X < XMin)
{
    Y= 1 / (XMin - X);
    XMin= X;
}

This was sometimes raising a division by zero error, because when the comparison was carried out, the full 80 bits representation (Intel FPU) was used, while later when the subtraction was performed, values were truncated to the 32 bits representation, possibly being equal.

这有时会增加除零误差,因为在进行比较时,使用了完整的80位表示(Intel FPU),而后来执行减法时,值被截断为32位表示,可能相等。

#3


7  

Yes, you can use -ffast-math on normal projects, for an appropriate definition of "normal projects." That includes probably 95% of all programs written.

是的,您可以在正常项目中使用-ffast-math,以获得“正常项目”的适当定义。这可能包括95%的所有程序。

But then again, 95% of all programs written would not benefit much from -ffast-math either, because they don't do enough floating point math for it to be important.

但话说回来,95%的所有程序编写都不会从-ffast-math中获益,因为它们没有做足够的浮点数学,因为它很重要。

#4


5  

Given that IEEE 754 floating point has rounding errors, the assumption is that you are already living with inexact calculations.

鉴于IEEE 754浮点具有舍入误差,假设您已经生活在不精确的计算中。

The question you should answer is not whether the program expects inexact computations (it had better expect them, or it will break with or without -ffast-math), but whether the program expects approximations to be exactly those predicted by IEEE 754, and special values that behave exactly as predicted by IEEE 754 as well; or whether the program is designed to work fine with the weaker hypothesis that each operation introduces a small unpredictable relative error.

你应该回答的问题不是程序是否需要不精确的计算(它更好地期望它们,或者它会在有或没有-ffast-math的情况下中断),但是程序是否期望近似值与IEEE 754预测的那些相近,并且特殊这些值的行为与IEEE 754的预测完全相同;或者该程序是否设计为与较弱的假设一起工作,即每个操作引入一个小的不可预测的相对误差。

Many algorithms do not make use of special values (infinities, NaN) and are designed to work well in a computation model in which each operation introduces a small nondeterministic relative error. These algorithms work well with -ffast-math, because they do not use the hypothesis that the error of each operation is exactly the error predicted by IEEE 754. The algorithms also work fine when the rounding mode is other than the default round-to-nearest: the error in the end may be larger (or smaller), but a FPU in round-upwards mode also implements the computation model that these algorithms expect, so they work more or less identically well in these conditions.

许多算法不使用特殊值(无穷大,NaN),并且被设计为在计算模型中很好地工作,其中每个操作引入小的非确定性相对误差。这些算法适用于-ffast-math,因为它们不使用每个操作的错误正好是IEEE 754预测的错误的假设。当舍入模式不是默认的舍入模式时,算法也可以正常工作。最近:最后的误差可能更大(或更小),但是向上舍入模式的FPU也实现了这些算法所期望的计算模型,因此它们在这些条件下或多或少地相同地工作。

Other algorithms (for instance Kahan summation, “double-double” libraries in which numbers are represented as the sum of two doubles) expect the rules to be respected to the letter, because they contain smart shortcuts based on subtle behaviors of IEEE 754 arithmetic. You can recognize these algorithms by the fact that they do not work when the rounding mode is other than expected either. I once asked a question about designing double-double operations that would work in all rounding modes (for library functions that may be pre-empted without a chance to restore the rounding mode): it is extra work, and these adapted implementations still do not work with -ffast-math.

其他算法(例如Kahan求和,“双倍”库,其中数字表示为两个双精度的总和)期望规则被尊重为字母,因为它们包含基于IEEE 754算法的微妙行为的智能快捷方式。您可以通过以下事实识别这些算法:当舍入模式不是预期时,它们不起作用。我曾经问过一个关于设计双重操作的问题,这些操作可以在所有舍入模式下工作(对于可能会被抢占而没有机会恢复舍入模式的库函数):这是额外的工作,而这些改编的实现仍然没有使用-ffast-math。

#5


5  

The short answer: No, you cannot safely use -ffast-math except on code designed to be used with it. There are all sorts of important constructs for which it generates completely wrong results. In particular, for arbitrarily large x, there are expressions with correct value x but which will evaluate to 0 with -ffast-math, or vice versa.

简短的回答:不,除了设计用于它的代码之外,你不能安全地使用-ffast-math。有各种各样的重要结构,它们会产生完全错误的结果。特别是,对于任意大的x,存在具有正确值x的表达式,但是使用-ffast-math将其评估为0,反之亦然。

As a more relaxed rule, if you're certain the code you're compiling was written by someone who doesn't actually understand floating point math, using -ffast-math probably wrong make the results any more wrong (vs. the programmer's intent) than they already were. Such a programmer will not be performing intentional rounding or other operations that badly break, probably won't be using nans and infinities, etc. The most likely negative consequence is having computations that already had precision problems blow up and get worse. I would argue that this kind of code is already bad enough that you should not be using it in production to begin with, with or without -ffast-math.

作为一个更宽松的规则,如果你确定你正在编译的代码是由一个实际上不了解浮点数学的人编写的,那么使用-ffast-math可能会导致结果错误(与程序员的意图相比) )比他们已经。这样的程序员不会进行故意舍入或其他严重破坏的操作,可能不会使用nans和infinities等。最可能的负面后果是计算已经有精确问题爆炸并变得更糟。我认为这种代码已经足够糟糕,你不应该在生产中使用它,无论有没有-ffast-math。

From personal experience, I've had enough spurious bug reports from users trying to use -ffast-math (or even who have it buried in their default CFLAGS, uhg!) that I'm strongly leaning towards putting the following fragment in any code with floating point math:

根据个人经验,我有足够的虚假错误报告来自尝试使用-ffast-math的用户(甚至是将其隐藏在默认CFLAGS中的人,呃!)我强烈倾向于在任何代码中放入以下片段浮点数学:

#ifdef __FAST_MATH__
#error "-ffast-math is broken, don't use it"
#endif

If you still want to use -ffast-math in production, you need to actually spend the effort (lots of code review hours) to determine if it's safe. Before doing that, you probably want to first measure whether there's any benefit that would be worth spending those hours, and the answer is likely no.

如果你仍然想在生产中使用-ffast-math,你需要花费精力(大量的代码审查时间)来确定它是否安全。在此之前,您可能想首先衡量是否有值得花费这些时间的任何好处,答案可能是否定的。

#6


-5  

Yes, they can be used safely, provided that you know what you are doing. This implies that you understand that they represent magnitudes, not exact values. This means:

是的,只要您知道自己在做什么,它们就可以安全使用。这意味着您了解它们代表的是数量级,而不是精确值。意即:

  1. You always do a sanity check on any external fp input.
  2. 您总是对任何外部fp输入进行健全性检查。
  3. You never divide by 0.
  4. 你永远不会除以0。
  5. You never check for equality, unless you know it is an integer with an absolute value below the max value of the mantissa.
  6. 你永远不会检查是否相等,除非你知道它是一个绝对值低于尾数最大值的整数。
  7. etc.
  8. 等等

In fact, I would argue the converse. Unless you are working in very specific applications where NaNs and denormals have meaning, or if you really need that tiny incremental bit of reproduceability, then -ffast-math should be on by default. That way, your unit tests have a better chance of flushing out errors. Basically, whenever you think fp calculations have either reproduceability or precision, even under ieee, you are wrong.

事实上,我认为反过来。除非你在非常特殊的应用程序中工作,其中NaNs和非正规数有意义,或者如果你真的需要那么微小的可再现性,那么-ffast-math应该默认启用。这样,您的单元测试有更好的机会来清除错误。基本上,每当你认为fp计算具有可重现性或精确度时,即使在ieee下,你也错了。