SetFPCompareAccuracy: Difference between revisions
From MultiCharts
(Created page with "Sets floating point compare accuracy by specifying the tolerance value to be used when floating point values are compared. Two floating point values will be considered equal...") |
|||
| (4 intermediate revisions by one other user not shown) | |||
| Line 13: | Line 13: | ||
<code>Accuracy</code> - an FPC constant or a numerical expression specifying a tolerance value as follows: | <code>Accuracy</code> - an FPC constant or a numerical expression specifying a tolerance value as follows: | ||
{| | {| class="wikitable" | ||
<code>fpcVeryLowAccuracy</code>||0|| | |<code>fpcVeryLowAccuracy</code>|| 0 ||2.2204460492503131e-8 | ||
|- | |- | ||
<code>fpcLowAccuracy</code>||1|| | |<code>fpcLowAccuracy</code>|| 1 ||2.2204460492503131e-10 | ||
|- | |- | ||
<code>fpcMedAccuracy</code>||2|| | |<code>fpcMedAccuracy</code>|| 2 ||2.2204460492503131e-12 (Default) | ||
|- | |- | ||
<code>fpcHighAccuracy</code>||3|| | |<code>fpcHighAccuracy</code>|| 3 ||2.2204460492503131e-14 | ||
|- | |- | ||
<code>fpcVeryHighAccuracy</code>||4|| | |<code>fpcVeryHighAccuracy</code>|| 4 ||2.2204460492503131e-16 | ||
|- | |- | ||
<code>fpcExact</code>||5|| | |<code>fpcExact</code>|| 5 ||0.00 | ||
|} | |} | ||
== Example == | == Example == | ||
Latest revision as of 16:00, 19 April 2021
Sets floating point compare accuracy by specifying the tolerance value to be used when floating point values are compared.
Two floating point values will be considered equal if "abs(Value1 – Value2) <= ε", where ε is the tolerance value.
By default, the tolerance value is 2.2204460492503131e-012.
Usage
SetFPCompareAccuracy(Accuracy)Parameters
Accuracy - an FPC constant or a numerical expression specifying a tolerance value as follows:
fpcVeryLowAccuracy |
0 | 2.2204460492503131e-8 |
fpcLowAccuracy |
1 | 2.2204460492503131e-10 |
fpcMedAccuracy |
2 | 2.2204460492503131e-12 (Default) |
fpcHighAccuracy |
3 | 2.2204460492503131e-14 |
fpcVeryHighAccuracy |
4 | 2.2204460492503131e-16 |
fpcExact |
5 | 0.00 |
Example
Set floating point compare accuracy to High:
SetFPCompareAccuracy(3);Set floating point compare accuracy to High:
SetFPCompareAccuracy(fpcHighAccuracy);