Difference between revisions of "SetFPCompareAccuracy"

From MultiCharts
Jump to navigation Jump to search
 
(3 intermediate revisions by one other user not shown)
Line 14: Line 14:
  
 
{| class="wikitable"
 
{| class="wikitable"
|<code>fpcVeryLowAccuracy</code>||0||02.2204460492503131e-8  
+
|<code>fpcVeryLowAccuracy</code>||&nbsp;0&nbsp;||2.2204460492503131e-8  
 
|-
 
|-
|<code>fpcLowAccuracy</code>||1||12.2204460492503131e-10  
+
|<code>fpcLowAccuracy</code>|| &nbsp;1&nbsp; ||2.2204460492503131e-10  
 
|-
 
|-
|<code>fpcMedAccuracy</code>||2||22.2204460492503131e-12 (Default)   
+
|<code>fpcMedAccuracy</code>||&nbsp;2&nbsp;||2.2204460492503131e-12 (Default)   
 
|-
 
|-
|<code>fpcHighAccuracy</code>||3||32.2204460492503131e-14  
+
|<code>fpcHighAccuracy</code>||&nbsp;3&nbsp;||2.2204460492503131e-14  
 
|-
 
|-
|<code>fpcVeryHighAccuracy</code>||4||42.2204460492503131e-16  
+
|<code>fpcVeryHighAccuracy</code>||&nbsp;4&nbsp;||2.2204460492503131e-16  
 
|-
 
|-
|<code>fpcExact</code>||5||50.00
+
|<code>fpcExact</code>||&nbsp;5&nbsp;||0.00
 
|}
 
|}
  

Latest revision as of 16:00, 19 April 2021

Sets floating point compare accuracy by specifying the tolerance value to be used when floating point values are compared.

Two floating point values will be considered equal if "abs(Value1 – Value2) <= ε", where ε is the tolerance value.

By default, the tolerance value is 2.2204460492503131e-012.

Usage

SetFPCompareAccuracy(Accuracy)

Parameters

Accuracy - an FPC constant or a numerical expression specifying a tolerance value as follows:

fpcVeryLowAccuracy  0  2.2204460492503131e-8
fpcLowAccuracy  1  2.2204460492503131e-10
fpcMedAccuracy  2  2.2204460492503131e-12 (Default)
fpcHighAccuracy  3  2.2204460492503131e-14
fpcVeryHighAccuracy  4  2.2204460492503131e-16
fpcExact  5  0.00

Example

Set floating point compare accuracy to High:

SetFPCompareAccuracy(3);

Set floating point compare accuracy to High:

SetFPCompareAccuracy(fpcHighAccuracy);