当前位置：网站首页>Why does LabVIEW lose precision in floating point numbers

Why does LabVIEW lose precision in floating point numbers

2022-07-02 11:07:00 【LabVIEW development】

LabVIEW Why floating point numbers lose precision

When using floating-point numbers or double precision floating-point numbers in the programming environment, I find that the value of numbers will change slightly . Such as input 2.4 It shows that 2.3999999999999999. Why does this inaccuracy happen to floating point numbers ？

answer : There are two main reasons for the imprecision of floating-point numbers ：

The binary representation of decimal numbers may not be accurate

It is not uncommon that floating-point numbers or double precision floating-point numbers cannot be accurately represented . The reason why floating-point values cannot be accurately expressed in decimal is due to CPU The method of representing floating point numbers . In this case, some accuracy may be sacrificed , Some floating-point operations also introduce errors . Take the situation mentioned above as an example ,2.4 The binary representation of is not exactly 2.4. Instead, the closest binary representation is 2.3999999999999999. The reason is that floating point numbers consist of two parts ： Index and mantissa . The value of a floating-point number is actually calculated by a specific mathematical formula .

If you want to get more information about this formula or mantissa , You can refer to “What is Mantissa？” article .

The accuracy loss encountered will be encountered in any operating system and programming environment .

Be careful ： have access to BinaryCoded Decimal (BCD) Library to maintain accuracy .BCD Digital coding method will encode each decimal digit separately . If you want to know BCD More information about , You can refer to the article “What is Binary Coded Decimal(BCD)?”

For more information about this accuracy loss, please refer to IEEE/ANSI Standard 754-1985, This article includes Standardfor Binary Floating Point Arithmetic.

Type mismatch

Floating point number and double precision floating point number types may be mixed . Please make sure that all data types are the same when performing mathematical operations .

Be careful ：float The only variables of type are 7 Bit accuracy , and double Variables of type are 15 Bit accuracy .