Single precision, double precision and precision (Reprint)

Floating point numbers are one of the most commonly used data types on computers , Some languages even have floating point numbers （Perl,Lua Classmate, don't run , It's about you ）.

Commonly used floating-point numbers are double precision and single precision . besides , There is also a semi precision Dongdong .

Double precision 64 position , Single precision 32 position , Half precision is naturally 16 Yes. .

Half precision is NVIDIA in 2002 It came out in , Double precision and single precision are used to calculate , The purpose of semi precision is to reduce the cost of data transmission and storage .

In many scenes, the accuracy requirements are not so high , For example, distributed deep learning , If you use half precision , Compared with single precision, it can save half of the transmission cost . Considering that the model of deep learning may have hundreds of millions of parameters , Using half precision transmission is still very valuable .

Google Of TensorFlow Is to use 16 Floating point number of bits , However, they do not use the standard proposed by NVIDIA , But directly 32 The decimal part of the floating-point number of digits is truncated . It's said to be for less computation expensive...

Compare the following floating point numbers layout:

Double precision floating point

Single-precision floating-point

Semi precision floating point number

They all share 3 part , Sign bit , Index and mantissa . Different precision only means that the length of digits and trailing digits are different .

Parsing a floating point number is 5 Bar rule

If the exponent is all zero , The trailing digits are all zeros , That means 0
If the exponent is all zero , The trailing digit is non-zero , It means a very small number （subnormal）, Calculation method (−1)^signbit × 2^−126 × 0.fractionbits
If the index bits are all 1, The trailing digits are all zeros , Indicates positive and negative infinity
If the index bits are all 1, The trailing digit is non-zero , Indicates that it is not a number NAN
The remaining calculation method is (−1)^signbit × 2^(exponentbits−127) × 1.fractionbits
Almost all commonly used languages do not provide half precision floating-point numbers , At this time, we need to transform ourselves

Original address ：https://blog.csdn.net/sinat_24143931/article/details/78557852