first_page the funky knowledge base
personal notes from way, _way_ back and maybe today

Review of the Data Type Summary: Floating Point Numbers

The online help for VBA has a topic called "Data Type Summary." For those of us who are not computer scientists we should take note of the following with regard to floating point numbers:

The use of scientific notation in the help file topic to denote range implies that floating point values are stored as exponents.

The terms "single precision" and "double precision" refer to the numbers left of the decimal point that will not be rounded when VBA allocates memory between the exponent and the significant digits.

Because floating point numbers are stored as exponents, their "precision" affects large numbers as well as small numbers. Large numbers exceeding the precision can be inadvertently rounded. For example,

CSng("123,123,123") = 1.231231E+08 = 123,123,100

because the Single data type is accurate to seven significant digits. This means that values (without decimals) greater than or equal to 10 million will be rounded. Also we can see that

CDbl("123,123,123,123,123,123") = 1.23123123123123E+17 = 123,123,123,123,123,000

because the Double data type is accurate up to 15 significant digits. This implies that values (without decimals) greater than 999 trillion will be rounded.

The double precision may be safe to use for large numbers in "real world" applications but keep in mind the size of the number stored drops when decimal places in the original value need to be included. For example, suppose you want to store a value in a DAO database as type Currency. The Currency data type holds four decimal places. This implies that values greater than 99 billion will be rounded if CDbl() is used to prepare data for storage in the Currency field.

In the "real world" of, say, financial applications, 99 billion is not an unusually high number.

mod date: 1998-06-13T06:03:15.000Z