The problem of writing high-digit numbers into a file and its solution

In this post, the problem of writing a real number with high digit to a file in scientific and numerical computing is presented and discussed.

At the end of this post, you will understand and aware about the problem of writing high-digit numbers to a file and how to solve this problem.

It is common that many number crunching computations require to export all values into a file for further processing or to store the data values. However, there is an important aspect need to be considered!

The number that is written to a file is truncated or rounded to reduce the digits.

This digit reduction performed by a programming language uses, for example MATLAB or python, will cause the value written to the file is not the same with the value in the memory before being written to the file.

The difference between values in the memory and values written to a file may be only one or few decimal digits. However, for heavy number crunching, these few values differences will be propagated and will cause a totally incorrect results or even no solution on a scientific computation, for example in optimisation problem.

This problem is always difficult to debug in any scientific computing!

This post will show a real case study on the mentioned problem as well as how to solve the problem. MATLAB programming language is used for this demonstration.

READ MORE: The problematic aspect of number crunching in programming: Case study in GNSS calculations

• Numerical Python: Scientific Computing and Data Science Applications - A book Leverage the numerical and mathematical modules in Python and its standard library

Real case study: Writing a high-digit number to a file with MATLAB

Dealing with high-digit numbers is very common in any scientific computing that involves number crunching processes of real data.

High-digit numbers are commonly used in a problem where the integer part of the value is very large as well as the decimal part of the values. That is, a value with high numerical value and with high precision presentation.

For example, in GNSS or earth orbit computation, the measure of earth is in “km” with precision up to “mm”. Another example is in distance measurement where value usually in “metre”, but the precision representation can be up to “micro-metre”.

In this case study, suppose we have a value of 11284297.29672563.

In this value, there are 16 digits to represent both the integer part and the fractional part of the value.

To estimate what is the minimum bit required to represent this 16-digit value is calculated as follow:

Where $D$ is the number of digits to represent and $b$ is the number of binary bits to sufficiently represent number of digits $D$.

In this case, since we have 16 digits, hence the number of bits to represent this digit is calculated as follow:

Hence, the minimum number of bits to represent the values is 56 bits.

When, we write the value to a file, the programming language may not represent the value with 56 bits, instead the language may represent the bit of, for example, 32 bits (so that the value will be < 1 digits).

Hence, the value written to the file will be truncated or rounded to reduce the number of digits of the value.

In the next section, we will show how the problem example and how to solve this problem. MATLAB programming language is used.

However, this problem may occur to other programming languages. So, we need to be cautious when dealing with this problem!

(Detailed explanation of this bit requirements can be seen in the previous post).

• Numerical Python: Scientific Computing and Data Science Applications - A book Leverage the numerical and mathematical modules in Python and its standard library

Method 1: the problem of writing high-digit numbers to a file

In this MATLAB implementation, we assign 11284297.29672563 to a variable $val$ that will be automatically set as double precision (8 bytes=64 bits).

Since the $val$ has 64 bits, this variable can sufficiently represent the 16-digit number in memory (since the number require at least 54 bits for sufficient representation).

After that, we directly write the value into a text file.

The MATLAB code is as follow:

%Assign high-digit value to a variable
val=11284297.29672563;
%The value above contains 16 digits.
%8-digit for number (integer) values
%8-digit for decimal (fractional) values
 
%Write into a file
FileNameOut='data1.txt';
fid = fopen(FileNameOut,'w');
fprintf(fid,'%f',val);
fclose(fid);

In this implementation, the written value in the file is 11284297.296726 (rounded to be 14 digits) instead of 11284297.29672563 (16 digits).

These small value differences can cause significant problems in scientific computing and cause incorrect results or incorrect performances of an algorithm. In reality, the effect can be critical!

The result of the written value is shown as below:

The problem above is difficult to spot or debug! As it is usually overlooked!

Method 2: The solution to write high-digit numbers to a file

To solve the above problem of writing high-digit numbers to a file, we cannot directly write the value into a file.

Instead, we need to convert the value into a “string” type variable and adjust how may digit we want to represent the digit in the string.

In this case, we can adjust the string to represent 16 digits so that the full value can be sufficiently stored with “num2str(val1,16)”.

Hence, we write the string value into a file.

The MATLAB code for the problem solution is as follow:

%Assign high-digit value to a variable
val=11284297.29672563;
 
%Write into a file
FileNameOut='data2.txt';
fid = fopen(FileNameOut,'w');
fprintf(fid,'%s',num2str(val,16));
% "16" parameter on the num2str() set a total of 16 digit representations
fclose(fid);

As can be seen from the above MATLAB code, we first convert the value into a string variable with sufficient digit representations (16 digits) and then write the string value to a file.

By doing this, the written value to the file is exactly 11284297.29672563.

The result of the written value after implementing the solution is shown as below:

Hence, there is no missing value or information when we write the value to the file for future use.

READ MORE: Common problems in numerical computation: from data overflow, rounding error, poor conditioning to memory leak

Conclusion

In this post, the problem of writing high-digit numbers to a file is presented and discussed. Then, the solution of this problem is presented.

A real case study showing the writing of a 16-digit real number to a file is presented and discussed in detail. From this case study, the problem is clearly highlighted so that readers will be aware about this problem in their scientific computing implementations.

The problem is that the written value of a variable is not the same with the value in a computer memory before writing. Because there is a truncation or rounding process by a programming language before writing to a file.

This type of problem is difficult to spot and can be frustrated to debug. Because it usually causes algorithms do not work as expected or the computation process crashes and cause a system halt!

The solution is that to write a number with high digit, this number should be converted first to string data type with sufficient digit representations before writing to a file. With this solution the value with high digit in a memory can be equally written to a file for further uses or processes.

You may find some interesting items by shopping here.