About Calculation Accuracy

About Calculation Accuracy

M-Calc can display up to 15 digits. Internally, 80 bit floating point type is used. For countermeasures against floating point roundoff error, we will deal with rounding digits by the contents of each calculation. For example, in the case of adding or subtracting one digit of decimal part, since the decimal part of the result can not be more than two digits, the second decimal place is rounded. Similarly, in the case of multiplication, it can not be more than three digits, so the third decimal place is rounded. For division etc, round to the internal 18 digits. As the number of digits is finite, an error of ± 1 occurs in the least significant digit of the mantissa part in principle. If you calculate continuously, the error accumulates and becomes large.

Loan repayment amount calculation

    Calculation results are approximate. How to deal with the number of days from borrowing to first repayment, how to process fractions, etc. depends on financial institution. For that reason we do not consider them at all. There is an error in monthly repayment amount, interest total etc etc.

Unit conversion

For example, if it is a unit of length, it converts it via the standard unit such as meters. For this reason, if the scale of the unit greatly departs like micrometres and light years, errors may occur. Also, the original conversion data itself is not always used for actual transactions. Therefore it is for reference level, please do not use in commercial transactions.

Statistical function of trigonometric calculation / calculation history

It is simply making calculations based on formulas. I think that there is no problem with practical accuracy, but errors peculiar to floating point arithmetic are given out.

Hexadecimal and decimal conversion

Since M-Calc itself supports up to 15 digits, it can not convert all the range of 64 - bit integers. Also, I do not plan to deal with calculators with more than 15 digits.

Hexadecimal number with decimal point

Even though it can be represented by a decimal number, there are values that become cyclic decimal numbers in hexadecimal. And this function is used less often, and the operation verification is not sufficiently done. The precision of the display digit can not be converted.