Working with decimals in uBasic - Script Writing - CHDK Forum

# Working with decimals in uBasic

• 1 Replies
• 3313 Views

#### fbonomi

• 469
• A570IS SD1100/Ixus80
##### Working with decimals in uBasic
« on: 22 / May / 2008, 16:37:08 »
Hah! you fell for that, you fools!

uBasic variables are INTEGERS, and therefore cannot have decimals.

But there are a few workarounds that can help you have better results.

They will be obvious for some of you, but I hope they will be anyway useful for somoene

1) Fixed point decimals
To put it simply, work with your numbers multiplied by (e.g.) 100 and pretend the last 2 digits are decimal numbers.

When you add and subtract, it's completely transparent:
1.34 + 5.82 = 8.16
is represented as
134 + 582 = 816

When you multply, divide by 100 after the operation:
12.36 * 3.14 = 38.81
is represented as
1236 * 314 = 388104
388104 / 100 = 3881

When you divide, multiply by 100 before the operation:
12.36 / 3.14 = 3.93
is represented as
1236 * 100 = 123600
123600 / 314 = 393

(depending on how many digits you want, you can use 10, 100, or 1000 as a divisor)

2) use rounding, not truncating
At the end, when you will have to use the results of your calculations, you will probably need to divide them by 100 in order to get "real numbers".

If in uBasic you do a=399/100 you get 3, which is a VERY bad approximation of the result. A better aproximation would be 4...

Converting 3.99 to three is truncating, converting it to 4 is rounding: when rounding, everyting from 3.50 upwards is converted to 4, not to 3

To achieve rounding, just add 50 to the number before dividing it.

(399+50)/100 gives 4
(372+50)/100 gives 4
(351+50)/100 gives 4
(349+50)/100 gives 3
(311+50)/100 gives 3

see?

3) first multpily, then divide

If you have these instructions

a = b / c
a = a * d

You will get better results if you execute FIRST the multpilication and then the division:

a = b * d
a = a / c

In other words, (b / c)*d gives a poorer result than (b * d) / c

If you think about it, it's rather obvious: dividing creates an approximation error, so if you first divide and then multpily you first create an error and then you magnify it with multpilication.

If you do the opposite (multiply, then divide) the error will be produced at the end of the calculaton, and will not be magnified....

When using huge numbers (especially if you artificially add digits as I suggested above) remember that the numbers contained in variables have a limit in size. If you go above that size you get odd results (most probably negative numbers!)

This limit is really huge as the range of valid numbers is ?2,147,483,648 to 2,147,483,647. (note that I haven't actually tried these numbers, as my camera is at the moment busy on a night-long time-lapse. I bet these are the limits, anyway)

So for most practical uses this won't be a problem, but always keep your mind to the maximum value your numbers can get to.

To convince yourself, run this little script:

a=1
:loop
print a
a=a*10
sleep 500
goto "loop"

I hope this helps someone!

#### wontolla

• 413
• S3 & G9 & A720
##### Re: Working with decimals in uBasic
« Reply #1 on: 22 / May / 2008, 17:10:09 »
Interesting.

Who the heck needs Floating Point Units!

This could be implemented in the ubasic interpreter, so when you write say "18.43" it automatically performs the trick needed.