Jump to content
  • 0

Non-accurate results of integers division


CmP

Question

Greetings, I have experienced strange problem with integers division while writing a script.

Dividing one integer by another always resulted in a number without fractional part, even when it is supposed to be present.
Let me illustrate this with an example. Here is the code:

local test = {}
test = gg.prompt({'Input number:'}, {}, {'number'})
print(test[1])
print(tonumber(test[1]))
print(tonumber(test[1]) / 7)

And the result of it's execution:

 image.thumb.png.9e240e54108b790c2c4b60c689c88140.png

As you see, fractional part is not there, athough it should have been.

I have figured out a solution to fix this. Adding ".0" to an integer and passing this to "tonumber" function seems to produce a floating-point number, I guess.
Replacing last line from code above with this one:

print(tonumber(test[1] .. '.0') / 7)

results in next output (division result with fractional part):

image.thumb.png.687a335b5dcfb8e64933ab11a2fa4d0f.png


So my questions are next.
Why is this happening?
Are there other ways to deal with this problem?

Thank you in advance!

P.S. This problem does not appear when executing practically same code as above with Lua interpreter.
Code:

local test = {}
test[1] = 100
print(test[1])
print(tonumber(test[1]))
print(tonumber(test[1]) / 7)

Output:
image.png.3d53a4cb19c1098cac383ae38ef608b4.png

Link to comment
Share on other sites

1 answer to this question

Recommended Posts

Archived

This topic is now archived and is closed to further replies.

×
×
  • Create New...

Important Information

We have placed cookies on your device to help make this website better. You can adjust your cookie settings, otherwise we'll assume you're okay to continue.