Comparing 2 float values in squish

Question

I am trying to compare 2 values (float) using squish and it works fine sometimes, but fails few times. It is very inconsistent. Can someone help me how to use isclose for comparing float values.

Following is my script:

 Xposition_set = waitForObject("{id='textBoxGoto_Sample_X'}").text
 Xposition_setValue = round(float(Xposition_set), 2)

 Xposition_displayed = waitForObject("{id='dbxX' nativeObject.DataContext.Name='" + str(positionList[Index])+ "'}").text
 Xposition_displayedValue = round(float(Xposition_displayed), 2)

 test.compare(Xposition_setValue, Xposition_displayedValue,  GetTestCaseNumber() + "X -Positions matches")

Answer 1

What you are trying should work in general, but because of missing information it is not possible to find out what particular problem you are running into with the above code.

Here is an example Squish test script (for Squish for Windows, which you seem to use) which does what you are asking for with the Squish Addressbook example:

import os
import math

def main():
    startApplication("%s/examples/win/addressbook/Addressbook" % os.getenv("SQUISH_PREFIX"))
    o = waitForObject("{type='Window'}")

    # Obtaining fake value 1:
    o.nativeObject.Text = "1.0"
    v1 = round(float(o.nativeObject.Text), 2)

    # Obtaining fake value 2:
    o.nativeObject.Text = "2.0"
    v2 = round(float(o.nativeObject.Text), 2)

    # Pass:
    test.verify(math.isclose(v1, v2, abs_tol=1))

    # Fail on purpose:
    test.verify(math.isclose(v1, v2, abs_tol=0.9))

Please note that this test script is for Python 3.5.x due to math.isclose(), so it won't work with a default Squish binary package as those ship with Python 2.7.x (contact Squish technical support to obtain a Squish binary package with a different Python version).

Answer 2

Check out the documentation for math.isclose():

math.isclose(a, b, *, rel_tol=1e-09, abs_tol=0.0)
Return True if the values a and b are close to each other and False otherwise.

Also note at the bottom:

New in version 3.5.

So you should be sure you're running Python 3.5+ to use the function. But with that said, all you need to do is pass in your values. A standard example of floating point arithmetic error is usually adding 0.1 + 0.2:

>>> 0.1 + 0.2
0.30000000000000004

So if we checked if that calculation is equal to 0.3:

>>> 0.1 + 0.2 == 0.3
False

With the new math.isclose() function, we can remedy that:

>>> import math
>>> math.isclose(0.1 + 0.2, 0.3)
True

Before Python 3.5, you can just do this manually. You just want to check if the absolute difference between the two numbers is very very small. The default tolerance for math.isclose() is 1e-09, so we can use that:

>>> abs(0.1+0.2 - 0.3) < 1e-9
True

In fact the difference between those two floats is far smaller than that even:

>>> abs(0.1+0.2 - 0.3)
5.551115123125783e-17

So you could use a stronger threshold, either for manual checking or if using math.isclose() and passing in a value for rel_tol.