Discuss alternative float comparison algorithms.

_pytest/python.py

@@ -1344,50 +1344,55 @@ class RaisesContext(object):
 
 class approx(object):
     """
-    Assert that two numbers (or two sets of numbers) are equal to each
+    Assert that two numbers (or two sets of numbers) are equal to each other
-    other within some margin.
+    within some tolerance.
 
-    Due to the intricacies of floating-point arithmetic, numbers that we would
+    Due to the `intricacies of floating-point arithmetic`__, numbers that we
-    intuitively expect to be the same are not always so::
+    would intuitively expect to be equal are not always so::
 
         >>> 0.1 + 0.2 == 0.3
         False
 
+    __ https://docs.python.org/3/tutorial/floatingpoint.html
+
     This problem is commonly encountered when writing tests, e.g. when making
     sure that floating-point values are what you expect them to be.  One way to
     deal with this problem is to assert that two floating-point numbers are
-    equal to within some appropriate margin::
+    equal to within some appropriate tolerance::
 
         >>> abs((0.1 + 0.2) - 0.3) < 1e-6
         True
 
     However, comparisons like this are tedious to write and difficult to
     understand.  Furthermore, absolute comparisons like the one above are
-    usually discouraged in favor of relative comparisons, which can't even be
+    usually discouraged because there's no tolerance that works well for all
-    easily written on one line.  The ``approx`` class provides a way to make
+    situations.  ``1e-6`` is good for numbers around ``1``, but too small for
-    floating-point comparisons that solves both these problems::
+    very big numbers and too big for very small ones.  It's better to express
+    the tolerance as a fraction of the expected value, but relative comparisons
+    like that are even more difficult to write correctly and concisely.
+
+    The ``approx`` class performs floating-point comparisons using a syntax
+    that's as intuitive as possible::
 
         >>> from pytest import approx
         >>> 0.1 + 0.2 == approx(0.3)
         True
 
-    ``approx`` also makes is easy to compare ordered sets of numbers, which
+    The same syntax also works on sequences of numbers::
-    would otherwise be very tedious::
 
         >>> (0.1 + 0.2, 0.2 + 0.4) == approx((0.3, 0.6))
         True
 
-    By default, ``approx`` considers two numbers to be equal if the relative
+    By default, ``approx`` considers numbers within a relative tolerance of
-    error between them is less than one part in a million (e.g. ``1e-6``).
+    ``1e-6`` (i.e. one part in a million) of its expected value to be equal.
-    Relative error is defined as ``abs(x - a) / x`` where ``x`` is the value
+    This treatment would lead to surprising results if the expected value was
-    you're expecting and ``a`` is the value you're comparing to.  This
+    ``0.0``, because nothing but ``0.0`` itself is relatively close to ``0.0``.
-    definition breaks down when the numbers being compared get very close to
+    To handle this case less surprisingly, ``approx`` also considers numbers
-    zero, so ``approx`` will also consider two numbers to be equal if the
+    within an absolute tolerance of ``1e-12`` of its expected value to be
-    absolute difference between them is less than one part in a trillion (e.g.
+    equal.  Infinite numbers are another special case.  They are only
-    ``1e-12``).
+    considered equal to themselves, regardless of the relative tolerance.  Both
-
+    the relative and absolute tolerances can be changed by passing arguments to
-    Both the relative and absolute error thresholds can be changed by passing
+    the ``approx`` constructor::
-    arguments to the ``approx`` constructor::
 
         >>> 1.0001 == approx(1)
         False
@@ -1396,12 +1401,12 @@ class approx(object):
         >>> 1.0001 == approx(1, abs=1e-3)
         True
 
-    Note that if you specify ``abs`` but not ``rel``, the comparison will not
+    If you specify ``abs`` but not ``rel``, the comparison will not consider
-    consider the relative error between the two values at all.  In other words,
+    the relative tolerance at all.  In other words, two numbers that are within
-    two numbers that are within the default relative error threshold of 1e-6
+    the default relative tolerance of ``1e-6`` will still be considered unequal
-    will still be considered unequal if they exceed the specified absolute
+    if they exceed the specified absolute tolerance.  If you specify both
-    error threshold.  If you specify both ``abs`` and ``rel``, the numbers will
+    ``abs`` and ``rel``, the numbers will be considered equal if either
-    be considered equal if either threshold is met::
+    tolerance is met::
 
         >>> 1 + 1e-8 == approx(1)
         True
@@ -1409,6 +1414,46 @@ class approx(object):
         False
         >>> 1 + 1e-8 == approx(1, rel=1e-6, abs=1e-12)
         True
+
+    If you're thinking about using ``approx``, then you might want to know how
+    it compares to other good ways of comparing floating-point numbers.  All of
+    these algorithms are based on relative and absolute tolerances, but they do
+    have meaningful differences:
+
+    - ``math.isclose(a, b, rel_tol=1e-9, abs_tol=0.0)``:  True if the relative
+      tolerance is met w.r.t. either ``a`` or ``b`` or if the absolute
+      tolerance is met.  Because the relative tolerance is calculated w.r.t.
+      both ``a`` and ``b``, this test is symmetric (i.e.  neither ``a`` nor
+      ``b`` is a "reference value").  You have to specify an absolute tolerance
+      if you want to compare to ``0.0`` because there is no tolerance by
+      default.  Only available in python>=3.5.  `More information...`__
+
+      __ https://docs.python.org/3/library/math.html#math.isclose
+
+    - ``numpy.isclose(a, b, rtol=1e-5, atol=1e-8)``: True if the difference
+      between ``a`` and ``b`` is less that the sum of the relative tolerance
+      w.r.t. ``b`` and the absolute tolerance.  Because the relative tolerance
+      is only calculated w.r.t. ``b``, this test is asymmetric and you can
+      think of ``b`` as the reference value.  Support for comparing sequences
+      is provided by ``numpy.allclose``.  `More information...`__
+
+      __ http://docs.scipy.org/doc/numpy-1.10.0/reference/generated/numpy.isclose.html
+
+    - ``unittest.TestCase.assertAlmostEqual(a, b)``: True if ``a`` and ``b``
+      are within an absolute tolerance of ``1e-7``.  No relative tolerance is
+      considered and the absolute tolerance cannot be changed, so this function
+      is not appropriate for very large or very small numbers.  Also, it's only
+      available in subclasses of ``unittest.TestCase`` and it's ugly because it
+      doesn't follow PEP8.  `More information...`__
+
+      __ https://docs.python.org/3/library/unittest.html#unittest.TestCase.assertAlmostEqual
+
+    - ``a == pytest.approx(b, rel=1e-6, abs=1e-12)``: True if the relative
+      tolerance is met w.r.t. ``b`` or the if the absolute tolerance is met.
+      Because the relative tolerance is only calculated w.r.t. ``b``, this test
+      is asymmetric and you can think of ``b`` as the reference value.  In the
+      special case that you explicitly specify an absolute tolerance but not a
+      relative tolerance, only the absolute tolerance is considered.
     """
 
     def __init__(self, expected, rel=None, abs=None):
@@ -1427,6 +1472,9 @@ class approx(object):
 
     @property
     def expected(self):
+        # Regardless of whether the user-specified expected value is a number
+        # or a sequence of numbers, return a list of ApproxNotIterable objects
+        # that can be compared against.
         from collections import Iterable
         approx_non_iter = lambda x: ApproxNonIterable(x, self.rel, self.abs)
         if isinstance(self._expected, Iterable):
@@ -1439,12 +1487,17 @@ class approx(object):
         self._expected = expected
 
 
-
 class ApproxNonIterable(object):
     """
     Perform approximate comparisons for single numbers only.
 
-    This class contains most of the 
+    In other words, the ``expected`` attribute for objects of this class must
+    be some sort of number.  This is in contrast to the ``approx`` class, where
+    the ``expected`` attribute can either be a number of a sequence of numbers.
+    This class is responsible for making comparisons, while ``approx`` is
+    responsible for abstracting the difference between numbers and sequences of
+    numbers.  Although this class can stand on its own, it's only meant to be
+    used within ``approx``.
     """
 
     def __init__(self, expected, rel=None, abs=None):
@@ -1528,7 +1581,6 @@ class ApproxNonIterable(object):
         return max(relative_tolerance, absolute_tolerance)
 
 
-
 #
 #  the basic pytest Function item
 #