Issue2237

Ticket opened to cover these aspects of #1861:

[Corrected list v2 :( ]

test_cmath.py:            # FIXME uncomment tests for Jython
test_cmath.py:    @unittest.skipIf(is_jython, "FIXME: not working in Jython")

test_complex.py:        # FIXME: The following currently crashes on Alpha
test_complex.py:    @unittest.skipIf(test_support.is_jython, "FIXME: not working in Jython")
test_complex.py:    @unittest.skipIf(test_support.is_jython, "FIXME: not working in Jython")
test_complex.py:    @unittest.skipIf(test_support.is_jython, "FIXME: not working in Jython")
test_complex.py:        #FIXME: these are not working in Jython.
test_complex.py:        #FIXME: these are not working in Jython.
test_complex.py:        #FIXME: this is not working in Jython.
test_complex.py:        #FIXME: this is not working in Jython.
test_complex.py:        #FIXME: this is not working in Jython.
test_complex.py:        #FIXME: these are raising wrong errors in Jython.
test_complex.py:        #FIXME: these are raising wrong errors in Jython.
test_complex.py:        #FIXME: not working in Jython.
test_complex.py:        #FIXME: these are not working in Jython.
test_complex.py:        #FIXME: these are not working in Jython.
test_complex.py:        #FIXME: this is not working in Jython.
test_complex.py:        #FIXME: this is not working in Jython.
test_complex.py:        #FIXME: this is not working in Jython.
test_complex.py:        #FIXME: these are raising wrong errors in Jython.
test_complex.py:        #FIXME: these are raising wrong errors in Jython.
test_complex.py:        #FIXME: not working in Jython.
test_complex.py:    @unittest.skipIf(test_support.is_jython, "FIXME: str.__complex__ not working in Jython")
test_complex.py:                         "FIXME: not working in Jython")
test_complex.py:    @unittest.skipIf(test_support.is_jython, "FIXME: not working in Jython")
test_complex.py:    @unittest.skipIf(test_support.is_jython, "FIXME: not working in Jython")

test_float.py:        #FIXME: not raising ValueError on Jython:
test_float.py:        #FIXME: Jython fails some mod edge cases.
test_float.py:            #FIXME: Jython fails this.
test_float.py:            #FIXME: Jython fails these.
test_float.py:            #FIXME: Jython fails some of this.
test_float.py:            #FIXME: Jython fails some of this.
test_float.py:            #FIXME: Jython fails these.
test_float.py:            #FIXME: Jython fails this.
test_float.py:            #FIXME: Jython fails this.

I've polished off the float ones and most of the complex cases. This one is in the CPython suite, so I'm leaving it.

test_complex.py:        # FIXME: The following currently crashes on Alpha

This one's amusing:
>>> 0j, -0j
(0j, 0j)
>>> -0j, 0j
(-0j, -0j)

We don't distinguish between 0.0 and -0.0 when rationalising constants during code generation: took ages to find. I wonder if there are other places where we have the wrong kind of Object.equals()?

TO DO list:

1. Our test_cmath is out of date relative to CPython:                 self.rAssertAlmostEqual(0., z.imag) -> self.assertEqual(0., z.imag)
in test_cmath_matches_math.

2. test_complex failed to detect my error complex("inf") not valid.

3. cmath.java hyp/trig functions could be more efficient. e.g. sin(z) calls Math.exp twice for same values. Possibly needless testing for nan/inf.

4. Object comparison is a maze of twisty passages.

Some are definitely not in scope for this issue, and some are -- I'm parking them here as I noticed them in the course of the work.

The float and complex test failures are done now, but the cmath library part of this turns out to be quite demanding. We're only skipping a couple of tests: when I un-skiped the test against cmath_testcases.txt,
and refused to bail on the first error, I counted 961 failures.

In general, I would say failures occur for edge cases: arguments near zero, near one (e.g. for acosh), or large/infinite. 

I've checked-in my progress on this in a dozen change sets leading up to https://hg.python.org/jython/rev/69826acfb4a9 . A couple of them are worth a remark.

Some of the cmath tests are against (real-line) mathematical functions from the math module: I had real :) trouble with this until I recognised that CPython's test_math does not actually demand an accurate result. And in a couple of places our 'passing' math module wasn't giving one! I've fixed that in https://hg.python.org/jython/rev/705cee780e1f .

Nice to see, however, that the Java.Math treatment of inf and nan arguments is almost always what Python requires. We can strip out much of the testing for special values if we want to. I've done this where I've made changes, but not generally.

Many failing cmath functions depend heavily on the complex square root, I eventually realised. And cmath.sqrt was not accurate for small argument. Reworking cmath.sqrt is mostly about scaling to manage the potential overflow or loss of precision in sqrt(a**2 + b**2), or spotting when we don't have to.
https://hg.python.org/jython/rev/d2fa71e7ca54

The score now stands at 945 failures against cmath_testcases.txt.

I've noticed that our implementation of the (real) math module is at the root of many of our cmath problems. sinh(x) = (exp(x) - exp(-x)) / 2 in theory, but it's not accurate to compute this way for small x. For abs(x) < 2**-53, it just evaluates to zero.

Yet Jython math passes the tests in test_math.py . Why?

I decided to look at the range of x (not counting 0, 1, inf and nan) over which these methods are actually tested. Test values come from the same file cmath_testcases.txt as is used for cmath. test_math.MathTests.test_testfile() reads the file, then uses only the cases where the argument is real. The resultant coverage is this:

test_testfile (__main__.TestCaseCoverage) ... 
acos       5e-324                   <= x <= 0.9999999999999999      
acosh      1.0000000000000002       <= x <= 1.5653640340214026e+308 
asin       1e-323                   <= x <= 0.9999999999999999      
asinh      5e-324                   <= x <= 1.6025136110019349e+308 
atan       1.0516056964171069e+308  <= x <= 1.440812624377215e+308  
atanh      1e-323                   <= x <= 0.9999999999999999      
cos        None                     <= x <= None                    
cosh       None                     <= x <= None                    
exp        745.0                    <= x <= 745.0                   
log        1e-323                   <= x <= 1.440860577601428e+308  
log10      1e-323                   <= x <= 1.440860577601428e+308  
sin        None                     <= x <= None                    
sinh       None                     <= x <= None                    
sqrt       1e-323                   <= x <= 1e+299                  
tan        None                     <= x <= None                    
tanh       None                     <= x <= None                    

None means they aren't tested at all. This happens when all the non-trivial examples use a complex argument. In addition, test_math.py only requires the library to get within 1e-5 of the reference answer, quite unlike the more demanding test_cmath.py .

So that's why we pass. I'll add to our cmath_testcases.txt, some examples I generate to address that.

But this also implies CPython (real) math is not tested properly, except to the extent cmath relies on it. I think improved rigour here is something we could give back.

#2244 opened as a focus for discussion of the weak testing of the math library.

On both OSX 10.10.1 and Ubuntu 14.10:

ERROR: test_abs (__main__.CMathTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "dist/Lib/test/test_cmath.py", line 450, in test_abs
    self.assertEqual(abs(complex(1.4e308, 1.4e308)), INF)
OverflowError: absolute value too large

----------------------------------------------------------------------
Ran 11 tests in 0.125s

FAILED (errors=1, skipped=1)
Traceback (most recent call last):
  File "dist/Lib/test/test_cmath.py", line 494, in <module>
    test_main()
  File "dist/Lib/test/test_cmath.py", line 491, in test_main
    run_unittest(CMathTests)
  File "/Users/jbaker/jythondev/jython27/dist/Lib/test/test_support.py", line 1274, in run_unittest
    _run_suite(suite)
  File "/Users/jbaker/jythondev/jython27/dist/Lib/test/test_support.py", line 1257, in _run_suite
    raise TestFailed(err)
test.test_support.TestFailed: Traceback (most recent call last):
  File "dist/Lib/test/test_cmath.py", line 450, in test_abs
    self.assertEqual(abs(complex(1.4e308, 1.4e308)), INF)
OverflowError: absolute value too large

Sorry about that. I fixed a bug:
    https://hg.python.org/jython/rev/69826acfb4a9
but I missed the special Jython clause:
    https://hg.python.org/jython/file/69826acfb4a9/Lib/test/test_cmath.py#l449
Already fixed but not pushed.

Currently 921 failures against cmath_testcases.txt.

I claim this little batch of test failures is fixed in the sequence of change sets leading up to this merge:
https://hg.python.org/jython/rev/9c81bac53d7a

The two cmath errors were a bit more involved than I expected ...