Skip to content

Overflow in matthews_corrcoef on a 64-bit mac #9622

Closed
@sam-s

Description

@sam-s

Description

I see

sklearn/metrics/classification.py:538: RuntimeWarning: overflow encountered in long_scalars

and

sklearn/metrics/classification.py:538: RuntimeWarning: invalid value encountered in sqrt

in sklearn.metrics.matthews_corrcoef for large input vectors.

Steps/Code to Reproduce

Use functions from #2806 and then:

def mcc_test(n_points):
    y_true, y_pred = random_ys(n_points)
    mcc_safe = matthews_corrcoef(y_true, y_pred)
    mcc_unsafe = sklearn.metrics.matthews_corrcoef(y_true, y_pred)
    try:
        assert(abs(mcc_safe - mcc_unsafe) < 1e-8)
    except AssertionError:
        print('Error: mcc_safe=%s, mcc_unsafe=%s, n_points=%s' % (
            mcc_safe, mcc_unsafe, n_points))

>>> mcc_test(100)
>>> mcc_test(1000)
>>> mcc_test(10000)
>>> mcc_test(100000)
/Users/sds/.virtualenvs/algorisk/lib/python2.7/site-packages/sklearn/metrics/classification.py:538: RuntimeWarning: overflow encountered in long_scalars
  mcc = cov_ytyp / np.sqrt(cov_ytyt * cov_ypyp)
Error: mcc_safe=0.898760987225, mcc_unsafe=1.75544683417, n_points=100000

Expected Results

nothing printed, like above for mcc_test(1000).

Actual Results

see messages above.

Versions

Darwin-16.7.0-x86_64-i386-64bit
('Python', '2.7.13 (default, Jul 18 2017, 09:17:00) \n[GCC 4.2.1 Compatible Apple LLVM 8.1.0 (clang-802.0.42)]')
('NumPy', '1.13.1')
('SciPy', '0.19.1')
('Scikit-Learn', '0.19.0')

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions