Skip to content

random malloc error on test_k_means_plus_plus_init_2_jobs test #1407

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
erg opened this issue Nov 24, 2012 · 7 comments · Fixed by #1430
Closed

random malloc error on test_k_means_plus_plus_init_2_jobs test #1407

erg opened this issue Nov 24, 2012 · 7 comments · Fixed by #1430
Labels
Bug
Milestone
0.13

Comments

@erg
Copy link
Contributor

erg commented Nov 24, 2012

Happens on current master 3c46eba on MacOSX 10.8.2 64bit. Happens like 1/10th of the time?

modern:scikit-learn erg$ [master*] nosetests -v
Affinity Propagation algorithm ... ok
Tests the DBSCAN algorithm with a similarity array. ... ok
Tests the DBSCAN algorithm with a feature vector array. ... ok
Tests the DBSCAN algorithm with a callable metric. ... ok
sklearn.cluster.tests.test_dbscan.test_pickle ... ok
Check that we obtain the correct solution for structured ward tree. ... ok
Check that we obtain the correct solution for unstructured ward tree. ... ok
Check that the height of ward tree is sorted. ... ok
Check that we obtain the correct number of clusters with Ward clustering. ... ok
Check that we obtain the correct solution in a simplistic case ... ok
Test scikit ward with full connectivity (i.e. unstructured) vs scipy ... ok
Check that connectivity in the ward tree is propagated correctly during ... ok
Check non regression of a bug if a non item assignable connectivity is ... ok
sklearn.cluster.tests.test_k_means.test_square_norms ... ok
sklearn.cluster.tests.test_k_means.test_kmeans_dtype ... ok
sklearn.cluster.tests.test_k_means.test_labels_assignement_and_inertia ... ok
Check that dense and sparse minibatch update give the same results ... ok
sklearn.cluster.tests.test_k_means.test_k_means_plus_plus_init ... ok
sklearn.cluster.tests.test_k_means.test_k_means_new_centers ... ok
sklearn.cluster.tests.test_k_means.test_k_means_plus_plus_init_2_jobs ... Python(33971) malloc: *** error for object 0x10dae4000: pointer being freed already on death-row
*** set a breakpoint in malloc_error_break to debug
Python(33972) malloc: *** error for object 0x10daa3000: pointer being freed already on death-row
@ogrisel
Copy link
Member

ogrisel commented Nov 28, 2012

Can you reproduce it if you only run this specific test with the following?

nosetests sklearn/cluster/tests/test_k_means.py:test_k_means_plus_plus_init_2_jobs

@ogrisel
Copy link
Member

ogrisel commented Nov 28, 2012

I cannot reproduce it with:

for i in {1..100}; do nosetests sklearn/cluster/tests/test_k_means.py:test_k_means_plus_plus_init_2_jobs; done

I am running the same architecture with a Python installed with homebrew and numpy 1.6.1 compiled from source using the default build env of OSX 10.8.

@erg
Copy link
Contributor Author

erg commented Dec 1, 2012

I still get it with the newest numpy and scikit. Happened on the first make during nosetests, and then I had to run it several hundred times to have it happen again.

make

I: Seeding RNGs with 2110279685
...................Python(19564) malloc: *** error for object 0x1046d5000: pointer being freed already on death-row
*** set a breakpoint in malloc_error_break to debug

Note: this is not exactly your line, I'm running all of the tests here. It still might break with your test verbatim, but I didn't run it enough times.

for i in {1..100}; do nosetests sklearn/cluster/tests/test_k_means.py; done

.....................................
----------------------------------------------------------------------
Ran 37 tests in 2.310s

OK
.....................................
----------------------------------------------------------------------
Ran 37 tests in 2.040s

OK
......Python(22636) malloc: *** error for object 0x10c7b3000: pointer being freed already on death-row
*** set a breakpoint in malloc_error_break to debug

Um, I'll try to debug it more.

@ogrisel
Copy link
Member

ogrisel commented Dec 1, 2012

This might be a new version of: #636 that used to happen on lion 10.7 but it's much rarer under 10.8. Maybe we should skip this test by default when sys.platform == 'darwin' and recommend OSX users to build numpy / scipy / scikit-learn against altas or MKL if they want to use multiprocessing.

@ogrisel
Copy link
Member

ogrisel commented Dec 1, 2012

You can try to see if this makes it possible to reproduce the crash:

https://gist.github.com/2027412

@erg
Copy link
Contributor Author

erg commented Dec 1, 2012

Yes, this is exactly bug #636! The gist doesn't crash for the sizes I tried (32, 64, etc).

+1 for disabling the unit test on mac or tracking down the people who could fix it for real.

@ogrisel
Copy link
Member

ogrisel commented Dec 1, 2012

Please feel free to open new PR for this.

@erg erg mentioned this issue Dec 1, 2012
Merged
@ogrisel ogrisel closed this as completed in eb64f18 Dec 2, 2012
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
Bug
Projects
None yet
Milestone
0.13
Development

Successfully merging a pull request may close this issue.

BUG: Don't test test_k_means_plus_plus_init_2_jobs on Mac OSX >= 10.7 be... erg/scikit-learn
2 participants
@erg @ogrisel