We should expect this to work, but currently we get an inappropriate error message.

>>> from sklearn import *
>>> model_selection.cross_validate(linear_model.LogisticRegression(), np.random.rand(10, 2), ['no'] * 3 + ['yes'] * 7, scoring='roc_auc')
...
ValueError: Data is not binary and pos_label is not specified

_binary_clf_curve is assuming that pos_label=1 if not specified. But it should be assuming the same ordering as the classifier, i.e. the greater string is the positive class label.

Ping @qinhanmin2014