I don't see how we can not support returning a numeric array

But does it need to be the default?
Because basically having the case of always returning a numeric array means a) encoding categorical features and b) dropping string features. So that means that you don't get the dataset you asked for as it is described / accessible online, but a processed version.
I personally don't like that to happen silently or to be the default behaviour (we could also have it as an option to return a strictly numerical array, and by default the object array if there are mixed dtypes).

rth or amueller or other, please have another look at this in terms of encoding nominals.

I am happy @jorisvandenbossche joined in on this, because I don't have much to say about it :)

I feel very uncomfortable that we are blocking a release for these decisions, because we need such a loader to simply facilitate examples. I wish we had resolved this much earlier.

There are a few of us who wish we could make dataframes and heterogeneous data first-class citizens, but it's a big change and we're not there yet. I look forward to being able to return a DataFrame (I don't think frame=True or fetch_openml_frame is a great imposition on the user), but I think returning an object array, or deciding whether or not to use object dtype on the basis of the presence of nominals, is a very poor solution in the interim, especially when applied to sparse matrices. Or we could return a dict of arrays, but this is not well facilitated by liac-arff, and not supported by ColumnTransformer; or a struct array, but these have issues with object dtypes, and are not supported by ColumnTransformer.

I think we can add a nominal_encoding={'ordinal', 'one-hot'} switch soon (where 'default' would be added when we support returning dataframes, and would use pd.Categorical), and perhaps a way of encoding strings too (but this is fraught; I'd rather just export them in dataframes).

I hope you come to agree with me that this is the best we can do for now in terms of compatibility and extensibility, while supporting our immediate needs. But please do raise alternatives (and required extra features before 0.20 final) if you have them.

In seeing the enormous difference between 'titanic' version 1 and 2, I think we need to require version when fetching by name...

Instead we can have an issue after merge: Raise a warning along the lines of "Multiple dataset versions match the name BLAH. Versions may be very different. Getting version x."

I think this is acceptable as is, and I would like to merge this. Can I hear +1/-1 for merge?

I'll also prepare a deprecation of fetch_mldata based on this... I hope that's not hard to do.

A problem: scikit-learn doesn't support string targets for multilabel data. So I can't get the yeast example to work... Do we therefore also need to encode targets, or only when there are multiple? :(

I get a few DeprecationWarnings at import with Python 3.6,

In [1]: from sklearn.datasets import fetch_openml
sklearn/externals/_arff.py:204: DeprecationWarning: Flags not at the start of the expression '(?x)\n        ,      ' (truncated)
  ''' % {'value_re': value_re})
sklearn/externals/_arff.py:204: DeprecationWarning: Flags not at the start of the expression '(?x)\n        ,      ' (truncated)
  ''' % {'value_re': value_re})
sklearn/externals/_arff.py:204: DeprecationWarning: Flags not at the start of the expression '(?x)\n        ,      ' (truncated)
  ''' % {'value_re': value_re})
sklearn/externals/_arff.py:219: DeprecationWarning: Flags not at the start of the expression '(?x)\n        (?:^\\s*' (truncated)
  ''' % {'value_re': value_re})
sklearn/externals/_arff.py:219: DeprecationWarning: Flags not at the start of the expression '(?x)\n        (?:^\\s*' (truncated)
  ''' % {'value_re': value_re})
sklearn/externals/_arff.py:219: DeprecationWarning: Flags not at the start of the expression '(?x)\n        (?:^\\s*' (truncated)
  ''' % {'value_re': value_re})

In the datasets I tried that are currently used in examples, titanic v0 works, v1 raises exception about unsupported string field (as discussed above), "minst_748", "shuttle", "iris", "leukemia" works.

For "yeast" I get,

{...
 'target': array(['MIT', 'MIT', 'MIT', ..., 'ME2', 'NUC', 'CYT'], dtype=object)
 ...
}

with 
In [15]: np.unique(data['target'])
Out[15]: 
array(['CYT', 'ERL', 'EXC', 'ME1', 'ME2', 'ME3', 'MIT', 'NUC', 'POX',
       'VAC'], dtype=object)

is this just the first label?

Overall I share the sentiment of #11419 (comment), some things are not currently not that great (e.g. default nominal encoding, sparse datasets loading is very slow etc) but this is a good basis, it's marked as experimental, and we need this to fix broken examples CI. We can improve various points in subsequent PRs after the examples are migrated, even if it means changing the API a bit. +1 for merge.

OK, the version parameter can really change the output completely...

Maybe leave the multilabel targets as they are, and do the manual encoding in the single example where we need them for now? I am also not so keen on doing default encoding if we can avoid it...

This, and examples based off it, are passing in all tested platforms. Merge?

This, and examples based off it, are passing in all tested platforms. Merge?

Great! Someone needs to press the green button.

If the targets all have value {FALSE, TRUE} then it's fairly clear we
should encode it.

Fair enough, also running some detection on targets is probably not too expensive.

Should we error if no version is specified and there are multiple? Or
simply warn?

+1 for warn (in subsequent PR)

Fair enough, also running some detection on targets is probably not too expensive.

It's negligible as the {FALSE, TRUE} information is in metadata.

I'd like to squash and merge, but I don't know who GitHub will attribute the commit to! I'd like it to be @janvanrijn. I can go squash and merge manually...

According to someone somewhere, "GitHub takes the info of the PR author". Let's do it.

Thank you @janvanrijn!!

Wonderful! Thanks @janvanrijn and also @jnothman for doing major fixes on the fly in related projects !

Thanks guys, was a nice experience to work with you :)

See you all in Paris this year!