Hmmmm... Have you tried submitting a patch to sphinx-doc? I'd be more comfortable evaluating this change if I could see that patch directly at least.

Fundamentally, the only addition to the process_generate_options (line 100 in custom_files_autosummary.py of this PR) is:

try:
    filename = app.config.custom_autosummary_file_map[name] + suffix
except KeyError:
    filename = name + suffix

It is a little crazy this is all that needs to be changed for this PR to work. scikit-learn must be doing something special to be able to link sklearn.cluster.dbscan to sklearn.cluster.dbscan_lowercase.html correctly. When I created a barebones example to debug this issue, I needed to do a little more fine tuning of the references to get links to point correctly.

Anyways, I could try to patch sphinx, but that patch would most likely be on the latest version. Is there something preventing us from using the latest version of sphinx?

@jnothman I updated this PR with a smaller LOC fix to our issue. It mocks out os.path.join temporarily with a function that maps filenames to our custom filenames. Again, this is not the most elegant solution.

I updated this PR with zero configuration approach to detecting if a file has already been generated with the same case insensitive name. In these cases the filename will be appended with the hash of the name.

This can be seen here: https://43873-843222-gh.circle-artifacts.com/0/doc/modules/generated/sklearn.cluster.optic35a4574d84b18e6ad28c8c022895e64bb2242daf.html#sklearn.cluster.optics

sklearn.cluster.optic is appended with a sha1 hash of sklearn.cluster.optic.

You have valid points. I’ll look into reducing the scope of this PR to prioritize the uppercase filename and add a predefined string to the lowercase filename.

Here comes another update to this PR. The class names now take priority over the function names. When a function name matches a class name, then a prefix -lowercase will be added to the end of the function's generated filename. This makes an two assumptions that should be valid for sklearn:

1. Class names always begins with a capitalized letter.
2. If a function name matches a class name (case-insensitive), then it is the
   only function that matches.

You are correct, this implementation still assumes that the class is generated first. It turns out this is always the case because sphinx sorts the names here.

In creating this PR, I was trying to create a no configuration fix, so future PR's do not need to worry about this.

Regarding a change to sphinx-doc, this problem stems from not being able to trust os.path.isfile on case insensitive file systems. Every solution I come up with to patch sphinx, feels hacky. The least hacky solution was the solution with a configuration dict that maps a name to another name. This dictionary may need to be passed to the autodoc extension to make sure it knows about the mapping.

I suspect os.path.isfile is used in autosummary.generate because it needs to work with the sphinx-autogen cli. It uses the filesystem to figure out if a file was already generated.

I have been wrestling with the following two approaches:

1. Two configuration options:

custom_autosummary_names_with_new_suffix = {
    "sklearn.cluster.dbscan",
    "sklearn.cluster.optics",
    "sklearn.covariance.oas",
    "sklearn.decomposition.fastica"
}
custom_autosummary_new_suffix = "-lowercase.rst"

This leads to a simpler patch, but future contributors running into this issue would need to know to add their function to custom_autosummary_filenames_with_new_suffix.

2. No configuration options and take advantage of the fact that classes are generated first. This leads to a slightly more complicated patch.

@jnothman What do you think?

As a reference, I opened another PR: #13022 that uses the configuration options. It feels less hacky because it does less and allow a user to configure the filenames.

I am closing this PR in favor of #13022.