The problem with that is what you can currently observe already when searching on google: It'll find old pages for you, because those are the ones most links direct to. (e.g. google "matplotlib+imshow+example"). Relevant: #10016 and #10854

So ideally the non versioned link would link to the latest available version of a page. Not sure if you can do that with some sort of fancy redirect.

Would it be possible to add noindex to older versions of the docs? That would prevent google from showing these results.

On the downside, we would have to modify all doc pages once a newer version comes out.

I would suggest using robots.txt (which currently doesn't exist).

If you put all your docs in directories named after the version /docs/3.0.2/ and then only allow robots to crawl the /current/ edition (which is just an invisible pointer to /3.0.2/ for instance), that would go a long way to reducing end-user discovery of old docs. The old versions would remain perpetually available for links.

Does somebody know if it's possible to integrate the version picker from readthedocs into other sites?

IMHO that would really help with version confusion.

So on other projects, the RTD theme has an option canonical_url that can be used to set a path to the version we want crawled--it adds links with the rel="canonical" attribute that apparently tells the crawlers what we want. I set it in conf.py as, e.g.:

{'canonical_url': 'https://matplotlib.org/latest/'}

though that would require us to have something like https://matplotlib.org/ work as a root for the latest for all doc pages--I'm not sure if that's the case. I'm also not sure if the canonical_url is a standard option or if it could easily be grafted onto our theme.

I think #13592 was incorrect, and canonical should point to our top level. I think we should also have a sitemap.xml that puts a lower weight on the old versions versus the current one. Do we have anyone who knows enough about how to do this automatically? Maybe using this extension: https://github.com/jdillard/sphinx-sitemap

Re-milestoning because our search results are really all over the place.

See also https://stackoverflow.com/questions/63542354/readthedocs-robots-txt-and-sitemap-xml

See matplotlib/matplotlib.github.com#47 for most of this....

See matplotlib/matplotlib.github.com#48 for an alternative to matplotlib/matplotlib.github.com#47

@floggle Do you have SEO experience? Is matplotlib/matplotlib.github.com#48 what you were suggesting? Note the sitemap.xml down-weights the old versions, versus hiding them. We could also hide the old versions and only allow /stable/...

Now see matplotlib/matplotlib.github.com#49