Trying older versions of Matplotlib, the problem first appears in 3.6.0.

As noted by the leading underscore, _Backend has never been intended to be a mechanism for the implementation of third-party backends. The API to implement backends prior to matplotlib 3.6 was documented (somewhat poorly, admittedly) in https://matplotlib.org/stable/api/backend_template_api.html; in matplotlib 3.6 this API has been streamlined and there is a PR to describe the new API at #24218; see https://output.circle-artifacts.com/output/job/77219d62-0e62-45c1-9e6b-c3ec83e68401/artifacts/0/doc/build/html/users/explain/writing_a_backend_pyplot_interface.html for the rendered docs.
Feel free to let us know if that documentation could be improved.

Is there a minimal working example of a backend that would work with 3.6.0 and up?

matplotlib/backends/backend_template.py is intended as such an example, although I can see it's not entirely up to date...

Actually the breakage did show a bug in the new backend API implementation; #24527 should fix that (and it also updates the backend_template example).
(But the point about _Backend not being intended for external use remains.)

If I replace MyBackend in the above example with

FigureCanvas  = FigureCanvasAgg
FigureManager = MyFigureManager

def show(*, block=None):
    print("MyBackend.show()")
    for manager in Gcf.get_all_fig_managers():
        manager.show()
    print("MyBackend.show() after manager loop")

I still get

matplotlib.backend_bases.NonGuiException: Matplotlib is currently using module://mpl, which is a non-GUI backend, so cannot show the figure.

Is that the breakage that #24527 is supposed to fix? Or am I doing something wrong?

If I remove show entirely, which is what I think that issue is suggesting, I still get the same warning. But I think that's the wrong route for me. I'm quite explicitly after plt.show() working.

Looking at it again, the problem in your case is different from the one fixed by #24527. Rather, in order to register the manager class, setting FigureManager on _Backend doesn't work anymore (it can be kept if you want backcompat while keeping the use of _Backend). On matplotlib 3.6+, you need to subclass FigureCanvasAgg and set the manager_class class attribute, e.g.

class MyCanvas(FigureCanvasAgg):
    manager_class = MyFigureManager

(this is assuming you don't want to use the old, old method of defining new_figure_manager and related functions).

Ah, victory! That works. Thanks a lot for your help.

@mrzv, as someone who wrote a custom backend, any comments you had on #24281 would be very valuable. Thanks!

@jklymak It's probably too strong to call what I did a custom backend. I just call savefig() to get a png out. I've run in the past into weird formatting/layout issues (in a different backend), but I can't reproduce them right now. I'll come back to that issue, if I encounter such problems again. Thanks for pointing me to it.