The problem is your expectation. The path of a FancyArrowPatch in data coordinates depends on the transform, which is not known until it is attached to an Axes. Here is the method:

    def get_path(self):
        """
        Return the path of the arrow in the data coordinates. Use
        get_path_in_displaycoord() method to retrieve the arrow path
        in display coordinates.
        """
        _path, fillable = self.get_path_in_displaycoord()

        if cbook.iterable(fillable):
            _path = concatenate_paths(_path)

        return self.get_transform().inverted().transform_path(_path)

This is calling self.get_path_in_displaycoord(), where more transform-dependent magic happens.

I agree that get_path() is returning the correct result in each case, but I don't think the path should depend on the transform the transform attached to the patch should change when it is added to the axis (for the Cartesian axis case). Fundamentally, if I ask for a arrow with a head_length of 0.2, I expect the head length to be 0.2 in data co-ordinates.

Extracting the path before plotting the arrow works fine, but simple adding the FancyArrowPatch doesn't do what I expect:

import matplotlib.pyplot as plt
import matplotlib.patches as mpatch

# Create simple arrow
arrowstyle = mpatch.ArrowStyle.Simple(head_length=0.2, head_width=0.2, tail_width=0.1)
arrow = mpatch.FancyArrowPatch((0.5, 0), (0.5, 1), arrowstyle=arrowstyle)

# Extract the path of the arrow
path = arrow.get_path()
pathpatch = mpatch.PathPatch(path)

fig, [ax1, ax2] = plt.subplots(2, 1, tight_layout=True)
ax1.add_artist(arrow)
ax2.add_artist(pathpatch)

ax1.set_title('FancyArrowPatch')
ax2.set_title('Path before adding FancyArrowPatch to ax1')
plt.show()

I don't understand why this is bothering you. What is the problem you are trying to solve?

The "Fancy" things and their derivatives, annotations, are extremely complex, hard to understand (at least for me), and poorly documented. But they work, and serve important functions. So you seem to be pointing to an oddity, something calling for better documentation, not a bug. Correct? I still don't understand how you stumbled across this. Is it that you were trying to figure out what the units of the arrow parameters are?

Regarding units: I think your expectation of specifying arrow properties in data coordinates is not right; the point of these sorts of things is that they inherently use a mix of coordinate systems and units. For example, you may want an arrow to connect two points specified in data coordinates, but normally one wants the arrow characteristics other than length to be in some sort of display coordinates and units, e.g. points or inches, or maybe pixels.

If you have a chunk of time and the motivation, then going through the "Fancy" code, figuring out how it all works, and documenting that along with what all of the kwargs really mean and do, would be a great contribution. The code was written quite some time ago by Jae-Joon Lee, and we are very grateful to him for that, but he has completely dropped out of sight as far as mpl is concerned.

I stumbled across this in my attempts to clean up arrow plotting as a whole in Matplotlib. In order to deprecate FancyArrow (#7213), ax.arrow() needs to be switched over to using FancyArrowPatch. But if there's no way of knowing what the size head the head shape kwargs to FancyArrowPatch actually result in (this bug) then I don't think that would be possible.

What I broadly intend on doing in the long run is sketched out here, which is definitely open to discussion and comment: https://github.com/matplotlib/matplotlib/wiki/MEP-29-(arrows).

re. a mixture of display and data co-ordiante systems, I kind of understand the point, but think it's a bit confusing. Once the patch is added to an axis it's co-ordinates are fixed rigidly in data co-ordinates, so the arrow head changes size when any zooming is done. If the arrow head truly existed in 'display space' I would expect it's size on the screen to be independent of any zooming in the figure.

I also think (but may be wrong) that the concept of 'data space' is much more intuitive to an average user, and specifying all the arrow properties in the same units (data units) is much less confusing than some in 'data units' (arrow start, arrow end), and some in 'display units' (arrow head properties).

Given the plan is to use FancyArrowPatch as the go to for arrow drawing in the future, as a first step I'll try and document what it actually does at the moment before trying to change it at all.

FancyArrowPatch really is a huge rabbit hole...

It turns out that the head doesn't change shape in display co-ordinates, so I have crossed out my false statement in the above comment.

I think I now understand what is going on and think it makes sense, apologies for all the confusion, and thanks @efiring for being patient and explaining! I've opened #8703 to clarify how the patch behaves, and will try and work out what the units are for the ArrowStyle kwargs.