This is trading an exception for another one, isn't it? Would this really solve the linked issue, if it's solvable?

Yes, instead of a TypeError that confuses the user by telling them that something is not a callable, they would now get an exception simply saying that linking failed. I don't think that the issue is actually solvable.

We could then throw a more specific exception, telling that symlink is disabled.
And I also think Composer should consider not trying to symlink when the function doesn't exist.

Alternatively, we could replace the passed 'symlink' string with a closure:

$symlink = static function($originDir, $targetDir) {
    return symlink($originDir, $targetDir);
};

if (!self::box($symlink, $originDir, $targetDir)) {
    $this->linkException($originDir, $targetDir, 'symbolic');
}

This way, the user would get a fatal error Call to undefined function symlink(). PHP 8.1 would allow us to achieve the same thing more elegantly, see #46408 for a patch against the 6.1 branch.

I also think Composer should consider not trying to symlink when the function doesn't exist.

Symlinking could fail for other reasons, so attempting the symlink and catching Symfony's IOException should be the proper workflow. Let's throw this exception also when the function is disabled.

I've added a more specific error message.

If we go with #46408, we should ensure that the behavior will be the same in both branches.

Using Closure::fromCallable will throw TypeError: Failed to create closure from callable: function "foo" not found or invalid function name
Using foo(...) will throw Error: Call to undefined function foo()

We need to chose one from TypeError, Error or IOException :)

I think we could remove the callable type hint on the box() method (replace it by string), and do the check there, with a nice IOException.

What I like about the current approach is that we'd do an early exit and skip the whole operation if the function is unavailable.

Let's do both then: add the same check in box after changing its type hint, just in case any other methods are disabled?

Agreed. 👍🏻