@axel-rutz thanks for reporting this issue. Let me ask you if I've understood you right. You want us to change this:

private static function isBinaryString($value)
{
    return preg_match('/[^\x09-\x0d\x20-\xff]/', $value);
}

By something like this?

private static function isBinaryString($value)
{
    $isValidUTF8 = preg_match('//u', $string);

    return $isValidUTF8 && preg_match('/[^\x09-\x0d\x20-\xff]/', $value);
}

The intent is to have a rock-solid unicode check, so that i can throw whatever in and can be sure i get no invalid YAML. And, otoh, to throw whatever unicode in and get no unreadable binary.

It's even simpler, see above SO link (which i did only basically test though).

private static function isBinaryString($value)
{
    return FALSE === preg_match('//u', $value);
}

The thing is that the YAML dumper right now is able to also dump strings that are no UTF-8 strings (for example, the ISO-8859-1 charset used in Western Europe). However, as dumping binary strings must be explicitly enabled by the user I think we can deprecate dumping non UTF-8 encoded strings as YAML strings and dump them as binary data if the flag is passed (see #18294 for a patch).

The thing is that the YAML dumper right now is able to also dump strings that are no UTF-8 strings (for example, the ISO-8859-1 charset used in Western Europe).

I don't see how this can work in general, as you can easily construct valid ISO-8859-1 strings that are invalid as UTF-8.

Let's elaborate it a bit more: The claim is that it is important that the binary-check is specific in the following sense. (For the sake of clarity i'll replace "binary" with "invalid UTF-8".)

• a) There MUST NOT be any false negatives (i.e. invalid UTF-8 that is considered valid, thus not encoded)
  Reason: Non-encoded invalid UTF-8 is not allowed in valid JSON.
• b) There SHOULD NOT be any false positives (i.e. valid UTF-8 that is considered invalid, thus encoded)
  Reason: DX. Developers might expect to be able to look at string data in an unencoded form. (I have some projects where this is important.)

This is why i think it is important to use a proper utf-8-check. Given another check it will always be possible to create a string that will violate a) or b).