Fix #61072: inconsistent fullmatch results with regex alternation #61343
Conversation
…nation in PyArrow strings Fixes an issue where regex patterns with alternation (|) produce different results between str dtype and string[pyarrow] dtype. When using patterns like "(as)|(as)", PyArrow implementation would incorrectly match "asdf" while Python's implementation correctly rejects it. The fix adds special handling to ensure alternation patterns are properly parenthesized when using PyArrow-backed strings
5463b28 to
9b917a2
Compare
| ) | ||
| if is_pyarrow and "|" in pat: | ||
|
|
||
| def _is_fully_wrapped(pattern): |
There was a problem hiding this comment.
This appears to be an improper solution to me. Can you explain why the current code path fails?
There was a problem hiding this comment.
I have already fixed the error in the pull request, why do u think is it a improper solution?
Thanks for the feedback! I agree the issue stems from PyArrow’s match_substring_regex not enforcing full-string matching. To ensure consistent Series.str.fullmatch behavior, I propose a pandas workaround: wrap patterns in ^...$ for PyArrow arrays (e.g., pat = f"^{pat}$") while keeping the | grouping logic. I’ll: Implement the workaround. |
in PyArrow strings
Fixes an issue where regex patterns with alternation (|) produce different results between str dtype and string[pyarrow] dtype. When using patterns like "(as)|(as)", PyArrow implementation would incorrectly match "asdf" while Python's implementation correctly rejects it. The fix adds special handling to ensure alternation patterns are properly parenthesized when using PyArrow-backed strings