Skip to content

gh-101955: Fix SystemError in possesive quantifier with alternative and group #111362

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Conversation

serhiy-storchaka
Copy link
Member

@serhiy-storchaka serhiy-storchaka commented Oct 26, 2023

@serhiy-storchaka
Copy link
Member Author

@animalize Could you please make a review of this PR?

@serhiy-storchaka serhiy-storchaka added the needs backport to 3.13 bugs and security fixes label May 9, 2024
@wjssz
Copy link

wjssz commented Oct 16, 2024

How about this patch? It's simpler:

 Modules/_sre/sre_lib.h | 20 ++++++++++++++++++++
 1 file changed, 20 insertions(+)

diff --git a/Modules/_sre/sre_lib.h b/Modules/_sre/sre_lib.h
index 97fbb0a75e5..2d124ab5ac3 100644
--- a/Modules/_sre/sre_lib.h
+++ b/Modules/_sre/sre_lib.h
@@ -1294,6 +1294,18 @@ SRE(match)(SRE_STATE* state, const SRE_CODE* pattern, int toplevel)
                pointer */
             state->ptr = ptr;
 
+            /* Set state->repeat to non-NULL */
+            ctx->u.rep = (SRE_REPEAT*) PyObject_Malloc(sizeof(*ctx->u.rep));
+            if (!ctx->u.rep) {
+                PyErr_NoMemory();
+                RETURN_ERROR(SRE_ERROR_MEMORY);
+            }
+            ctx->u.rep->count = -1;
+            ctx->u.rep->pattern = NULL;
+            ctx->u.rep->last_ptr = NULL;
+            ctx->u.rep->prev = state->repeat;
+            state->repeat = ctx->u.rep;
+
             /* Initialize Count to 0 */
             ctx->count = 0;
 
@@ -1308,6 +1320,10 @@ SRE(match)(SRE_STATE* state, const SRE_CODE* pattern, int toplevel)
                 }
                 else {
                     state->ptr = ptr;
+                    /* Restore state->repeat */
+                    state->repeat = ctx->u.rep->prev;
+                    PyObject_Free(ctx->u.rep);
+
                     RETURN_FAILURE;
                 }
             }
@@ -1380,6 +1396,10 @@ SRE(match)(SRE_STATE* state, const SRE_CODE* pattern, int toplevel)
                 }
             }
 
+            /* Restore state->repeat */
+            state->repeat = ctx->u.rep->prev;
+            PyObject_Free(ctx->u.rep);
+
             /* Evaluate Tail */
             /* Jump to end of pattern indicated by skip, and then skip
                the SUCCESS op code that follows it. */

@serhiy-storchaka
Copy link
Member Author

@wjssz, would not it leak memory in case of error (when user interrupted evaluation by pressing Ctrl-C)?

There are two differences of this PR from your patch:

  • It tries to avoid a memory leak in case of error.
  • It tries to not allocate a new SRE_REPEAT block if not needed.

But I have suspect that it does not work. It is difficult to test the former case, and the latter case may work incorrectly in case of nested possessive repetitions. So it may be safer to use your simpler patch and fix memory leaks later. Do you mind to create a PR @wjssz?

@serhiy-storchaka serhiy-storchaka marked this pull request as draft November 14, 2024 10:50
@wjssz
Copy link

wjssz commented Nov 14, 2024

I'm @animalize, this is my new GitHub account.

would not it leak memory in case of error (when user interrupted evaluation by pressing Ctrl-C)?

You are right. Maybe RETURN_FAILURE; is better than RETURN_ERROR(SRE_ERROR_MEMORY) for this reason.

So it may be safer to use your simpler patch and fix memory leaks later. Do you mind to create a PR @wjssz?

Sorry. I'm not going to sign CLA.
Don't forget to change from PyObject_Malloc/Free() to PyMem_Malloc/Free().

fix memory leaks later

I have a patch for this. I'll post it after.

@serhiy-storchaka
Copy link
Member Author

You are welcome @wjssz/@animalize. It was pleasure to work with you last time.

Right now I'm writing a code which would help to test user interruption and catch memory leaks.

@serhiy-storchaka serhiy-storchaka marked this pull request as ready for review November 18, 2024 10:24
@serhiy-storchaka
Copy link
Member Author

Now, after fixing leaks and using memory pool for SRE_REPEAT, the simpler patch #111362 (comment) can be used. Thanks @wjssz.

@serhiy-storchaka serhiy-storchaka merged commit f9c5573 into python:main Nov 18, 2024
41 checks passed
@miss-islington-app
Copy link

Thanks @serhiy-storchaka for the PR 🌮🎉.. I'm working now to backport this PR to: 3.12, 3.13.
🐍🍒⛏🤖

@serhiy-storchaka serhiy-storchaka deleted the re-possesive-alternative-and-group branch November 18, 2024 11:43
miss-islington pushed a commit to miss-islington/cpython that referenced this pull request Nov 18, 2024
…tive and group (pythonGH-111362)

(cherry picked from commit f9c5573)

Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
Co-authored-by: <wjssz@users.noreply.github.com>
@bedevere-app
Copy link

bedevere-app bot commented Nov 18, 2024

GH-126962 is a backport of this pull request to the 3.13 branch.

@bedevere-app bedevere-app bot removed the needs backport to 3.13 bugs and security fixes label Nov 18, 2024
miss-islington pushed a commit to miss-islington/cpython that referenced this pull request Nov 18, 2024
…tive and group (pythonGH-111362)

(cherry picked from commit f9c5573)

Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
Co-authored-by: <wjssz@users.noreply.github.com>
@bedevere-app
Copy link

bedevere-app bot commented Nov 18, 2024

GH-126963 is a backport of this pull request to the 3.12 branch.

@bedevere-app bedevere-app bot removed the needs backport to 3.12 only security fixes label Nov 18, 2024
serhiy-storchaka added a commit that referenced this pull request Nov 18, 2024
…ative and group (GH-111362) (GH-126963)

(cherry picked from commit f9c5573)

Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
serhiy-storchaka added a commit that referenced this pull request Nov 18, 2024
…ative and group (GH-111362) (GH-126962)

(cherry picked from commit f9c5573)

Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
ebonnal pushed a commit to ebonnal/cpython that referenced this pull request Jan 12, 2025
…tive and group (pythonGH-111362)

Co-authored-by: <wjssz@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
topic-regex type-bug An unexpected behavior, bug, or error
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants