Skip to content

Fetching DF segfaults when outputtypehandler is present #486

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
mauropagano opened this issue Apr 24, 2025 · 11 comments
Closed

Fetching DF segfaults when outputtypehandler is present #486

mauropagano opened this issue Apr 24, 2025 · 11 comments
Labels
bug Something isn't working patch available

Comments

@mauropagano
Copy link

mauropagano commented Apr 24, 2025 •
edited
Loading

  1. What versions are you using?
    3.1.0, repro in python 3.9 and 3.12, issue is on client side I think, DB version is irrelevant

  2. Is it an error or a hang or a crash?
    Hard crash, segfault

  3. What error(s) or behavior you are seeing?
    Segmentation fault

  4. Does your application call init_oracle_client()?
    Reproduces both in thin and thick mode

  5. Include a runnable Python script that shows the problem.

import oracledb

# this is copy&paste from doc                                                                                                                                                                                                                                                                                                                                                                
def output_type_handler(cursor, metadata):
    def out_converter(d):
        if isinstance(d, str):
            return f"{d} was a string"
        else:
            return f"{d} was not a string"

    if metadata.type_code is oracledb.DB_TYPE_NUMBER:
        return cursor.var(
            oracledb.DB_TYPE_VARCHAR,
            arraysize=cursor.arraysize,
            outconverter=out_converter,
        )


conn = oracledb.connect("...")
# comment out the line below and the issue goes away
conn.outputtypehandler = output_type_handler
sql = "select 1 n from dual"
t = conn.fetch_df_all(sql)
print(t)
@mauropagano mauropagano added the bug Something isn't working label Apr 24, 2025
@cjbj
Copy link
Member

cjbj commented Apr 24, 2025

Thanks for the report. We can replicate it.

One item on our task list is to look at type handling etc. Were you wanting something similar, or does your code base have the handler/converter in place for non-dataframe queries?

@mauropagano
Copy link
Author

mauropagano commented Apr 24, 2025 •
edited
Loading

I do have already a handler in place for non-df, I hit this when porting existing code to df fetching.

(I like that issue you linked! 😉 )

@anthony-tuininga
Copy link
Member

This patch can be used to avoid the segfault. More effort is needed to get to the true source of the issue, but this is will "work" for you. The other workaround is to simply not specify an output type handler -- since it won't do you any good anyway!

diff --git a/src/oracledb/impl/base/cursor.pyx b/src/oracledb/impl/base/cursor.pyx
index 3a78fa40..0815be2b 100644
--- a/src/oracledb/impl/base/cursor.pyx
+++ b/src/oracledb/impl/base/cursor.pyx
@@ -306,6 +306,8 @@ cdef class BaseCursorImpl:
         cdef:
             BaseConnImpl conn_impl
             object type_handler
+        if self.fetching_arrow:
+            return None
         if self.outputtypehandler is not None:
             type_handler = self.outputtypehandler
         else:

anthony-tuininga added a commit that referenced this issue May 12, 2025
@anthony-tuininga
Fixed a bug resulting in a segfault when attempting to use an output
1b5a4cb 
type handler while fetching data frames (#486).
@anthony-tuininga
Copy link
Member

I have pushed a patch that corrects this issue and have initated a build from which you can download pre-built development wheels once it completes. You can also build from source if you prefer. If you can test your scenario and confirm the patch works as expected, that would be appreciated!

@mauropagano
Copy link
Author

Thanks @anthony-tuininga , will test as soon as it's out!

@mauropagano
Copy link
Author

@anthony-tuininga just to be explicit, the fix is supposed to prevent the crash OR obey the outputtypehandler specified?

@anthony-tuininga
Copy link
Member

It prevents the crash. It does not obey the output type handler -- doing so would incur significant overhead in a number of cases and in others there may not be an appropriate data type. The data types for Arrow do not match Oracle database types.

@mauropagano
Copy link
Author

Makes sense, thanks! Also likely faster to do it in Arrow anyway

Will this be documented or at least print a warning?
I'm thinking I know it's ignored, might not be obvious for other users

@anthony-tuininga
Copy link
Member

@cjbj will take a look at the documentation and see if something needs to be changed there. Thanks!

@cjbj
Copy link
Member

cjbj commented May 12, 2025

The general plan is add some kind of support. Suggestions about what it looks like & behaves are welcome.

anthony-tuininga added a commit that referenced this issue May 15, 2025
@anthony-tuininga
Fixed a bug resultging in a segfault when attempting to use an output
c73ba53 
type handler while fetching data frames (#486).
@anthony-tuininga
Copy link
Member

This was included in python-oracledb 3.1.1 which was just released.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working patch available
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@cjbj @anthony-tuininga @mauropagano