Hello @petronny! Thank you for your contribution 💪

As it's your first contribution be sure to check out the patch review checklist.

If you're fixing a ticket from Trac make sure to set the "Has patch" flag and include a link to this PR in the ticket!

If you have any design or process questions then you can ask in the Django forum.

Welcome aboard ⛵️!

@petronny We should change tests/requirements/oracle.txt.

buildbot, test on oracle.

@petronny Please push fixes in separate commits. It's hard to review after force-push.

Please push fixes in separate commits.

OK.

The last change only replaces oracledb.OBJECT with oracle.DB_TYPE_OBJECT. These two are same but the first one is deprecated.

This won't fix the failed test in oragis. I've created an issue oracle/python-oracledb#43 for it on oracledb.

view_tests.tests.test_debug.DebugViewQueriesAllowedTests.test_handle_db_exception crashes when we introspect traceback frames after catching DatabaseError. Precisely when we try to get f_back (the next outer frame) and assign it to the current_frame:

On Jenkins f_back is missing after ThinCursorImpl.execute and it crashes when assigning current_frame = current_frame.f_back:

0: <frame at 0x7f1774ed3e20, file 'src/oracledb/impl/thin/cursor.pyx', line 138, code oracledb.thin_impl.ThinCursorImpl.execute>
1: None

on my local machine:

0: <frame at 0x7f234f53ad40, file 'src/oracledb/impl/thin/cursor.pyx', line 138, code oracledb.thin_impl.ThinCursorImpl.execute>
1: <frame at 0x7f234e91d300, file '/lib/python3.10/site-packages/oracledb/cursor.py', line 378, code execute>
2: <frame at 0x7f234ea2ebc0, file '/django/db/backends/oracle/base.py', line 553, code execute>
...

So it looks like Python 3.11 changed stuff with the frame object:

https://docs.python.org/3/whatsnew/3.11.html#pyframeobject-3-11-hiding

It also looks like python-oracledb is using cython for which there is the following issue: cython/cython#5222

Yes, python-oracledb is using cython. So does this mean something needs to change in Django to handle the changes to the frames? Is this specific to Python 3.11 then?

I tried running the test suite on my machine using a local database (running 21.3) and it raised a number of exceptions but didn't segfault. I don't know precisely how you are running the test suite and what you have in your settings file. If you are able to share that information I can try replicate the issue here. Currently I see the following results:

Ran 16529 tests in 2056.447s

FAILED (failures=44, errors=3, skipped=1255, expected failures=6)

This is the contents of my settings file:

# This is an Oracle test settings file for use with the Django test suite.
#

DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.oracle',
        'NAME': 'localhost/DB21PDB1',
        'USER': 'django',
        'PASSWORD': 'dev',
        'TEST': {
            'USER': 'default_django_test',
            'TBLSPACE': 'default_django_test_ts',
            'TBLSPACE_TMP': 'default_django_temp_ts'
        }
    },
    'other': {
        'ENGINE': 'django.db.backends.oracle',
        'NAME': 'localhost/DB21PDB1',
        'USER': 'django',
        'PASSWORD': 'dev',
        'TEST': {
            'USER': 'other_django_test',
            'TBLSPACE': 'other_django_test_ts',
            'TBLSPACE_TMP': 'other_django_temp_ts'
        }
    },
}

DEFAULT_AUTO_FIELD='django.db.models.BigAutoField'
USE_TZ=False
SECRET_KEY="my_secret"

If any of this is incorrect (for example, a lot of failures are about the "wrong" AutoField) then please advise. I'd like to get to the bottom of this issue if at all possible!

Ok. I tweaked the settings file to follow the suggested file from @petronny. I now get this:

Ran 16529 tests in 1947.899s

FAILED (failures=23, errors=1, skipped=1255, expected failures=6)

Running the test that causes the segfault runs just fine on my machine. This is on Linux x86_64, Python 3.11.4 with the python-oracledb 1.3.2 prebuilt binary that was released publicly. Adding print statements in the get_traceback_frame_variables() method shows that it is indeed being called without any difficulty. @felixxm, what platform does the CI use, what environment does it set, etc. Is there a configuration file somewhere? I'll see if I can find it myself but I am not familiar with Django build procedures.

You can see the CI job here: django-oracle

The configuration of interest would be database=oracle19,label=oracle,python=python3.11.

And looking at the latest build, specifically the full console output, we can see that it uses:

• Ubuntu 20.04 LTS (Focal Fossa)
• CPython 3.11.1
• x86-64

It may be that this has been fixed in 3.11.2 - see release notes. This sounds suspiciously relevant:

• gh-99110: Initialize frame->previous in frameobject.c to fix a segmentation fault when accessing frames created by PyFrame_New().

If you can install Python 3.11.1 and reproduce and then see if 3.11.2 fixes the issue it'd be helpful to confirm.

I could download and install 3.11.1 but is there a reason why the CI job doesn't get upgraded to 3.11.4? I am unable to replicate with 3.11.4 and your releaase notes reference does indeed seem quite relevant. Cython is likely using PyFrame_New() to build its frames.

I bumped Python versions on CI nodes, we will see.

All tests have passed with python 3.11.4. 🎉

1. Do we need to make a note stating that version 3.11.1 should not be used?
2. Do we need to also test the thick mode?

Do we need to make a note stating that version 3.11.1 should not be used?

No, we only officially supports the latest version.

Do we need to also test the thick mode?

TBH, we don't have the time and resources to extensively test for Oracle drivers.

TBH, we don't have the time and resources to extensively test for Oracle drivers.

Then do we need to note that we only support the thin mode?

Then do we need to note that we only support the thin mode?

It should work fine in Thick mode (as it has been doing so for many years), so I don't think the word 'support' is useful to add. You could say it was tested with Thin mode if you want to say anything. If any Thick mode issues are logged we can address them.

Rebased.

@petronny Thanks 👍 Welcome aboard ⛵ I pushed small edits.

Yay! A big thanks to all.

Superb. Let us (@oracle) know if there are features that require attention.