ctypes.pythonapi issues; getting AttributeError: undefined symbol #1866
Comments
|
Not sure what's going on here, but it isn't impossible that this symbol is missing for some environment-related reason. I'll try to look into it. Presumably this has worked for some people though, @AndreMiras any thoughts? |
|
I wanted to chime in and say I ran into the same problem, with a nearly identical stack trace. Any help is appreciated! |
|
I'm currently using |
|
I've tried playing with some pycryptodome primitives with the following code. from typing import List
import Crypto
from Crypto.Cipher import AES
from Crypto.Random import get_random_bytes
def encrypt(key: bytes, data: bytes) -> List[bytes]:
"""
Given a key and data returns a list containing nonce, tag and ciphertext.
"""
cipher = AES.new(key, AES.MODE_EAX)
ciphertext, tag = cipher.encrypt_and_digest(data)
encrypted = [x for x in (cipher.nonce, tag, ciphertext)]
return encrypted
def decrypt(key: bytes, data: bytes) -> bytes:
"""
Given a key and data (nonce, tag and ciphertext), returns decrypted data.
"""
nonce, tag, ciphertext = data
cipher = AES.new(key, AES.MODE_EAX, nonce)
decrypted = cipher.decrypt_and_verify(ciphertext, tag)
return decrypted
def main():
print(f'Crypto.version_info: {Crypto.version_info}')
key = get_random_bytes(16)
data = b'Testing pycryptodome on Python for Android'
print(f'data: {data}')
encrypted = encrypt(key, data)
print(f'encrypted: {encrypted}')
decrypted = decrypt(key, encrypted)
print(f'decrypted: {decrypted}')
assert data == decrypted
if __name__ == '__main__':
main()And it produced the following output on device. So I guess next up is to mess up with |
|
And I could confirm the issue with Code was: import ctypes
def main():
print(f'ctypes.__version__: {ctypes.__version__}')
print(f'ctypes.pythonapi.PyObject_GetBuffer: {ctypes.pythonapi.PyObject_GetBuffer}')
if __name__ == '__main__':
main()Edit: And in the p4a build. So it's as if we were not linking to it. |
|
Initially, in pycryptodome, ctypes is used as fallback to cffi, because of the "ImportError: CFFI with optimize=2 fails due to pycparser bug." Is there any way to make cffi work in python-for-android, so that pycryptodome can work despite ctypes issues? I've seen in the current master some cmd line params like "no_compile_pyo" and "optimize_python", but I don't really get how/when they are used in the compilation toolchain (the pycryptodome files I find in the build directory are "pyc" and not "pyo", but they seem to be stripped from their docstrings anyway). On the other hand, the fallback from cffi to ctypes, above, looks at sys.flags.optimize only, not the real state of some files... is there any way to make p4a run the python interpreter in non optimized mode, to see if it solves problems? EDIT: I forced both flags in p4a to skip bytecode optimization, but files compiled into "_python_bundle/" are still without docstring, and anyway the python interpreter seems to be forced to use PYTHONOPTIMIZED=2 in multiple parts of the toolchain... so cffi seems unusable. |
|
This is not related to the main ctypes.pythonapi issue (which has maybe existed for very long), but if anyone needs to compile the latest pycryptodome for android, commenting out a few "Setenv("PYTHONOPTIMIZE", "2")" in start.c and in PythonActivity.java files allows it to use cffi and not hit that ctypes bug. |
|
For what it's worth, I've just noticed that other cross-compilation attempts to android did precisely patch "pythonapi" in ctypes, maybe it's what's missing in P4A? The post: http://mdqinc.com/blog/2011/09/cross-compiling-python-for-android/ The patch: https://bitbucket.org/gabomdq/ignifuga/raw/tip/tools/patches/python.android.diff |
|
I'm facing the exact same issue. In which directory can I find the start.c that you've modified? Also what do I have to do exactly to fix the bug? Also @AndreMiras does downgrading pycryptodome help or does downgrading ctypes help (if that is possible) P.S. I am using pycryptodomex, but I don't think that should make a difference since the only difference I am aware of is the namespace. |
|
@ShreeSinghi my workaround works by searching the line |
|
Some other project's patch of python to support ctypes on Android is now here : https://github.com/gabomdq/ignifuga/blob/master/tools/patches/python.android.diff Copy here in case it disappears again |
|
For what it's worth : I've monkey-patched ctypes from the beginning of my main app, to force the loading of the proper python DLL, and it seems to work fine (PyCryptodome 3.9.9 works with this); thus no need to rely on CFFI by disabling PYTHONOPTIMIZE in P4A. What would be the proper way to integrate this into P4A though? With this I could update the PyCryptodome recipe. |
|
I am having this error with Python 3.11.9 |
|
In January 2025 I still find the comment from pakal to be the solution. To clarify, here is what that solution comprises: In your python app's code, before importing anything from pycryptodome, insert the following two lines: import ctypes, sys
ctypes.pythonapi = ctypes.PyDLL("libpython%d.%d.so" % sys.version_info[:2]) # replaces ctypes.PyDLL(None)Here is a working example of main.py with pycryptodome==3.19.1: from kivymd.app import MDApp
from kivy.lang import Builder
class SampleApp(MDApp):
def build(self):
try:
# patch for pycryptodome, from https://github.com/kivy/python-for-android/issues/1866#issuecomment-927157780
import ctypes, sys
ctypes.pythonapi = ctypes.PyDLL("libpython%d.%d.so" % sys.version_info[:2]) # replaces ctypes.PyDLL(None)
import Crypto
text = f"{'.'.join([str(num) for num in Crypto.version_info])}"
except Exception as e:
text = str(e)
self.appKv=f'''
MDScreen:
MDLabel:
text:'{text}.'
multiline:True
color:"blue"
halign:'center'
'''
AppScreen=Builder.load_string(self.appKv)
return AppScreen
SampleApp().run() |
Using the new python3 toolchain, I am trying to use PyCryptodome (3.7+) as a dependency.
As in #1685, PyCryptodome >= 3.6.4 runs into compilation issues; I am trying to fix these too, though this is not that relevant here.
PyCryptodome >=3.6.0 crashes at runtime (since commit Legrandin/pycryptodome@f5aa2c1). So actually the currently pinned version in the recipe does not work:
python-for-android/pythonforandroid/recipes/pycryptodome/__init__.py
Line 5 in 80e4f05
The issue at runtime is with ctypes.
Say I have a main script that just does the following (https://github.com/Legrandin/pycryptodome/blob/95ccce7ae82d3a36f1a8652dd2c645222d0128dd/lib/Crypto/Util/_raw_api.py#L200):
This works with cpython on my laptop, but with the p4a-compiled python on Android it fails:
I have also tried to access some other attributes of
ctypes.pythonapi, such asPy_IncRef, which raises the same exception.Any idea what's going on here? Why is
ctypes.pythonapinon-functional?The text was updated successfully, but these errors were encountered: