Skip to content

Commit 2be9ca0

Browse files
opacamopacam
committed
Merge branch 'master' of https://github.com/kivy/python-for-android into python-core-bootstraps
2 parents a4fca80 + ba82d8a commit 2be9ca0

File tree

107 files changed

+2098
-1368
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

107 files changed

+2098
-1368
lines changed

‎.travis.yml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -18,7 +18,7 @@ env:
1818
- ANDROID_SDK_HOME=/opt/android/android-sdk
1919
- ANDROID_NDK_HOME=/opt/android/android-ndk
2020
matrix:
21-
- COMMAND='. venv/bin/activate && cd testapps/ && python setup_testapp_python3.py apk --sdk-dir $ANDROID_SDK_HOME --ndk-dir $ANDROID_NDK_HOME --requirements sdl2,pyjnius,kivy,python3'
21+
- COMMAND='. venv/bin/activate && cd testapps/ && python setup_testapp_python3_sqlite_openssl.py apk --sdk-dir $ANDROID_SDK_HOME --ndk-dir $ANDROID_NDK_HOME --requirements libffi,sdl2,pyjnius,kivy,python3,openssl,requests,sqlite3,setuptools'
2222
# overrides requirements to skip `peewee` pure python module, see:
2323
# https://github.com/kivy/python-for-android/issues/1263#issuecomment-390421054
2424
- COMMAND='. venv/bin/activate && cd testapps/ && python setup_testapp_python2_sqlite_openssl.py apk --sdk-dir $ANDROID_SDK_HOME --ndk-dir $ANDROID_NDK_HOME --requirements sdl2,pyjnius,kivy,python2,openssl,requests,sqlite3,setuptools'

‎Dockerfile.py2

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -26,7 +26,7 @@ RUN apt -y update -qq \
2626

2727

2828
ENV ANDROID_NDK_HOME="${ANDROID_HOME}/android-ndk"
29-
ENV ANDROID_NDK_VERSION="16b"
29+
ENV ANDROID_NDK_VERSION="17c"
3030
ENV ANDROID_NDK_HOME_V="${ANDROID_NDK_HOME}-r${ANDROID_NDK_VERSION}"
3131

3232
# get the latest version from https://developer.android.com/ndk/downloads/index.html
@@ -104,7 +104,7 @@ RUN dpkg --add-architecture i386 \
104104
# specific recipes dependencies (e.g. libffi requires autoreconf binary)
105105
RUN apt -y update -qq \
106106
&& apt -y install -qq --no-install-recommends \
107-
autoconf automake cmake gettext libltdl-dev libtool pkg-config \
107+
libffi-dev autoconf automake cmake gettext libltdl-dev libtool pkg-config \
108108
&& apt -y autoremove \
109109
&& apt -y clean
110110

‎Dockerfile.py3

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -26,7 +26,7 @@ RUN apt -y update -qq \
2626

2727

2828
ENV ANDROID_NDK_HOME="${ANDROID_HOME}/android-ndk"
29-
ENV ANDROID_NDK_VERSION="16b"
29+
ENV ANDROID_NDK_VERSION="17c"
3030
ENV ANDROID_NDK_HOME_V="${ANDROID_NDK_HOME}-r${ANDROID_NDK_VERSION}"
3131

3232
# get the latest version from https://developer.android.com/ndk/downloads/index.html
@@ -104,7 +104,7 @@ RUN dpkg --add-architecture i386 \
104104
# specific recipes dependencies (e.g. libffi requires autoreconf binary)
105105
RUN apt -y update -qq \
106106
&& apt -y install -qq --no-install-recommends \
107-
autoconf automake cmake gettext libltdl-dev libtool pkg-config \
107+
libffi-dev autoconf automake cmake gettext libltdl-dev libtool pkg-config \
108108
&& apt -y autoremove \
109109
&& apt -y clean
110110

‎README.rst

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -35,6 +35,25 @@ issues and PRs relating to this branch are still accepted. However, the
3535
new toolchain contains all the same functionality via the built in
3636
pygame bootstrap.
3737

38+
In the last quarter of 2018 the python recipes has been changed, the new recipe
39+
for python3 (3.7.1) has a new build system which has been applied to the ancient
40+
python recipe, allowing us to bump the python2 version number to 2.7.15. This
41+
change, unifies the build process for both python recipes, and probably solve
42+
some issues detected over the years, but unfortunately, this change breaks the
43+
pygame bootstrap (in a near future we will fix it or remove it). Also should be
44+
mentioned that the old python recipe is still usable, and has been renamed to
45+
`python2legacy`. This `python2legacy` recipe allow us to build with a minimum
46+
api lower than 21, which is not the case for the new python recipes which are
47+
limited to a minimum api of 21. All this work has been done using android ndk
48+
version r17c, and your build should success with that version...but be aware
49+
that the project is in constant development so...the ndk version will change
50+
at some time.
51+
52+
Those mentioned changes has been done this way to make easier the transition
53+
between python3 and python2. We will slowly phase out python2 support
54+
towards 2020...so...if you are using python2 in your projects you should
55+
consider to migrate it into python3.
56+
3857
Documentation
3958
=============
4059

‎doc/source/quickstart.rst

Lines changed: 20 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -93,11 +93,27 @@ named ``tools``, and you will need to run extra commands to install
9393
the SDK packages needed.
9494

9595
For Android NDK, note that modern releases will only work on a 64-bit
96-
operating system. If you are using a 32-bit distribution (or hardware),
97-
the latest useable NDK version is r10e, which can be downloaded here:
96+
operating system. The minimal, and recommended, NDK version to use is r17c:
97+
98+
- `Go to ndk downloads page <https://developer.android.com/ndk/downloads/>`_
99+
- Windows users should create a virtual machine with an GNU Linux os
100+
installed, and then you can follow the described instructions from within
101+
your virtual machine.
102+
103+
If you are using a 32-bit distribution (or hardware),
104+
the latest usable NDK version is r10e, which can be downloaded here:
98105

99106
- `Legacy 32-bit Linux NDK r10e <http://dl.google.com/android/ndk/android-ndk-r10e-linux-x86.bin>`_
100107

108+
.. warning::
109+
**32-bit distributions**
110+
111+
Since the python2 recipe updated to version 2.7.15, the build system has
112+
been changed and you should use an old release of python-for-android, which
113+
contains the legacy python recipe (v2.7.2). The last python-for-android
114+
release with the legacy version of python is version
115+
`0.6.0 <https://github.com/kivy/python-for-android/archive/0.6.0.zip>`_.
116+
101117
First, install a platform to target (you can also replace ``27`` with
102118
a different platform number, this will be used again later)::
103119

@@ -113,8 +129,8 @@ Then, you can edit your ``~/.bashrc`` or other favorite shell to include new env
113129

114130
# Adjust the paths!
115131
export ANDROIDSDK="$HOME/Documents/android-sdk-27"
116-
export ANDROIDNDK="$HOME/Documents/android-ndk-r10e"
117-
export ANDROIDAPI="27" # Target API version of your application
132+
export ANDROIDNDK="$HOME/Documents/android-ndk-r17c"
133+
export ANDROIDAPI="26" # Target API version of your application
118134
export NDKAPI="19" # Minimum supported API version of your application
119135
export ANDROIDNDKVER="r10e" # Version of the NDK you installed
120136

‎pythonforandroid/archs.py

Lines changed: 67 additions & 28 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,6 @@
1-
from os.path import (exists, join, dirname)
1+
from os.path import (exists, join, dirname, split)
22
from os import environ, uname
3+
from glob import glob
34
import sys
45
from distutils.spawn import find_executable
56

@@ -30,13 +31,30 @@ def include_dirs(self):
3031
d.format(arch=self))
3132
for d in self.ctx.include_dirs]
3233

33-
def get_env(self, with_flags_in_cc=True):
34+
@property
35+
def target(self):
36+
target_data = self.command_prefix.split('-')
37+
return '-'.join(
38+
[target_data[0], 'none', target_data[1], target_data[2]])
39+
40+
def get_env(self, with_flags_in_cc=True, clang=False):
3441
env = {}
3542

36-
env['CFLAGS'] = ' '.join([
37-
'-DANDROID', '-mandroid', '-fomit-frame-pointer'
38-
' -D__ANDROID_API__={}'.format(self.ctx.ndk_api),
39-
])
43+
cflags = [
44+
'-DANDROID',
45+
'-fomit-frame-pointer',
46+
'-D__ANDROID_API__={}'.format(self.ctx.ndk_api)]
47+
if not clang:
48+
cflags.append('-mandroid')
49+
else:
50+
cflags.append('-target ' + self.target)
51+
toolchain = '{android_host}-{toolchain_version}'.format(
52+
android_host=self.ctx.toolchain_prefix,
53+
toolchain_version=self.ctx.toolchain_version)
54+
toolchain = join(self.ctx.ndk_dir, 'toolchains', toolchain, 'prebuilt', 'linux-x86_64')
55+
cflags.append('-gcc-toolchain {}'.format(toolchain))
56+
57+
env['CFLAGS'] = ' '.join(cflags)
4058
env['LDFLAGS'] = ' '
4159

4260
sysroot = join(self.ctx._ndk_dir, 'sysroot')
@@ -45,6 +63,8 @@ def get_env(self, with_flags_in_cc=True):
4563
# https://android.googlesource.com/platform/ndk/+/ndk-r15-release/docs/UnifiedHeaders.md
4664
env['CFLAGS'] += ' -isystem {}/sysroot/usr/include/{}'.format(
4765
self.ctx.ndk_dir, self.ctx.toolchain_prefix)
66+
env['CFLAGS'] += ' -I{}/sysroot/usr/include/{}'.format(
67+
self.ctx.ndk_dir, self.command_prefix)
4868
else:
4969
sysroot = self.ctx.ndk_platform
5070
env['CFLAGS'] += ' -I{}'.format(self.ctx.ndk_platform)
@@ -82,8 +102,20 @@ def get_env(self, with_flags_in_cc=True):
82102
env['NDK_CCACHE'] = self.ctx.ccache
83103
env.update({k: v for k, v in environ.items() if k.startswith('CCACHE_')})
84104

85-
cc = find_executable('{command_prefix}-gcc'.format(
86-
command_prefix=command_prefix), path=environ['PATH'])
105+
if clang:
106+
llvm_dirname = split(
107+
glob(join(self.ctx.ndk_dir, 'toolchains', 'llvm*'))[-1])[-1]
108+
clang_path = join(self.ctx.ndk_dir, 'toolchains', llvm_dirname,
109+
'prebuilt', 'linux-x86_64', 'bin')
110+
environ['PATH'] = '{clang_path}:{path}'.format(
111+
clang_path=clang_path, path=environ['PATH'])
112+
exe = join(clang_path, 'clang')
113+
execxx = join(clang_path, 'clang++')
114+
else:
115+
exe = '{command_prefix}-gcc'.format(command_prefix=command_prefix)
116+
execxx = '{command_prefix}-g++'.format(command_prefix=command_prefix)
117+
118+
cc = find_executable(exe, path=environ['PATH'])
87119
if cc is None:
88120
print('Searching path are: {!r}'.format(environ['PATH']))
89121
raise BuildInterruptingException(
@@ -93,20 +125,20 @@ def get_env(self, with_flags_in_cc=True):
93125
'installed. Exiting.')
94126

95127
if with_flags_in_cc:
96-
env['CC'] = '{ccache}{command_prefix}-gcc {cflags}'.format(
97-
command_prefix=command_prefix,
128+
env['CC'] = '{ccache}{exe} {cflags}'.format(
129+
exe=exe,
98130
ccache=ccache,
99131
cflags=env['CFLAGS'])
100-
env['CXX'] = '{ccache}{command_prefix}-g++ {cxxflags}'.format(
101-
command_prefix=command_prefix,
132+
env['CXX'] = '{ccache}{execxx} {cxxflags}'.format(
133+
execxx=execxx,
102134
ccache=ccache,
103135
cxxflags=env['CXXFLAGS'])
104136
else:
105-
env['CC'] = '{ccache}{command_prefix}-gcc'.format(
106-
command_prefix=command_prefix,
137+
env['CC'] = '{ccache}{exe}'.format(
138+
exe=exe,
107139
ccache=ccache)
108-
env['CXX'] = '{ccache}{command_prefix}-g++'.format(
109-
command_prefix=command_prefix,
140+
env['CXX'] = '{ccache}{execxx}'.format(
141+
execxx=execxx,
110142
ccache=ccache)
111143

112144
env['AR'] = '{}-ar'.format(command_prefix)
@@ -123,12 +155,13 @@ def get_env(self, with_flags_in_cc=True):
123155
env['READELF'] = '{}-readelf'.format(command_prefix)
124156
env['NM'] = '{}-nm'.format(command_prefix)
125157

126-
hostpython_recipe = Recipe.get_recipe('hostpython2', self.ctx)
127-
128-
# This hardcodes python version 2.7, needs fixing
158+
hostpython_recipe = Recipe.get_recipe(
159+
'host' + self.ctx.python_recipe.name, self.ctx)
129160
env['BUILDLIB_PATH'] = join(
130161
hostpython_recipe.get_build_dir(self.arch),
131-
'build', 'lib.linux-{}-2.7'.format(uname()[-1]))
162+
'build', 'lib.linux-{}-{}'.format(
163+
uname()[-1], self.ctx.python_recipe.major_minor_version_string)
164+
)
132165

133166
env['PATH'] = environ['PATH']
134167

@@ -147,12 +180,18 @@ class ArchARM(Arch):
147180
command_prefix = 'arm-linux-androideabi'
148181
platform_dir = 'arch-arm'
149182

183+
@property
184+
def target(self):
185+
target_data = self.command_prefix.split('-')
186+
return '-'.join(
187+
['armv7a', 'none', target_data[1], target_data[2]])
188+
150189

151190
class ArchARMv7_a(ArchARM):
152191
arch = 'armeabi-v7a'
153192

154-
def get_env(self, with_flags_in_cc=True):
155-
env = super(ArchARMv7_a, self).get_env(with_flags_in_cc)
193+
def get_env(self, with_flags_in_cc=True, clang=False):
194+
env = super(ArchARMv7_a, self).get_env(with_flags_in_cc, clang=clang)
156195
env['CFLAGS'] = (env['CFLAGS'] +
157196
(' -march=armv7-a -mfloat-abi=softfp '
158197
'-mfpu=vfp -mthumb'))
@@ -166,8 +205,8 @@ class Archx86(Arch):
166205
command_prefix = 'i686-linux-android'
167206
platform_dir = 'arch-x86'
168207

169-
def get_env(self, with_flags_in_cc=True):
170-
env = super(Archx86, self).get_env(with_flags_in_cc)
208+
def get_env(self, with_flags_in_cc=True, clang=False):
209+
env = super(Archx86, self).get_env(with_flags_in_cc, clang=clang)
171210
env['CFLAGS'] = (env['CFLAGS'] +
172211
' -march=i686 -mtune=intel -mssse3 -mfpmath=sse -m32')
173212
env['CXXFLAGS'] = env['CFLAGS']
@@ -180,8 +219,8 @@ class Archx86_64(Arch):
180219
command_prefix = 'x86_64-linux-android'
181220
platform_dir = 'arch-x86_64'
182221

183-
def get_env(self, with_flags_in_cc=True):
184-
env = super(Archx86_64, self).get_env(with_flags_in_cc)
222+
def get_env(self, with_flags_in_cc=True, clang=False):
223+
env = super(Archx86_64, self).get_env(with_flags_in_cc, clang=clang)
185224
env['CFLAGS'] = (env['CFLAGS'] +
186225
' -march=x86-64 -msse4.2 -mpopcnt -m64 -mtune=intel')
187226
env['CXXFLAGS'] = env['CFLAGS']
@@ -194,8 +233,8 @@ class ArchAarch_64(Arch):
194233
command_prefix = 'aarch64-linux-android'
195234
platform_dir = 'arch-arm64'
196235

197-
def get_env(self, with_flags_in_cc=True):
198-
env = super(ArchAarch_64, self).get_env(with_flags_in_cc)
236+
def get_env(self, with_flags_in_cc=True, clang=False):
237+
env = super(ArchAarch_64, self).get_env(with_flags_in_cc, clang=clang)
199238
incpath = ' -I' + join(dirname(__file__), 'includes', 'arm64-v8a')
200239
env['EXTRA_CFLAGS'] = incpath
201240
env['CFLAGS'] += incpath

‎pythonforandroid/bootstrap.py

Lines changed: 7 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -269,14 +269,13 @@ def strip_libraries(self, arch):
269269
return
270270
strip = sh.Command(strip)
271271

272-
if self.ctx.python_recipe.name == 'python2':
273-
filens = shprint(sh.find, join(self.dist_dir, 'private'),
274-
join(self.dist_dir, 'libs'),
275-
'-iname', '*.so', _env=env).stdout.decode('utf-8')
276-
else:
277-
filens = shprint(sh.find, join(self.dist_dir, '_python_bundle', '_python_bundle', 'modules'),
278-
join(self.dist_dir, 'libs'),
279-
'-iname', '*.so', _env=env).stdout.decode('utf-8')
272+
libs_dir = join(self.dist_dir, '_python_bundle',
273+
'_python_bundle', 'modules')
274+
if self.ctx.python_recipe.name == 'python2legacy':
275+
libs_dir = join(self.dist_dir, 'private')
276+
filens = shprint(sh.find, libs_dir, join(self.dist_dir, 'libs'),
277+
'-iname', '*.so', _env=env).stdout.decode('utf-8')
278+
280279
logger.info('Stripping libraries in private dir')
281280
for filen in filens.split('\n'):
282281
try:

‎pythonforandroid/bootstraps/common/build/build.py

Lines changed: 1 addition & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -46,7 +46,6 @@ def get_bootstrap_name():
4646
# Try to find a host version of Python that matches our ARM version.
4747
PYTHON = join(curdir, 'python-install', 'bin', 'python.host')
4848
if not exists(PYTHON):
49-
print('Could not find hostpython, will not compile to .pyo (this is normal with python3)')
5049
PYTHON = None
5150

5251
BLACKLIST_PATTERNS = [
@@ -151,7 +150,6 @@ def make_python_zip():
151150

152151
if not exists('private'):
153152
print('No compiled python is present to zip, skipping.')
154-
print('this should only be the case if you are using the CrystaX python')
155153
return
156154

157155
global python_files
@@ -465,7 +463,7 @@ def make_package(args):
465463
join(res_dir, 'values/strings.xml'),
466464
**render_args)
467465

468-
if exists("custom_rules.tmpl.xml"):
466+
if exists(join("templates", "custom_rules.tmpl.xml")):
469467
render(
470468
'custom_rules.tmpl.xml',
471469
'custom_rules.xml',

‎pythonforandroid/bootstraps/common/build/jni/application/src/start.c

Lines changed: 13 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -104,6 +104,10 @@ int main(int argc, char *argv[]) {
104104
LOGP(env_argument);
105105
chdir(env_argument);
106106

107+
#if PY_MAJOR_VERSION < 3
108+
Py_NoSiteFlag=1;
109+
#endif
110+
107111
Py_SetProgramName(L"android_python");
108112

109113
#if PY_MAJOR_VERSION >= 3
@@ -144,10 +148,6 @@ int main(int argc, char *argv[]) {
144148
#if PY_MAJOR_VERSION >= 3
145149
wchar_t *wchar_paths = Py_DecodeLocale(paths, NULL);
146150
Py_SetPath(wchar_paths);
147-
#else
148-
char *wchar_paths = paths;
149-
LOGP("Can't Py_SetPath in python2, so crystax python2 doesn't work yet");
150-
exit(1);
151151
#endif
152152

153153
LOGP("set wchar paths...");
@@ -161,6 +161,12 @@ int main(int argc, char *argv[]) {
161161
Py_Initialize();
162162

163163
#if PY_MAJOR_VERSION < 3
164+
// Can't Py_SetPath in python2 but we can set PySys_SetPath, which must
165+
// be applied after Py_Initialize rather than before like Py_SetPath
166+
#if PY_MICRO_VERSION >= 15
167+
// Only for python native-build
168+
PySys_SetPath(paths);
169+
#endif
164170
PySys_SetArgv(argc, argv);
165171
#endif
166172

@@ -183,7 +189,9 @@ int main(int argc, char *argv[]) {
183189
*/
184190
PyRun_SimpleString("import sys, posix\n");
185191
if (dir_exists("lib")) {
186-
/* If we built our own python, set up the paths correctly */
192+
/* If we built our own python, set up the paths correctly.
193+
* This is only the case if we are using the python2legacy recipe
194+
*/
187195
LOGP("Setting up python from ANDROID_APP_PATH");
188196
PyRun_SimpleString("private = posix.environ['ANDROID_APP_PATH']\n"
189197
"argument = posix.environ['ANDROID_ARGUMENT']\n"

0 commit comments

Comments
 (0)