Playing with `on_device_unit_test_app` #2046 #2073

opacam · 2020-02-29T20:31:07Z

This is not to be reviewed nor merged, of course commented if you want, but the purpose of this is to play a little with some recipes and the on_device_unit_test_app (#2046).

Some of the commits I will make in here, will be to test some active PRs. But I also will make some other commits which may be future PRs...if we like them. BTW, each gh-actions run should produce four apks, one for each arch, but probably it won't be the case...we will have failures on some/all archs (but this is fine...since is the first step to find a future solution). Also You should know that the generated artifacts via gh-actions are .zip compressed files with the apk inside them.

So, this is what I'm planning to do in here:

Don't stop builds in case that one of the matrix jobs fails
Test the current Pillow recipe, v5.2.0
Test the updated Pillow recipe, v7.0.0 ([recipes] Update Pillow to v7.0.0 #2067)
Test current matplotlib's recipe (I think that we have it broken alongside with some direct dependencies...due to the NDKr19 upgrade we made some months ago)
Add KiwisolverTestCase for on_device_unit_test_app and test it
Test kiwisolver fix (Fix recipe kiwisolver (add cppy recipe) #2075)
Test updated matplotlib's recipe (Fix matplotlib and update to v3.1.3 #2076)
Test the updated Pillow and Matplotlib recipes with updated libraries ([recipes] Update freetype & add zlib support #2068 and [recipes] Update harfbuzz to v2.6.4 #2069)
Test the updated numpy recipe (Update numpy to v1.18.1 (add cython recipe) #2077)

Notes:

I will completely ignore travis results in here, because this is to test gh-actions generated artifacts
The planning may change along the way, so I will try to keep it updated
This tests will be useful because we are testing with latest develop branch (at the time of writing), so we are testing with our current default python version 3.8.1 (Python 3.8 support on Android #2044)

So depending on the supplied recipes, a different test app will be ran. The supported test apps are: - kivy - flask - no-gui

for our CI providers and `rebuild_updated_recipes.py` because we recently removed the old test app that we made use of.

Remove sentence regarding python2 compatibility... because python2 it's almost at the end of his life

So we can get an apk via gh-actions Notes: - this way the reviewers can install the flask apk without making the build - these changes will be reverted in the next commit...since we want a kivy app as our default)

This reverts commit 0ed4d70.

in case that we detect that p4a will build an `service_only` app. Note: if we don't do this we wil get errors once we try to create our apk with the target distribution

and: - remove Python 2 test apps because: + we already test Python 2 with travis + Python 2 it's almost at the end of his life - change gh-actions job title (because it looks better)

Because it looks better and is more readable.

Because reducing filename to `setup.py` will make our cli arguments shorter and source code cleaner

So our matrix gh-actions builds will not be cancelled if some arch fails

We also add `harfbuzz` to requirements to have freetype library built with harfbuzz support.

opacam · 2020-02-29T21:50:00Z

Hahahaha, first bugs found, for archs x86 and x86_64...when building harfbuzz recipe in both cases:

BTW, the first commit I made here 9fca4b0 (not included in #2046), allow us to go on if some arch fails to build...:thinking:...I think that we should include that in #2046, since it will be difficult to test at runtime all the generated artifacts (precisely those failed archs are the ones that I cannot test at runtime, so I don't usually test the build in my local machine...and it will probably be the same case for other members/contributors, since those kind of CPUs, AFAIK are not common on smartphones...).

About the errors:

x86_64: it looks like that we should include -fPIC to our CFLAGS, probably we should do that at class level at Archx86_64
x86: This is not so clear to me...:thinking:

Anyway, let's go on and test the updated Pillow recipe #2067 and see what happens...

opacam · 2020-02-29T23:00:22Z

Ok, as expected, same harfbuzz errors for mentioned archs: x86 and x86_64 for the updated Pillow recipe (v7.0.0), so let's merge the updated one, since it cannot broke what is already broken and, probably, it has better Python 3.8.1 compatibility 😉

I tested the generated apks at runtime for the updated Pillow recipe, both works with my android device running Android 10.

Here are the generated apks, zip file compressed

BTW, I never uploaded an image of how it will look like the on device unit test app, here you have the one I got with one of those generated apk via gh-actions (only showing one screen of the app in portrait mode, split into two screenshots):

AndreMiras · 2020-02-29T23:03:24Z

It looks amazing, thanks for sharing 👏

Since, at this point, we know that we need to add the mentioned flags at some point, let's be brave and try to add it at Arch level (since we need `-fPIC` for `ArchARMv7_a` it may be also the case for `Archx86_64`) See also: kivy#2073 (comment)

opacam · 2020-03-01T14:16:54Z

Ok, matplotlib recipe doesn't even have a chance to be built, fails installing kiwisolver, we may needed a cppy recipe...so let's fix this...

To do so, I will create a couple of PRs, which then I will merge the corresponding branch into here. This way, we can play a little more with this, having the fixes ready to review into the proper PR and the testing in here, since the matplotlib fix will probably fail the CI tests, because it will depend of the kiwisolver fix.

This reverts commit e504d1b

opacam · 2020-03-02T12:24:17Z

Yeah, the same situation that with the old freetype and harfbuzz libraries. Here are the generated apks. In this case I tested, at runtime, arch armeabi-v7a, works fine.

So, so far we have this situation:

harfbuzz is broken for x86 (I don't know what is happening here...needs investigation)
Pillow doesn't build for x86_64 (not find zlib headers)
numpy doesn't compile for MacOS (at least in our CI tests)
x86_64 may need some extra CFLAGS at Arch level (7d61143)

So now I will remove harfbuzz again, but also Pillow so we should see if we can get a succesful matplotlib build for all archs.

Because the updated `matplotlib` recipe could work now...let's test it

opacam · 2020-03-04T11:26:56Z

More thoughts:

Successfully built all archs for matplotlib recipe (without harfbuzz) 😄
MacOs build with travis is still failing (of course...nothing made to change that)
Yeah, rebuild_updated_recipes has no problem now with matplotlib recipe 🎉. BTW, if you look to the commit I made eb5e35a , you will see that the matplotlib recipe has hardcoded version for freetype library, and for png as well. That is not ideal and we probably want to get the version dynamically...anyway doesn't seem to affect the matplotlib compilation/installation....I suppose that matplotlib build only checks for a minimal version...still...before the merging of Fix matplotlib and update to v3.1.3 #2076, I'm planning to do a couple more of commits to address these things 😉
I cannot test, at runtime, the generated apks for x86 and x86_64...will work at runtime? 😶

Because the `matplotlib` recipe should work now in our `CI` tests

opacam · 2020-03-05T14:25:39Z

Ok, we have matplotlib recipe merged into develop branch, with the issues mentioned before already fixed in there. BTW, thanks for the fast review @AndreMiras 👍 :)

So now I want to do one last test in here, the numpy update #2077, because matplotlib recipe depends on it...so better not to broke that, right? So...let's continue the investigation 🔎

Because the latest version of numpy requires Python>=3.5

opacam · 2020-03-14T14:04:59Z

Yeah, The new numpy recipe plays nice with the matpltolib recipe for armeabi-v7a and arm64-v8a (Here are the generated apks)

BTW, the error we got are the expected ones:

MacOsX test app build fails for travis
Python2 fails because the new numpy recipe is not compatible with it.

So, as a conclusion for the numpy test:
The new numpy recipe is not compatible with python2. Given that python2 will not be maintained anymore, and the fact that we have the numpy recipe posted and approved without merging for several days without anyone complaining, I think that the show must go on, so let's merge numpy into develop branch 😄

Since, at this point, we know that we need to add the mentioned flags at some point, let's be brave and try to add it at Arch level (since we need `-fPIC` for `ArchARMv7_a` it may be also the case for `Archx86_64`) **See also:** - kivy#2073 (comment)

Since, at this point, we know that we need to add the mentioned flags at some point, let's be brave and try to add it at Arch level (since we need `-fPIC` for `ArchARMv7_a` it may be also the case for `Archx86_64`) **See also:** - #2073 (comment)

So our matrix gh-actions builds will not be cancelled if some arch fails which is very useful to fix specific arch issues. See also: kivy#2073

Fix our `CI` travis tests by explicitly omit the numpy build for MacOsX. Yeah...not actually a solution for MacOsX...but at least we will have the `green tick mark` for our `CI` tests...but we have a problem with MacOsX and numpy recipe :< See also: kivy#2073

So our matrix gh-actions builds will not be cancelled if some arch fails which is very useful to fix specific arch issues. See also: kivy#2073

* 📦 Refactor (move) `PythonTestMixIn` into `tests.mixin` * 📦 Unify most of the test apps into one app So depending on the supplied recipes, a different test app will be ran. The supported test apps are: - kivy - flask - no-gui * 🏗️ Make test app `on_device_unit_tests` the default... for our CI providers and `rebuild_updated_recipes.py` because we recently removed the old test app that we made use of. * 📝 Update docs * ✅ Create a `x86_64` testapp for gh-actions * 📝 Fix typo errors and... Remove sentence regarding python2 compatibility... because python2 it's almost at the end of his life * ✅ Make a `flask` testapp... So we can get an apk via gh-actions Notes: - this way the reviewers can install the flask apk without making the build - these changes will be reverted in the next commit...since we want a kivy app as our default) * ✅ Revert "Make a `flask` testapp..." This reverts commit 0ed4d70. * 🔥 Remove `orientation` kwarg... in case that we detect that p4a will build an `service_only` app. Note: if we don't do this we wil get errors once we try to create our apk with the target distribution * ✅ Create a `x86` testapp for gh-actions and change gh-actions job title (because it looks better) * 🏗️ Rename test app Because it looks better and is more readable. * 📝 Fix a syntactical error at docstring * 🚚 Rename `setup_test_app.py` to `setup.py` Because reducing filename to `setup.py` will make our cli arguments shorter and source code cleaner * 🔧 Disable gh-actions `fail-fast` So our matrix gh-actions builds will not be cancelled if some arch fails which is very useful to fix specific arch issues. See also: #2073 * 💚 Add `testapps-with-numpy/%` & ... make `testapps/%` support multiple archs Notes: this completes the job started at 9ea53fc & afd4413

* 📦 Refactor (move) `PythonTestMixIn` into `tests.mixin` * 📦 Unify most of the test apps into one app So depending on the supplied recipes, a different test app will be ran. The supported test apps are: - kivy - flask - no-gui * 🏗️ Make test app `on_device_unit_tests` the default... for our CI providers and `rebuild_updated_recipes.py` because we recently removed the old test app that we made use of. * 📝 Update docs * ✅ Create a `x86_64` testapp for gh-actions * 📝 Fix typo errors and... Remove sentence regarding python2 compatibility... because python2 it's almost at the end of his life * ✅ Make a `flask` testapp... So we can get an apk via gh-actions Notes: - this way the reviewers can install the flask apk without making the build - these changes will be reverted in the next commit...since we want a kivy app as our default) * ✅ Revert "Make a `flask` testapp..." This reverts commit 0ed4d70. * 🔥 Remove `orientation` kwarg... in case that we detect that p4a will build an `service_only` app. Note: if we don't do this we wil get errors once we try to create our apk with the target distribution * ✅ Create a `x86` testapp for gh-actions and change gh-actions job title (because it looks better) * 🏗️ Rename test app Because it looks better and is more readable. * 📝 Fix a syntactical error at docstring * 🚚 Rename `setup_test_app.py` to `setup.py` Because reducing filename to `setup.py` will make our cli arguments shorter and source code cleaner * 🔧 Disable gh-actions `fail-fast` So our matrix gh-actions builds will not be cancelled if some arch fails which is very useful to fix specific arch issues. See also: kivy#2073 * 💚 Add `testapps-with-numpy/%` & ... make `testapps/%` support multiple archs Notes: this completes the job started at 9ea53fc & afd4413

iruriksl · 2021-05-28T17:36:35Z

The issue is still persistent as I faced it while building for armeabi-v7a & arm64-v8a android arch using buildozer, which is as the results are also crossed in checks link of the post.
I m using python==3.8.2, buildozer==1.2.0.dev0, kivy==2.0.0 on MacOS==11.3.1

Is their any work around for mentioned arch till?

opacam added 18 commits February 23, 2020 15:08

[recipes] Update Pillow to v7.0.0

b5f99e8

[recipes] Update freetype & add zlib support

9eb5eb9

[recipes] Update harfbuzz to v2.6.4

6137f31

[testapp] Refactor (move) PythonTestMixIn into tests.mixin

e9a6f97

[testapp] Unify most of the test apps into one app

5aaf248

So depending on the supplied recipes, a different test app will be ran. The supported test apps are: - kivy - flask - no-gui

[testapp] Make test app on_device_unit_tests the default...

5196451

for our CI providers and `rebuild_updated_recipes.py` because we recently removed the old test app that we made use of.

[testapp] Update docs

099b099

[testapp] Enhance gh-actions with a matrix for our testapps

692be94

[testapp] Fix typo errors and...

86de274

Remove sentence regarding python2 compatibility... because python2 it's almost at the end of his life

[testapp] Make a flask testapp...

f34566f

So we can get an apk via gh-actions Notes: - this way the reviewers can install the flask apk without making the build - these changes will be reverted in the next commit...since we want a kivy app as our default)

Revert "[testapp] Make a flask testapp..."

d86c2ff

This reverts commit 0ed4d70.

[testapp] Remove orientation kwarg...

3dea1d1

in case that we detect that p4a will build an `service_only` app. Note: if we don't do this we wil get errors once we try to create our apk with the target distribution

[testapp] Build test app for all archs

ef50835

and: - remove Python 2 test apps because: + we already test Python 2 with travis + Python 2 it's almost at the end of his life - change gh-actions job title (because it looks better)

[testapp] Rename test app

cb17076

Because it looks better and is more readable.

[testapp] Fix a syntactical error at docstring

48a9475

[testapp] Rename setup_test_app.py to setup.py

b42643f

Because reducing filename to `setup.py` will make our cli arguments shorter and source code cleaner

[testapp] Disable gh-actions fail-fast

9fca4b0

So our matrix gh-actions builds will not be cancelled if some arch fails

[testapp] Let's test Pillow recipe

665be80

We also add `harfbuzz` to requirements to have freetype library built with harfbuzz support.

opacam added the WIP label Feb 29, 2020

Merge branch 'feature-update-pillow' into testing-testapps-rework

05ccb3a

opacam added 3 commits March 1, 2020 01:06

[testapp] Let's test matplotlib recipe

6666f8c

[recipes] Fix recipe kiwisolver (add cppy)

25b3e3d

opacam mentioned this pull request Mar 1, 2020

Fix recipe kiwisolver (add cppy recipe) #2075

Merged

[recipes] Fix matplotlib and update to v3.1.3

5552fdc

opacam mentioned this pull request Mar 1, 2020

Fix matplotlib and update to v3.1.3 #2076

Merged

opacam added 3 commits March 1, 2020 23:35

[testapp] Let's be brave now, re-add harfbuzz

5032aa6

This reverts commit e504d1b

Merge branch 'hotfix-matplotlib' into testing-testapps-rework

c0c3fa8

Merge branch 'feature-update-harfbuzz' into testing-testapps-rework

4182ad7

opacam mentioned this pull request Mar 2, 2020

Update numpy to v1.18.1 (add cython recipe) #2077

Merged

[testapps] Remove Pillow and harfbuzz

fab33f5

[CI] Remove matplotlib from broken recipes

b2a7c05

Because the updated `matplotlib` recipe could work now...let's test it

opacam added 2 commits March 4, 2020 12:32

[CI] Remove matplotlib from broken recipes

484181f

Because the `matplotlib` recipe should work now in our `CI` tests

[recipes] Set library version dynamically for *.pc.template

32e6ccb

opacam added 4 commits March 5, 2020 15:32

[recipes] Update numpy to v1.18.1 (add cython recipe)

801ce4f

[recipes] Move CI numpy tests to Python 3

0b3cffc

Because the latest version of numpy requires Python>=3.5

[recipes] Move CI numpy tests to Python 3 (linux)

e473134

Merge branch 'hotfix-matplotlib' into testing-testapps-rework

b943edc

opacam force-pushed the testing-testapps-rework branch from dbec040 to b943edc Compare March 5, 2020 14:38

opacam mentioned this pull request Mar 14, 2020

🐛 Add -fPIC to CFLAGS for Arch x86_64 #2085

Merged

opacam added a commit to opacam/python-for-android that referenced this pull request Mar 14, 2020

🚀 Disable gh-actions fail-fast

cbebd95

So our matrix gh-actions builds will not be cancelled if some arch fails which is very useful to fix specific arch issues. See also: kivy#2073

This was referenced Mar 14, 2020

Numpy on MacOsX fails in our CI tests #2087

Closed

Unify most of the test apps into on device unit test app #2046

Merged

opacam added a commit to opacam/python-for-android that referenced this pull request Mar 30, 2020

🔧 Disable gh-actions fail-fast

293d009

So our matrix gh-actions builds will not be cancelled if some arch fails which is very useful to fix specific arch issues. See also: kivy#2073

opacam added a commit to opacam/python-for-android that referenced this pull request Mar 30, 2020

🔧 Disable gh-actions fail-fast

6365157

So our matrix gh-actions builds will not be cancelled if some arch fails which is very useful to fix specific arch issues. See also: kivy#2073

inclement closed this in #2046 Mar 31, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Playing with `on_device_unit_test_app` #2046 #2073

Playing with `on_device_unit_test_app` #2046 #2073

Uh oh!

opacam commented Feb 29, 2020 •

edited

Loading

Uh oh!

opacam commented Feb 29, 2020

Uh oh!

opacam commented Feb 29, 2020

Uh oh!

AndreMiras commented Feb 29, 2020

Uh oh!

opacam commented Mar 1, 2020

Uh oh!

opacam commented Mar 2, 2020

Uh oh!

opacam commented Mar 4, 2020

Uh oh!

opacam commented Mar 5, 2020

Uh oh!

opacam commented Mar 14, 2020

Uh oh!

iruriksl commented May 28, 2021 •

edited

Loading

Uh oh!

Uh oh!

Playing with on_device_unit_test_app #2046 #2073

Playing with on_device_unit_test_app #2046 #2073

Uh oh!

Conversation

opacam commented Feb 29, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

opacam commented Feb 29, 2020

Uh oh!

opacam commented Feb 29, 2020

Uh oh!

AndreMiras commented Feb 29, 2020

Uh oh!

opacam commented Mar 1, 2020

Uh oh!

opacam commented Mar 2, 2020

Uh oh!

opacam commented Mar 4, 2020

Uh oh!

opacam commented Mar 5, 2020

Uh oh!

opacam commented Mar 14, 2020

Uh oh!

iruriksl commented May 28, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Playing with `on_device_unit_test_app` #2046 #2073

Playing with `on_device_unit_test_app` #2046 #2073

opacam commented Feb 29, 2020 •

edited

Loading

iruriksl commented May 28, 2021 •

edited

Loading