[Maybe bug] The todos app crashed immediately after I run "buildozer -v android debug deploy run"

[diyism]

The todos app crashed immediately after I run "buildozer -v android debug deploy run",
while the "adb shell logcat" says:

04-22 03:28:28.306 14372 14405 I python  : Android kivy bootstrap done. __name__ is __main__
04-22 03:28:28.306 14372 14405 I python  : AND: Ran string
04-22 03:28:28.306 14372 14405 I python  : Run user program, change dir and execute entrypoint
04-22 03:28:28.309 14372 14372 W SDLThread: type=1400 audit(0.0:865753): avc: granted { execute } for path="/data/data/com.pywebview.todos.pywebviewtodos/files/app/_python_bundle/modules/math.cpython-311.so" dev="dm-4" ino=1812842 scontext=u:r:untrusted_app:s0:c245,c256,c512,c768 tcontext=u:object_r:app_data_file:s0:c245,c256,c512,c768 tclass=file
04-22 03:28:28.313 14372 14372 W SDLThread: type=1400 audit(0.0:865754): avc: granted { execute } for path="/data/data/com.pywebview.todos.pywebviewtodos/files/app/_python_bundle/modules/_datetime.cpython-311.so" dev="dm-4" ino=1812670 scontext=u:r:untrusted_app:s0:c245,c256,c512,c768 tcontext=u:object_r:app_data_file:s0:c245,c256,c512,c768 tclass=file
04-22 03:28:28.353 10613 10678 D SharedPreferencesImpl: Time required to fsync /data/user/0/com.google.android.gms/shared_prefs/com.google.android.gms.update.storage.xml: [<1: 1786917, <2: 3297541, <4: 1342907, <8: 53011, <16: 18054, <32: 840, <64: 48, <128: 6, <256: 1, <512: 2, <1024: 1, <2048: 0, <4096: 0, <8192: 0, <16384: 0, >=16384: 0]
04-22 03:28:28.403 14372 14405 I python  : Traceback (most recent call last):
04-22 03:28:28.403 14372 14405 I python  :   File "/tmp/tmpe2ewy8c2/main.py", line 3, in <module>
04-22 03:28:28.403 14372 14405 I python  :   File "/home/malcolm/pywebview/examples/todos/.buildozer/android/platform/build-arm64-v8a/build/python-installs/pywebviewTodos/arm64-v8a/webview/__init__.py", line 23, in <module>
04-22 03:28:28.403 14372 14405 I python  : ModuleNotFoundError: No module named 'proxy_tools'
04-22 03:28:28.403 14372 14405 I python  : Python for android ended.

[r0x0r]

you have to include proxy_tools in your requirements
requirements = python3,kivy,bottle,proxy-tools,typing_extensions,cryptography. I have updated todos spec file.

[diyism]

I've git-pulled the update, now the app won't crash, but it shows:
The webpage at http://127.0.0.1:48437/index.html could not be loaded because: net:ERR_CLEARTEXT_NOT_PERMITTED
and when I open http://127.0.0.1:48437/index.html in the android chrome browser, it shows:
Error: 404 Not Found: Sorry, the request URL 'http://127.0.0.1:48437/index.html' caused an error: File does not exist.

[r0x0r]

This looks like you do not have index.html bundled with the apk. Look for source.include_exts and android.add_assets properties.

[diyism]

Keep configs in buildozer.spec:

source.include_exts = py,png,jpg,kv,atlas,html,jar
android.add_jars = /home/malcolm/miniconda3/lib/python3.10/site-packages/webview/lib/pywebview-android.jar

Uncommented 2 more configs in buildozer.spec:

android.add_assets = assets
android.extra_manifest_application_arguments = extra_manifest_application_arguments.xml

And added a new file(extra_manifest_application_arguments.xml) in pywebview/examples/todos/, the file content:
android:usesCleartextTraffic="true"

Install the latest buildozer(1.5.1.dev0, the old one has bug about extra_manifest_application_arguments):

pip3 uninstall buildozer
git clone https://github.com/kivy/buildozer.git
cd buildozer
python3 setup.py install
python3.11 -m pip install --force-reinstall pexpect

Then run "buildozer android clean", and re-run "buildozer android debug deploy run"

But now the "adb shell logcat" says:

04-24 00:36:03.067 30742 30771 I python  : Run user program, change dir and execute entrypoint
04-24 00:36:03.067  1410  1505 D ArtManagerInternalImpl: /data/misc/iorapd/com.pywebview.todos.pywebviewtodos/10211/org.kivy.android.PythonActivity/compiled_traces/compiled_trace.pb doesn't exist
04-24 00:36:03.067  1410  1479 D EventSequenceValidator: Transition from ACTIVITY_LAUNCHED to ACTIVITY_FINISHED
04-24 00:36:03.068 30742 30771 I python  : Traceback (most recent call last):
04-24 00:36:03.068 30742 30771 I python  :   File "/tmp/tmpcqbhz6jg/main.py", line 3, in <module>
04-24 00:36:03.068 30742 30771 I python  : ModuleNotFoundError: No module named 'webview'
04-24 00:36:03.068 30742 30771 I python  : Python for android ended.

I can't figure out what's wrong currently.

[r0x0r]

My bad. Of course you need to include pywebview as well in requirements
requirements = python3,pywebview,kivy,bottle,proxy-tools,typing_extensions,cryptography

[diyism]

Modified requirements into "requirements = python3,pywebview,kivy,bottle,proxy-tools,typing_extensions,cryptography",
Modified android.add_assets into "android.add_assets = assets/index.html:index.html",
But the todos app still shows "Error: 404 Not Found: Sorry, the request URL 'http://127.0.0.1:53097/index.html' caused an error: File does not exist."

I used the "apktool d ./bin/pywebviewTodos-0.1-arm64-v8a-debug.apk" to see the apk directory structure:

|-- assets/
    |-- index.html
    |-- private.tar
|-- lib
    |-- arm64-v8a
        |-- libcryto1.1.so
...
|-- AndroidManifest.xml
|-- apktool.yml

I can see the index.html is in the assets directory.

I still can't figure out what's wrong.

[r0x0r]

This looks like a bug. I will look into this.

[r0x0r]

I have spent way more time investigating this than I have intended and still could not find what's wrong here. Some intermediate findings so far:

• The problem is in packaging. The resulting apk file's assets/private.tar archive has only main.py.
• assets/index.html and the rests should be packed inside private.tar. APK's assets are not related to todos's assets.
• It is working with another buildozer.spec setup run from pywebview's main folder, but not with in examples/todos. both buildozer.spec are similar to each other.

[diyism]

Great thanks for your effort,
I'll make some time to investigate this problem.

And could we invite others to help us?

[r0x0r]

Some more information. This seems to be the cause https://python-for-android.readthedocs.io/en/latest/distutils.html namely this bit Only your ``main.py`` from your app’s ``–private`` data is copied into the .apk!.
I have troubles understanding this documentation, but it seems that assets need to be included in the dedicated setup.py or pyproject.yaml

[diyism]

I've tried modify "p4a.setup_py = false" into "p4a.setup_py = true", but when I run "buildozer android debug deploy run", it shows errors:

    metadata = build.util.project_wheel_metadata(path)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/malcolm/.local/lib/python3.11/site-packages/build/util.py", line 49, in project_wheel_metadata
    env.install(builder.get_requires_for_build('wheel'))
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/malcolm/.local/lib/python3.11/site-packages/build/_builder.py", line 219, in get_requires_for_build
    with self._handle_backend(hook_name):
  File "/usr/lib/python3.11/contextlib.py", line 158, in __exit__
    self.gen.throw(typ, value, traceback)
  File "/home/malcolm/.local/lib/python3.11/site-packages/build/_builder.py", line 345, in _handle_backend
    raise BuildBackendException(exception, f'Backend subprocess exited when trying to invoke {hook}') from None
build._exceptions.BuildBackendException: Backend subprocess exited when trying to invoke get_requires_for_build_wheel

[diyism]

There's a temporary fix:
keep the "source.dir = .",
keep the "p4a.setup_py = false",
modify "source.include_exts = py,png,jpg,kv,atlas,html,jar" into "source.include_exts = py,png,jpg,kv,atlas,html,jar,js,css",
and delete the setup.py file,
run "buildozer android debug deploy run",
run "apktool d ./bin/pywebviewTodos-0.1-arm64-v8a-debug.apk",
run "tar -tf pywebviewTodos-0.1-arm64-v8a-debug/assets/private.tar", it will show:

Makefile
assets/
assets/index.html
assets/logo.jpg
assets/script.js
assets/styles.css
doc/
doc/Makefile
include/
include/Makefile
libtool
main.pyc
man/
man/Makefile
sitecustomize.pyc
stamp-h1
test.pyc
testsuite/
testsuite/Makefile
p4a_env_vars.txt

It seems that only 5 Makefile files are redundant.
And the app can run in the phone and show the kitty normally.

[diyism]

Or:
$ mkdir build
$ cp -r assets build/
$ cp main.py build/
modify "source.dir = ." into "source.dir = ./build"
$ buildozer android debug deploy run
$ apktool d ./bin/pywebviewTodos-0.1-arm64-v8a-debug.apk
$ tar -tf pywebviewTodos-0.1-arm64-v8a-debug/assets/private.tar
it shows:

assets/
assets/index.html
assets/logo.jpg
assets/script.js
assets/styles.css
main.pyc
sitecustomize.pyc
p4a_env_vars.txt

and the app can also run normally.