Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

plan for maru-0.7 #154

Open
utzcoz opened this issue Nov 14, 2020 · 51 comments
Open

plan for maru-0.7 #154

utzcoz opened this issue Nov 14, 2020 · 51 comments

Comments

@utzcoz
Copy link
Member

utzcoz commented Nov 14, 2020

Hi @pdsouza , we know @pintaf has ported maruos to Android 9.0, can you push base LOS to repositories with maru-0.7 branch, and @pintaf and other people start to send pr with porting patches? The people of community can help to test and review, and you just need to review lastly and merge the pr. If we can start recent days, maybe we can release maru-0.7 the Q1 of the next year.

@pdsouza
Copy link
Member

pdsouza commented Nov 15, 2020

@utzcoz Sounds good! I will get maru-0.7 branches set up soon and will update this issue as I make progress. 2021 Q1 is a good target for 0.7.

@twaik
Copy link

twaik commented Nov 16, 2020

Something about mflinger. I can provide libX11, libXfixes, libXext, libXdamage, libXi and libXrandr source to be built with Android tree so someone can try to merge mflinger and mclient into one program.

@pintaf
Copy link
Member

pintaf commented Nov 16, 2020

@pdsouza if you wanna base yourself on my manifest: https://github.com/pintaf/manifest.

It's already building some devices. It's not working 100% especially nash, but it's on the good path. PME workd quite well.
I have introduced more changes to los envsetup script or similar to remove these lineage options and get a cleaner lunch menu.

Let me know if you have any questions.

@pdsouza
Copy link
Member

pdsouza commented Nov 17, 2020

@pintaf Thanks, I was already referencing your manifest as soon as I started. I'm just getting the official maru-0.7 to a buildable state; then you can send me PRs for most of the changes from your repos and I'll review and merge them in.

@pdsouza
Copy link
Member

pdsouza commented Nov 22, 2020

Alright guys, I've pushed up a manifest for maru-0.7.

Everything builds without error when I tested with a Nexus 5 (hammerhead). The core services like mflinger, perspectived look like they are starting up fine. I think at this point we just need to cherry-pick @utzcoz's dynamic input switching patches and the PerspectiveManager batches in frameworks/base.

Happy to take PRs from @pintaf on any other modifications he thinks are worth including for 0.7. Just send over the PRs and I'll review and merge.

@pintaf
Copy link
Member

pintaf commented Nov 26, 2020

Hi @pdsouza Excellent news!
I'll work on this this week-end, and integrate for example the repos for HTC10.

Once everything seems nice do you think about building the devices officialy with the new buster image ?

@utzcoz , looking forward working with you on integrating your external settings. It would be nice that this external settings could download the container image and decompress it. It would reduce rom size, and enable us to provide more easilly diferrent images. Debian, ubuntu...

@pdsouza
Copy link
Member

pdsouza commented Nov 26, 2020

@pintaf That sounds great! I can't wait to ship an official image for the HTC10. And yes, absolutely, as soon as 0.7 is stable and all is working well I will start up the automated builds for maru-0.7 for all working devices and we'll get it out as official images on the website so everyone can try them out.

@pintaf
Copy link
Member

pintaf commented Nov 27, 2020

@pdsouza for me, out of the box, the only device you shared in your 0.7 manifest, the hammerhead is failing to build:

Out of space? The tree size of /mnt/disk/maru07/out/target/product/hammerhead/obj/PACKAGING/target_files_intermediates/maru_hammerhead-target_files-eng.ubuntu/SYSTEM is 1133555712 bytes (1081 MB), with reserved space of 0 bytes (0 MB).
The max image size for filsystem files is 1073741824 bytes (1024 MB), out of a total partition size of 1073741824 bytes (1024 MB).

Seems like a partition size issue.
Appart from that device-specific error it seems to build correctly so I will integrate the htc10 and Aquarius X2.
Maybe also the Pixel XL 2 from bootlessxfly.

@pdsouza
Copy link
Member

pdsouza commented Nov 27, 2020 via email

@pintaf
Copy link
Member

pintaf commented Nov 27, 2020

@pdsouza I do have an issue while building HTC10 pme:

libsepol.report_failure: neverallow on line 1385 of system/sepolicy/public/domain.te (or line 11546 of policy.conf) violated by allow mcprepare mcprepare:capability { dac_override };
libsepol.check_assertions: 1 neverallow failures occurred

In the past I had solved it in this commit:
pintaf/android_system_sepolicy@ffcf3df

Do you have any suggestion about why I hit this error with PME and not with hammerhead ? This seems so weird because the issue is linked to mcprepare allow rules, so it shouldn't be device specific. At first I though that you had forgot to call maru code in the hamerhead device repo, but no, you called it.

Maybe is it because you disabled selinux on your build ?

@pintaf
Copy link
Member

pintaf commented Nov 28, 2020

Just posting comments as documentation for my future PRs:
I have hit another issue.

device/htc/pme/devicesettings/res/values/arrays.xml:20: error: resource string/action_none (aka org.lineageos.settings.device:string/action_none) not found.
device/htc/pme/devicesettings/res/values/arrays.xml:21: error: resource string/action_launch_camera (aka org.lineageos.settings.device:string/action_launch_camera) not found.
device/htc/pme/devicesettings/res/values/arrays.xml:22: error: resource string/action_torch (aka org.lineageos.settings.device:string/action_torch) not found.
device/htc/pme/devicesettings/res/values/styles.xml:38: error: resource layout/preference_category_material_settings (aka org.lineageos.settings.device:layout/preference_category_material_settings) not found.
device/htc/pme/devicesettings/res/values/styles.xml:49: error: resource layout/preference_material_settings (aka org.lineageos.settings.device:layout/preference_material_settings) not found.
error: failed linking references.

Solved by adding
<project name="LineageOS/android_packages_resources_devicesettings" path="packages/resources/devicesettings" remote="github" revision="lineage-16.0" />
to the manifest

@utzcoz
Copy link
Member Author

utzcoz commented Nov 28, 2020

Just posting comments as documentation for my future PRs:
I have hit another issue.

device/htc/pme/devicesettings/res/values/arrays.xml:20: error: resource string/action_none (aka org.lineageos.settings.device:string/action_none) not found.
device/htc/pme/devicesettings/res/values/arrays.xml:21: error: resource string/action_launch_camera (aka org.lineageos.settings.device:string/action_launch_camera) not found.
device/htc/pme/devicesettings/res/values/arrays.xml:22: error: resource string/action_torch (aka org.lineageos.settings.device:string/action_torch) not found.
device/htc/pme/devicesettings/res/values/styles.xml:38: error: resource layout/preference_category_material_settings (aka org.lineageos.settings.device:layout/preference_category_material_settings) not found.
device/htc/pme/devicesettings/res/values/styles.xml:49: error: resource layout/preference_material_settings (aka org.lineageos.settings.device:layout/preference_material_settings) not found.
error: failed linking references.

Solved by adding
<project name="LineageOS/android_packages_resources_devicesettings" path="packages/resources/devicesettings" remote="github" revision="lineage-16.0" />
to the manifest

@pintaf , could you send a PR to manifest repo maru-0.7? I can fetch and rebuild my local code.

@pintaf
Copy link
Member

pintaf commented Nov 28, 2020

Yes, will do

@utzcoz
Copy link
Member Author

utzcoz commented Nov 28, 2020

@pdsouza I do have an issue while building HTC10 pme:

libsepol.report_failure: neverallow on line 1385 of system/sepolicy/public/domain.te (or line 11546 of policy.conf) violated by allow mcprepare mcprepare:capability { dac_override };
libsepol.check_assertions: 1 neverallow failures occurred

In the past I had solved it in this commit:
pintaf/android_system_sepolicy@ffcf3df

Do you have any suggestion about why I hit this error with PME and not with hammerhead ? This seems so weird because the issue is linked to mcprepare allow rules, so it shouldn't be device specific. At first I though that you had forgot to call maru code in the hamerhead device repo, but no, you called it.

Maybe is it because you disabled selinux on your build ?

I remember the SELinux is permissive default. I have discuss this problem with @pdsouza in issue maruos/vendor_maruos#4.

@utzcoz utzcoz changed the title plan for maru0.7 plan for maru-0.7 Nov 28, 2020
@pintaf
Copy link
Member

pintaf commented Nov 28, 2020

PR made. It includes HTC 10 device, the devicesettings repo solving the missing string issue.
it does not include my sepolicy repo that solves the sepolicy issue I had.
I don't know what to do about that.

@pintaf
Copy link
Member

pintaf commented Dec 6, 2020

@pdsouza can you add me as maintainer for:
android_kernel_htc_msm8996,
android_device_htc_pme,
android_device_google_marlin,
android_kernel_google_marlin

As of today, I cannot push new maru-0.7 branches:
loicpoisot@pop-os:~/code/android_kernel_htc_msm8996$ git push maruos maru-0.7 ERROR: Permission to maruos/android_kernel_htc_msm8996.git denied to pintaf. fatal: Impossible de lire le dépôt distant.

@pdsouza
Copy link
Member

pdsouza commented Dec 7, 2020

@pintaf Just added you to two teams within our org (marlin and pme) that have access to those repos.

@utzcoz
Copy link
Member Author

utzcoz commented Dec 7, 2020

@pdsouza can you add me as maintainer for:
android_kernel_htc_msm8996,
android_device_htc_pme,
android_device_google_marlin,
android_kernel_google_marlin

As of today, I cannot push new maru-0.7 branches:
loicpoisot@pop-os:~/code/android_kernel_htc_msm8996$ git push maruos maru-0.7 ERROR: Permission to maruos/android_kernel_htc_msm8996.git denied to pintaf. fatal: Impossible de lire le dépôt distant.

@pintaf if you have finished the work of porting marlin and sailfish, you can push LOS 16 source code to maru-0.7 branch, and send your patches with a PR, I can test it locally.

@pintaf
Copy link
Member

pintaf commented Dec 7, 2020

@utzcoz I probably won't have time to PR this until wednesday.

So if you want to try it out as of today, you can head over this: https://github.com/pintaf/manifest/blob/maru-0.7-pintaf/default.xml
I was planing to push my maru changes directly to maru-0.7.
I don't see the advantage in pushing just LOS16 sources, then the rest in PR ?

@utzcoz
Copy link
Member Author

utzcoz commented Dec 7, 2020

@utzcoz I probably won't have time to PR this until wednesday.

So if you want to try it out as of today, you can head over this: https://github.com/pintaf/manifest/blob/maru-0.7-pintaf/default.xml
I was planing to push my maru changes directly to maru-0.7.
I don't see the advantage in pushing just LOS16 sources, then the rest in PR ?

Okay, I will test after pushed. The pr can help to show the real changes of device and kernel to boot Maruos, and can be searched directly.

@pintaf
Copy link
Member

pintaf commented Dec 8, 2020

two PR made for pixel kernel and device sources.
Manifest PR updated as requested and including new pixel repos,

@utzcoz
Copy link
Member Author

utzcoz commented Dec 19, 2020

@pdsouza @pintaf , I transfer maru settings project to maruos organization, because I'm struggling with non-boot of pixel, and I don't have time to process it. @pdsouza if you think its fine, you can accept the transferring, and @pintaf maybe help improve it on its htc device.

@utzcoz
Copy link
Member Author

utzcoz commented Dec 19, 2020

The sailfish and marlin boot correctly now with extracted vendor blobs. But the rootfs didn't been extracted correctly, so I will try to fix this problem.

@pintaf
Copy link
Member

pintaf commented Dec 19, 2020

Hello @utzcoz So you ended up on the one thing I was not able to fix.
Indeed, at the end of migrating to 0.7, it was working correctly on my HTC 10, only problem, mcprepare was "incorrectly" preparing things. but If I deleted the destination folders and run mcprepare from shell after first boot, everything was working well.
I never found out why it was not working correctly on first boot.
I had modified mcprepare to have more logs and understand how and where it could fail, if it's of any interest.

@pintaf
Copy link
Member

pintaf commented Dec 19, 2020

Also, I did not spend too much time on that, because since we plan to transition to your external settings, the mcprepare could potentially be run after downloading the rootfs image. Instead of including it at build time.

@utzcoz
Copy link
Member Author

utzcoz commented Dec 19, 2020

Hello @utzcoz So you ended up on the one thing I was not able to fix.
Indeed, at the end of migrating to 0.7, it was working correctly on my HTC 10, only problem, mcprepare was "incorrectly" preparing things. but If I deleted the destination folders and run mcprepare from shell after first boot, everything was working well.
I never found out why it was not working correctly on first boot.
I had modified mcprepare to have more logs and understand how and where it could fail, if it's of any interest.

type=1400 audit(616392.856:115): avc: denied { read } for pid=866 comm="busybox" name="maru" dev="sda35" ino     =1048582 scontext=u:r:mcprepare:s0 tcontext=u:object_r:system_data_file:s0 tclass=dir permissive=0

Looks like its a selinux problem.

@utzcoz
Copy link
Member Author

utzcoz commented Dec 19, 2020

When I try to port perspective service in frameworks/base, the Android.bp needs the libperspective defined with Android.bp. But after I change Android.mk of libperspective to Android.bp, it needs liblxc defined in Android.bp. The lxc-android provides an example to use Android.bp to build lxc-3, maybe we can try.

@pintaf
Copy link
Member

pintaf commented Dec 20, 2020

I though that maybe we could get in touch with @ppoffice from lxc-android.
Maybe he would be able to help us on those topics and help transition from lxc1 to lxc3.

@utzcoz
Copy link
Member Author

utzcoz commented Jan 17, 2021

I'm trying to integrate MaruSettings to maru-0.7, but it makes AudioFlinger starting failed, I will try to fix it later. @pintaf do you have encounter this problem before?

@utzcoz
Copy link
Member Author

utzcoz commented Jan 17, 2021

I'm trying to integrate MaruSettings to maru-0.7, but it makes AudioFlinger starting failed, I will try to fix it later. @pintaf do you have encounter this problem before?

Okay, I found the reason, the MaruSettings misses some permissions, and it caused zygote dead and reboot.

@pdsouza
Copy link
Member

pdsouza commented Jan 24, 2021

I'm trying to integrate MaruSettings to maru-0.7, but it makes AudioFlinger starting failed, I will try to fix it later. @pintaf do you have encounter this problem before?

Okay, I found the reason, the MaruSettings misses some permissions, and it caused zygote dead and reboot.

Maybe this was causing @pintaf's boot hang issue?

@utzcoz
Copy link
Member Author

utzcoz commented Jan 24, 2021

I'm trying to integrate MaruSettings to maru-0.7, but it makes AudioFlinger starting failed, I will try to fix it later. @pintaf do you have encounter this problem before?

Okay, I found the reason, the MaruSettings misses some permissions, and it caused zygote dead and reboot.

Maybe this was causing @pintaf's boot hang issue?

I remember @pintaf has pointed out the permission problem. Maybe he missed something after repo sync.
The integrating for MaruSettings is pushed for reviewing, and I will start to add notification for desktop running to MaruSettings, the part deleted from frameworks/base porting.

@utzcoz
Copy link
Member Author

utzcoz commented Jan 30, 2021

Just a note :) and they can confirm
Hi @pdsouza , we know @pintaf has ported maruos to Android 9.0, can you push base LOS to repositories with maru-0.7

I was again the one who ported it to Los 16.1 :)) and I send them the patches , but because he failed to apply the patches I also send them the full modified sources they can just build :)
:)
Was ported sometimes in 2019
v@C30 /media/v/hdd/lineage_16/device/motorola $ ls -al
total 12
drwxr-xr-x 3 root root 4096 Apr 25 2019 .
drwxr-xr-x 9 root root 4096 Apr 25 2019 ..
drwxr-xr-x 28 root root 4096 Apr 25 2019 nash

@dianaxxyyzz welcome back. I'm sorry to miss it. I also can confirm it, and I have saw your work and your sharing in maruos' Google Groups. So sorry to miss it.

@pintaf
Copy link
Member

pintaf commented Jan 30, 2021

Yes true. I was thankful that @dianaxxyyzz shared the full sources, because there were modifications that he forgot to share as patches, so full source helped me to put them in git repos and make it available for build to everyone.

@twaik
Copy link

twaik commented Jan 31, 2021

Hey guys. I think I have 2 possible solutions for low system space problem.

  1. Switch system and data partitions. In most cases data partition is much bigger. But you need to make sure recovery knows about it.
  2. Use external sdcard's second partition as /system. Some kernels do not support it.
    It should be enough for most cases.

@twaik
Copy link

twaik commented Feb 1, 2021

@dianaxxyyzz I hadn't. But somebody here had.

the hammerhead is failing to build:

Out of space? The tree size of /mnt/disk/maru07/out/target/product/hammerhead/obj/PACKAGING/target_files_intermediates/maru_hammerhead-target_files-eng.ubuntu/SYSTEM is 1133555712 bytes (1081 MB), with reserved space of 0 bytes (0 MB).
The max image size for filsystem files is 1073741824 bytes (1024 MB), out of a total partition size of 1073741824 bytes (1024 MB).

Seems like a partition size issue.

@twaik
Copy link

twaik commented Feb 1, 2021

It is possible to download tarballs from external server and not to ship it with system. Or maybe it is possible to build Debian container from scratch on device using something like Linux Deploy.

@pdsouza
Copy link
Member

pdsouza commented Feb 2, 2021

It is possible to download tarballs from external server and not to ship it with system. Or maybe it is possible to build Debian container from scratch on device using something like Linux Deploy.

Downloading container images is the next step once we have our base MaruSettings app which @utzcoz is doing some great work on with help from @pintaf. See #127.

The /system space issue is mostly on our legacy devices like hammerhead. Our old Nexus 7 had space issues even on maru 0.6, which is why we have discontinued it for now. We should be able to bring these old devices back online once we download the images from the Maru app.

@utzcoz
Copy link
Member Author

utzcoz commented Feb 23, 2021

I will start to port dynamic input switch based on @twaik 's suggestion.

@utzcoz
Copy link
Member Author

utzcoz commented Apr 7, 2021

I will start to port dynamic input switch based on @twaik 's suggestion.

Update: the new version for dynamic input switching based on @twaik 's suggestion with InputManager disable input device API has been merged (maruos/android_platform_frameworks_base#10). It passes basic tests. The next step I will work is to add desktop state notification to MaruSettings, which is more important than adding API to dismiss pointer immediately from my opinion.

@pdsouza
Copy link
Member

pdsouza commented Apr 8, 2021

Sounds good @utzcoz. It is really cool to be using the new InputManager APIs as @twaik suggested instead of adding our own patches. I too noticed the pointer not immediately disappearing but we can fix that after the desktop state notification.

@utzcoz
Copy link
Member Author

utzcoz commented May 30, 2021

After tile service merged into MaruSettings, I think all features are all merged or refactored into maru-0.7. There is a found issue that showing internal problem dialog caused by matrix compatibility on Pixel devices. I will try to fix it recent days. Hi @pdsouza maybe we can prepare a alpha release for maru-0.7.

@pdsouza
Copy link
Member

pdsouza commented May 30, 2021 via email

@pdsouza
Copy link
Member

pdsouza commented Jun 3, 2021

Just merged security updates from LOS16 through May 2021 into maru-0.7. After @utzcoz successfully moved all our Settings code to MaruSettings, the work needed to merge in updates is much reduced - the only repos that I needed to update were frameworks/base, frameworks/native, build, and manifest.

Remember to repo sync on your next builds to get these changes!

@utzcoz
Copy link
Member Author

utzcoz commented Jun 4, 2021

I will test with my sailfish and Marlin devices.

@utzcoz
Copy link
Member Author

utzcoz commented Jun 5, 2021

Another thing, @pdsouza could you help to change default branch to maru-0.7? I think it is ready.

@pdsouza
Copy link
Member

pdsouza commented Jun 5, 2021

Another thing, @pdsouza could you help to change default branch to maru-0.7? I think it is ready.

Done! All maru-0.7 repositories, including manifest, should have default updated to maru-0.7.

@bootlessxfly
Copy link

bootlessxfly commented Nov 3, 2021

@pintaf @pdsouza @utzcoz
Im not sure this is the correct thread to post this on, but I have a solution for the neverallow rule that blocks mcprepare from being allowed to dac_override.

The solution requires us to add mcprepare to dac_override_allowed so that mcprepare is allowed to use dac_override. This is inside of domain.te. The one big issue I came across with this solution was that mcprepare type is not loaded early enough, so simply adding mcpreapre to dac_override_allowed causes the build to fail since mcprepare is an unknown type at that point(gets loaded later in the sepolicy check).

As it is though, I think it may be beneficial to create a maru specific system/sepolicy project, as this would allow for the above problem to be solved. I would simply moved the check into mcprepare.te, add a custom dac_override_allowed which includes mcprepare and comment it out inside domain.te. This allows us to move the check to a point where sepolicyundertands what mcprepare type is instead of failing when adding to the allowed list. This keeps the original check in place, it just moves into into vendor/maruos.
Here is the PR for mcprepare:

Here is the system/sepolicy change:

Also, I've hit two issues where system_server is explicitly nerverallowed. This causes two issues:
allow system_server default_android_hwservice:hwservice_manager { find };
allow system_server default_android_service:service_manager { add };

Im working on a vendor/sepolicy specific work around for this by adding a wrapper to default_android_hwservice and default_android_service
Here is the PR:

I also submitted a PR to vendor/maruos that should fix the sepolicy issues with treble devices first noted by @pintaf:

@utzcoz
Copy link
Member Author

utzcoz commented Nov 4, 2021

@bootlessxfly Thanks your work, I think this thread is the correct place for sharing. I have left some comments about your PRs.

For tracking Pixel 2 and Pixel 2L, you can contact @pdsouza to grant permission to create related projects under MaruOS, and push your source code to those projects. We can discuss how to process vendor binaries at email.

I have a solution for the neverallow rule that blocks mcprepare from being allowed to dac_override.

Sorry, I know dac_override from your comment and commit right now. But after some searching, I found granting permission for dac_override is dangerous: https://danwalsh.livejournal.com/79643.html, https://danwalsh.livejournal.com/80232.html. Is there any reason to grant dac_override for mcprepare? From what I learnt about dac_override, we can change file permissions to avoid mcprepare.

Also, I've hit two issues where system_server is explicitly nerverallowed. This causes two issues:
allow system_server default_android_hwservice:hwservice_manager { find };
allow system_server default_android_service:service_manager { add };

I believe those rules work correctly at my Pixel 3a and Pixel 3a XL. Could you post your failed avc denied log at your related PR page? We can discuss it at that page.

@utzcoz
Copy link
Member Author

utzcoz commented Feb 21, 2022

Potential new features we can do for 0.7(ideas from @twaik):

  1. Modify scrcpy to mirror MaruOS to desktop(see Add a MobilePhone View #2 (comment)).
  2. Add a daemon to grant permissions for users added by /etc/passwd(see Some suggestions for future releases #153 (comment)).

@utzcoz
Copy link
Member Author

utzcoz commented Feb 22, 2022

Update for 0.7: we can't release 0.7 alpha successfully last year, but we will try it again this year.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

6 participants
@pdsouza @pintaf @twaik @bootlessxfly @utzcoz and others