Add armhf dependencies for Pegasus

[mmatyas]

At the moment the Pegasus binaries are built for the traditional 32-bit ARM platform, and as such link to the 32-bit system libraries. On 64-bit distros however these will be missing, so the script should explicitly install the 32-bit armhf variant of the dependencies.

[mmatyas]

@darksaviorx Could you try this out?
Related: mmatyas/pegasus-frontend#1021

[cmitu]

I don't think the proposed changes will work, since armhf packages are not known to apt unless the proper repositories are added and dpkg knows about the armhf arch (i.e. via dpkg --add-architecture..).
But the solution is sub-optimal and IMHO the aarch64 version for Pegasus should be installed to run natively. I understand that it's not working (see the linked issue), but adding it via 32bit lib/arch support is IMHO not a proper solution.

[darksaviorx]

Correct, it will not work until the armhf architecture is added with that command.

[mmatyas]

Yeah, I remember RetroPie having an armhf-specific SDL build, but I wasn't sure whether armhf itself is enabled by default on the system.

I fully agree that a proper solution would be to provide a 64-bit build. On the other hand, as the devices can run both 32 and 64 bit distros (eg. Pi 3 and 4), setting up and providing a build for every device + platform combination is IMHO might not be the optimal strategy either. The patch here is a workaround for the issue, as currently Pegasus won't launch on aarch64 without these dependencies.

As far as I know the current RetroPie releases are all based on armhf, so this problem should only affects those who manually install it. As an alternative solution, I'm okay with ignoring the package on aarch64, until there's a better build. Which one would you prefer?

[cmitu]

Yeah, I remember RetroPie having an armhf-specific SDL build, but I wasn't sure whether armhf itself is enabled by default on the system.

Yes, we have, but that's just for enabling specific video drivers in SDL (RPI on Pi models or mali on SBC's having the Mali GPU drivers). In both cases, a source installation is available and verified working on both 32bit and 64bit ARMs.

I fully agree that a proper solution would be to provide a 64-bit build.

Yes, not necessarily from the released assets in the Pegasus repo; the pegasus-fe relies on the pre-built packages from your repository, so installing from source is not an option at the moment in RetroPie.

As far as I know the current RetroPie releases are all based on armhf, so this problem should only affects those who manually install it.

Yes 64bit installations are just a manual option right now.

As an alternative solution, I'm okay with ignoring the package on aarch64, until there's a better build. Which one would you prefer?

My preference would be to have a source install available, which we can add to the scriptmodule. 64bit support is still labeled as beta in RetroPie (for ARM).

I don't have an aarch64 install at hand, but I've tested a source build and (leaving aside the hunt for the packages needed to be installed - thanks Debian!) I had a similar issue with the main window not displaying anything. Turns out I was missing quite lot of QML related modules, which are to be installed separately (qml-module-qtgraphicaleffects qml-module-qtmultimedia qml-module-qtqml qml-module-qtqml-models2 qml-module-qtquick-controls qml-module-qtquick-controls2 qml-module-qtquick-layouts qml-module-qtquick-templates2 qml-module-qtquick-window2 qml-module-qtquick2). Maybe @darksaviorx's issue is similar, we may want to wait and see what's the issue causing it, before adding the 32bit libs instead.

I noticed also the similar size, the binary is small since it dynamically linked - does PEGASUS_STATIC_CXX build option works only clang ?

I can take a look at a source aarch64 build, but I'll have to install a aarch64 system first.

[darksaviorx]

I'm fine with building it myself. I just need a complete list of all the required packages and whatever else that might be needed.

I tried again today with @cmitu 's listed modules and I also added these libqt5multimedia5-plugins qml-module-qt-labs-qmlmodels. The Settings text now shows.

Pegasus is still having issues. Loading a game loads to a black screen. Running Pegasus remotely I can see this error: [e] Could not queue DRM page flip on screen HDMI1 (Device or resource busy). I load pegasus with QT_QPA_EGLFS_KMS_CONFIG=/home/pi/egl.json pegasus-fe -platform eglfs. The json file contents: { "outputs": [ { "name": "HDMI1", "mode": "2560x1440" } ] }
I also tried with only pegasus-fe -platform eglfs with the same results.

Log file: https://pastebin.com/vMVHvgNv
Log file when attempting to load a game: https://pastebin.com/sjqSz9bX

[cmitu]

I'm fine with building it myself. I just need a complete list of all the required packages and whatever else that might be needed.

I don't have a list available, but might get one if I start from scratch the build process.

I tried again today with @cmitu 's listed modules and I also added these libqt5multimedia5-plugins qml-module-qt-labs-qmlmodels. The Settings text now shows.

Yes, qml-module-qt-labs-qmlmodels was missing from my list and it's needed for the Settings window.

Pegasus is still having issues. Loading a game loads to a black screen

I'm seeing this also on my build, so it's not specific to aarch64. My guess is that, when starting runcommand to launch the game, pegasus-fe is not releasing the DRM master and no DRM/KMS programs are able to start since they cannot acquire a video context. The runcommand launch menu (which runs in text mode) is not shown after launch either and the emulators that are started are complaining they cannot initialize the video (and failing). After exiting, pegasus-fe tries again to initialize the video device and fails with the errors shown by @darksaviorx (Could not queue DRM page flip on screen..).

If I use the armhf build from the pegasus repo, this doesn't happen (i.e. the game starts), so it's clearly something related to the qt5 libraries on the host, either the build options or something related to options passed to the eglfs platform.

[darksaviorx]

Might this be needed? https://github.com/mmatyas/pegasus-frontend/blob/master/etc/rpi4/kms_launch_fix.diff

Again, I'm not familiar with most of this so I'm not sure if: This is needed, this is already compiled, or if this needs to be manually added.

[cmitu]

Investigating further the DRM master is not released theory, this is what is logged when a game is started:

...
[ 4333.167603] [drm:drm_mode_setcrtc [drm]] [CONNECTOR:41:HDMI-A-2]
[ 4333.433432] [drm:drm_mode_addfb2 [drm]] [FB:339]
[ 4333.443867] [drm:drm_mode_addfb2 [drm]] [FB:344]

# game is started here, pegasus is trying to disable the video output to release the DRM master lease.
[ 4343.025256] vc4-drm gpu: [drm:drm_mode_rmfb_work_fn [drm]] Removing [FB:344] from all active usage due to RMFB ioctl
[ 4343.025602] vc4-drm gpu: [drm:drm_framebuffer_remove [drm]] Disabling [PLANE:95:plane-4] because [FB:344] is removed

# this is from the emulator trying to initiliaze the video, but failing since the `drm` master is still help by QT.
[ 4343.085055] vc4-drm gpu: [drm:drm_mode_getconnector [drm]] User-space requested a forced probe on [CONNECTOR:32:HDMI-A-1] but is not the DRM master, demoting to read-only probe
[ 4343.085255] vc4-drm gpu: [drm:drm_mode_getconnector [drm]] User-space requested a forced probe on [CONNECTOR:41:HDMI-A-2] but is not the DRM master, demoting to read-only probe
[ 4348.388580] vc4-drm gpu: [drm:drm_mode_getconnector [drm]] User-space requested a forced probe on [CONNECTOR:32:HDMI-A-1] but is not the DRM master, demoting to read-only probe
[ 4348.388796] vc4-drm gpu: [drm:drm_mode_getconnector [drm]] User-space requested a forced probe on [CONNECTOR:41:HDMI-A-2] but is not the DRM master, demoting to read-only probe
[ 4348.389031] vc4-drm gpu: [drm:drm_mode_getconnector [drm]] User-space requested a forced probe on [CONNECTOR:32:HDMI-A-1] but is not the DRM master, demoting to read-only probe
[ 4348.389506] vc4-drm gpu: [drm:drm_mode_getconnector [drm]] User-space requested a forced probe on [CONNECTOR:41:HDMI-A-2] but is not the DRM master, demoting to read-only probe

# this is pegasus re-initializing the video.
[ 4349.081603] [drm:drm_mode_addfb2 [drm]] [FB:339]
[ 4349.450128] [drm:drm_mode_addfb2 [drm]] [FB:341]

Clearly there's a conflict somewhere. bullseye has Qt 5.12.2 and the static builds are using 5.15.3, could be something that's fixed in the later versions.

[darksaviorx]

Running qmake --version gives me: QMake version 3.1 Using Qt version 5.15.2 in /usr/lib/aarch64-linux-gnu

[mmatyas]

When Pegasus was added, the issue with source build was that the Qt libraries of the system were years behind, and similarly to SDL, weren't configured well for embedded graphics. The situation seems to be better now, Qt 5 will likely receive only patch updates, and recently I've seen a -gles variant of the Qt libs too (eg. libqt5gui5-gles). If they indeed work, then yes, a source build would be a possible option.

When using KMS, like on the Pi 4, you do need the KMS fix applied before the build. The program has to completely disconnect from the driver for the launching to work, but it requires some workarounds to access the Qt internals to do this. This might fix the DRM issues you encounter.

The PEGASUS_STATIC_CXX links the C++ libraries statically (ie. libstdc++), nothing else. If you'd like to link something else statically, you'll need to tell that to CMake. For the dynamic build on RetroPie, the flags you likely want are PEGASUS_USE_SDL2_GAMEPAD=1 PEGASUS_USE_SDL2_POWER=1 PEGASUS_ENABLE_LTO=1. For running Pegasus, you might need the QT_QPA_EGLFS_KMS_ATOMIC=1 QT_QPA_EGLFS_FORCE888=1 env vars set, though these are also handled by the install script.

[cmitu]

@mmatyas thank you for the pointers, that got me on the right track.

Indeed, the missing KMS patch is what's causing the issue.
So, on top of the existing dependencies that I already had installed, I had to add the qtbase5-private-dev package, then build using qmake (cmake doesn't add the necessary includes):

cd $PEGASUS_SRC_DIR/
mkdir build && cd build
qmake .. QMAKE_LIBS_LIBDL=-ldl USE_SDL_GAMEPAD=1 INSTALLDIR=/opt/retropie/supplementary/pegasus-fe
make -j3
sudo make install

# the binary should be now at /opt/retropie/supplementary/pegasus-fe/pegasus-fe, just like a binary install on 'armhf'.

To launch, I'm using

QT_QPA_EGLFS_ATOMIC=1 /opt/retropie/supplementary/pegasus-fe/pegasus-fe -- -platform eglfs

and I guess the other option can be added.
@darksaviorx try the build steps above and see if the new binary solves the game launching problem.

[darksaviorx]

I reverted to 1080p so I can test without the json file. I installed Pegasus from RetroPie-Setup so it installs a pegasus-fe in /usr/bin with the contents:

#!/bin/bash

if [[ $(id -u) -eq 0 ]]; then
    echo "Pegasus should not be run as root. If you used 'sudo pegasus-fe' please run without sudo."
    exit 1
fi

# save current tty/vt number for use with X so it can be launched on the correct tty
tty=$(tty)
export TTY="${tty:8:1}"

export QT_QPA_EGLFS_FORCE888=1  # improve gradients
export QT_QPA_EGLFS_KMS_ATOMIC=1  # use the atomic DRM API on Pi 4

clear
"/opt/retropie/supplementary/pegasus-fe/pegasus-fe" "$@"

Made the new build. No change with those new steps. Loading a game still gets the same error: [e] Could not queue DRM page flip on screen HDMI1 (Invalid argument).

Now, if I run the new build with just pegasus-fe -platform eglfs, the game attempts to load and then exits back to Pegasus. The tv does not show the runcommand loading screen. If I execute Pegasus remotely then I can only see the loading screen on my remote machine. I don't see any error messages with Pegasus but I noticed a seg fault with runcommand.

Runcommand log: https://pastebin.com/SCpfR7Ua

[cmitu]

Made the new build. No change with those new steps. Loading a game still gets the same error: [e] Could not queue DRM page flip on screen HDMI1 (Invalid argument).

Not sure if you're using the binary in this case. I get no more errors from the DRM layer and the runcommand runs fine. It's true that I don't start it automatically, I just run the new binary (with the env options for QT) from a simple command prompt. I'll try to see if aarch64 makes a difference here, since I've done all of this on just an armhf test bullseye install.

[darksaviorx]

I started over fresh just in case. Same results as before. If the kms patch is required then I'm not sure how to add it.

[cmitu]

I started over fresh just in case. Same results as before. If the kms patch is required then I'm not sure how to add it.

The patch is required, I thought that was obvious from my previous comment. To apply it:

git clone --depth 1 https://github.com/mmatyas/pegasus-frontend
cd pegasus-frontend/
wget https://raw.githubusercontent.com/mmatyas/pegasus-frontend/master/etc/rpi4/kms_launch_fix.diff
patch -p1 < kms_launch_fix.diff 
# proceed with `qmake` and `make`

[darksaviorx]

It finally works. Thank you both.