My recent ARM experience with Ubuntu 22.04 bare (no Docker) and Handbrake custom profiles - I hope it helps somebody.

[DZaremba14600]

I just wanted to share my recent experience with ARM. Hopefully this is of help to others and/or the development folks.

I've been able to get ARM running on Ubuntu 22.04 bare (no Docker) which is more recent than 20.04 and I've also been able to easily access Handbrake GUI custom profiles in real-time (no importing/exporting) from the ARM web interface to allow easy selection between multiple custom profiles as needed.

This post is quite long, but its all summed up in the last 10 lines or so. Hope its a contribution that helps.

I have about 340/350 blu-ray and 4k disks I want to put into JellyFish.

My hardware is two i7-12700k systems with 64 meg. Each has an ASUS 4k blu-ray drive, one SATA attached, one attached via SATA to USB3.

I installed Ubuntu 20.04 onto a partition, ran the docker-setup.sh script for an ARM docker install and all seemed well. The only issue was getting QuickSync working.
I used the ubuntu-quicksync.sh script and got Quicksync to work one time, but it stopped working. I reinstalled everything and could never get it working again. ARM itself worked fine, just not Quicksync.

I wondered if it was my system, so I decided to take a different tack to troubleshoot. I decided to not use docker and run ARM on top of Ubuntu. I know it's not supported but thought I'd give it a try.

I decided to use Ubuntu 22.04 as a base and go from there. Here's how it turned out.

I took the ubuntu-20.04-install.sh, used that as a base. I commented out the MakeMKV and Handbrake installs, I'll handle those later. I took the Dockerfile from ARM dependencies and basically merged the lists of modules and python libraries into ubuntu-20.04-install.sh to form a master list. I ran this merged/modified ubuntu-20.04-install.sh and, aside from some odd, non-displayable characters that terminated the script prematurely (only seen when I edited the .sh in nano and removed them), the script ran fine and ARM was running! Because it had terminated, the rules for the blu-ray disk didn't seem to work correctly. I added them in manually and it worked! Probably an issue with the script terminating. I built and installed Handbrake (with GUI and QuickSync support) and MakeMKV manually from source and Voila! everything was working on a Ubuntu 22.04 base outside of Docker!.

I then decided to do this all over again, this time scripting everything and seeing if it would run to completion without failing.

I created a script to install the appropriate Ubuntu 22.04 Intel graphics driver for the i7-12700K.
I created a script to build and install Handbrake with GUI and Quicksync support from source.
I created a script to build and install MakeMKV from source (actually this script came from the MakeMKV forum).
I created a script to build and install the Intel Media SDK from source.
I double checked the ubuntu-20.04-install.sh (I made sure the ARM user got added to the groups video, render, sudo, root, and arm). This last script requires requirements.txt in the same directory.

Last night I decided I'd run all this again from scratch and make sure it all works and is repeatable.

One Ubuntu install from .ISO, and then run 5 scripts.

I installed Ubuntu 22.04 from .iso onto a clean partition. I updated the system after the initial boot.
I ran the script to install the Intel graphics driver for Ubuntu 22.04.
I ran the script to build and install Handbrake GUI with QuickSync support. (this also installs everything required to build this project from source).
I ran the script to build and install MakeMKV.
I ran the script to build and install the Intel Media SDK.
I ran the script ubuntu-20.04-install.sh to install ARM.

End result - no aborts, no deal breaker errors, no issues. ARM was accessed via the browser and was running fine!!

This now got me thinking - Can I create custom profiles in Handbrake for various configurations and easily access them from ARM?

I sure can!! I go into Handbrake GUI, create a custom profile and save it. I loaded the 'Roku 1080p30 Surround' profile, changed a few settings such as framerate, encoder, output format, RF (constant quality) and the audio passthrough settings, and then saved it as a custom preset named 'Roku 1080p30 Surround Custom'.
To use it in ARM, I set the HB_PRESET to 'Roku 1080p30 Surround Custom' (without the quotes) and set the HB_ARGS_BD to '--preset-import-gui' (again without the quotes) and just that, nothing else, and voila! the custom preset is now used!

By having only --preset-import-gui on the HB_ARGS_BD line, I can change the HB_PRESET to any preset I want, including other custom presets, and be able to use any number of preset settings by changing only the HB_PRESET line. (also, any other command line args used in addition to --preset-import-gui, such as --quality 18.0 will override whatever is in the custom preset, for a quick change). This is great for changing settings between Blu-ray disks and 4K/HDR disks which may require different settings.

You can also do things like process with TRANSCODE FALSE and manually run Handbrake GUI manually with the custom profile on SuperFast to check how certain settings will work or check that audio passthrough is working or that GPU acceleration is working.

How might the developers use this for the Docker image? Well, there is a HandBrake Docker image that allows access to the Handbrake Docker GUI through a browser (https://github.com/jlesage/docker-handbrake). This could probably (?), easily (?) be integrated into the ARM docker image and allow easy access to multiple custom Handbrake profiles from the ARM interface. It could allow more custom settings instead of using the HB_ARGS_BD line which can get clumsy when used with a lot of settings and/or wanting to switch settings for different projects or needs.

So, in conclusion, ARM can now run on Ubuntu 22.04 as a base without docker if anyone wants to, and its a newer version of Ubuntu than 20.04. Also, ARM can have access to multiple custom Handbrake profiles quite easily.

Thanks for listening and I hope this helps somebody down the road.

And thanks for a great project in ARM - Automatic Ripping Machine.

PS - any suggestions on settings in Handbrake for Blu-ray and 4K HDR disks video and audio related to processing 340 or so disks going forward would be most welcome.

Thanks.

David

[ekodante]

hello,

I am interested ! :D

did you post the script ?

best regards :)

[charmarkk]

@microtechno9000 I also wouldn't mind helping to get the wiki etc. updated based on these scripts here, give a shout if you need help.

[charmarkk]

Any word on if we can get the wiki updated for those of us who don't use Docker?

[shitwolfymakes]

Starting around 2.6ish, we dropped official support for installing as a thick client in favor of Docker, and this will only become more pronounced once this becomes a docker compose app with 3.0.

You're welcome to try and run it as a thick client, but we won't provide any official support for doing so

[charmarkk]

That's unfortunate - previous comments here made it seem that there was some understanding that not everyone prefers Docker. I know "official support" was dropped, but figured due to this discussion, at least the wiki would be updated with a "do at your own risk/understanding" warning - as the instructions there have been incomplete even before "official support" was dropped.

[shitwolfymakes]

Ah, I misunderstood the question. Yes, that's basically what will happen.

[DZaremba14600]

ARM-ubuntu-scripts.zip

Let me first be clear that I'm a Docker advocate and love Docker, love the concept and love its implementation. So ARM being implemented in Docker is fine with me.
I installed the ARM Docker container and all worked well, except the QuickSync funtionality. I saw the unbuntu-quicksync script and used it and actually got QuickSync working at one point but found it would be available, then not, then available, then not, and then finally, I could not get it to work again, no matter what I did. I know it was me because it HAD worked, but I was really frustrated. So I decided to forget the docker implementation and try to get ARM working as a straight application install on Ubuntu and just make sure everything was working for me, the hardware, the OS, the GPU drivers, the ARM install, everything (I suspect my issues were related to the VA driver operation between the container and the base OS. Then I'd go back to installing the Docker container and getting it working. After all, if Docker was going to be the supported path going forward, I needed to settle on that eventually.

I decided that I'd start with the ubuntu-20.04-install.sh install. The Ubuntu 20.04 drivers for my i7-12700k were already deprecated, so I made the decision to use Ubuntu 22.04 as a base, use those drivers, and then update the ubuntu-20.04-install.sh to work on 22.04. I mean, really, how different would it be?

Since Docker was the supported method going forward, I decided I'd use the dockerfile script from the ARM dependencies, the dockerfile from ARM itself, and reconcile the modules being installed in all three scripts into a new ubuntu-22.04-install.sh. As it turned out this was actually very easy and very quick. Copy/past the module and delete the duplicate modules.

I also decided I wanted to build Handbrake from source to enable QuickSync support and to provide the GUI. This would also allow me to create custom profiles and use them in ARM (so my original post). I decided to do the same for MakeMKV. I broke those out of the ARM script and put together a small script for each; the Handbrake script was a few cut and pastes from their site, and the MakeMKV script I got from the MakeMKV forums. Both scripts build and install with GUI support. From the ubuntu-quicksync script, I decided to create a script to build and install the Intel Media SDK.

I commented out the Handbrake and MakeMKV from the ARM script, and I commented out the Docker and pycharm-community installs (not needed for this implementation).

None of the core ARM logic or flow was changed, so there was little to debug. I had an issue with some non-displayable characters that got into the new script, probably during the copy/paste that caused the script to keep aborting for no apparent reason, but once I found what the cause was all was well. I had a little cleanup left related to the udev rules and permissions, but everything looked well.

First run has a few glitches, but it complete and ARMui was running and I could actually rip and transcode.

I wanted to install this on a second machine, so this was going to be the big test. An install from scratch - install Ubuntu, the GPU drivers, install Handbrake, MakeMKV, Media SDK, and then ARM. All using these few scripts.

How did it go? Last night from start to finish it too me two hours and everything was up and running and I ripped and transcoded a blu-ray without having to make any changes. For good measure, this morning I wiped the partition and repeated the whole install - Ubuntu 22.04 from ISO, then updates, then GPU driver install, then Handbrake, then MakeMKV, then MediaSDK, then ARM itself. from start to finish this morning it was 1hr 45min. I kicked off a blu-ray rip and headed to work.

I've attached the scripts in a .ZIP file. I don't actually see the attachment. If there's an issue, let me know.

Here's the process:

1. Install Ubuntu 22.04 from ISO. reboot and log in as the user specified during the install. I used user 'ubuntu'.
2. Run the Ubuntu updates utility OR do an apt-get update/apt-get upgrade and get the system updated properly
3. copy all of the attached files into your home folder - (Install_Intel_GPU.sh, Install_Handbrake.sh, Install_MakeMKV.sh, Install_Intel_Media_SDK.sh, ubuntu-22.04-install.sh, and requirements.txt)
4. open a terminal/command prompt
5. sudo ./Install_Intel_GPU.sh (note: this is for my GPU, yours may differ. Just install the most updated GPU driver for your system).
6. reboot
   7 Get back to home directory and open a terminal/command prompt
7. sudo ./Install_Handbrake.sh (this downloads, builds and installs Handbrake GUI with QSV support) - NOTE: this should be run first as it installs prerequisites use in the remaining scripts for building and installing from source.
8. sudo ./Install_MakeMKV.sh (this downloads, builds, and installs MakeMKV GUI). There will be a text screen that will need to be scrolled down, key in q and then yes to accept).
9. sudo ./Install_Intel_Media_SDK.sh (this downloads, builds, and installs the Intel Media SDK).
10. reboot for good measure
11. Get back to home directory and open a terminal/command prompt
12. sudo ./ubuntu-22.04-install.sh (this downloads and installs ARM. There will be prompts for user Arm password, postfix (I selected no config) and DVD css updating).

At this point, ARMui is running, udev rulls are active, and you should be able to rip. (pun intended)
14) Give user Arm a bash shell for default shell, unless you want something different.
15) reboot and login as Arm. Point browser to IP:8080/setup, then login. The working directories will be in home/media (raw, transcode, completed).

In my original post, I also documented how to now be able to use custom Handbrake profiles and not have a convoluted HB_ARGs field.

Any questions or comments, please feel free.

Again, I hope this helps somebody. I found it all useful.

[dani190]

Thank you for writing this up and providing your scripts! Worked perfectly for me!

[DZaremba14600]

-->I responded to your comment and the maintainer’s question. I attached the scripts and provided detailed instructions. Take a look and if you have any questions or comments, please feel free!! Thanks and hope this helps.   

…

--> hello,I am interested ! :D did you post the script ?best regards :)—

[maccampb]

Thanks very much for this... I need to update my ARM installation (non-docker) and will give this a go...

[DZaremba14600]

ARM-ubuntu-scripts.zip

I realized I did not include those instructions in my comment in the .Zip files with the scripts. Here's an updated .ZIP.

Post back your experience good or bad, hopefully good. As I reported, I ran through this twice and on my system it went fine. Hopefully yours too.

And a good report might be of help to others.

Good Luck!!

[charmarkk]

Looking forward to trying this too. Finally getting around to updating and the docker method just is NOT working for me. Glad to see a regular install method still out there, thanks @DZaremba14600. @maccampb anything good/bad to report?

[charmarkk]

@DZaremba14600 well that was quick - I've been troubleshooting the docker install for an hour now, but your script worked (after purging docker and its mess) immediately - took all of 10 minutes! The script choked when trying to make the /home/arm/* directories (not really sure why - it just said the permissions were denied) so I made them on my own, rebooted, and voila I'm back in business!! I'm still on 20.04 and it worked like a charm. I feel like I'd change the status updates (Installing Dependencies, Overwriting this, Removing that) to green or something less frightening than red - an issue I have w/ the Docker script too - but this is great!!

Maintainers, I'd love to see an updated script install (I'd help where I could, though I'm no Python expert), as it seems much more straightforward to me. I get why lots of folks like Docker, but it just doesn't click for me.

Off to get abcde, arm.yaml, and handbrake settings put back - thank you so much for the tweaks/updates @DZaremba14600!

[charmarkk]

Actually it seems like this script isn't creating the YAML file. Gonna see if I Can figure out where that isn't happening.

Removed my arm user then let the script readd it and it seemed like that did it, I see

creating config file /etc/arm/config/arm.yaml
creating config file /etc/arm/config/apprise.yaml
Non-default port specified, updating arm config...

in the output now. Weird.
ETA2: Seems like adding something in the script giving the arm folder write permissions before making the media etc. folders might be necessary.

The next problem is figuring out why the service fails when restarting! ETA3: This might have been a non issue - it seems like it restarts constantly for about 3-8 mins til it finally gets the server to stick. Maybe that's a change since I last had it installed.

[DZaremba14600]

charmarkk - sorry for taking so long to respond to your series of comments.

I am really glad that somebody tried the scripts, had success, and found them useful. From your comments, it seems that you found them very helpful. I'm glad!

To address some of your issues -

When I originally began this, I referred to the wiki. The wiki said the manual install was no longer supported. However, it was last supported on Ubuntu 20.04. I began by using Ubuntu 20.04 and using the scripts from the wiki.

To be honest, I encountered similar issues such as permissions and the service having startup glitches related to permissions. I could not track it down and frankly I didn't want to spend time on an older Ubuntu release and an older script. I decided, why stick with 20.04 when there was a newer 22.04 (and now a 23.10 even)? The 20.04 script couldn't be so behind that it wouldn't work with 22.04 could it? So, everything I did was going to be done on 22.04.

So while I cant be certain its related to 20.04 (you said that's what you were using), there might be something there.

As for my testing, I'd start from scratch and image a blank partition from the 22.04 downloaded ISO, I'd apply all the updates, I'd place all the script in my new, initial default user's home directory (I used user 'Ubuntu') and then ran all the scripts using sudo. I've run that process 4 times on 2 machines and it worked every time without a glitch. During development of this updated script, if the script aborted and I corrected something, re-running the script usually worked and picked up processing from where it bombed. Usually but not always. But on a clean run its been reliable.

Like I said, I never figured out why the service had permissions issues on 20.04 using the original script. Same with the permissions for the folder in the Arm home directory. But when I follow the step and start by running the scripts in the user created during Ubuntu 22.04 install directory's home folder, everything has consistently worked.

I'm glad you were easily able to resolve any issues. Maybe that points to some odd, non-obvious logic issue or something. Time will tell.

Anyway, again, I'm happy it proved helpful for you.

Once thing I've really found helpful is what I referred to in my original post, being able to create and use custom handbrake presets. I can create and test a preset in GUI handbrake, test it using very fast settings, and then set it to slow or medium for actual use, and never have to change the ARM handbrake preset setting in the web UI. once I define and use the custom preset and enter its name in the ARM web UI, I can use handbrake GUI and the custom preset, test on a test MKV and get my preset just right and then start ripping. I don't have to remember to change preset names back and forth. I hate taking a few hours to process a disk only to find I forgot to change something I had set because of a test. I don't have to deal with the convoluted settings normally used for HB_ARGS. I was personally primarily testing setting for audio tracks.

Anyway, I found this gives me a little more flexibility when processing disks.

But I'm glad to have been able to have made a useful contribution.

Thanks for the feedback.

[DZaremba14600]

<!-- /* Font Definitions */ @font-face {font-family:"Cambria Math"; panose-1:2 4 5 3 5 4 6 3 2 4;} @font-face {font-family:Calibri; panose-1:2 15 5 2 2 2 4 3 2 4;} /* Style Definitions */ p.MsoNormal, li.MsoNormal, div.MsoNormal {margin:0in; font-size:11.0pt; font-family:"Calibri",sans-serif;} a:link, span.MsoHyperlink {mso-style-priority:99; color:blue; text-decoration:underline;} .MsoChpDefault {mso-style-type:export-only;} @page WordSection1 {size:8.5in 11.0in; margin:1.0in 1.0in 1.0in 1.0in;} div.WordSection1 {page:WordSection1;} -->Thanks for the feedback! I’m glad this has proved useful for some. Sent from Mail for Windows From: DaniSent: Sunday, December 31, 2023 9:54 AMTo: automatic-ripping-machine/automatic-ripping-machineCc: DZaremba14600; MentionSubject: Re: [automatic-ripping-machine/automatic-ripping-machine] My recent ARM experience with Ubuntu 22.04 bare (no Docker) and Handbrake custom profiles - I hope it helps somebody. (Discussion #960) Thank you for writing this up and providing your scripts! Worked perfectly for me!—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you were mentioned.Message ID: ***@***.***>