Mac M1 binaries

[vorburger]

For MariaDB4j to "just" work on M1 Mac... either:

A. Someone contributes binaries for MariaDB binaries JAR for M1

-OR-

B. #296 is implemented to support running MariaDB in a container instead of from binaries in a JAR

I welcome community contributions for A. (but won't work on that myself), but think B. is a better way to pursue for the future.

[vorburger]

docker/for-mac#5389 has something about (now solved?) performance issues on M1.

https://stackoverflow.com/questions/69896059/is-it-possible-to-get-mariadb4j-to-work-on-an-m1-mac asks for this.

[vorburger]

Just like for #298 re. ARM, https://downloads.mariadb.org doesn't actually seem to have DB binaries for M1, but presumably someone interested in supporting this knows of some place where prebuilt binaries for Mac M1 arch can be downloaded from.

[balamsoft]

Well, there's a really easy way to build MariaDB on Mac using Homebrew.
brew install mariadb --build-from-source

As of now this command will build MariaDB 10.6.4. I built it using an M1 Mac Mini and I have the binaries now. Would those be of any use?

[vorburger]

brew install mariadb --build-from-source
Would those be of any use?

The only problem with this approach is that you cannot easily contribute that as a PR to this repo and make it possible for myself or anyone sel to easily build from source accross platforms - that will only work on a Mac (which e.g. I or other others don't have / use).

I'm unfortuantely not willing (anymore) to trust 3rd-party and include native binaries built by strangers on the Internet in this Git repo; I hope you understand. (I will hower accept PRs which DL binaries from "reputable sources" such as downloads.mariadb.com.)

A possible "compromise" I can think of to unblock this could be to make a PR which builds M1 binaries IF RUN ON A M1 MAC, and then let Mac end-users of this project decide if they want to mvn install | deploy that themselves. I'm happy to review and merge a PR for that. But I won't be able to locally run it myself and deploy it to Maven Central with future release.

[balamsoft]

I'm unfortuantely not willing (anymore) to trust 3rd-party and include native binaries built by strangers on the Internet in this Git repo; I hope you understand.

Understood.

[balamsoft]

Maybe another way to approach this would be detect if the mariadb binaries are already available at the host.

[balamsoft]

if anyone's interested I was able to build the latest MariaDB4j version for Mac M1 processors.
Steps:

1. Clone MariaDB4j.
2. Build MariaDB from source (arch -arm64 brew install mariadb@10.2 --build-from-source).
3. Go to folder /opt/homebrew/Cellar/mariadb@10.2/10.2.41 and copy all the required binaries from the bin and script subfolders to MariaDB4j/DBs/mariaDB4j-db-mac64-10.2.11/src/main/resources/ch/vorburger/mariadb4j/mariadb-10.2.11/osx/bin.
4. In a terminal go to MariaDB4j/DBs/mariaDB4j-db-mac64-10.2.11 and do a mvn install.

That's it, now your tests will run in your M1 :)

[vorburger]

brew install mariadb --build-from-source
Maybe another way to approach this would be detect if the mariadb binaries are already available at the host.

This may actually be the simplest way forward on this topic...

Watch new issue #560 (which is about the generalization of this idea, not Mac & M1 specific).

[vorburger]

PS: Anyone reading this and using MariaDB4j, please Sponsor my maintenance this project! I would very much appreciate it.

[stevenodb]

I was trying to add the AppleSilicon (arm64) binaries, but the current structure only supports win32, linux, osx; adding another architecture for osx isn't supported, for instance. It would need some refactoring to how platform detection is currently designed. Do you agree?

[vorburger]

Steven, you can just name it something like osx-arm64/ perhaps, without needing to refactor the existing one? BTW please note that I cannot accept PR with binaries, so if you are willing to make a contribution for this (thank you!), then it would have to download an archive from a reputable source and unzip it during the build, using something similar to here: https://github.com/vorburger/MariaDB4j/blob/master/DBs/mariaDB4j-db-10.3/mariaDB4j-db-mac64-10.3/prepare.xml

[Renevith]

The Homebrew recipe at https://formulae.brew.sh/formula/mariadb says "Bottle (binary package) installation support provided for: [...] Apple Silicon". Does anyone know if Homebrew would allow the MariaDB4j build to point to their bottle URL, and if so, what URL to use? The URLs behind the "Bottle JSON API" link e.g. this one just return an "authentication required" error for me.

[vorburger]

#560 is currently the recommended approach for how to use this library on Mac M1 models.

[stevenodb]

I have a draft PR with working support for macos-arm64 in the manner you suggested, but no leads on a reputable source to host an apple silicon build of mariadb.

[vorburger]

I have a draft PR with working support for macos-arm64 in the manner you suggested,

@stevenodb thanks for #628!

but no leads on a reputable source to host an apple silicon build of mariadb.

Yeah that's still the crux here, given that https://mariadb.org/download doesn't seem to have Mac builds. So I guess the options are:

A. Someone to find and post here a publicly accessible URL to Homebrew (or any other reputable site) from where I can (have the Ant script in #628) myself download a MariaDB version for Apple Silicon.

B. Explain to me how I could myself use brew on Linux to install the an Apple Silicon version of MariaDB on Linux, which I can then package. (But I'm unclear about how Homebrew exactly works and if it even support this kind of cross installation; and it sounds like a PITA.)

C. Go on vacation to a friend who has a M1 Mac who can install MariaDB and copy the files from there, and commit them into this Git repository. I don't particularly like this idea, and don't really have much time for doing that.

D. Automate C. with scripts in this repo and execute those on a GitHub supported Runner on Mac OS to produce what we want. I guess if this sounds like your idea of fun, you could go for it... this sounds like it could be a "least PITA" and "most future proof maintenable" route forward here.

E. Have the MariaDB4j code assume Mac M1 users have brew installed (or even automate download its installation?), and execsomething like brew install mariadb@10.6 (to a temporary directory?) to solve this conundrum. Possible. Contrived. Not sure how useful this is. Try, if you are so inclined.

F. Give up on this entirely? 😈 Is running brew install mariadb@10.6 yourselves and then using https://github.com/vorburger/MariaDB4j#how-using-existing-native-mariadb-binaries really THAT hard and inconvenient? Why?

For full disclosure and expliciteness, I don't really have the time to investigate these further - but I'm happy to review "community initiaves" by readers of this post who would like to explore doing any of this.

PS: How about someone SPONSOR this? 😹 E.g. Tidelift, IssueHunt, LFX Crowdfunding.

[ranma2913]

if anyone's interested I was able to build the latest MariaDB4j version for Mac M1 processors. Steps:

1. Clone MariaDB4j.

2. Build MariaDB from source (`arch -arm64 brew install mariadb@10.2 --build-from-source`).

3. Go to folder `/opt/homebrew/Cellar/mariadb@10.2/10.2.41` and copy all the required binaries from the `bin` and `script` subfolders to `MariaDB4j/DBs/mariaDB4j-db-mac64-10.2.11/src/main/resources/ch/vorburger/mariadb4j/mariadb-10.2.11/osx/bin`.

4. In a terminal go to `MariaDB4j/DBs/mariaDB4j-db-mac64-10.2.11` and do a `mvn install`.

That's it, now your tests will run in your M1 :)

Hi, I wanted to share my complete command list which should work on anyone's Apple M1/Arm64 Chip.

Apple M1/arm64 Chips: Build MariaDB from Source and Override Binaries in local MariaDB4j

1. Check mariadb major.minor.patch version in your error logs and update the variables in the commands.
   
2. Update the MariaDB4j_Workspace variable to point to your project workspace directory.
3. Run commands for the required mariadb version:

export MariaDB4j_MAJORMINOR="10.2" && export MariaDB4j_VERSION="${MariaDB4j_MAJORMINOR}.11"
brew install mariadb@$MariaDB4j_MAJORMINOR --build-from-source
export MariaDB4j_Workspace=/Users/$(whoami)/WORK/RIPTiDE/IntelliJ_Workspace && export MariaDB4j_Repo=${MariaDB4j_Workspace}/MariaDB4j 
env | grep MariaDB4j_
git clone https://github.com/vorburger/MariaDB4j ${MariaDB4j_Repo}
yes | cp -fv $(brew --prefix)/opt/mariadb@$MariaDB4j_MAJORMINOR/bin/* ${MariaDB4j_Repo}/DBs/mariaDB4j-db-mac64-$MariaDB4j_VERSION/src/main/resources/ch/vorburger/mariadb4j/mariadb-$MariaDB4j_VERSION/osx/bin/
yes | cp -fv $(brew --prefix)/opt/mariadb@$MariaDB4j_MAJORMINOR/scripts/* ${MariaDB4j_Repo}/DBs/mariaDB4j-db-mac64-$MariaDB4j_VERSION/src/main/resources/ch/vorburger/mariadb4j/mariadb-$MariaDB4j_VERSION/osx/bin/
mvn install -f ${MariaDB4j_Repo}/DBs/mariaDB4j-db-mac64-$MariaDB4j_VERSION/pom.xml

While this does work. fine for local. Be aware that when MariaDB4j makes a next release if the version of MariaDB changes you'll need to run these commands again for the new version.

[abrousseau001]

@vorburger Are you still interested in doing the least PITA option of building a GitHub repository that uses GitHub's Actions to build the macOS version of MariaDB? I was just playing around with the idea locally on my M2 MacBook Pro and it seems pretty straight forward for building (at least the version without all the extras included). I am willing to put up a test repository with packages from the action for both intel and arm64 (if the GitHub Action supports both containers).

[vorburger]

@abrousseau001 how do you mean building a GitHub repository? As in another repo/project? I am happy to review Pull Requests for THIS repo (https://github.com/vorburger/MariaDB4j) with proposed contributions to make this possible... 😄

[vorburger]

BTW as mentioned on #713, https://github.com/ankane/setup-mariadb seems like a great GitHub Action for this project.

It's interesting to note how they use Homebrew to locally install a MariaDB. Perhaps that could be used to "package" it...

[abrousseau001]

@abrousseau001 how do you mean building a GitHub repository? As in another repo/project? I am happy to review Pull Requests for THIS repo (https://github.com/vorburger/MariaDB4j) with proposed contributions to make this possible... 😄

Yes, I meant a Pull Request, which is a fork until you bring it back it. The only contribution to start with would be providing a package for MariaDB. Keeping the possible testing dependency down the road once that is established.

[abrousseau001]

BTW as mentioned on #713, https://github.com/ankane/setup-mariadb seems like a great GitHub Action for this project.

It's interesting to note how they use Homebrew to locally install a MariaDB. Perhaps that could be used to "package" it...

I think using that action might be a good idea for testing in GitHub actions, but I would think a better approach, if using homebrew, would be to either require homebrew as a dependency to install MariaDB locally, eliminating the above package and building, or getting with the Homebrew folks to see if there is any way to get the pre-built binaries without using the brew command. As a macOS user, I am more than happy to run a single command to get the dependency beforehand, but then you have to look at various scenarios:

• I am unable to access homebrew (network restrictions)
• I downloaded this this Java library to test various versions and have cleanup after the fact

I think ultimately there is a smaller population using MariaDB in macOS, I can be wrong on this, so requiring homebrew and installation beforehand seems like a fine solution to me especially when MariaDB themselves don't provide the package for macOS and tell you to use homebrew.

[vorburger]

@abrousseau001 re. a Pull Request, which is a fork until you bring it back it, the trouble is that I still would need a Mac to mvn deploy a Mac artifact, no? And I don't have one. Unless I trust somebody (you? or @TheKnowles or so?) to deploy a Mac artifact for MariaDB4j and add a dependency? Or am I totally misunderstanding what you may be suggesting?

@abrousseau001 re. requiring homebrew and installation beforehand seems like a fine solution to me especially when MariaDB themselves don't provide the package for macOS and tell you to use homebrew, do you mean Option F. above, with https://github.com/vorburger/MariaDB4j#how-using-existing-native-mariadb-binaries? I personally would actually be fine with leaving it at that, but I think the project is missing at least 2-3 things for that to be "fully done":

1. test coverage for Mac on CI - see GitHub Action to run Tests on Mac #714, initially maybe just "hard-coded"
2. some "automation" along the lines of Add latest MariaDB v10.6+ (incl. support for OpenSSL 3) #699 (comment), for this Java library to "just to the right thing based on where it's running"; incl. some built-in "guidance" for users; on Mac, that "right thing" might be trying to find a local installation, and if N/A then abort printing something clear and helpful, such as: _"Dude, brew install mariadb, as per ... (URL)".

While I personally won't work on this (without Sponsoring), I certainly would review any PRs about this!

PS: If you use MariaDB4j professionally, please consider 💸 support on OpenCollective, via GitHub Sponsoring or through a Tidelift subscription to ensure active maintenance of this project used by hundreds, since 2011! 🫶_

[vorburger]

I just noticed that https://brew.sh is actually available on Linux as well now, and not just Mac only anymore, and (apparently, not yet actually tested, just read) can "cross" `brew fetch --os ... --arch ... its MariaDB formula...

... maybe that's a way forward here worth further exploring to allow me to (eventually, once it works) mvn deploy this, from Linux.

@abrousseau001 given your comment on #714 last week, are you still interested in creating such a PR? I would definitely welcome the contribution, and be happy to review it.

Testing of a PR for this would require to first have #714.

And if that doesn't work, then running Brew on a Mac GitHub Runner could... but that would then require to be able to directly release from there, which is more work.

[abrousseau001]

Give me a bit @vorburger and I will see what I can do on my MacBook Pro to ensure when the request comes in it should work. Ping me again in a couple of weeks if you haven't heard anything new.