Provide and publish s390x images #67
Comments
|
Any updates to this? An official image for |
|
@ivandov Unfortunately there are serious errors in compiling CouchDB correctly on S390X, as some unit tests fail. The problem is related to endianness in snappy, see https://issues.apache.org/jira/browse/COUCHDB-3261 for details. We refuse to release a broken image on principle. Until an S390-supplied engineer is ready to step up and fix these issues properly we are stalled on this. No one on the CouchDB team is working this issue. |
|
@wohali thank you for that information. Let me forward this along to the Linux on Z "ecosystem" team, to see if they can look into a fix for this. I am personally planning on creating an image for s390x following this guide: https://github.com/linux-on-ibm-z/docs/wiki/Building-CouchDB It seems that the build process for s390x has already been verified - at least directly on a VM - I just need to make it into a portable image. Would you like to me to retry some test suites during the image build process? |
|
@ivandov The modifications in that walkthrough are unacceptable. You can't just change the "right answer" in a test case without delving into why things are the way they are. For instance, your solution would not allow people to take We will not release an s390x build until this problem is resolved. Further, note #118 which now builds CouchDB in Docker using our .deb binary packages. We won't be building these on qemu with s390x until the build process is successful on an unmodified source set. The work for that happens over in the apache/couchdb-pkg repository. |
|
To make it clear - don't waste your time building a Dockerfile for s390x that doesn't install from the .deb, we won't host it here or at the official Fix the snappy code correctly, get CouchDB building and passing all tests without modification to our source code. Then, review and help in the Then, we can update the At that point, this Dockerfile will just "work," we'll just enable manifest multi-arch building here which is entirely a build-time step. |
|
To be clear, I did not write the documentation that I linked, I was just planning on attempting to follow it to see if a Docker Image could successfully be built. I understood that would be a private Docker Image and not one that would contribute back to the multi-arch effort for the official As I said, I will make sure the Linux on Z ecosystem team is aware of the blocking bug, and hopefully they can work on creating a fix to allow the unit tests to pass as they are currently written. I'll follow-up soon. |
|
Also, I just read through the the JIRA issue you linked earlier: https://issues.apache.org/jira/browse/COUCHDB-3261 The final comment by Paul may be the best we can do for snappy support on s390x, because of the endian-ness issue, and as mentioned in that thread, this is a consistent problem across libraries that need cross-architecture support. The requirement for a That JIRA thread sounds like it needs some updating / re-running for CouchDB |
|
@ivandov It really depends how far you s390x people want to go. snappy could be extended to handle reverse-endianness (with a performance hit) if desired. I could see it being an upselling point that you could "outgrow" your amd64/ppc64le Couch instances and want to move to s390x without having to replicate the data over, but instead just mounting your IBM Cloud storage device over to the new hardware node. But that's really your call. For reference @davisp is also IBM so you can bug him if you want about what he thinks might be acceptable now - though that JIRA ticket you linked was at least 2.0.0 already, so there's not much different architecturally between then and the imminent 2.3.0 release. Sorry for coming across angrily, I just don't want a slapped-together public image. I hope you understand and appreciate my attention to detail on this point :) Help in improving the apache/couchdb-ci and apache/couchdb-pkg repos to support s390x/debian stretch would still be the next steps after the snappy work is done. Thanks again! |
|
Hi @wohali, Any update on this? It looks like most of the issues listed initially have been taken care of?
Looks like it was fixed with commit: 4dbd7c17e6e7e7894b64595b2b7fdde7ead823f6
Issue 1121 seems to have been closed as well.
I’ve built and run all the tests, on Ubuntu 16.04( as well as others, but not Debian), they all pass.
The PR was merged here?: I think it's been a while since anyone has looked at this...so I'm wondering what would be needed to get version 3.x images for s390x? Thank you |
|
@adpopescu update: I had missed the fix to the couch compress tests, that's good. We don't have an s390x build target in Jenkins today. apache/couchdb#1121 and was tracking overall support for multi-platform building in our Jenkins environment - and was closed because that landed. That's the next step. @davisp is in charge of the IBM/SoftLayer machines for us, and has helped us set up ppc64le and x86_64 targets there. You'll need to liaise with him (internal to IBM is fine) to get us an s390x run environment. Parallel to that, I can build an s390x Docker image that contains our build chain. I'll start work on that this week. Next step after that lands is to add s390x to our main platform matrix build here: https://github.com/apache/couchdb/blob/master/build-aux/Jenkinsfile.full Specifically, it'd make sense to clone this section. Once that is passing and is green for a few commits, we can add s390x in this repo, and push support downstream to the main Docker repo. Thanks for offering to lend a hand! |
|
Our CI image, couchdbdev/s390x-debian-buster-erlang-20.3.8.25-1, is now available. |
|
@wohali I believe late last year, Paul Davis from IBM had shared access to an Ubuntu Community cloud VM for s390x architecture, to be included as build node into the Jenkins CI/CD for community s390x builds. I gather that the VM is still up and active(online). Could you please confirm if the s390x builds are still running without any issues ? Please let know if any support is required in getting the s390x docker image published through CI. |
|
@SriramaSharma I was never given login access to those machines, though I did sign paperwork. Please advise. You can contact me at wohali@apache.org directly, rather than doing so through this webpage. |
|
Travis now also has builds for `s390x` architectures.
https://blog.travis-ci.com/2019-11-12-multi-cpu-architecture-ibm-power-ibm-z
…On Wed, Jun 16, 2021, 4:20 PM Joan Touzet ***@***.***> wrote:
@SriramaSharma <https://github.com/SriramaSharma> I was never given login
access to those machines, though I did sign paperwork. Please advise. You
can contact me at ***@***.*** directly, rather than doing so
through this webpage.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#67 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/ABWECVKJ36UPIEAFIQGCYLDTTEBSRANCNFSM4EOZHSYA>
.
|
|
We no longer use Travis due to the length of the queue for Apache projects.
|
|
Thanks to @nickva ! Through CouchDB CI a new s390x image was pushed to docker hub. However, for announcing the support all the tests have to be passing. So apache/couchdb#3660 is being used to investifate and track closure of test failures on s390x. |
|
Hi @wohali , I think all the pre-requisites which you mentioned in #67 (comment) are available now. The official s390x .deb binary is available from v3.3.2 afterwards and s390x jobs are running "all-green" on community CI. I think it's time to update The current Dockerfile can work out-of-the-box on s390x. We have verified the generated s390x docker image of CouchDB v3.3.2 on local VMs, it worked fine. Could the community consider to publish the official s390x docker image in the next release? I can raise a PR to submit necessary changes in Thank you very much! |
|
Hi @nickva , Could you please have a look when you have a chance? ☝️ Thank you very much! |
A couple of requests have come in for
s390ximages, see #66 for instance.This is blocked until:
s390xare resolved, such as JIRA COUCHDB_3261.s390xbuild platform (currently, unavailable), see Add multi-platform support to CI configuration couchdb#1121.The text was updated successfully, but these errors were encountered: