request: add darwin/arm64 build for macOS ARM

[sneak]

Apple Silicon M1 Macs are out, and a native ARM version of syncthing would be awesome. AIUI, golang already has support for darwin/arm64 (edit: accidentally wrote darwin/amd64 originally by accident) , so hopefully this isn't much work on your end. I'm happy to help with testing if you need any as I have the hardware in hand.

[calmh]

Go supports darwin-arm64 (in the macOS/M1 sense) in the "tip" (current development) version only, not in the released 1.15. We have a preview build with this, it's available in the latest release candidate for example: https://github.com/syncthing/syncthing/releases/tag/v1.12.0-rc.3

I don't yet have an M1 Mac so this is completely untested, might not even run. There are new requirements around code signing for example that I'm not sure if we fulfill or not. Testing appreciated.

[sneak]

Will test today and report back.

[sneak]

Off Topic: just noticed the changelog, and: oh wow, untrusted encrypted devices in 1.12, that's such a great feature I've hoped for for so long! Thanks so much for all of your hard work!

[ssgelm]

@calmh FYI I tried the new build and get this panic:

$ ./syncthing
panic: qtls.ClientHelloInfo doesn't match

goroutine 1 [running]:
github.com/marten-seemann/qtls-go1-15.init.0()
	github.com/marten-seemann/qtls-go1-15@v0.1.0/unsafe.go:20 +0x1b4

[calmh]

Thanks for testing. Of course the quic package blows up, that will probably need to be disabled until it's updated for Go 1.16.

[ssgelm]

I've been having an issue where the amd64 version of syncthing spikes to 100% cpu and hangs so I had to turn it off for now so if you'd like me to test anymore arm64 versions please let me know! 😄

[calmh]

See if this one runs: https://build.syncthing.net/buildConfiguration/Syncthing_BuildMac/86021?buildTab=artifacts&guest=1

[ssgelm]

Nope same exact panic.

[calmh]

That's odd because it shouldn't be building that package at all. Maybe it's included by something else. I guess I'll have to experiment when I have M1 hardware, or when Go 1.16 is released.

[ssgelm]

If I have a chance I will try to see if I can get it to build and run locally.

[homeisfar]

With the latest arm builds provided, I get the same error as #7128 (comment)

I tried installing the latest Syncthing through MacPorts, which theoretically is a fresh compilation job. I'm not sure what happened (no output from MacPorts) but I figured after about three hours the build system was spinning, not actually doing anything useful. Not that I expect this to be fixed here, just mentioning it for awareness. I think the issue is, as already mentioned, Go's currrent lack of support for the new architecture.

The amd64 build does work fine on my M1 Mac with Rosetta. I copied the configuration directly from my old Mac. I haven't noticed any CPU spikes or hangs, but I also don't share many files or generate much activity.

[calmh]

Apparently the zip build didn't take the extra build tags into account, but now it should so see if this helps: https://build.syncthing.net/buildConfiguration/Syncthing_BuildMac/86187?buildTab=artifacts&guest=1

I read something about having to set GODEBUG=asyncpreemptoff=1 when running, not sure if that was for amd64+Rosetta or arm64 native or not at all.

[homeisfar]

The arm build you provided launches successfully (./syncthing) but with filesystem watcher errors: Error while trying to start filesystem watcher for folder "org" (uykt5-no3s9), trying again in 1m0s: watching is not supported

The amd64 build in your latest link launches with Rosetta successfully with no errors.

[calmh]

Yeah, that's expected, no fs notifications due to cross compilation. Will need a native build for that, or more complicated build setup.

[lenlo]

Apparently the zip build didn't take the extra build tags into account, but now it should so see if this helps: https://build.syncthing.net/buildConfiguration/Syncthing_BuildMac/86187?buildTab=artifacts&guest=1

I read something about having to set GODEBUG=asyncpreemptoff=1 when running, not sure if that was for amd64+Rosetta or arm64 native or not at all.

FWIW, the amd64 version started up fine on my M1 based Mac mini, but soon after ran into the following error:

[EB7QQ] 16:29:21 INFO: Completed initial scan of sendreceive folder Xxxxx
assertion failed [abi_info.kind == AbiKind::TranslatedCode]: emulated forward to an arm pc that isn't in translated code. arm_pc=0x105bc44d8 abi_kind=6 emulation_interval=[0x105c094fc,0x105c09510) instruction_interval=[0x105c094e8, 0x105c09510) x86_rip=0x404e70d
(ThreadContextRegisterState.cpp:677 move_to_instruction_boundary)

Restarting it with GODEBUG=asyncpreemptoff=1 seemed to take care of that problem (thanks for that tip!), but file watching would still not work.

I'm now running the arm64 version with file watching disabled. Apart from:

[EB7QQ] 16:59:46 INFO: Listener for quic://0.0.0.0:22000: unknown address scheme "quic"

it seems to be working well.

[FiloSottile]

Is this fix committed anywhere? I still get the panic at bf7d03d when building with Homebrew (after patching --HEAD Homebrew/homebrew-core#65703).

Note that this is not a darwin/arm64 issue, but a Go 1.16 one. quic-go relies on internal implementation details of the standard library, so can break at every Go release.

The CPU spikes are probably golang/go#42700 and indeed can be worked around by GODEBUG=asyncpreemptoff=1.

[FiloSottile]

Ah, adding --tags noquic got me a working build.

May I suggest gating the quic files on go1.14,!go1.16 instead? This will ensure that the build works with any version automatically, and lets you opt in to building quic with a new version after you tested it.

[calmh]

The noquic tag is a quick hack to deal with quic package oddities, not something we really want to perpetuate or systematise I think... That is, I think I prefer requiring an action to confirm that you’re creating a broken build (lacking quic support) rather than silently excluding quic.

[imsodin]

@FiloSottile

May I suggest gating the quic files on go1.14,!go1.17 instead? This will ensure that the build works with any version automatically, and lets you opt in to building quic with a new version after you tested it.

That sounds like a good idea. Would you mind suggesting that on https://github.com/marten-seemann/qtls-go1-15 instead? That would make a runtime panic become a build error, which is so much better. And all dependencies of quic would actually profit, instead of everyone having to work around quic badness on their own. There's a better chance that they will take it seriously coming from the go security lead (there's a bit of a track record there of ignoring people until a big player says the same thing as the people).

[FiloSottile]

Hmm, it would definitely be better to fix this upstream, but in this case it's a bit tricky. Until Go 1.16 is actually released they have two options: tag !go1.16 and just break anyone trying to build with Go tip regardless of whether it works (which is bad because it would stop you, us, and anyone from testing Syncthing with Go tip); or tag !go1.17 and then change it to !go1.16 every time I change something in crypto/tls, which can happen multiple times per cycle, and which would anyway introduce lag during which the panic would come back.

Instead, since QUIC is optional in Syncthing, you can just tag !go1.16 until Go 1.16 is finalized (say, until Go 1.16rc1) and QUIC support was tested. Builds with Go tip would always drop QUIC, but that's much better than not working at all.

[vitobotta]

Hi all, I read the thread but I am not sure of what to try since I am not familiar with Go etc.... I am trying to get syncthing sync to work on file system changes on a Mac with Apple silicon. I am actually trying to get Okteto to work, and Okteto uses syncthing under the hood. Can anyone please tell me if you actually got the realtime sync working, and how to achieve it? Thanks!

[calmh]

Once there is a stable Go release for the M1 we'll release binaries. In the meantime your best bet is:

• Get the latest development version of Go ("tip") and build it using the appropriate procedure, and
• Get the latest development version of Syncthing and build it using our build instructions and -tags noquic.

In theory it might work. None of us on the team have the requisite hardware to test yet, so you're all early adopters for now.

[vitobotta]

Thanks @calmh. I am trying to compile Go from the master branch but it doesn't seem to be dying anything.... The script to build is not returning any errors but it's taking ages and the go process being used to build the new go is using just 2% of the cpu, so I don't think it's working properly :(

[calmh]

We cannot support Go's build process here, you need to turn to the Go project for that, or potentially do some research and testing on your own as this is all still in development and unreleased at this point.

[vitobotta]

Thanks.

[calmh]

This is a cross compiled arm64 build that might have working fs notifications, I'd be happy to know how/if it works or not:

https://build.syncthing.net/buildConfiguration/Syncthing_BuildMac/87329?buildTab=artifacts&guest=1

[vitobotta]

Hi @calmh, just tested and it seems that it doesn't pick up the file system changes in realtime unfortunately. For now I have configured a rescan every 2 seconds as a workaround since the project isn't too big and the rescan is pretty quick. I am going to try and use this build you linked to because the one that I can install with brew and that runs under rosetta is not very stable (it crashes after a while). Thanks!

[calmh]

My M1 arrived today, and the current build works for me with fs notifications, so that's cool. I added it to the 1.12.1-rc.1 release we just made, I encourage everyone to try that build and file any oddities you discovery. (We'll probably not actually make a 1.12.1 stable release for arm64 until we have quic and stable Go in place, but we can all surf the RCs and nightlies in the meantime.)

[vitobotta]

Hi @calmh which build is working for you with fs notifications? I would like to give it a try. Thanks!

[vitobotta]

@calmh the latest from here? https://build.syncthing.net

[calmh]

https://github.com/syncthing/syncthing/releases/tag/v1.12.1-rc.1

[vitobotta]

@calmh it seems to work fine, a huge thanks! :) What value do you use for fsWatcherDelayS?

[calmh]

I don't change the defaults.

[vitobotta]

Uhm maybe I was celebrating too soon. The default is 10 seconds, right? With that value the changes are propagated after ten seconds if I understood correctly. How can I have changes propagated in realtime? Is it OK to set fsWatcherDelayS to a lower number? Thanks

[calmh]

This is not the general settings help thread, please feel free to open a discussion on the forum for tweaking fs notification settings.

[AudriusButkevicius]

Is this question related to the new silicon? If not, general support questions go on the forum please.

[vitobotta]

Sorry.