Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

iOS 13 + 14 arm64e fixes: support for architecture-specific schemas and schema-specific toolchains #574

Draft
wants to merge 6 commits into
base: master
Choose a base branch
from

Conversation

kabiroberai
Copy link
Member

@kabiroberai kabiroberai commented Feb 16, 2021

(Thanks for sending a pull request! Please make sure you click the link above to view the contribution guidelines, then fill out the blanks below.)

What does this implement/fix? Explain your changes.

This PR adds support for modifying the Theos schema depending on the architecture, including the ability to build a single architecture multiple times using different schemas. It also makes variables like PREFIX, SDKBINPATH, THEOS_PLATFORM_SDK_ROOT respect schema overrides, so you can specify a custom toolchain based on the schema+arch combination.

Does this close any currently open issues?

Closes #563.

Any relevant logs, error output, etc?

(If it’s long, please paste to https://ghostbin.com/ and insert the link here.)
Nope

Any other comments?

You do unfortunately need to use a patched lipo if you're building the multiple schemas for the same architecture. As a temporary solution, the patched binary linked here should work: #563 (comment).

Using these changes, you can build a binary containing slices with both the iOS 13 and iOS 14 arm64e ABIs by putting something like this at the top of your Makefile (note that the older toolchain must correspond to the schema that's listed first for that cpu subtype in ARCHS, so, e.g. ARCHS = arm64 arm64e:v2 arm64e:v1 would not work – at least for apps if not for libraries):

ARCHS = arm64 arm64e:v1 arm64e:v2

TARGET_LIPO = /path/to/patched/lipo
V1.THEOS_PLATFORM_SDK_ROOT = /Applications/Xcode_11.app/Contents/Developer

include $(THEOS)/makefiles/common.mk

# ...

A few things we should do before merging this:

  • Confirm that everything works on Linux (particularly schema-specific SDKBINPATH/SWIFTBINPATH/PREFIX variables).
  • Confirm that this works with static linking (cc @sbingner).
  • Discuss the architectural strategy of using schemas for this before we merge – I do personally think it's an elegant solution but I'm open to alternatives too.
  • Figure out a long-term solution for patching lipo (maybe maintaining our own fork?)

Where has this been tested?

Operating System: macOS 11.1 (20C69)

Platform: macOS

Target Platform: iOS (arm64, arm64e iOS 13, arm64e iOS 14)

Toolchain Version: Xcode 11.7 + Xcode 12.3

SDK Version: iOS 13.7 + iOS 14.3

@Matchstic
Copy link

Whilst I’m happy to leave it online for now, the patched lipo probably wants hosting somewhere a bit safer than Dropbox. Long term I believe the idea should be that is handled by something a bit less hacky?

@kabiroberai
Copy link
Member Author

For sure, sorry if that wasn't clear – I do want to figure out a more long-term solution for lipo before merging. I'll add that to the list of to-dos in the PR.

@r-plus
Copy link
Contributor

r-plus commented Feb 17, 2021

Looks great 👍

This is the list that seems to be good to add as todo for after discuss architecture phase.

  • Add newly variables as option (commented out by default) to NIC templates is maybe helpful.
  • Add output warning when developer build without this multiple arm64e slice for deployment version is less than iOS 14.0.

@danpashin
Copy link

How about supporting FINALPACKAGE? I think something is missed somewhere 'cause it perfectly compiles without this flag but when it is set it compiles both shemas in the one architecture directory.

Without flag
without_flag

With flag
with_flag

@kabiroberai
Copy link
Member Author

@r-plus it turns out iOS 13 arm64e libraries (including tweaks) might work on iOS 14 so if that's the case we probably won't update the template. We're still investigating the matter.

@danpashin thanks for the report, I'll look into the issue.

@kabiroberai
Copy link
Member Author

@danpashin I believe 85a8cd6 should fix your issue; could you please confirm that this is indeed the case?

@danpashin
Copy link

@kabiroberai yup, works fine. Thanks :)

@r-plus
Copy link
Contributor

r-plus commented Feb 17, 2021

nicely works on M1 mac! (Rosetta; patched lipo binary contains only x86_64 for now)
with schema arm64e:x11 and un-schema arm64e

Makefile

export ARCHS = armv7 arm64 arm64e:x11 arm64e
export TARGET = iphone:clang:11.2:8.0
TARGET_LIPO = /usr/local/bin/plipo
X11.THEOS_PLATFORM_SDK_ROOT = /Applications/Xcode_117.app/Contents/Developer

include $(THEOS)/makefiles/common.mk
...

result confirmations

$ file .theos/obj/debug/DeleteCut.dylib
.theos/obj/debug/DeleteCut.dylib: Mach-O universal binary with 4 architectures: [arm_v7:Mach-O dynamically linked shared library arm_v7] [arm64:Mach-O 64-bit dynamically linked shared library arm64] [arm64e:Mach-O 64-bit dynamically linked shared library arm64e] [arm64e:Mach-O 64-bit dynamically linked shared library arm64e]
.theos/obj/debug/DeleteCut.dylib (for architecture armv7):	Mach-O dynamically linked shared library arm_v7
.theos/obj/debug/DeleteCut.dylib (for architecture arm64):	Mach-O 64-bit dynamically linked shared library arm64
.theos/obj/debug/DeleteCut.dylib (for architecture arm64e):	Mach-O 64-bit dynamically linked shared library arm64e
.theos/obj/debug/DeleteCut.dylib (for architecture arm64e):	Mach-O 64-bit dynamically linked shared library arm64e
$ otool -h .theos/obj/debug/DeleteCut.dylib
.theos/obj/debug/DeleteCut.dylib (architecture armv7):
Mach header
      magic  cputype cpusubtype  caps    filetype ncmds sizeofcmds      flags
 0xfeedface      12          9  0x00           6    22       2596 0x00100085
.theos/obj/debug/DeleteCut.dylib (architecture arm64):
Mach header
      magic  cputype cpusubtype  caps    filetype ncmds sizeofcmds      flags
 0xfeedfacf 16777228          0  0x00           6    22       3024 0x00100085
.theos/obj/debug/DeleteCut.dylib (architecture arm64e):
Mach header
      magic  cputype cpusubtype  caps    filetype ncmds sizeofcmds      flags
 0xfeedfacf 16777228          2  0x00           6    22       3024 0x00100085
.theos/obj/debug/DeleteCut.dylib (architecture arm64e):
Mach header
      magic  cputype cpusubtype  caps    filetype ncmds sizeofcmds      flags
 0xfeedfacf 16777228          2  0x80           6    23       3024 0x00100085

@foxfortmobile
Copy link

@kabiroberai Is it necessary to have both Xcode 11 and 12 installed to build a tweak? I have Xcode 12 installed but have an Xcode 11 toolchain in my theos directory. How can i specify theos to use that toolchain to compile the v1 slice?
Tried something like this:

ARCHS = arm64 arm64e:v1 arm64e:v2
TARGET_LIPO = /usr/local/bin/plipo
V1.THEOS_PLATFORM_SDK_ROOT = $(THEOS)/toolchain/Xcode11.xctoolchain/

but it fails:

...
==> Compiling Tweak.xm (arm64e:v1)…
bash: -x: command not found

@r-plus How do you manage to run otool on an executable with multiple arm64e slices? I always got this error if i try:

otool -h .theos/obj/debug/Translomatic.dylib 
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/objdump: 
error: '.theos/obj/debug/Translomatic.dylib': truncated or malformed fat file (contains two of the same architecture (cputype (16777228) cpusubtype (2)))

@kabiroberai
Copy link
Member Author

@foxfortmobile using a default Xcode toolchain outside the host Xcode app is not supported and is prone to breakage. You should have both Xcode versions installed if you want to build two arm64e slices. Like I said before though, there's a chance Xcode 11 alone should be enough to build tweaks for both iOS 13 and 14, which is something we'll try to confirm soon. If that's indeed the case, dual-toolchain support will only be required for apps and command line tools that need to use arm64e for whatever reason.

@kirb
Copy link
Collaborator

kirb commented Feb 18, 2021

@foxfortmobile THEOS_PLATFORM_SDK_ROOT expects a Developer directory rather than a bare xctoolchain, since this value gets passed to xcrun when we later start calling things in the toolchain.

You may be able to make it work by creating a structure like:

theos/toolchains/Xcode11Developer/Toolchains/XcodeDefault.xctoolchain/

But no guarantees whether xcrun will understand that.

@kabiroberai
Copy link
Member Author

@foxfortmobile THEOS_PLATFORM_SDK_ROOT expects a Developer directory rather than a bare xctoolchain, since this value gets passed to xcrun when we later start calling things in the toolchain.

You may be able to make it work by creating a structure like:

theos/toolchains/Xcode11Developer/Toolchains/XcodeDefault.xctoolchain/

But no guarantees whether xcrun will understand that.

You could also set V1.PREFIX instead and provide the path to the toolchain's bin directly but again using a default toolchain outside Xcode is absolutely not recommended – for example, it tends to break Swift entirely. I do want to look into whether it's possible to use one of the OSS Swift toolchains instead, because those don't require a "host" Xcode installation.

@foxfortmobile
Copy link

@foxfortmobile using a default Xcode toolchain outside the host Xcode app is not supported and is prone to breakage. You should have both Xcode versions installed if you want to build two arm64e slices. Like I said before though, there's a chance Xcode 11 alone should be enough to build tweaks for both iOS 13 and 14, which is something we'll try to confirm soon. If that's indeed the case, dual-toolchain support will only be required for apps and command line tools that need to use arm64e for whatever reason.

Some time ago there was the issue that many devs were having regarding Xcode 12 compiled tweaks were not working on A12+ devices. One workaround suggested on r/jailbreakdevelopers was to use an xcode 11 toolchain as theos prefix
That's how i have been compiling my tweaks for a while. I was not aware that it was not a good solution.

I tried doing the following but it had same errror.
V1.THEOS_PLATFORM_SDK_ROOT = $(THEOS)/toolchain/Xcode11.xctoolchain/usr/bin

So i set it to full Xcode 11 developer as advised and it compiled successfully.

(base) foxfortmobile@Foxforts-MacBook-Pro vizion % make package FINALPACKAGE=1 DEBUG=0
> Making all for bundle com.foxfort.vizion…
==> Copying resource directories into the bundle wrapper…
==> Warning: No files to link - creating a bundle containing only resources
> Making all for tweak vizion…
==> Preprocessing Tweak.xm…
==> Compiling Tweak.xm (arm64e:v1)…
==> Compiling AverageStackingFilter.m (arm64e:v1)…
==> Linking tweak vizion (arm64e:v1)…
ld: warning: building for iOS, but linking in .tbd file (/Users/foxfortmobile/theos/vendor/lib/CydiaSubstrate.framework/CydiaSubstrate.tbd) built for iOS Simulator
==> Generating debug symbols for vizion…
==> Stripping vizion (arm64e:v1)…
rm /Users/foxfortmobile/Documents/workspace/blacksight/vizion/.theos/obj/V1/arm64e/Tweak.xm.mm
==> Preprocessing Tweak.xm…
==> Compiling Tweak.xm (arm64e:v2)…
==> Compiling AverageStackingFilter.m (arm64e:v2)…
==> Linking tweak vizion (arm64e:v2)…
ld: warning: building for iOS, but linking in .tbd file (/Users/foxfortmobile/theos/vendor/lib/CydiaSubstrate.framework/CydiaSubstrate.tbd) built for iOS Simulator
==> Generating debug symbols for vizion…
==> Stripping vizion (arm64e:v2)…
rm /Users/foxfortmobile/Documents/workspace/blacksight/vizion/.theos/obj/V2/arm64e/Tweak.xm.mm
==> Merging tweak vizion…
==> Signing vizion…
> Making stage for bundle com.foxfort.vizion…
> Making stage for tweak vizion…
dm.pl: building package `com.foxfort.vizion:iphoneos-arm' in `./packages/com.foxfort.vizion_0.0.1_iphoneos-arm.deb'
file  .theos/obj/vizion.dylib
.theos/obj/vizion.dylib: Mach-O universal binary with 3 architectures: [arm64:Mach-O 64-bit dynamically linked shared library arm64] [arm64e:Mach-O 64-bit dynamically linked shared library arm64e] [arm64e:Mach-O 64-bit dynamically linked shared library arm64e]
.theos/obj/vizion.dylib (for architecture arm64):       Mach-O 64-bit dynamically linked shared library arm64
.theos/obj/vizion.dylib (for architecture arm64e):      Mach-O 64-bit dynamically linked shared library arm64e
.theos/obj/vizion.dylib (for architecture arm64e):      Mach-O 64-bit dynamically linked shared library arm64e

How do verify it using otool?

 otool -h .theos/obj/vizion.dylib
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/objdump: 
error: '.theos/obj/vizion.dylib': truncated or malformed fat file (contains two of the same architecture (cputype (16777228) cpusubtype (2)))

By the way i have some tweaks linked against frameworks like opencv compiled with xcode 12. Will it need special handling on my side when used in a tweak with multiple arm64e slices?

@kabiroberai
Copy link
Member Author

You might want to include an Xcode 11 arm64e slice in the framework.

@r-plus
Copy link
Contributor

r-plus commented Feb 21, 2021

@foxfortmobile FYI about otool

  • M1 mac mini 11.2.1
  • Xcode 12.3
$ otool --version
llvm-otool(1): Apple Inc. version cctools-977.1
otool(1): Apple Inc. version cctools-977.1
disassmbler: LLVM version 12.0.0, (clang-1200.0.32.28)

arm64

arm64e version of otool -h (Mach header) will display only single header.

$ otool -h .theos/obj/debug/DeleteCut.dylib
.theos/obj/debug/DeleteCut.dylib:
Mach header
      magic  cputype cpusubtype  caps    filetype ncmds sizeofcmds      flags
 0xfeedfacf 16777228          0  0x00           6    22       3024 0x00100085

So we can confirm it by -f universal header.

$ otool -f .theos/obj/debug/DeleteCut.dylib
Fat headers
fat_magic 0xcafebabe
nfat_arch 4
architecture 0
    cputype 12
    cpusubtype 9
    capabilities 0x0
    offset 16384
    size 84176
    align 2^14 (16384)
architecture 1
    cputype 16777228
    cpusubtype 0
    capabilities 0x0 <-- ABI version 0 of arm64e by Xcode 11
    offset 114688
    size 85264
    align 2^14 (16384)
architecture 2
    cputype 16777228
    cpusubtype 2
    capabilities 0x0
    offset 212992
    size 84912
    align 2^14 (16384)
architecture (illegal duplicate architecture) 3
    cputype 16777228
    cpusubtype 2
    capabilities 0x80 <-- yah! ABI version 128 of arm64e by Xcode 12!
    offset 311296
    size 84656
    align 2^14 (16384)

x64

x64 version of otool can confirm it simply by -h option. (-f also works)

$ otool -h .theos/obj/debug/DeleteCut.dylib
.theos/obj/debug/DeleteCut.dylib (architecture armv7):
Mach header
      magic  cputype cpusubtype  caps    filetype ncmds sizeofcmds      flags
 0xfeedface      12          9  0x00           6    22       2596 0x00100085
.theos/obj/debug/DeleteCut.dylib (architecture arm64):
Mach header
      magic  cputype cpusubtype  caps    filetype ncmds sizeofcmds      flags
 0xfeedfacf 16777228          0  0x00           6    22       3024 0x00100085
.theos/obj/debug/DeleteCut.dylib (architecture arm64e):
Mach header
      magic  cputype cpusubtype  caps    filetype ncmds sizeofcmds      flags
 0xfeedfacf 16777228          2  0x00           6    22       3024 0x00100085
.theos/obj/debug/DeleteCut.dylib (architecture arm64e):
Mach header
      magic  cputype cpusubtype  caps    filetype ncmds sizeofcmds      flags
 0xfeedfacf 16777228          2  0x80           6    23       3024 0x00100085

intel x64

But yes, otool on intel mac display error you showed.
tested macOS 10.15.4 + Xcode 12.1, cctools- 973.0.1
not yet tested on BigSur.

Update: BigSur contained otool version cctools-977.1 executable for this purpose on intel mac.

@NSExceptional
Copy link
Contributor

NSExceptional commented Mar 1, 2021

You should have both Xcode versions installed if you want to build two arm64e slices.

@kabiroberai Surely there's a way to do this for the next several without requiring two copies of Xcode? 😕 (Such as using an OSS toolchain like you said and mentioned to me)

I guess that's the feedback that I have on this feature:

  1. Ideally shouldn't require two versions of Xcode, as having two copies of Xcode is almost more frustrating than it's worth
  2. If 1. comes through, Theos should standardize where it expects you to put the v1 toolchain so that you can compile a tweak for iOS 13+ for arm64 arm64e without any additional flags in your Makefile

@kabiroberai
Copy link
Member Author

@NSExceptional see #563 (comment) and the comments prior to that

@lizhongkan
Copy link

Xcode11 can not running on macOS Monterey.

@kirb
Copy link
Collaborator

kirb commented Nov 10, 2021

You can still use its command line tools by using xcode-select -switch, you just won’t be able to use Xcode.app itself. See our arm64e docs for instructions on that. Of course, can’t guarantee it’ll work forever, unfortunately. (As-is it’s already a little difficult to get Xcode 11 working on Apple Silicon Macs due to it running in Rosetta, and now we have this inconvenience of the app being blocked on Monterey.)

@lizhongkan
Copy link

lizhongkan commented Nov 10, 2021

I used the lipo on https://www.dropbox.com/s/6h85hlc4sm14zeg/lipo?dl=0. tested on macOS Monterey 12.0.1, Xcode 13.1, Xcode 11.5. When compiling the tweak, it failed:
==> Signing AbcTweak…
error: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/codesign_allocate: fat file: /path/to/tweak/.theos/obj/debug/AbcTweak.dylib.47ba6b93.unsigned contains two of the same architecture (cputype (16777228) cpusubtype (2))
ldid/ldid.cpp(989): _assert(WEXITSTATUS(status) == 0); errno=0

My makefile added:
ARCHS = armv7s arm64 arm64e:x11 arm64e
TARGET_LIPO = /path/to/pathed/lipo
x11.THEOS_PLATFORM_SDK_ROOT = /Applications/Xcode_11.5.app/Contents/Developer

Please help me to fix this error?

@prathumca
Copy link

You can still use its command line tools by using xcode-select -switch, you just won’t be able to use Xcode.app itself. See our arm64e docs for instructions on that. Of course, can’t guarantee it’ll work forever, unfortunately. (As-is it’s already a little difficult to get Xcode 11 working on Apple Silicon Macs due to it running in Rosetta, and now we have this inconvenience of the app being blocked on Monterey.)

@kabiroberai @r-plus

My DEV Environment:

Device: 16" M1 Max, 10C/32C GPU

Operating System: macOS 12.0.1

Platform: macOS

Target Platform: iOS (arm64, arm64e iOS 13, arm64e iOS 14)

Toolchain Version: Xcode 11.7 + Xcode 13.1

SDK Version: iOS 13.7 + iOS 15

Issue 1: THEOS_OBJ_DIR_NAME is causing some sort of trouble & below is the log

prathumca@M1-Max-MacBook-Pro LockWidgets % make
swift-driver version: 1.26.9 > Making all in liblockwidgets…
swift-driver version: 1.26.9 > Making all for library liblockwidgets…
swift-driver version: 1.26.9 swift-driver version: 1.26.9 swift-driver version: 1.26.9 swift-driver version: 1.26.9 swift-driver version: 1.26.9 ==> Compiling MRYIPCCenter.m (arm64e:x11)…
==> Compiling MRYIPCCenter.m (arm64e)…
==> Compiling LockWidgetsConfig.m (arm64e:x11)…
==> Compiling LockWidgetsConfig.m (arm64e)…
==> Compiling LockWidgetsView.mm (arm64e)…
==> Compiling LockWidgetsView.mm (arm64e:x11)…
==> Linking library liblockwidgets (arm64e)…
==> Generating debug symbols for liblockwidgets…
==> Linking library liblockwidgets (arm64e:x11)…
==> Generating debug symbols for liblockwidgets…
==> Compiling MRYIPCCenter.m (arm64)…
==> Compiling LockWidgetsConfig.m (arm64)…
==> Compiling LockWidgetsView.mm (arm64)…
==> Linking library liblockwidgets (arm64)…
==> Generating debug symbols for liblockwidgets…
==> Merging library liblockwidgets…
fatal error: /usr/local/bin/plipo: can't open input file: /Users/prathumca/Desktop/LockWidgets/.theos/../output/x11/arm64e/liblockwidgets.dylib (No such file or directory)
make[3]: *** [/opt/theos/makefiles/instance/library.mk:51: /Users/prathumca/Desktop/LockWidgets/.theos/../output/liblockwidgets.dylib.47ba6b93.unsigned] Error 1
make[2]: *** [/opt/theos/makefiles/instance/library.mk:37: internal-library-all_] Error 2
make[1]: *** [/opt/theos/makefiles/master/rules.mk:117: liblockwidgets.all.library.variables] Error 2
make: *** [/opt/theos/makefiles/master/aggregate.mk:12: internal-all] Error 2
prathumca@M1-Max-MacBook-Pro LockWidgets %

The entire code is available here along with all MakeFiles.

Issue 2: The generated dylibs are not compatible on iOS 13.5 (iPhone 11 Pro Max). I'm not certain the linked patched lipo is causing trouble or something else.

I see below long on console from my device:

Error loading /Library/PreferenceBundles/LockWidgetsPreferences.bundle/LockWidgetsPreferences: dlopen(/Library/PreferenceBundles/LockWidgetsPreferences.bundle/LockWidgetsPreferences, 265): no suitable image found. Did find:
/Library/PreferenceBundles/LockWidgetsPreferences.bundle/LockWidgetsPreferences: no matching architecture in universal wrapper
/Library/PreferenceBundles/LockWidgetsPreferences.bundle/LockWidgetsPreferences: no matching architecture in universal wrapper

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Xcode 12 toolchain generated binary for arm64e is not compatible lower than iOS 14.
10 participants