Thoughts on creating a new macOS desktop client

[claucambra]

Before starting, this is a discussion thread -- not an announcement! After some brief internal discussion we thought it best to continue our discussions about this in public.

Over the past year I have been working on implementing native virtual files into the desktop client via a new "File Provider module", which leverages Apple's system frameworks to provide a synchronisation experience akin to VFS on other platforms. We've received a lot of feedback and there is still work left to do before it is out of beta, but things are moving along well! 🙂

The File Provider module will be launched as part of a major desktop client release in the next few months. However, over the process of developing this module, I've come to the conclusion that we might be better off splitting the module off into a new, independent desktop client.

VFS on Windows leverages the desktop client's existing user interface components and core synchronisation engine, integrating it with Windows-specific APIs to present virtual files to the user in a seamless way via Explorer. However, this is not at all the case on macOS. Using the File Provider framework has meant building a new sync engine based around the File Provider APIs and NextcloudKit, the framework used by the iOS Nextcloud client.

The File Provider module exists as an entirely different process, which only communicates with the main desktop client to request authentication details, and with recent additions, to handle presenting settings in the desktop client UI. Everything else is handled by the file provider module independently -- file sync, push notification handling, file sharing, etc etc. Due to the nature of the File Provider APIs this is going to continue being the case; Apple gives us a specific way to present UIs to the user via Finder modals, we have to interact with File Provider via Objective-C/Swift, and so on.

Building this stuff has also required building a ton of code to find ways to connect these two halves, the client and the File Provider module. First, lots of Objective-C++ classes abstracting away the internal Objective-C code from our normal Qt code; second, a new socket-based interface allowing the File Provider module to request information from the desktop client; third, an XPC interface allowing the desktop client to request information from the File Provider module.

Detaching the File Provider module from the current desktop client and building it into a new, independent, macOS-specific client would achieve a few things.

First, it would allow us to simplify both the File Provider module and the desktop client, eliminating the need for the first and second components discussed in the previous paragraph. This would, by extension, also help eliminate a significant source of bugs; our socket based communication, for example, is finicky and requires some hacky workarounds.

Second, it would allow us to speed up development and improve maintenance. Currently, our desktop client development team is primarily a C++/Qt based team -- which is great, as that's what makes up the overwhelming majority of the client code! However, creating a desktop client for macOS that further leverages the technologies used on the Apple side of things, we will be able to integrate this new desktop client's development more closely with the iOS team that is already building the framework that makes up the backbone of the File Provider module. This would also allow me to continue development of this module without bugging the multiplatform desktop client's other developers for reviews on code that is unfamiliar to them and that they can't test. Additionally, we would be able to use well-established and well-documented tooling during development and in debugging that we currently cannot use due to the funky way in which we are currently forced to build and package the client with the file provider module. This has been a huge pain point for the last year. Adjacent to this, we would stop dealing with the particularities of Qt->Objective-C++->Objective-C->App Extension interfacing that I deal with now, which has slowed down development and is an area generally poorly documented. Additionally, we'd also be able to leverage existing build and deployment infrastructure used for the iOS client.

Third, it would improve our integrations with macOS. Rather than using Qt components for our UI-facing components, we'd be able to leverage AppKit everywhere, making the macOS desktop client feel just like Apple's own applications. Additionally, we'd be able to more easily integrate with future macOS APIs, and would be able to leave behind many of the components using deprecated macOS APIs in the multiplatform desktop client.

Fourth, it would allow us to more easily publish the client on the Mac App Store.

The good news is that all of this would come at a relatively low development cost. If we ignore synchronisation features (which we'd have to reimplement if we kept the File Provider module attached to the desktop client anyway), there are only a couple of components we'd have to duplicate from the multiplatform desktop client: authentication, user settings, and activities. Fortunately, most of the core code needed to implement this is already present in NextcloudKit, meaning we'd mainly just need to build the UI components.

The migration path would also be trivial, as we'd be able to upgrade users in-place from the multiplatform desktop client to the File Provider-based desktop client seamlessly.

Thoughts welcome! @tobiasKaminsky

[marcotrevisan]

Thanks for the thorough explanation and for your decision to collect feedback from the community!
Even if I'm not a NC team developer I'm in still an IT professional, and it's very clear to me that the above proposal makes a lot of sense.
I would like to emphasize, if not already clear, that the proposed change would generally remove project risks (by lowering its complexity), in addition to speeding up the development life cycle as described.

It's just an idea in my mind but the new standalone implementation may also help for finding a newer architecture to use on the other plaftorms if needed. I mean: once the Swift code has stabilized, it might be an idea to mimic the framework behaviour in a cross platform "IoC micro container" that would call the client code. It may be a laughable idea but let me throw it... :-)

Thanks again and regards!

[hrdwdmrbl]

This sounds like an extremely promising development! Given the great work that @claucambra has been doing, he has my full backing.

Additionally, I think integrating with the native platform, in all cases, is generally going to give the best user experience.

Kudos to all working hard to make this and everything else at Nextcloud a reality. 👍

[Saifudaullah]

I agreed on all of the points you brought up. It is indeed better to develop a separate client for macos so it will be easier to debug the code.

[wrench12435678]

I agree with this as well, if wheels were set into motion, how long do you think it would be until a first alpha? (weeks, months etc)

Either way thanks for sticking with development throughout this :)

[claucambra]

Way too early to tell at the moment. I suspect the more momentum this discussion gets, the more likely it'll happen (and more quickly) :)

[marcotrevisan]

I also see repos like "SomethingKit" popping up, which is a very nice thing in my humble opinion.

[dp466]

that would be so nice
count me in