Tree selector for ignore patterns

[p0l0us]

Purpose

As a user, I wish to have an easy way to select which files from other devices will be downloaded and managed in my syncthing folder.

The main problem with current ignores is, that a user doesn't know what files exists in the global tree without downloading all files to the device or checking other device(s). So I want to expose the global item tree in the folder ignore editor tab in a tree component, allowing the user to select files which should be downloaded (not ignored). The feature target is not complex ignores management - just simply choose what to download.

Tree selection overrides the ignore patterns by something like:

// Generated by tree selector
!/FolderToDownload
!/AnotherFolder/FileToDownload.bar
!/AnotherFolder/FileToDownload_2.bar
**

The tree component is not supposed to parse any other ignore patterns than those generated by itself. It is each user's responsibility to not screw format up when using the tree selector. If the first line "// Generated by tree selector" is missing or incorrect, all patterns will be replaced by "**" and the user can un-ignore items one by one.

// Generated by tree selector
**

A broader view of the feature will be clarified by the following PR draft:
#9130

Testing

Describe what testing has been done, and how the reviewer can test the change
if new tests are not included.

Screenshots

Not receive-only folders:

Receive-only folders:

Tree expanded:

Documentation

No API changes.

[acolomb]

Thank you for working on this. Have you seen #7145? It is a previous attempt at providing a similar feature, although sadly never finished. Might give you some inspiration or a basis to make your contribution most useful.

[tomasz1986]

One major issue with the previous attempt was that performance with larger numbers of files/folders was abysmal. It literally brought the whole GUI down, making it completely unresponsive, in the worst case for minutes.

Possibly related:

• Restore Versions modal extremely laggy and RAM-hungry with many items #7393
• gui: restore versions dialog fails to handle a significant amount of files #8835

[p0l0us]

One major issue with the previous attempt was that performance with larger numbers of files/folders

I just tried with 10 000 and 100 000 files in random folders, and it's not that bad. There is about 2s lag with 100 000 files when loading the tree. But I would expect such behavior.

Of course with higher number of ignore patterns parsing would take longer. What is reasonable performance we're aiming to ?

For now, I'm planning following improvements:

• Tree data loading spinner or a message when loading the tree
• Design similar to suggested PR with Basic and Advanced tabs.
• Remove limitation to receive-only folders if possible.

[tomasz1986]

I just tried with 10 000 and 100 000 files in random folders, and it's not that bad. There is about 2s lag with 100 000 files when loading the tree. But I would expect such behavior.

Of course with higher number of ignore patterns parsing would take longer. What is reasonable performance we're aiming to ?

2s lag with 100,000 files is totally fine, I would say. With the restore versions tree, the lag starts with 1,000+ files already, and the same was true for the tree generated in the other PR.

I'm going to test your PR later on using a similar setup as before, as it would be absolutely great if you've actually managed to solve the tree lag problem.

Just for the record, in those other trees the lag happens when there are 1,000+ files in the same folder (e.g. the root folder which is always loaded).

[p0l0us]

I did more testing of the performance and did some tweaks:

• I believe, limited height of FancyTree improved the initialization performance when 10 000 files in root folder since UI creates less items. It is also the reason why all nodes should be closed by default (TBD).
• I used web worker to parse data from db/browse service to FancyTree format.
• Data for FancyTree are pre-formatted in background once loaded. Loader is triggered by Edit Folder modal open. So most likely FancyTree data will be prepared before user even opens Ignore Patterns tab.

Limitation to receive-only folders removed.

Tree data loading message

Switching between Basic and Advanced patterns in Advanced settings tab and stored in folder config

Tree now takes whole space of Ignore Pattern tab and height is resizable.

[tomasz1986]

The styles here seem like a copy from #restoreTree-container. Is this correct? If yes, then I think it would be nice to just create a common class with these to add to the both elements to avoid the repetition.

[p0l0us]

Yes, I just copied that.

I will look at this later, when I know the style is final. min-height and max-height might need some tuning.

[ismay]

Is this PR still on the table? Would be great to have, it's been such a highly anticipated feature.

[EmberLightVFX]

Writing a push on this PR also. Getting this into Syncthing would change a lot on how we're currently using it and for the average user this tree view instead will be way easier than learning git-ignore patterns.

@p0l0us What's the state of this PR? Anything you feel is missing?

[galilley]

Is it possible to add some user defined global ignore patterns to exclude temporary files like ".~", ".pyc" etc?

Also I would like to offer some extension to see the size (remote/local) of folders in the tree.