Disk Usage migration towards Android 11

This is the current status of migration of disk usage for Android 11 where direct usage of public folders using File API is no longer allowed.

Status is as of 16.03.21: SAF is available on release branch and related issues should be fixed there.

Implementation status is tracked in GitHub project: https://github.com/cgeo/cgeo/projects/17

Abbrevations are used as defined in Disk Usage Wiki page here: https://github.com/cgeo/cgeo/wiki/Disk-usage-structure

Concepts

Principal concept

The concept of a "persistable" Folder is introduced (class "PersistableFolder"). For each "public folder" an (enum) instance of this class is created

• Each public folder is in principal user-definable by the end user. However, this option is only offered so far for directories BASE, OFFLINE_MAPS and OFFLINE_MAPS_THEMES.
• Each public folder has a Default-Location which is used when the user has not (or cannot) select an own location
  • Default-Locations cannot be SAF/Documents-based because those have to be explicitely selected. So they are either file-based or based on another public folder's location (which in turn may be SAF-based).
  • All folders have a sensible "default" setting which is taken in case the user has not done her own selection OR the currently user-defined location is not accessible.
  • Folders may have multiple default options. In this case the first default which is still accessible is used. That way we can "gracefully" migrate away from the existing "/cgeo" base folder while still providing access to it as long as Android 11 permits.
  • If none of the default options is accessible then a folder in the PRIVATE cgeo section is chosen as a last-resort. This way it is ensured that each public folder is always accessible to c:geo
• Each folder MAY store a User-defined location as well. This is a location which was explicitely selected by user using SAF dialogs.
  • If the user self-selects a user-defined folder, it is always a SAF/Document-based folders
  • For legacy-reasons user-defined location may be file-based locations too (in case of existing settings for offline-maps/-themes dir not yet changed by user), but those are then only assigend by system (not by user).
• Migration
  • On each user-initiated folder-change , the user gets the option to move or copy files from old folder to new one. Default is "do nothing" (TODO: other values may be discussed, "Move" was suggested)
  • This, combined with the default-mechanism to legacy-folders above, gives full migratability to users
  • On each c:geo startup, it is checked whether the BASE-folder is used-defined AND accessible. If not, a reminder is triggered to the user to change this.
  • This startup reminder shall be integrated into the Installation Wizard

Folder selection

Each folder has one of three "Types"

• T (transient): folder is selected on-the-fly by user, used for single use case and then forgotten
• D (defaulted): folder has a fixed default value which can not be overridden by user
• U (user-selectable): folder can be freely selected by user, but has a default location as fallback in case the user doesn't do that

Folder	Type	Default(s)	Special
base	U	file:/cgeo, file://documents/cgeo	-
backup	D	/backup	-
gpx	U	/gpx	gpx import and gpx export are merged into one dir. dir is used as initial dir for gpx import, also for exports of cache-gpx, tracks, ind routes and trailhistory. Has legacy user-defined value: legacy gpx-import is preferred over gpx-export if both are set.
logfiles	D	/logfiles	Stores logfiles, heap dump, LogWriter-files and log configuration
field-notes	D	/field-notes	-
Offline Log images	-	No longer publicly stored	-
maps	U	/maps	Has legacy user-defined value
map themes	U	/themes	Has legacy user-defined value
backup restore	T	subdirs of "backup"	Init dir: backup. Needed to restore other backup than default
Spoilers	D	/GeocachePhotos	See https://manual.cgeo.org/en/spoilersync
Test Folder	U	(none)	Special folder used for unit testing of SAF framework. Only visible in Debug-mode

Single File Selection

Single File	Persistent/Transient	Initial dir on select
GPX import	T	gpx import
Track	P	gpx import
Ind. Route	T	gpx import
Log image	T	none (decided by system)

Using the framework

All Framework classes can be found in package cgeo.geocaching.storage.

• A File or Directory in SAF is represented by its Uri (android.net.Uri).
• To unify Folder handling though, directories were further abstracted in the new class cgeo.geocaching.storage.Folder
• All public folders have an instance in the enum PersistableFolder
• To read from or write into files, methods readFromUri resp writeToUri of class ContentStorage are used

A code example showing basic function usage can be found in test class cgeo.geocaching.storage.ContentStorageTest, method testSimpleExample

    public void testSimpleExample() throws IOException {

        //This is a simple example for usage of contentstore

        //List the content of the LOGFILES directory, write for each file its name, size and whether it is a directory
        final List<ContentStorage.FileInformation> files = ContentStorage.get().list(PersistableFolder.LOGFILES);
        for (ContentStorage.FileInformation file : files) {
            Log.i("  File: " + file.name + ", size: " + file.size + ", isDir: " + file.isDirectory);
        }

        //Create a new subfolder with name "my-unittest-subfolder" in LOGFILES
        final Folder mysubfolder = Folder.fromPersistableFolder(PersistableFolder.LOGFILES, "my-unittest-subfolder");
        ContentStorage.get().ensureFolder(mysubfolder, true);

        //Create a new file in this subfolder
        final Uri myNewFile = ContentStorage.get().create(mysubfolder, "myNewFile.txt"); //in prod code: check for null!

        //write something to that new File
        Writer writer = null;
        try {
            final OutputStream os = ContentStorage.get().openForWrite(myNewFile); //in prod code: check for null!
            writer = new OutputStreamWriter(os, "UTF-8");
            writer.write("This is a test");

        } finally {
            IOUtils.closeQuietly(writer);
        }

        //read the same file out again
        BufferedReader reader = null;
        try {
            final InputStream is = ContentStorage.get().openForRead(myNewFile); //in prod code: check for null!
            reader = new BufferedReader(new InputStreamReader(is, "UTF-8"));
            final String line = reader.readLine();
            assertThat(line).isEqualTo("This is a test");

        } finally {
            IOUtils.closeQuietly(reader);
        }

        //delete the created file
        ContentStorage.get().delete(myNewFile);

        //delete the created folder
        ContentStorage.get().delete(mysubfolder.getUri());

        //check out the other functions of ContentStorage
        //More complex operations (e.g. copyAll, deleteAll) can be found in class FolderUtils.
        
    }