NPI-3197 Transfer download functions from ginan repo

[ronaldmaj]

Recently we had an update to the auto_download_PPP.py script over on the Ginan repo. A number of new and useful functions were added there but have a more appropriate home here on gnssanalysis.

This PR transfers some of those useful functions from https://github.com/GeoscienceAustralia/ginan/blob/develop-weekly/scripts/auto_download_PPP.py togn_download.py.

This should be the first in a series of PRs where we slowly move to using the newer download functions. This will include:

• moving existing utilities that use the old download functions to using the new functions (on gnssanalysis and ginan)
• deleting old functions that have been superseded with new functions
• updating / documenting existing functions that aren't deleted

Happy to discuss this further in the comments of this PR or in the discussions tab of this repo

[seballgeyer]

A few possible changes

[seballgeyer]

Maybe a bit too much if/elif

def generate_uncompressed_filename(filename: str) -> str:
    """Returns a string of the uncompressed filename given the [assumed compressed] filename

    :param str filename: Original filename of compressed file
    :return str: The uncompressed filename based on input (returns input filename if compression not recognised)
    """
    # Define a dictionary to map file extensions to their corresponding actions
    actions = {
        ".tar.gz": lambda f: _tarfile.open(f, "r").getmembers()[0].name,
        ".tar": lambda f: _tarfile.open(f, "r").getmembers()[0].name,
        ".crx.gz": lambda f: f[:-6] + "rnx",
        ".gz": lambda f: f[:-3],
        ".Z": lambda f: f[:-2],
        ".bz2": lambda f: f[:-4],
    }

    # Iterate over the dictionary items
    for ext, action in actions.items():
        if filename.endswith(ext):
            return action(filename)

    # If no matching extension is found, log a debug message and return the original filename
    logging.debug(f"{filename} not compressed - extension not a recognized compression format")
    return filename

or if you assume there is no . in the filename

def generate_uncompressed_filename(filename: str) -> str:
    """Returns a string of the uncompressed filename given the [assumed compressed] filename

    :param str filename: Original filename of compressed file
    :return str: The uncompressed filename based on input (returns input filename if compression not recognised)
    """
    # Define a dictionary to map file extensions to their corresponding actions
    actions = {
        "tar.gz": lambda f: _tarfile.open(f, "r").getmembers()[0].name,
        "tar": lambda f: _tarfile.open(f, "r").getmembers()[0].name,
        "crx.gz": lambda f: f[:-6] + "rnx",
        "gz": lambda f: f[:-3],
        "Z": lambda f: f[:-2],
        "bz2": lambda f: f[:-4],
    }

    # Split the filename on the '.' and use the last part as the extension
    ext = filename.split('.')[-1]

    # Use the dictionary to get the action for the extension, if it exists
    action = actions.get(ext)

    # If an action was found, execute it, otherwise log a debug message and return the original filename
    if action:
        return action(filename)
    else:
        logging.debug(f"{filename} not compressed - extension not a recognized compression format")
        return filename

[ronaldmaj]

Running this through testing I realised that there's a bigger problem with how this has been written up: the tarfile.open function only works when there is an actual file to open. The generate_uncompressed_filename function is supposed to work on strings so this will fail unless the filename variable actually points at an existing file. We may have to make an exception for tar.gz files in that we cannot generate its uncompressed filename

[treefern]

Having no . in the filename seems... very unlikely? I shouldn't need to handle e.g. a file named bz2.

[treefern]

It's probably not realistic to reliably predict the filename of a compressed directory/collection - only the case where a single file is compressed. And even then, there is no universal convention for naming a compressed directory vs a compressed single file. So when provided with only a filename, we'd still have to make assumptions as to whether it decompresses to single file of the same name (minus suffix) or a directory, based on what is typical of specific sources.

[seballgeyer]

return epoch >= long_filename_cutoff

will have the same results as your if statement

    if epoch >= long_filename_cutoff:
        return True
    else:
        return False

[seballgeyer]

typehint of the return is None in the declaration