Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

new socializer code for pruning posts fails to prune posts and images #349

Closed
rbreesems opened this issue May 10, 2024 · 1 comment · Fixed by #350
Closed

new socializer code for pruning posts fails to prune posts and images #349

rbreesems opened this issue May 10, 2024 · 1 comment · Fixed by #350

Comments

@rbreesems
Copy link
Contributor

rbreesems commented May 10, 2024

**Edit - I have linked a pull request to fix this issue.

We have tested the new socializer code for pruning posts/images and verified that the CleanupService is running with the correct values from the config file. It was tested with two users making posts overnight, with posts, images > 12 hours old, and the threshold was set to 8 hours.

However, it is not deleting image post directoires and it is always reporting 0 posts deleted.. Log messages verify the task is running but always reporting 0 posts, 0 images deleted. The problem with images is that it is trying to delete files from wwwroot.images but this directory contains post directories and the 'u' avatar directory, so it is not finding any files to delete. I have appended a function at the end of this that could be used to delete post directories that meet the threshold.

I believe that the post DB entries are indeeded being identified using the following code and being deleted:
var oldPosts = dbContext.Posts.Where(p => p.CreatedUtc < threshold);

However, I believe the reason the post deletion count is being reported as 0 is because the {oldPosts.Count() is being called after the posts are deleted and thus will always return zero.

However, I have confirmed that the post image directories are definitely not being deleted.

Here is the example function to delete post files in the image directory::

// it is assumed that startDir is wwwroot/images
static int deleteFiles(System.DateTime threshold, string startDir)
{
    int count = 0;
    foreach (var dirName in Directory.GetDirectories(startDir))
    {
        var fullPath = Path.GetFullPath(dirName).TrimEnd(Path.DirectorySeparatorChar);
        var lastDirname = fullPath.Split(Path.DirectorySeparatorChar).Last();
        if (lastDirname == "u") continue;  // skip 'u' directory as that contains avatar images, never delete these
        if (File.GetCreationTimeUtc(dirName) > threshold) continue;
        // found a post directory that meets the criteria. First delete all files in the directory, then the directory
        // we know that there are only files in this directory, it does not contain subdirectories
        foreach (var file in Directory.GetFiles(fullPath))
        {
            Console.WriteLine($"Deleting {file}");  // replace this with File.Delete(file);
            count++;
        }
        // now delete the post directory, will be empty
        // we could just delete the entire directory+files in one call but there have been bug reports about this 
        // not working consistently across all platforms, safer to delete the files first to get an empty directory
        Console.WriteLine($"Deleting {fullPath}"); // replace this with Directory.Delete(fullPath);
        count++;
    }
    return count;
}


@rbreesems
Copy link
Contributor Author

Complete function for deleting files with error handling:

// This starts in wwwroot/images, so directories will either be post directories or 'u', skip 'u'
static int deleteFiles(System.DateTime threshold, string startDir)
{
    int count = 0;
    foreach (var dirName in Directory.GetDirectories(startDir))
    {
        var fullPath = Path.GetFullPath(dirName).TrimEnd(Path.DirectorySeparatorChar);
        var lastDirname = fullPath.Split(Path.DirectorySeparatorChar).Last();
        if (lastDirname == "u") continue;  // skip 'u' directory as that contains avatar images, never delete these
        if (File.GetCreationTimeUtc(dirName) > threshold) continue;
        // found a post directory that meets the criteria. First delete all files in the directory, then the directory
        // we know that there are only files in this directory, it does not contain subdirectories
        foreach (var file in Directory.GetFiles(fullPath))
        {
            try
            {
                File.Delete(file);  
                count++;
            }
            catch (Exception ex)
            {
                logger.LogError($"Could not delete file {file}: {ex.Message}");
            }
        }
        // now delete the post directory, will be empty
        // we could just delete the entire directory+files in one call but there have been bug reports about this 
        // not working consistently across all platforms, safer to delete the files first to get an empty directory
        try
        {
            Directory.Delete(fullPath);
        }
        catch (Exception ex)
        {
            logger.LogError($"Could not delete directory {fullPath}: {ex.Message}");
        }

        count++;
    }
    return count;
}

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant