Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add: day wise log split and log files management #3

Merged
merged 4 commits into from Feb 28, 2023

Conversation

mratanusarkar
Copy link
Owner

@mratanusarkar mratanusarkar commented Feb 28, 2023

Improvisation of the existing implementation. This will enable the logs to be split day-wise and also manage the files and have some sort of retention policy or auto-delete after a certain number of days.

This fixes issue: #2

Topic Breakdown:

  • fix the issue of logs not getting split into date-wise logs
  • implement log management (so that the log files don't overflow taking up memory)
  • implement an auto-delete or auto-archival of the logs
  • implement a retention policy

TODO (for future reference):

  • make the implementation values dynamic and configurable (remove hardcoded values)
  • make the existing metadata dynamic and configurable (remove hardcoded values)

@mratanusarkar mratanusarkar self-assigned this Feb 28, 2023
@mratanusarkar mratanusarkar added the bug Something isn't working label Feb 28, 2023
@mratanusarkar
Copy link
Owner Author

  • tried modifying new Date() placement in logger.js to see if the files get split.
  • tried deleting log files or the log text content inside the log files to see if the files get recreated with a new date.
  • realized that this doesn't work because of how Winston is created. The config file is called one time at the beginning of the application start, and it's like an object.
  • searched through GitHub issues and forums to tackle this type of requirement.

@mratanusarkar
Copy link
Owner Author

I found the following GitHub issues relevant to this problem:

It seems like we have to rely on another package by the Winston foundation:
winston-daily-rotate-file

Implementation of the above package would take care of the following requirement:

  1. split log files according to date, time, or memory size.
  2. delete older log files (say 30 days or 60 days old files)

@mratanusarkar
Copy link
Owner Author

mratanusarkar commented Feb 28, 2023

  • implemented winston-daily-rotate-file
  • tried out various configurations for "log file rotate", "split logs datewise" and "auto deletion of old logs"
  • working on playing with config and testing the outcome and behavior matches with the expectation

Note: we can also set the log rotation frequency to anything else and not necessarily daily.
This should be a good generalization point for future TODOs.

@mratanusarkar mratanusarkar added the feature New feature addition label Feb 28, 2023
@mratanusarkar mratanusarkar marked this pull request as ready for review February 28, 2023 18:26
@mratanusarkar mratanusarkar linked an issue Feb 28, 2023 that may be closed by this pull request
Copy link
Owner Author

@mratanusarkar mratanusarkar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

merging PR as v1.0.1

@mratanusarkar mratanusarkar merged commit c714ec3 into main Feb 28, 2023
@mratanusarkar mratanusarkar deleted the feature/log-rotate branch February 28, 2023 18:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working feature New feature addition
Projects
None yet
Development

Successfully merging this pull request may close these issues.

log files are not generated day wise
1 participant