Second Minor Release - (still) Rockin' In The SnapRAID World

Another small release with two bugfixes.
Enjoy!

Fixed:

• Email still sent after removal / error if email not set - fixes #100
• Integer errors at the end of emails in some warning scenarios

Minor Release - (still) Rockin' In The SnapRAID World

This is a quick fix release to address two issues that have been recently reported after v3.3 release

• TOUCH command not running when using SnapRAID 12.3 #92
• if curl is not installed the script does not correctly check for it, causing to always show there's a new script version #93

Please note: this is a re-release of v3.3.1
I did not update the config file check. Now it's correct, apologies for the confusion.

What's Changed

• Align dev with master by @auanasgheps in #95
• Push 3.3.1 to master by @auanasgheps in #96
• Update script-config.sh to match current version by @DonkeeeyKong in #97

New Contributors

• @DonkeeeyKong made their first contribution in #97

Full Changelog: v.3.3...v3.3.1

Rockin' In The SnapRAID World

A long time has passed since the last release, so there's a lot to like!

• Support multiple script configuration files
  • When running the script you can specify the config file you'd like to use
  • If not specified, the script will use the default config file
• Check if SnapRAID config file exists
  • Alerts the user via email and notifications if it doesn't before exiting
• OpenMediaVault 7 Support #80, #85
  • OMV7 introduced changes to SnapRAID plugin: support for multiple array, new file location and naming
  • The script will search of one file in the new location and use it
  • If there are multiple files, will stop and alert the user
• Update Check
  • The script can let you know when there's an update #67
• Dependencies install
  • install all required dependencies when running a distro that uses apt
• Improvements to Docker management, thanks to @tehniemer #64
  • Local and remote containers can be managed at the same time
  • Can filter containers using wildcards
• Send SnapRAID Status and SMART outputs via notifications on each run - #50
• Improve Discord notification output and its reliability - fixes #55
  • Fix Discord notification not sent in specific conditions - thanks to @Sirver51
• Scrub and Sync errors now are reported as "Severe Warnings" #49
  • In these cases, the notification message tells the user to check logs.
• Detect if SnapRAID is already running before doing anything #62
• Fixed notifications not sent if email fields are empty #56
• Replace awk with bc, fixes errors at the end of emails #74
• Remove unused code for legacy spindown methods
• Other misc improvements, thanks to @tehniemer #64

What's Changed

• Update snapraid-aio-script.sh by @tehniemer in #48
• Rebase dev docs by @auanasgheps in #66
• Updates to make docker management more efficient and versitile. by @tehniemer in #64
• Update script-config.sh by @tehniemer in #79
• Update snapraid-aio-script.sh by @tehniemer in #81
• Merge v3.3 from dev in master by @auanasgheps in #88

Full Changelog: v3.2...v.3.3

Born to be SnapRAID

Here we go with a new round of features and improvements. Thanks to all contributions!

• Script output (log) retention feature
  • You can now keep detailed SnapRAID script output (called log) for a number of days, instead of overriding them at every run.
  • Logs will be rotated and can be stored in any folder you like
• Force zero size sync
  • Forces the operation of syncing a file with zero size that before was not. Useful when handling some system files that genuinely change their size to zero. Use with caution, disabled by default
• New Sync option "added to deleted ratio", thanks to @tehniemer #45
  • Authorize a Sync even if the delete threshold has been breached, but only if the ratio of added to deleted files is greater than the value set.
• Scrub new blocks, thanks to @tehniemer #45
  • New option to scrub data that has just been added to the array
• Customizable Healthchecks.io server URL, thanks to @phidauex #32
• Detailed hd-idle documentation, thanks to @phidauex #35

Upgrade Notice: If you are using a previous version of the script, you will have to move your preferences to the new script-config.sh. The old file can't be reused or the script will throw an error.

Turn it on again

This version has been in the works for quite a while. There are many changes, also thanks to the many contributions!
The config file has been completely redesigned and is a breaking change for existing users.

• Telegram notifications
  • Send short telegram messages about the script status. It's the same info sent to Healthchecks, which can be enabled at the same time.
  • You will have to create a bot to use this feature.
  • The whole email message does not fit due to space limitations.
• Discord notifications, thanks to @ranapushpender
  • Same feature set as Telegram
• Notification Hook, thanks to @Caedis
  • Allows to use an external service/script/mail binary not natively supported by this script
• Custom Hooks, thanks to @nzlov #27
  • Configure shell commands or scripts to run before and after SnapRAID operations
• Improved error handling
  • Exit with an error when SnapRAID files are not found during sanity check (parity, data)
  • New error messages for email subject and Healthchecks/Telegram
  • If the configuration file is not found, write a message to the default snapraid log location
• Configuration file
  • Configurations are now split in categories
  • You must update the configuration file, moving your configs to the new one
  • If you haven't updated the config file after an upgrade, the script will exit with an error.
  • You must update the configuration time every time is needed. This change ensures consistency and avoids unwanted errors.
• Remote Docker management improvements
  • Add configurable delay for actions to remote host to improve success rates, thanks to @tehniemer #22
  • Manage multiple remote hosts, thanks to @tehniemer #24
• Disks spin down has been fixed, now uses hd-idle
• Migrate python-markdown to python3-markdown, to enable compatibility with Debian 11 / OMV6.
• Fixed email formatting when Scrub job is not run

Upgrade Notice: If you are using a previous version of the script, you will have to move your preferences to the new script-config.sh. The old file can't be reused or the script will throw an error.

Back in Black

The script is back, better than ever!
Skipping v2.9 since there have been a lot of changes.

• Email rendering is accurate
  • Tabs, spaces and tables are finally displayed correctly. These sections have unformatted text (code blocs) and will appear slightly different.
• Docker management is back
  • You can choose to pause/unpause or stop/restart your containers
  • If Docker is running on another host, it can be managed too
• Added Healthchecks.io integration
  • Script result can be reported to Healthchecks.io. If the script ends with a WARNING message, it will report a DOWN. Useful to quickly acknowledge failures, since this service can alert you via many channels.
  • Email subject and file summary will be reported to Healtchecks.io
  • Requires curl, which will be installed if not found
• Added "Delayed Scrub" feature
  • Allows to run a scrub job every X runs instead of every time.
• Fixed detection to include all content files
  • Previously only the first file was checked
• Show the number of unchanged files to the summary
• Added more messages to syslog
• Improved many messages
• Code quality: thanks to generous contributions from @cmcginty, a lot has been rewritten and reorganized. The script is more robust, comments have been improved, proper formatting & best practices via shellcheck and more.

Upgrade Notice: If you are using a previous version of the script, you will have to move your preferences to the new script-config.sh. The old file can't be reused or the script will throw errors.

Can't Get Enough of you, SnapRAID

No new features in this release, it's focused on message improvements and general clean-up.

• Added a title for the SnapRAID Status section.
• Changed strings when the output is 0. This nonsense is gone:
  - The number of deleted/updated files, (0), is below the threshold
  - The number of threshold warning(s) (0) has reached/exceeded threshold (0)
  - 0 threshold warning(s) until the next forced sync
• Added a string when using Sync with threshold warning - before was only shown "Forced Sync"
• Moved "Sync is authorized" message for more clarity
• Minor tweaks to other messages
• Removed clean_desc and service_array_setup functions, used for container management

Stuff has been moved around

Stuff has been moved around.

• Divided script and config variables two different files
  • Making changes to your configuration is now easier since you don't have to deal anymore with the whole script
• Moved script introduction and changelog to GitHub
• The script will install Markdown if the package is not found
• Moved sync warn file in the same directory of the script. Previously was /tmp which would get cleared after a restart
• Improved threshold messages for clarity
• Added 'snapraid status'. Disabled by default, because the output is a little broken.

Going public

First public release here!

Here is the full changelog for all the earlier work that brought to this release.

• version 2.0 changelog by auanasgheps
  • Removed DIFF output from email (still present in logs) - credits to metagliatore
  • Fixed script for Debian 10/OMV5 - credits to sburke
  • Added alternative way of spinning down disks with hdparm
• version 2.1 changelog by auanasgheps
  • disabled disk spindown (code is commented) since it's not working properly
• version 2.2 changelog by auanasgheps
  • Redone changelog section
  • Added 'Prehash Data' feature - credits to Zack Reed
  • Added HTML formatting - NOTE: requires python-markdown
• version 2.3 changelog by auanasgheps
  • Removed TOUCH output from email (still present in logs)
  • Fixed small typos
• version 2.4 changelog by auanasgheps
  • Send an email alert if parity or content files are not found, then exit gracefully
• version 2.5 changelog by auanasgheps
  • Added configurable options for disk spindown and email verbosity
  • Added syslog features
• version 2.6 changelog by auanasgheps
  • Fixed sed error caused by a slash when updated/deleted threshold is breached
  • Fixed violation message not shown when threshold is reached but not exceeded
  • Fixed concurrent (deleted AND updated) violation message and mail subject
  • Added message and mail subject when sync is forced with breached thresholds (deleted, changed or both)
• version 2.6.1 changelog by auanasgheps
  • Disabled clean_desc function in main script, caused {out} file in /root
• version 2.6.2 changelog by auanasgheps
  • Added SnapRAID version to output
  • Added variable for script version
  • Removed timestamps from logs text since it's already added by the system
• version 2.6.3 changelog by auanasgheps
  • Small change to email subject when forcing syncs with violations
• version 2.6.4 changelog by auanasgheps
  • Removed unnecessary capitalized letters in email subject
• version 2.6.5 changelog by ozboss
  • Replaced tabs with spaces
  • Change default of 'SYNC_WARN_THRESHOLD' to '-1', will not force a sync
  • Added requirements
  • Added alternative spindown method (hd-idle)
  • Added automatic detection of rotational devices for spindown (hdparm and hd-idle)
• version 2.6.6 changelog by auanasgheps
  • removed code for Docker Container management, was unreliable