- Fix issue with emaail notifications and not being able to add from statement
- Change the way architectures are detected to re-enable backups with MSSQL and Influx2
- Allow _OPTS variabls to contain spaces
- Switch references of _DUMP_OPTS to _BACKUP_OPTS
- Fix issue with directories not properly being backed up (InfluxDB)
- Support backing up databases with spaces in them
- Seperate each job with its own temporary folder for isolation and to better cleanup jobs that backup as a directory instead of a flat file
- Skip blobxfer if either account or key is not present
- AWS Cli 1.31.5
- Switch to using PIP for installing AWS-Cli to remove deprecation warnings
- Switch to using actual crontab for cron expressions
- AWS CLI 1.31.4
- Fix #297 - Add parameters to blobxfer to restore functionality
- Fix issue with cron parsing and 0 being a value getting clobbered by sort command
- Resolve issue with custom notification scripts not executing
- Move cleanup_old_data routines to happen within backup_ function to properly accomodate for globals, and ALL DB_NAME use cases
- Fix for SQLite backups not being cleaned up properly due to a malformed base
- Update base image to support S6 Overlay 3.1.6.2 to solve shutdown issues specifically with MODE=MANUAL and MANUAL_RUN_FOREVER=TRUE
- Add some safety nets for Manual scheduling
- Make adjustments to cron scheduling feature to be able to handle whitespace properly"
4.0.18 2023-11-18 joergmschulz@github
- Fix loading msmtp configuration
- Provide more details when notifying via instant messages
- Switch to using msmtp instead of s-mail for notify()
- Fix cleanup of old backups
- Bugfix when PRE/POST scripts found not giving legacy warning
- Run pre / post scripts as root
- Check for any quotes if using MONGO_CUSTOM_URI and remove
- Allow creating schedulers if _MONGO_CUSTOM_URI is set and _DB_HOST blank
- Resolve issue with backing up ALL databases with PGSQL and MySQL
- Change environment variable parsing routines to properly accomodate for Passwords containing '=='
- Fix issue with quotes being wrapped around _PASS variables
- Tidy up file_encryption() routines
- Change environment variable _ENCRYPT_PUBKEY to _ENCRYPT_PUBLIC_KEY
- Add new environment variable _ENCRYPT_PRIVATE_KEY
- Add seperate permissions for _FILESYSTEM_PATH
- More output and debugging additions
- SQLite3 now backs up without running into file permission/access problems
- Cleanup old sqlite backups from temp directory
- Handle multiple SQLite3 backups concurrently
- Add additional DEBUG_ statements
- Fix issue with Influx DB not properly detecting the correct version
- Add undocumented DBBACKUP_USER|GROUP environment variables for troubleshooting permissions
- Add more verbosity when using DEBUG_ statements
- Change _FILESYSTEM_PERMISSION to 600 from 700
- Add support for restoring from different DB_ variables in restore script
- Resolve issue with _MYSQL_TLS_CERT_FILE not being read
- Properly use custom _S3_HOST variables
- Restore - Stop using DB_DUMP_TARGET and instead browse using DEFAULT_BACKUP_PATH
This is the fourth major release to the DB Backup image which started as a basic MySQL backup service in early 2017. With each major release brings enhancements, bugfixes, removals along with breaking changes and this one is no different.
This release brings functionality requested by the community such as multiple host backup support by means of independent scheduler tasks,blackout periods, better resource usage, better security via file encryption, file permissions, and more verbosity via log files. , and also merges contributions from other developers.
Upgrading to this image should for the most part work for most users, but will involve event upgrading environment variables as the formathas changed significantly. Old variables should continue to work, however are unsupported and will be removed with the 4.3.0 release, whenever that will be.
A significant amount of development hours were put in to accomodate for feature requests by the community. If you are using this in a commercial setting or find this image valuable, please consider sponsoring my work for a period of time or engaging in a private support offering. More details at https://www.tiredofit.ca/sponsor
- Backup Multiple Hosts in same image all with different options (scheduling, compression, destination, cleanup) (Use `DBXX_option` variables)
- Backup limits how many backup jobs run concurrently
- Backup Scheduling now allows using a timestamp (e.g. `Dec 12 2023 03:00:00`) - credit benvia@github
- Backup Scheduling now allows using a cron expression (e.g `00 03 * * *`)
- Backup Blackout period to skip backing up during a period of time
- Backup runs as dedicated user (no longer root)
- Backup can have specific file permissions set upon completion (e.g. `700` or `rwx------`)
- Backups can run with reduced priority mode to allow for fair scheduling across system
- Backups - MySQL/MariaDB now has ability to backup events
- Backups - Microsoft SQL server now has option to backup transaction logs
- Backups - Postgres now backs up globals - credit oscarsiles@github
- Backups with Azure synchronize storage before upload - credit eoehen@github
- Encrypt backup file with a passphrase or a GPG Public Key ability
- Log backup jobs to file along with log rotation
- Notification support upon job failure via Email, Mattermost, Matrix, Rocketchat
- Zabbix Metrics now auto discovers new jobs
- Zabbix Metrics sends metrics relating to backed up filename, checksum hash, and the duration of backup/compression time, checksum time, encryption time
- New Debug Capabilities
- Reworked Documentation
- Reworked all functions and renamed all variables
- Many variables now use a prefix of `DEFAULT_` to operate on all backup jobs
- Can be overridden per backup job by setting `DB_<option>` or to unset default variable `DB_<option>=unset`
- Renamed variables and terms
- `_DUMP_LOCATION` -> `_BACKUP_LOCATION`
- `_DUMP_BEGIN` -> `_BACKUP_BEGIN`
- `_DUMP_FREQ` -> `_BACKUP_INTERVAL`
- `_DUMP_TARGET`` -> `_FILESYSTEM_PATH`
- `_DUMP_ARCHIVE`` -> `_FILESYSTEM_PATH`
- `EXTRA_DUMP_OPTS`` -> `_EXTRA_BACKUP_OPTS`
- `TEMP_LOCATION`` -> `TEMP_PATH`
- Backups - AWS CLI updated to 1.29.78
- Backups - InfluxDB 2 Client version updated to 2.7.3
- Backups - Microsoft SQL server now compresses files post initial backup
- Backups - Manual backups handle aborting gracefully
- Checksum routines now complete in half the time
- Checksum variable now supports "NONE"
- Zabbix metrics sending occurs in one process as opposed to singular
- Cleanup Only cleanup files that match same backup name pattern
- Cleanup/Archive uses relative path instead of absolute with creating_latest_symlink
- A handful of code optimizations and cleanup
- `ENABLE_CHECKSUM` - has been wrapped into `_CHECKSUM=none`
3.12.0 2023-10-29 alwynpan@github
- Add DB_AUTH functionalith for Postgresql databases
- Add DB_AUTH environment for PGSQL
- Switch to tiredofit/alpine:edge for newer packages
- Postgresql 16 support
- Add --break-system-packages flag to pip when installing blobxfer
- Introduce aarch64 (ARMv8) support for Microsoft SQL Server backups
- Microsoft ODBC Driver 18.3.2.1-1
- Microsoft SQL Client 18.3.1.1-1
- Add option to drop existing data from MongoDB restore
- Fix some capabilities of not being able to select mongodb manually to restore
3.10.4 2023-10-11 thomas-negrault@github
- Use authentication database for MongoDB restores
3.10.3 2023-10-11 thomas-negrault@github
- Change sorting for restore script
3.10.2 2023-09-14 pimjansen@github
- Update to wording when sending files to blobxfer
- Remove --databases flag when backing up a single mysql/mariadb backup which allows to omit the "USE <db_name>" statement in the backup allowing for better restores
- Bugfix to 3.10.0 with syntax error revolving around unbraced variable
3.10.0 2023-09-13 jcdirks@github
- Add EXTRA_DUMP_OPTS and EXTRA_ENUMERATION_OPTS to add different arguments when checking for databases, vs doing the actual backup
- Perform additional checks for ENABLE_CHECKSUM=FALSE and skip executing actions for S3/BlobXfer
- AWS CLI 2.13.9
- Stop trying to move a non existent checksum file when ENABLE_CHECKSUM=FALSE
- Start compiling aws-cli instead of from packages to continue to support arm/v7
3.9.8 2023-08-20 ToshY@github
- Restore armv7 and aarch64 builds
- Cleanup check_exit_code parameter and reduce duplicate output
- Resolve issues introduced with 3.9.3. Split exit codes to be specific for backing up and moving. Uses parameter $11 for post backup scripts
- Start building Influx DB v1 manually due to being removed from Alpine repositories
- Add ability to use --rsyncable argument to zstd archives
- Add notification if blobxfer/s3 upload fails (credit @alwynpan)
- Add zip package
- Alpine 3.18 base
- Properly allow multiple _FILE environment variables to execute solving an issue with MySQL backups
- Fix _FILE functionality for DB_NAME variable
- Add support for _FILE environment variables
3.8.5 2023-04-11 tpansino@github
- Fix SQLite3, Influx, and MSSQL backups failing due to malformed/non existent ltarget
- Fix issue with Influx2 and MSSQL clients not installing properly
- Patchup for 3.8.2
- Patchup for 3.8.1
- Cleanup Dockerfile
- Fix issue with DB_ARCHIVE_TIME not firing correctly
- Introduce DB_DUMP_TARGET_ARCHIVE which works with DB_ARCHIVE_TIME to move backups older than (x) minutes from DB_DUMP_TARGET to DB_DUMP_TARGET_ARCHIVE for use with external backup systems and custom exclude rules
- Introduce CREATE_LATEST_SYMLINK which creates a symbolic link in DB_DUMP_TARGET of `latest-(DB_TYPE)-(DB_NAME)-(DB_HOST)`
3.7.7 2023-03-20 codemonium@github
- Simplify pg_isready usage
3.7.6 2023-03-14 toshy@github
- Remove EXTRA_OPT variable from MySQL/MariaDB check
- Add support for Docker Swarm mode Secrets for BLOBXFER_STORAGE_ACCOUNT_*_FILE
3.7.4 2023-02-22 gbe0@github
- Fix when running in MANUAL_RUN_FOREVER mode looping
- Make S3_KEY_ID and S3_KEY_SECRET optional should IAM roles be used (Credit to alwynpan@github)
- Bugfix for 3.7.1
- Add MYSQL_ENABLE_TLS environment variable to switch on and off
- Set default for MYSQL_TLS_CA_FILE to accomodate for most use cases
- Introduce support for connecting via TLS to MySQL / MariaDB Hosts with MYSQL_TLS_* variables - See README for more details
- Fix for cleaning up file systems that are syncing to Azure via blobxfer
- Switch to Alpine 3.17 base
- Switch to OpenSSL instead of LibreSSL
- Postgresql 15 Support
- Add failure if DB_TYPE empty or malformed
- Fix for S3 backups and trailing slashes (@greena13)
- Fix for Influx DB 1 backups when compression enabled
- Remove build dependencies for blobxfer making image size smaller
- Remove silencing commands limiting Postgres backups from working without DEBUG_MODE=TRUE
- Update Influxdb client to 2.4.0 (jauderho@github)
- Tighten up cleanup routines to not call blobxfer unless absolutely necessary
- Blobxfer / Microsoft Azure Support (credit: eoehen@github)
- Introduce MONGO_CUSTOM_URI environment variable for those not wanting to use DB_* variables
- Force filenames to be in lowercase
- Fix S3 Database cleanups (credit greenatwork@github)
- Remove MONGO_DB_TYPE variable as MONGO_CUSTOM_URI overrides
- Fix MSSQL Backups (credit eoehen@github)
- Seperate examples for MySQL and MSSQL
- Skip availability check for Mongo Atlas connections
- Introduce environment variables for SCRIPT_LOCATION_POST and SCRIPT_LOCATION_PRE for better separation
-
Introduce deprecation warning for the custom script paths
/assets/custom-scriptsand/assets/custom-scripts/pre. These paths will continue to work for now but support may be removed in the next major version release. To support the new default paths your scripts should be moved as follows:Script Type Old Path (Deprecated) New Environment Variable Environment Value Default Pre /assets/custom-scripts/preSCRIPT_LOCATION_PRE /assets/scripts/prePost /assets/custom-scriptsSCRIPT_LOCATION_POST /assets/scripts/post
- Add GZ_RSYNCABLE environment variable for better rsync compatibility (Credit teun95@github)
- Add Pre Backup Script Support
- Add MongoDB Atlas Support
- Fix Default Port for Influx 2 DB Hosts
- arifer612@github contributed a fix for incorrect case of "filesize" variable when using post backup scripts
3.3.11 2022-07-22 khoazero123@github
- Restore script not properly detecting postgres backups
- Remove MSSQL install packages properly
3.3.9 2022-07-09 fardeau@github
- Remaining work on interactive mode for entering port on restore script
- MSSQL Client Tools 18.0.1.1-1
- Allow overrides to actually override with the restore script
- Fix for restore script when using all 7 arguments
- Fix DB Port parameter not being able to be input in restore script
- Fix MongoDB restore questionnaire
3.3.4 2022-06-03 rozdzynski@github
- S3 backups failing with special characters in filename
- Alpine 3.16 base
- Add POST_SCRIPT_SKIP_X_VERIFY environment variables to allow for more host compatibility for post scripts
- Compressing silently was causing 0 byte backups
- Ability to auto clean old S3 / Minio Hosts like what occurs on filesystem
- Alert user how to turn off Zabbix Monitoring if fails
- Allow Zabbix Monitoring to work with S3
- Silence some more compression statements
- Fix for Redis not backing up properly
- Start sending checksums for S3 Outputs
- Cleanup some code functions
- FIx Container Log Level always in DEBUG
- Add -portable flag when backing up Influx
- Fix for bucket / db name InfluxDB 1.xx
- Minor aesthetics, spacing, spelling
- Restore script properly parses DB_PORT entry
- Influx Database ready performs different checks dependent on V1/V2
- Stop using weird database lowercase variables unnecessarily breaking Docker Secrets
- Fix a variety of issues iwth 3.2.0 relating to InfluxDB
- Install InfluxDB2 Client alongside version 1 (amd64 and arm64)
- Introduce InfluxDB 2 backup support
- Introduce multiple compression types other than Gzip for Influx 1/2
- Introduce compression for MSSQL backups
- Testing for Host availability for CouchDB and InfluxDB
- Fix for MariaDB not sending database name to post script
- Check for executible bit on post scripts both via environment variable and /assets/custom
- SPLIT_DB defaulted to TRUE
- �Fix for blank Notice when individual backup is completed (time taken)
- Resolve some issues with backups of Mongo and others not saving the proper timestamp
- Backup multiple databases by separating with comma e.g. db1,db2
- Backup ALL databases bu setting DB_NAME to ALL
- Exclude databases from being backed up comma separated when DB_NAME is all eg DB_NAME_EXCLUDE=db3,db4
- Backup timers execute per database, not per the whole script run
- Post scripts run after each database backup
- Checksum does not occur when database backup failed
- Database cleanup does not occur when any databases backups fail throughout session
- MongoDB now supported with 'restore' script - Credit to piemonkey@github
- Lots of reshuffling, optimizations with script due to botched 3.0 release
- ZSTD replaces GZ as default compression type
- Output is cleaner when backups are occurring
- Fix for SPLIT_DB not looping through all database names properly
- Rework compression function
- Fix for Bzip compression failing
- Rearrange Notice stating when next backup is going to start
- Add compression levels to debug mode
- Throw Errors for MANUAL mode when certain other CONTAINER_* services are enabled
- Fix for Parallel Compression
- Fix for restore script not taking "custom" usernames or passwords
- Switch to using parallel versions of compression tools all the time, yet explicitly state the threads in use (1 or ++)
- Add PARALLEL_COMPRESSION_THREADS environment variable to limit amount of threads when compressing - Currently auto detects however many processors are available to the container
- Strip unused LOG directives
- Fix for parallel compression
- Add more detail regarding manual modes
- Fix for 3.0.3
- Add documentation for Manual mode
- Revert Compression variables
- Cleanup of Zabbix Agent options
- Updated Zabbix template
- Split apart S3 options for better debugging and also cleaned up their variables
- Fixed issue with post scripts not outputting proper backup start time
- Cleaned up some notifications
- Rearranged code
- Fix for Environment variable not reading correctly for restore script for DB TYPE
- Rewrote entire image
- Ability to choose which file hash after backup (MD5 or SHA1)
- Restore Script (execute 'restore' in container)
- Allow to map custom CA certs for S3 backups
- Allow to skip certificate certification for S3 Backups
- Revamped Logging and parameters - File logs also exist in /var/log/container/container.log
- Added more functionality to send to Zabbix to track start, end, duration and status
- Ability to backup stored procedures for MySQL / MariaDB
- Ability to backup as a single transaction for MySQL / MariaDB
- Ability to execute "manually" and still allow container to execute to accommodate for Kubernetes cron usage
- Environment variables have changed! Specifically relating to COMPRESSION, PARALLEL COMPRESSION, CHECKSUMs
- Last release of 2.x series
- Fix timer for backups that take excessively long time, and allow it to start repetitively at the same time daily. What was happening is that if a backup took 30 minutes, time would shift by 30 minutes daily eventually taking backups mid day.
- Add additional debug statements
- Add debug statement around the scheduling component
- Rework to support new base image
- Refresh base image
2.11.1 2022-01-20 jacksgt@github
- Modernized S3 variables and sanity checks
- Change exit code to 0 when executing a manual backup
- Add capability to select `TEMP_LOCATION` for initial backup and compression before backup completes to avoid filling system memory
- Cleanup for MariaDB/MySQL DB ready routines that half worked in 2.10.3
- Code cleanup
- Change the way MariaD/MySQL connectivity check is performed to allow for better compatibility without requiring the DB_USER to have PROCESS privileges
- Remove logrotate configuration for redis which shouldn't exist in the first place
2.10.1 2021-12-22 milenkara@github
- Allow for choosing region when backing up to S3
- Revert back to Postgresql 14 from packages as its now in the repositories
- Fix for Zabbix Monitoring
- Fixup for Zabbix Autoagent registration
2.9.6 2021-12-03 alexbarcello@githuba
- Fix for S3 Minio backup targets
- Fix for annoying output on certain target time print conditions
- Fix for 2.9.3
- Add Zabbix auto register support for templates
- Alpine 3.15 base
2.9.2 2021-10-22 teenigma@github
- Fix compression failing on Redis backup
2.9.1 2021-10-15 sbrunecker@github
- Allow MariaDB 8.0 servers to be backed up
- Fixed DB available check getting stuck with empty password
- Postgresql 14 Support (compiled)
- MSSQL 17.8.1.1
- Change to using aws cli from Alpine repositories (fixes #81)
- Modernize image with updated environment variables from upstream
- Alpine 3.14 Base
- Fix for syntax error in 2.7.0 Release (Credit the1ts@github)
- Cleanup image and leftover cache with AWS CLI installation
- MongoDB Authentication Database support (DB_AUTH)
2.6.1 2021-06-08 jwillmer@github
- Fix for Issue #14 - SPLIT_DB=TRUE was not working for Postgres DB server
2.6.0 2021-02-19 tpansino@github
- SQLite support
- Fix xz backups with `PARALLEL_COMPRESION=TRUE`
- Multi Platform Build Variants (ARMv7 AMD64 AArch64)
- Alpine 3.13 Base
- Compile Pixz as opposed to relying on testing repository
- MSSQL Support only available under AMD64. Container exits if any other platform detected when MSSQL set to be backed up.
- Switch back to packges for Postgresql (now 13.1)
- Reapply S6-Overlay into filesystem as Postgresql build is removing S6 files due to edge containing S6 overlay
2.3.1 2020-11-11 bambi73@github
- Multiple Influx DB's not being backed up correctly
- Microsoft SQL Server support (experimental)
- Compiled Postgresql 13 from source to backup psql/13 hosts
2.2.2 2020-09-22 tpansino@github
- Patch for 2.2.0 release fixing Docker Secrets Support. Was skipping password check.
2.2.1 2020-09-17 alwynpan@github
- On-demand/Manual backup with `backup-now` was throwing errors not being able to find a proper date
2.2.0 2020-09-14 alwynpan@github
- Allow to use MariaDB and MongoDBs with no username and password while still allowing Docker Secrets
- Changed source of Alpine package repositories
2.1.1 2020-09-01 zicklag@github
- Add eval to POST_SCRIPT execution
- Add Exit Code variable to be used for custom scripts - See README.md for placement
- Add POST_SCRIPT environment variable to execute command instead of relying on custom script
- Reworked compression routines to remove dependency on temporary files
- Changed the way that MongoDB compression works - only supports GZ going forward
- Code cleanup (removed function prefixes, added verbosity)
- Removed Rethink Support
- Add zstd compression support
- Add choice of compression level
- Added EXTRA_OPTS variable to all backup commands to pass extra arguments
- Fix `backup-now` manual script due to services.available change
- Change to support tiredofit/alpine base image 5.0.0
- Bugfix to initialization routine
- Add S3 Compatible Storage Support
- Switch some variables to support tiredofit/alpine base image better
- Fix issue with parallel compression not working correctly
- Fix Auto Cleanup routines when using `root` as username
- Docker Secrets Support for DB_USER and DB_PASS variables
- Custom Script support to execute upon completion of backup
- Rework to allow passwords with spaces in them for MariaDB / MySQL
- Allow for passwords with spaces in them for MariaDB / MySQL
- Update image to support new tiredofit/alpine base images
- Quiet down Zabbix Agent
- Re Enable ZABBIX
- Fix spelling mistake in container initialization
- Stop compiling mongodb-tools as it is back in Alpine:edge repositories
- Cleanup Code
- Check to see if Database Exists before performing backup
- Fix for MysQL/MariaDB custom ports - Credit to spumer@github
- Added abaility to backup password protected Redis Hosts
- Switch to using locally built mongodb-tools from tiredofit/mongo-builder due to Alpine removing precompiled packages from repositories
- Fixed Postgres backup without SPLIT_DB enabled (credit MelwinKfr@github)
- Added DB_PORT reference to properly backup Postgres with non default ports (thanks Maxximus007@github)
- Fix for XZ Compression failing
- Fix for Urnary Operator Error
- Fix for InfluxDB for backing up and supporting DB_PORT variable - Thanks skylord123@github
- Switch from OpenSSL to LibreSSL
- Fix warnings on startup related to 1.7 Changes
- Added ability for Manual Backup (enter container and type
backup-now)
- Add Parallel Compression mode (Default TRUE
- Add Zabbix Checks
- Switch to Packages Postgres
- Fix to SPLIT_DB Postgresql Backup
- Remove Alpine postgres package and recompile version 10
- Syntax Error Fix
- Fix some environment variables for Postgres and Redis
- Added CouchDB
- Initial Release
- Alpine:Edge