restic 0.16.4

This release works around and improves detection of a bug in the compression library used by restic. The resulting issue only happens when using restic 0.16.3 and the max compression level (the default auto and off compression levels are not affected), and when the source files being backed up have specific data in them to trigger the bug. If you use max compression, you can use restic check --read-data to make sure you're not affected.

restic is distributed as a standalone binary: download the correct file for your operating system and architecture, extract the file and just run it. If you run into any issues, please report them at the GitHub issue tracker or visit the forum. If you already have restic >= 0.9.4, you can use restic self-update to get the latest version in a secure way.

The binaries released with each restic version are reproducible, which means that you can reproduce a byte identical version from the source code for that release. Instructions on how to do that in the Developer Documentation.

Changelog for restic 0.16.4 (2024-02-04)

The following sections list the changes in restic 0.16.4 relevant to restic users. The changes are ordered by importance.

Summary

• Fix #4677: Downgrade zstd library to fix rare data corruption at max. compression
• Enh #4529: Add extra verification of data integrity before upload

Details

• Bugfix #4677: Downgrade zstd library to fix rare data corruption at max. compression

  In restic 0.16.3, backups where the compression level was set to max (using --compression max) could in rare and very specific circumstances result in data corruption due to a bug in the library used for compressing data. Restic 0.16.1 and 0.16.2 were not affected.

  Restic now uses the previous version of the library used to compress data, the same version used by restic 0.16.2. Please note that the auto compression level (which restic uses by default) was never affected, and even if you used max compression, chances of being affected by this issue are small.

  To check a repository for any corruption, run restic check --read-data. This will download and verify the whole repository and can be used at any time to completely verify the integrity of a repository. If the check command detects anomalies, follow the suggested steps.

  #4677 #4679

• Enhancement #4529: Add extra verification of data integrity before upload

  Hardware issues, or a bug in restic or its dependencies, could previously cause corruption in the files restic created and stored in the repository. Detecting such corruption previously required explicitly running the check --read-data or check --read-data-subset commands.

  To further ensure data integrity, even in the case of hardware issues or software bugs, restic now performs additional verification of the files about to be uploaded to the repository.

  These extra checks will increase CPU usage during backups. They can therefore, if absolutely necessary, be disabled using the --no-extra-verify global option. Please note that this should be combined with more active checking using the previously mentioned check commands.

  #4529 #4681

restic 0.16.3

This release fixes a couple of bugs on Windows and in the restore command. It also works around an unlikely yet possible situation with rclone which could potentially result in data loss.

restic is distributed as a standalone binary: download the correct file for your operating system and architecture, extract the file and just run it. If you run into any issues, please report them at the GitHub issue tracker or visit the forum. If you already have restic >= 0.9.4, you can use restic self-update to get the latest version in a secure way.

The binaries released with each restic version are reproducible, which means that you can reproduce a byte identical version from the source code for that release. Instructions on how to do that in the Developer Documentation.

Changelog for restic 0.16.3 (2024-01-14)

The following sections list the changes in restic 0.16.3 relevant to restic users. The changes are ordered by importance.

Summary

• Fix #4560: Improve errors for irregular files on Windows
• Fix #4574: Support backup of deduplicated files on Windows again
• Fix #4612: Improve error handling for rclone backend
• Fix #4624: Correct restore progress information if an error occurs
• Fix #4626: Improve reliability of restoring large files

Details

• Bugfix #4560: Improve errors for irregular files on Windows

  Since Go 1.21, most filesystem reparse points on Windows are considered to be irregular files. This caused restic to show an error: invalid node type "" error message for those files.

  This error message has now been improved and includes the relevant file path: error: nodeFromFileInfo path/to/file: unsupported file type "irregular". As irregular files are not required to behave like regular files, it is not possible to provide a generic way to back up those files.

  #4560 #4620 https://forum.restic.net/t/windows-backup-error-invalid-node-type/6875

• Bugfix #4574: Support backup of deduplicated files on Windows again

  With the official release builds of restic 0.16.1 and 0.16.2, it was not possible to back up files that were deduplicated by the corresponding Windows Server feature. This also applied to restic versions built using Go 1.21.0-1.21.4.

  The Go version used to build restic has now been updated to fix this.

  #4574 #4621

• Bugfix #4612: Improve error handling for rclone backend

  Since restic 0.16.0, if rclone encountered an error while listing files, this could in rare circumstances cause restic to assume that there are no files. Although unlikely, this situation could result in data loss if it were to happen right when the prune command is listing existing snapshots.

  Error handling has now been improved to detect and work around this case.

  #4612 #4618

• Bugfix #4624: Correct restore progress information if an error occurs

  If an error occurred while restoring a snapshot, this could cause the restore progress bar to show incorrect information. In addition, if a data file could not be loaded completely, then errors would also be reported for some already restored files.

  Error reporting of the restore command has now been made more accurate.

  #4624 https://forum.restic.net/t/errors-restoring-with-restic-on-windows-server-s3/6943

• Bugfix #4626: Improve reliability of restoring large files

  In some cases restic failed to restore large files that frequently contain the same file chunk. In combination with certain backends, this could result in network connection timeouts that caused incomplete restores.

  Restic now includes special handling for such file chunks to ensure reliable restores.

  #4626 https://forum.restic.net/t/errors-restoring-with-restic-on-windows-server-s3/6943

restic 0.16.2

restic 0.16.2 restores support for ARMv5 which was accidentally not included in the previous one, and also fixes missing documentation on Read the Docs.

restic is distributed as a standalone binary: download the correct file for your operating system and architecture, extract the file and just run it. If you run into any issues, please report them at the GitHub issue tracker or visit the forum. If you already have restic >= 0.9.4, you can use restic self-update to get the latest version in a secure way.

The binaries released with each restic version are reproducible, which means that you can reproduce a byte identical version from the source code for that release. Instructions on how to do that in the Developer Documentation.

Changelog for restic 0.16.2 (2023-10-29)

The following sections list the changes in restic 0.16.2 relevant to restic users. The changes are ordered by importance.

Summary

• Fix #4540: Restore ARMv5 support for ARM binaries
• Fix #4545: Repair documentation build on Read the Docs

Details

• Bugfix #4540: Restore ARMv5 support for ARM binaries

  The official release binaries for restic 0.16.1 were accidentally built to require ARMv7. The build process is now updated to restore support for ARMv5.

  Please note that restic 0.17.0 will drop support for ARMv5 and require at least ARMv6.

  #4540

• Bugfix #4545: Repair documentation build on Read the Docs

  For restic 0.16.1, no documentation was available at https://restic.readthedocs.io/ .

  The documentation build process is now updated to work again.

  #4545

restic 0.16.1

restic 0.16.1 has been released! It contains a few usage and cosmetic improvements, but also one bugfix for a very unlikely but still possible data corruption issue when using the max compression level. Please read the changelog for more details and information on how to verify your repository if you used max compression. Please also note that the auto compression level (which is the default) was never affected.

restic is distributed as a standalone binary: download the correct file for your operating system and architecture, extract the file and just run it. If you run into any issues, please report them at the GitHub issue tracker or visit the forum. If you already have restic >= 0.9.4, you can use restic self-update to get the latest version in a secure way.

The binaries released with each restic version are reproducible, which means that you can reproduce a byte identical version from the source code for that release. Instructions on how to do that in the Developer Documentation.

Changelog for restic 0.16.1 (2023-10-24)

The following sections list the changes in restic 0.16.1 relevant to restic users. The changes are ordered by importance.

Summary

• Fix #4513: Make key list command honor --no-lock
• Fix #4516: Do not try to load password on command line autocomplete
• Fix #4523: Update zstd library to fix possible data corruption at max. compression
• Chg #4532: Update dependencies and require Go 1.19 or newer
• Enh #229: Show progress bar while loading the index
• Enh #4128: Automatically set GOMAXPROCS in resource-constrained containers
• Enh #4480: Allow setting REST password and username via environment variables
• Enh #4511: Include inode numbers in JSON output for find and ls commands
• Enh #4519: Add config option to set SFTP command arguments

Details

• Bugfix #4513: Make key list command honor --no-lock

  The key list command now supports the --no-lock options. This allows determining which keys a repo can be accessed by without the need for having write access (e.g., read-only sftp access, filesystem snapshot).

  #4513 #4514

• Bugfix #4516: Do not try to load password on command line autocomplete

  The command line autocompletion previously tried to load the repository password. This could cause the autocompletion not to work. Now, this step gets skipped.

  #4516 #4526

• Bugfix #4523: Update zstd library to fix possible data corruption at max. compression

  In restic 0.16.0, backups where the compression level was set to max (using --compression max) could in rare and very specific circumstances result in data corruption due to a bug in the library used for compressing data.

  Restic now uses the latest version of the library used to compress data, which includes a fix for this issue. Please note that the auto compression level (which restic uses by default) was never affected, and even if you used max compression, chances of being affected by this issue were very small.

  To check a repository for any corruption, run restic check --read-data. This will download and verify the whole repository and can be used at any time to completely verify the integrity of a repository. If the check command detects anomalies, follow the suggested steps.

  To simplify any needed repository repair and minimize data loss, there is also a new and experimental repair packs command that salvages all valid data from the affected pack files (see restic help repair packs for more information).

  #4523 #4530

• Change #4532: Update dependencies and require Go 1.19 or newer

  We have updated all dependencies. Since some libraries require newer Go standard library features, support for Go 1.18 has been dropped, which means that restic now requires at least Go 1.19 to build.

  #4532 #4533

• Enhancement #229: Show progress bar while loading the index

  Restic did not provide any feedback while loading index files. Now, there is a progress bar that shows the index loading progress.

  #229 #4419

• Enhancement #4128: Automatically set GOMAXPROCS in resource-constrained containers

  When running restic in a Linux container with CPU-usage limits, restic now automatically adjusts GOMAXPROCS. This helps to reduce the memory consumption on hosts with many CPU cores.

  #4128 #4485 #4531

• Enhancement #4480: Allow setting REST password and username via environment variables

  Previously, it was only possible to specify the REST-server username and password in the repository URL, or by using the --repository-file option. This meant it was not possible to use authentication in contexts where the repository URL is stored in publicly accessible way.

  Restic now allows setting the username and password using the RESTIC_REST_USERNAME and RESTIC_REST_PASSWORD variables.

  #4480

• Enhancement #4511: Include inode numbers in JSON output for find and ls commands

  Restic used to omit the inode numbers in the JSON messages emitted for nodes by the ls command as well as for matches by the find command. It now includes those values whenever they are available.

  #4511

• Enhancement #4519: Add config option to set SFTP command arguments

  When using the sftp backend, scenarios where a custom identity file was needed for the SSH connection, required the full command to be specified: -o sftp.command='ssh user@host:port -i /ssh/my_private_key -s sftp'

  Now, the -o sftp.args=... option can be passed to restic to specify custom arguments for the SSH command executed by the SFTP backend. This simplifies the above example to -o sftp.args='-i /ssh/my_private_key'.

  #4241 #4519

restic 0.16.0

restic 0.16.0 has been released!

restic is distributed as a standalone binary: download the correct file for your operating system and architecture, extract the file and just run it. If you run into any issues, please report them at the GitHub issue tracker or visit the forum. If you already have restic >= 0.9.4, you can use restic self-update to get the latest version in a secure way.

The binaries released with each restic version are reproducible, which means that you can reproduce a byte identical version from the source code for that release. Instructions on how to do that in the Developer Documentation.

Changelog for restic 0.16.0 (2023-07-31)

The following sections list the changes in restic 0.16.0 relevant to restic users. The changes are ordered by importance.

Summary

• Fix #2565: Support "unlimited" in forget --keep-* options
• Fix #3311: Support non-UTF8 paths as symlink target
• Fix #4199: Avoid lock refresh issues on slow network connections
• Fix #4274: Improve lock refresh handling after standby
• Fix #4319: Correctly clean up status bar output of the backup command
• Fix #4333: generate and init no longer silently ignore unexpected arguments
• Fix #4400: Ignore missing folders in rest backend
• Chg #4176: Fix JSON message type of scan_finished for the backup command
• Chg #4201: Require Go 1.20 for Solaris builds
• Enh #426: Show progress bar during restore
• Enh #719: Add --retry-lock option
• Enh #1495: Sort snapshots by timestamp in restic find
• Enh #1759: Add repair index and repair snapshots commands
• Enh #1926: Allow certificate paths to be passed through environment variables
• Enh #2359: Provide multi-platform Docker images
• Enh #2468: Add support for non-global Azure clouds
• Enh #2679: Reduce file fragmentation for local backend
• Enh #3328: Reduce memory usage by up to 25%
• Enh #3397: Improve accuracy of ETA displayed during backup
• Enh #3624: Keep oldest snapshot when there are not enough snapshots
• Enh #3698: Add support for Managed / Workload Identity to azure backend
• Enh #3871: Support <snapshot>:<subfolder> syntax to select subfolders
• Enh #3941: Support --group-by for backup parent selection
• Enh #4130: Cancel current command if cache becomes unusable
• Enh #4159: Add --human-readable option to ls and find commands
• Enh #4188: Include restic version in snapshot metadata
• Enh #4220: Add jq binary to Docker image
• Enh #4226: Allow specifying region of new buckets in the gs backend
• Enh #4375: Add support for extended attributes on symlinks

Details

• Bugfix #2565: Support "unlimited" in forget --keep-* options

  Restic would previously forget snapshots that should have been kept when a negative value was passed to the --keep-* options. Negative values are now forbidden. To keep all snapshots, the special value unlimited is now supported. For example, --keep-monthly unlimited will keep all monthly snapshots.

  #2565 #4234

• Bugfix #3311: Support non-UTF8 paths as symlink target

  Earlier restic versions did not correctly backup and restore symlinks that contain a non-UTF8 target. Note that this only affected systems that still use a non-Unicode encoding for filesystem paths.

  The repository format is now extended to add support for such symlinks. Please note that snapshots must have been created with at least restic version 0.16.0 for restore to correctly handle non-UTF8 symlink targets when restoring them.

  #3311 #3802

• Bugfix #4199: Avoid lock refresh issues on slow network connections

  On network connections with a low upload speed, backups and other operations could fail with the error message Fatal: failed to refresh lock in time.

  This has now been fixed by reworking the lock refresh handling.

  #4199 #4304

• Bugfix #4274: Improve lock refresh handling after standby

  If the restic process was stopped or the host running restic entered standby during a long running operation such as a backup, this previously resulted in the operation failing with Fatal: failed to refresh lock in time.

  This has now been fixed such that restic first checks whether it is safe to continue the current operation and only throws an error if not.

  #4274 #4374

• Bugfix #4319: Correctly clean up status bar output of the backup command

  Due to a regression in restic 0.15.2, the status bar of the backup command could leave some output behind. This happened if filenames were printed that are wider than the current terminal width. This has now been fixed.

  #4319 #4318

• Bugfix #4333: generate and init no longer silently ignore unexpected arguments

  #4333

• Bugfix #4400: Ignore missing folders in rest backend

  If a repository accessed via the REST backend was missing folders, then restic would fail with an error while trying to list the data in the repository. This has been now fixed.

  #235 #4400

• Change #4176: Fix JSON message type of scan_finished for the backup command

  Restic incorrectly set the message_type of the scan_finished message to status instead of verbose_status. This has now been corrected so that the messages report the correct type.

  #4176

• Change #4201: Require Go 1.20 for Solaris builds

  Building restic on Solaris now requires Go 1.20, as the library used to access Azure uses the mmap syscall, which is only available on Solaris starting from Go 1.20. All other platforms however continue to build with Go 1.18.

  #4201

• Enhancement #426: Show progress bar during restore

  The restore command now shows a progress report while restoring files.

  Example: [0:42] 5.76% 23 files 12.98 MiB, total 3456 files 23.54 GiB

  JSON output is now also supported.

  #426 #3413 #3627 #3991 #4314 https://forum.restic.net/t/progress-bar-for-restore/5210

• Enhancement #719: Add --retry-lock option

  This option allows specifying a duration for which restic will wait if the repository is already locked.

  #719 #2214 #4107

• Enhancement #1495: Sort snapshots by timestamp in restic find

  The find command used to print snapshots in an arbitrary order. Restic now prints snapshots sorted by timestamp.

  #1495 #4409

• Enhancement #1759: Add repair index and repair snapshots commands

  The rebuild-index command has been renamed to repair index. The old name will still work, but is deprecated.

  When a snapshot was damaged, the only option up to now was to completely forget the snapshot, even if only some unimportant files in it were damaged and other files were still fine.

  Restic now has a repair snapshots command, which can salvage any non-damaged files and parts of files in the snapshots by removing damaged directories and missing file contents. Please note that the damaged data may still be lost and see the "Troubleshooting" section in the documentation for more details.

  #1759 [#1714](https://github.com/restic/r...

Restic 0.15.2

We're very pleased to present you restic 0.15.2! This is just (another) small bugfix release.

restic is distributed as a standalone binary: download the correct file for your operating system and architecture, extract the file and just run it. If you run into any issues, please report them at the GitHub issue tracker or visit the forum. If you already have restic >= 0.9.4, you can use restic self-update to get the latest version in a secure way.

The binaries released with each restic version are reproducible, which means that you can reproduce a byte identical version from the source code for that release. Instructions on how to do that in the Developer Documentation.

Changelog for restic 0.15.2 (2023-04-24)

The following sections list the changes in restic 0.15.2 relevant to restic users. The changes are ordered by importance.

Summary

• Sec #4275: Update golang.org/x/net to address CVE-2022-41723
• Fix #2260: Sanitize filenames printed by backup during processing
• Fix #4211: Make dump interpret --host and --path correctly
• Fix #4239: Correct number of blocks reported in mount point
• Fix #4253: Minimize risk of spurious filesystem loops with mount
• Enh #4180: Add release binaries for riscv64 architecture on Linux
• Enh #4219: Upgrade Minio to version 7.0.49

Details

• Security #4275: Update golang.org/x/net to address CVE-2022-41723

  #4275 #4213

• Bugfix #2260: Sanitize filenames printed by backup during processing

  The backup command would previously not sanitize the filenames it printed during processing, potentially causing newlines or terminal control characters to mangle the status output or even change the state of a terminal.

  Filenames are now checked and quoted if they contain non-printable or non-Unicode characters.

  #2260 #4191 #4192

• Bugfix #4211: Make dump interpret --host and --path correctly

  A regression in restic 0.15.0 caused dump to confuse its --host=<host> and --path=<path> options: it looked for snapshots with paths called <host> from hosts called <path>. It now treats the options as intended.

  #4211 #4212

• Bugfix #4239: Correct number of blocks reported in mount point

  Restic mount points reported an incorrect number of 512-byte (POSIX standard) blocks for files and links due to a rounding bug. In particular, empty files were reported as taking one block instead of zero.

  The rounding is now fixed: the number of blocks reported is the file size (or link target size) divided by 512 and rounded up to a whole number.

  #4239 #4240

• Bugfix #4253: Minimize risk of spurious filesystem loops with mount

  When a backup contains a directory that has the same name as its parent, say a/b/b, and the GNU find command was run on this backup in a restic mount, find would refuse to traverse the lowest b directory, instead printing File system loop detected. This was due to the way the restic mount command generates inode numbers for directories in the mount point.

  The rule for generating these inode numbers was changed in 0.15.0. It has now been changed again to avoid this issue. A perfect rule does not exist, but the probability of this behavior occurring is now extremely small.

  When it does occur, the mount point is not broken, and scripts that traverse the mount point should work as long as they don't rely on inode numbers for detecting filesystem loops.

  #4253 #4255

• Enhancement #4180: Add release binaries for riscv64 architecture on Linux

  Builds for the riscv64 architecture on Linux are now included in the release binaries.

  #4180

• Enhancement #4219: Upgrade Minio to version 7.0.49

  The upgraded version now allows use of the ap-southeast-4 region (Melbourne).

  #4219

restic 0.15.1

We're very pleased to present you restic 0.15.1! This is just a small bugfix release, fixing some issues that arose after 0.15.0 was published.

restic is distributed as a standalone binary: download the correct file for your operating system and architecture, extract the file and just run it. If you run into any issues, please report them at the GitHub issue tracker or visit the forum. If you already have restic >= 0.9.4, you can use restic self-update to get the latest version in a secure way.

The binaries released with each restic version are reproducible, which means that you can reproduce a byte identical version from the source code for that release. Instructions on how to do that in the Developer Documentation.

Changelog for restic 0.15.1 (2023-01-30)

The following sections list the changes in restic 0.15.1 relevant to restic users. The changes are ordered by importance.

Summary

• Fix #3750: Remove b2_download_file_by_name: 404 warning from B2 backend
• Fix #4147: Make prune --quiet not print progress bar
• Fix #4163: Make self-update --output work with new filename on Windows
• Fix #4167: Add missing ETA in backup progress bar
• Enh #4143: Ignore empty lock files

Details

• Bugfix #3750: Remove b2_download_file_by_name: 404 warning from B2 backend

  In some cases the B2 backend could print b2_download_file_by_name: 404: : b2.b2err warnings. These are only debug messages and can be safely ignored.

  Restic now uses an updated library for accessing B2, which removes the warning.

  #3750 #4144 #4146

• Bugfix #4147: Make prune --quiet not print progress bar

  A regression in restic 0.15.0 caused prune --quiet to show a progress bar while deciding how to process each pack files. This has now been fixed.

  #4147 #4153

• Bugfix #4163: Make self-update --output work with new filename on Windows

  Since restic 0.14.0 the self-update command did not work when a custom output filename was specified via the --output option. This has now been fixed.

  As a workaround, either use an older restic version to run the self-update or create an empty file with the output filename before updating e.g. using CMD:

  type nul > new-file.exe restic self-update --output new-file.exe

  #4163 https://forum.restic.net/t/self-update-windows-started-failing-after-release-of-0-15/5836

• Bugfix #4167: Add missing ETA in backup progress bar

  A regression in restic 0.15.0 caused the ETA to be missing from the progress bar displayed by the backup command. This has now been fixed.

  #4167

• Enhancement #4143: Ignore empty lock files

  With restic 0.15.0 the checks for stale locks became much stricter than before. In particular, empty or unreadable locks were no longer silently ignored. This made restic to complain with Load(<lock/1234567812>, 0, 0) returned error, retrying after 552.330144ms: load(<lock/1234567812>): invalid data returned and fail in the end.

  The error message is now clarified and the implementation changed to ignore empty lock files which are sometimes created as the result of a failed uploads on some backends.

  Please note that unreadable lock files still have to cleaned up manually. To do so, you can run restic unlock --remove-all which removes all existing lock files. But first make sure that no other restic process is currently using the repository.

  #4143 #4152

restic 0.15.0

We're very pleased to present you restic 0.15.0!

In this version a new rewrite command has been implemented. This allows for removing unwanted data from existing snapshots, for example if one discovers that some files were unintentionally backed up, such as files with sensitive content or files that take up a lot of space and don't need to be backed up. Please see the corresponding documentation for more information and usage examples.

restic is distributed as a standalone binary: download the correct file for your operating system and architecture, extract the file and just run it. If you run into any issues, please report them at the GitHub issue tracker or visit the forum. If you already have restic >= 0.9.4, you can use restic self-update to get the latest version in a secure way.

The binaries released with each restic version are reproducible, which means that you can reproduce a byte identical version from the source code for that release. Instructions on how to do that in the Developer Documentation.

Changelog for restic 0.15.0 (2023-01-12)

The following sections list the changes in restic 0.15.0 relevant to restic users. The changes are ordered by importance.

Summary

• Fix #2015: Make mount return exit code 0 after receiving Ctrl-C / SIGINT
• Fix #2578: Make restore replace existing symlinks
• Fix #2591: Don't read password from stdin for backup --stdin
• Fix #3161: Delete files on Backblaze B2 more reliably
• Fix #3336: Make SFTP backend report no space left on device
• Fix #3567: Improve handling of interrupted syscalls in mount command
• Fix #3897: Fix stuck copy command when -o <backend>.connections=1
• Fix #3918: Correct prune statistics for partially compressed repositories
• Fix #3951: Make ls return exit code 1 if snapshot cannot be loaded
• Fix #4003: Make backup no longer hang on Solaris when seeing a FIFO file
• Fix #4016: Support ExFAT-formatted local backends on macOS Ventura
• Fix #4085: Make init ignore "Access Denied" errors when creating S3 buckets
• Fix #4100: Make self-update enabled by default only in release builds
• Fix #4103: Don't generate negative UIDs and GIDs in tar files from dump
• Chg #2724: Include full snapshot ID in JSON output of backup
• Chg #3929: Make unlock display message only when locks were actually removed
• Chg #4033: Don't print skipped snapshots by default in copy command
• Chg #4041: Update dependencies and require Go 1.18 or newer
• Enh #14: Implement rewrite command
• Enh #79: Restore files with long runs of zeros as sparse files
• Enh #1078: Support restoring symbolic links on Windows
• Enh #1734: Inform about successful retries after errors
• Enh #1866: Improve handling of directories with duplicate entries
• Enh #2134: Support B2 API keys restricted to hiding but not deleting files
• Enh #2152: Make init open only one connection for the SFTP backend
• Enh #2533: Handle cache corruption on disk and in downloads
• Enh #2715: Stricter repository lock handling
• Enh #2750: Make backup file read concurrency configurable
• Enh #3029: Add support for credential_process to S3 backend
• Enh #3096: Make mount command support macOS using macFUSE 4.x
• Enh #3124: Support JSON output for the init command
• Enh #3899: Optimize prune memory usage
• Enh #3905: Improve speed of parent snapshot detection in backup command
• Enh #3915: Add compression statistics to the stats command
• Enh #3925: Provide command completion for PowerShell
• Enh #3931: Allow backup file tree scanner to be disabled
• Enh #3932: Improve handling of ErrDot errors in rclone and sftp backends
• Enh #3943: Ignore additional/unknown files in repository
• Enh #3955: Improve backup performance for small files

Details

• Bugfix #2015: Make mount return exit code 0 after receiving Ctrl-C / SIGINT

  To stop the mount command, a user has to press Ctrl-C or send a SIGINT signal to restic. This used to cause restic to exit with a non-zero exit code.

  The exit code has now been changed to zero as the above is the expected way to stop the mount command and should therefore be considered successful.

  #2015 #3894

• Bugfix #2578: Make restore replace existing symlinks

  When restoring a symlink, restic used to report an error if the target path already existed. This has now been fixed such that the potentially existing target path is first removed before the symlink is restored.

  #2578 #3780

• Bugfix #2591: Don't read password from stdin for backup --stdin

  The backup command when used with --stdin previously tried to read first the password, then the data to be backed up from standard input. This meant it would often confuse part of the data for the password.

  From now on, it will instead exit with the message Fatal: cannot read both password and data from stdin unless the password is passed in some other way (such as --restic-password-file, RESTIC_PASSWORD, etc).

  To enter the password interactively a password command has to be used. For example on Linux, mysqldump somedatabase | restic backup --stdin --password-command='sh -c "systemd-ask-password < /dev/tty"' securely reads the password from the terminal.

  #2591 #4011

• Bugfix #3161: Delete files on Backblaze B2 more reliably

  Restic used to only delete the latest version of files stored in B2. In most cases this worked well as there was only a single version of the file. However, due to retries while uploading it is possible for multiple file versions to be stored at B2. This could lead to various problems for files that should have been deleted but still existed.

  The implementation has now been changed to delete all versions of files, which doubles the amount of Class B transactions necessary to delete files, but assures that no file versions are left behind.

  #3161 #3885

• Bugfix #3336: Make SFTP backend report no space left on device

  Backing up to an SFTP backend would spew repeated SSH_FX_FAILURE messages when the remote disk was full. Restic now reports "sftp: no space left on device" and exits immediately when it detects this condition.

  A fix for this issue was implemented in restic 0.12.1, but unfortunately the fix itself contained a bug that prevented it from taking effect.

  #3336 #3345 #4075

• Bugfix #3567: Improve handling of interrupted syscalls in mount command

  Accessing restic's FUSE mount could result in "input/output" errors when using programs in which syscalls can be interrupted. This is for example the case for Go programs. This has now been fixed by improved error handling of interrupted syscalls.

  #3567 #3694 #3875

• Bugfix #3897: Fix stuck copy command when -o <backend>.connections=1

  When running the copy command with -o <backend>.connections=1 the command would be infinitely stuck. This has now been fixed.

  [#3897](h...

restic 0.14.0

We're very pleased to present you restic 0.14.0! This is a great release which introduces a new repository version that adds the most requested feature: compression!

restic is distributed as a standalone binary: download the correct file for your operating system and architecture, extract the file and just run it. If you run into any issues, please report them at the GitHub issue tracker or visit the forum. If you already have restic >= 0.9.4, you can use restic self-update to get the latest version in a secure way.

The binaries released with each restic version are reproducible, which means that you can reproduce a byte identical version from the source code for that release. Instructions on how to do that in the Developer Documentation.

Changelog for restic 0.14.0 (2022-08-25)

The following sections list the changes in restic 0.14.0 relevant to restic users. The changes are ordered by importance.

Summary

• Fix #2248: Support self-update on Windows
• Fix #3428: List snapshots in backend at most once to resolve snapshot IDs
• Fix #3432: Fix rare 'not found in repository' error for copy command
• Fix #3685: The diff command incorrectly listed some files as added
• Fix #3681: Fix rclone (shimmed by Scoop) and sftp not working on Windows
• Fix #3720: Directory sync errors for repositories accessed via SMB
• Fix #3736: The stats command miscalculated restore size for multiple snapshots
• Fix #3861: Yield error on invalid policy to forget
• Fix #3716: Print "wrong password" to stderr instead of stdout
• Fix #3772: Correctly rebuild index for legacy repositories
• Fix #3776: Limit number of key files tested while opening a repository
• Chg #1842: Support debug log creation in release builds
• Chg #3295: Deprecate check --check-unused and add further checks
• Chg #3680: Update dependencies and require Go 1.15 or newer
• Chg #3742: Replace --repo2 option used by init/copy with --from-repo
• Enh #1153: Support pruning even when the disk is full
• Enh #21: Add compression support
• Enh #2162: Adaptive IO concurrency based on backend connections
• Enh #2291: Allow pack size customization
• Enh #2295: Allow use of SAS token to authenticate to Azure
• Enh #2696: Improve backup speed with many small files
• Enh #2907: Make snapshot directory structure of mount command customizable
• Enh #3114: Optimize handling of duplicate blobs in prune
• Enh #3465: Improve handling of temporary files on Windows
• Enh #3709: Validate exclude patterns before backing up
• Enh #3837: Improve SFTP repository initialization over slow links
• Enh #2351: Use config file permissions to control file group access
• Enh #3475: Allow limiting IO concurrency for local and SFTP backend
• Enh #3484: Stream data in check and prune commands
• Enh #2923: Improve speed of copy command
• Enh #3729: Display full IDs in check warnings
• Enh #3773: Optimize memory usage for directories with many files
• Enh #3819: Validate include/exclude patterns before restoring

Details

• Bugfix #2248: Support self-update on Windows

  Restic self-update would fail in situations where the operating system locks running binaries, including Windows. The new behavior works around this by renaming the running file and swapping the updated file in place.

  #2248 #3675

• Bugfix #3428: List snapshots in backend at most once to resolve snapshot IDs

  Many commands support specifying a list of snapshot IDs which are then used to determine the snapshots to be processed by the command. To resolve snapshot IDs or latest, and check that these exist, restic previously listed all snapshots stored in the repository. Depending on the backend this could be a slow and/or expensive operation.

  Restic now lists the snapshots only once and remembers the result in order to resolve all further snapshot IDs swiftly.

  #3428 #3570 #3395

• Bugfix #3432: Fix rare 'not found in repository' error for copy command

  In rare cases copy (and other commands) would report that LoadTree(...) returned an id [...] not found in repository error. This could be caused by a backup or copy command running concurrently. The error was only temporary; running the failed restic command a second time as a workaround did resolve the error.

  This issue has now been fixed by correcting the order in which restic reads data from the repository. It is now guaranteed that restic only loads snapshots for which all necessary data is already available.

  #3432 #3570

• Bugfix #3685: The diff command incorrectly listed some files as added

  There was a bug in the diff command, causing it to always show files in a removed directory as added. This has now been fixed.

  #3685 #3686

• Bugfix #3681: Fix rclone (shimmed by Scoop) and sftp not working on Windows

  In #3602 a fix was introduced to address the problem of rclone prematurely exiting when Ctrl+C is pressed on Windows. The solution was to create the subprocess with its console detached from the restic console.

  However, this solution failed when using rclone installed by Scoop or using sftp with a passphrase-protected private key. We've now fixed this by using a different approach to prevent Ctrl-C from passing down too early.

  #3681 #3692 #3696

• Bugfix #3720: Directory sync errors for repositories accessed via SMB

  On Linux and macOS, accessing a repository via a SMB/CIFS mount resulted in restic failing to save the lock file, yielding the following errors:

  Save(<lock/071fe833f0>) returned error, retrying after 552.330144ms: sync /repo/locks: no such file or directory Save(<lock/bf789d7343>) returned error, retrying after 552.330144ms: sync /repo/locks: invalid argument

  This has now been fixed by ignoring the relevant error codes.

  #3720 #3751 #3752

• Bugfix #3736: The stats command miscalculated restore size for multiple snapshots

  Since restic 0.10.0 the restore size calculated by the stats command for multiple snapshots was too low. The hardlink detection was accidentally applied across multiple snapshots and thus ignored many files. This has now been fixed.

  #3736 #3740

• Bugfix #3861: Yield error on invalid policy to forget

  The forget command previously silently ignored invalid/unsupported units in the duration options, such as e.g. --keep-within-daily 2w.

  Specifying an invalid/unsupported duration unit now results in an error.

  #3861 #3862

• Bugfix #3716: Print "wrong password" to stderr instead of stdout

  If an invalid password was entered, the error message was printed on stdout and not on stderr as intended. This has now been fixed.

  #3716 https://forum.restic.net/t/4965

• Bugfix #3772: Correctly rebuild index for legacy repositories

  After running rebuild-index on a legacy repository containing mixed pack files (that is, pack files which store both metadata and file data), check printed warnings like pack 12345678 contained in several indexes: .... This warning was not critical, but has now nonetheless been fixed by properly handling mixed...

restic 0.13.1

We're very pleased to present you restic 0.13.1! This is a small bugfix release which fixes two issues that were discovered right after 0.13.0.

restic is distributed as a standalone binary: download the correct file for your operating system and architecture, extract the file and just run it. If you run into any issues, please report them at the GitHub issue tracker or visit the forum. If you already have restic >= 0.9.4, you can use restic self-update to get the latest version in a secure way.

The binaries released with each restic version are reproducible, which means that you can reproduce a byte identical version from the source code for that release. Instructions on how to do that in the Developer Documentation.

Changelog for restic 0.13.1 (2022-04-10)

The following sections list the changes in restic 0.13.1 relevant to restic users. The changes are ordered by importance.

Summary

• Fix #3685: Fix the diff command
• Fix #3681: Fix rclone (shimmed by Scoop) and sftp stopped working on Windows

Details

• Bugfix #3685: Fix the diff command

  There was a bug in the diff command, it would always show files in a removed directory as added. We've fixed that.

  #3685 #3686

• Bugfix #3681: Fix rclone (shimmed by Scoop) and sftp stopped working on Windows

  In #3602 a fix was introduced to fix the problem that rclone prematurely exits when Ctrl+C is pressed on Windows. The solution was to create the subprocess with its console detached from the restic console. However, such solution fails when using rclone install by scoop or using sftp with a passphrase- protected private key. We've fixed that by using a different approach to prevent Ctrl-C from passing down too early.

  #3681 #3692 #3696