Verification sporadically fails on 7z creation

[akrabu]

Configuration

• Keka version: 1.3.3 (5230)
• macOS version: Ventura 13.4.1 (c)

Describe the bug

I was creating about 20 7z AES encrypted archives, with splitting turned on, and verification. One did not verify properly. When tested using 7z-cli, it checked out just fine.

To Reproduce

This happens sporadically, and I'm not certain I can reproduce it.

Expected behavior

Verification to succeed, provided it would succeed using 7z-cli.

Screenshots

Keka:

7z-cli:

7zz t /Volumes/Media\ Backup\ \(RAID-1\)/2010.7z.001 

7-Zip (z) 23.01 (x64) : Copyright (c) 1999-2023 Igor Pavlov : 2023-06-20
 64-bit locale=en_US.UTF-8 Threads:8 OPEN_MAX:256

Scanning the drive for archives:
1 file, 4697620480 bytes (4480 MiB)      

Testing archive: /Volumes/Media Backup (RAID-1)/2010.7z.001
           
Enter password: ******

--
Path = /Volumes/Media Backup (RAID-1)/2010.7z.001
Type = Split
Physical Size = 4697620480
Volumes = 9
Total Physical Size = 40991022614
----
Path = 2010.7z
Size = 40991022614
--
Path = 2010.7z
Type = 7z
Physical Size = 40991022614
Headers Size = 144502
Method = Copy 7zAES
Solid = -
Blocks = 3888

Everything is Ok                                                                             

Folders: 110
Files: 3888
Size:       40990848573
Compressed: 40991022614

Lastly, here's Keka's output log:

OS: Version 13.4.1 (c) (Build 22F770820d) (x86_64)
Keka: v1.3.3-r5230 (WEB) (NOT sandboxed) (en)
Binary used: keka7zz
Arguments: (
    a,
    "-t7z",
    "-snh",
    "-snl",
    "-v4697620480b",
    "-mx0",
    "-p",
    "-mhe",
    "-ms=off",
    "-bsp1",
    "-spd",
    "/Volumes/Media Backup (RAID-1)/2010.7z",
    "/Volumes/Media/Media/2010"
)

7-Zip (z) 23.01 (x64) : Copyright (c) 1999-2023 Igor Pavlov : 2023-06-20 : Modified by aone for Keka
 64-bit locale=en_US.UTF-8 Threads:8 OPEN_MAX:2560 temp_path:/Users/akrabu/Library/Application Support/Keka/Temp/

Scanning the drive:
  0M Scan  /Volumes/Media/Media/
110 folders, 3888 files, 40990848573 bytes (39 GiB)


Creating archive: /Volumes/Media Backup (RAID-1)/2010.7z

Add new data to archive: 110 folders, 3888 files, 40990848573 bytes (39 GiB)


___KEKA___PASSWORD___KEKA___:

  0%


Files read from disk: 3888
Archive size: 40991022614 bytes (39 GiB)
Volumes: 9
Everything is Ok

Verifying compression...
OS: Version 13.4.1 (c) (Build 22F770820d) (x86_64)
Keka: v1.3.3-r5230 (WEB) (NOT sandboxed) (en)
Binary used: keka7zz
Arguments: (
    t,
    "/Volumes/Media Backup (RAID-1)/2010.7z.001"
)

7-Zip (z) 23.01 (x64) : Copyright (c) 1999-2023 Igor Pavlov : 2023-06-20 : Modified by aone for Keka
 64-bit locale=en_US.UTF-8 Threads:8 OPEN_MAX:2560 temp_path:/Users/akrabu/Library/Application Support/Keka/Temp/

Scanning the drive for archives:
1 file, 4697620480 bytes (4480 MiB)

Testing archive: /Volumes/Media Backup (RAID-1)/2010.7z.001

___KEKA___PASSWORD___KEKA___:


--
Path = /Volumes/Media Backup (RAID-1)/2010.7z.001
Type = Split
Physical Size = 4697620480
Volumes = 9
Total Physical Size = 40991022614
----
Path = 2010.7z
Size = 40991022614
--
Path = 2010.7z
Type = 7z
Physical Size = 40991022614
Headers Size = 144502
Method = Copy 7zAES
Solid = -
Blocks = 3888


ERROR: CRC Failed in encrypted file. Wrong password? : 2010/2010-07-23 (...)/MVI_3248 [A5E63D1F].MOV


Sub items Errors: 1

Archives with Errors: 1

Sub items Errors: 1

Error code 340

[aonez]

Thanks a lot for the feedback @akrabu. Did you used a password for all the operations? Was it the same password on all of them?

[akrabu]

Sure did, and yes it was. I put in a password, and drug them all in at once.

Ps. Thanks for the follow-back on Mastadon haha :)

[aonez]

Just one last question to try to replicate, were all the operations ~40GB?

[akrabu]

^ Sorted by size, this is what I drug in.

Long story short, I was initially going to RAR up my photo/movie collection, split to ~5GB chunks, par2 everything, and burn to Blu-rays. RAR ended up being way too slow, even with no compression (about 80MB/s with Keka), so I switched to 7z without compression and with encryption (which worked at about 500MB/s, saturating the SATA link to my staging device). I had Verify and Archive Separately turned on. So the largest archive should be about 113GB.

So far it's written 632GB. Everything else verified just fine. I've seen the random verification issue pop up in the past, but just ignored it after verifying it again manually. Figured since it's happened 2-3 times now since the feature was added, a proper bug report was in order! (I always have Verify turned on, which means it doesn't happen often to me.)

[akrabu]

I also just extracted that specific file, and checked the CRC32 of it (I embed the CRC32 in the filenames), and it checked out just fine. So the archive really is fine, it appears!

I'm going to redo just 2010 and see if it does it again. Might help narrow it down.

[aonez]

Sure it feels like there is an issue with the verification + password. I'll do some tests tomorrow 🙌🏻

[akrabu]

Second attempt verified successfully. The file hashes ARE different - but I'm guessing 7z salts the passwords, making that a moot point.

[aonez]

I did roughly around 2TBs worth in tests and have not been able to reproduce the issue. Hashes should be different because of the encryption, that is expected. So at this point I'm not sure what the issue was there... I'll recheck the code and leave this issue open in case anyone replicates the issue.

Thanks for all the feedback and test @akrabu 🙇‍♂️

[gingerbeardman]

It's a lot of data. Could it be errors from a failing storage device?

[aonez]

Could it be errors from a failing storage device

@gingerbeardman Yeah also thought it may be because of the operation being in an external device so I did my tests using an external device. Maybe you’re right and it was just a temporary read issue, because later tests where valid.

@akrabu how that RAID disk connects to your Mac?

[akrabu]

It's a lot of data. Could it be errors from a failing storage device?

I was concerned about that at first, too. This is a Samsung T7 Shield 2TB external SSD. I've had it a little under a year. It's my main drive for personal photos / videos. Every file has a CRC32 code in the filename, courtesy of Rhash. As of last Friday, they also have the SHA256 code AND the last modified date as of summing, stored in extended attributes for each file, courtesy of Cshatag. I checked with both, and all ~900GB checks out fine. I don't think it's my SSD.

@akrabu how that RAID disk connects to your Mac?

The RAID-1 array is a SoftRAID volume. I have three 5TB drives, all mirrored, and mounted over USB. I've received no errors from the array. This is one of several backups (including a cloud backup, courtesy of Restic), cause I'm pretty paranoid about losing my photos haha

This verification "bug" has also happened to me on non-encrypted archives, I'm fairly certain. I'll turn on debug mode and just use it like that for awhile. Perhaps I'll catch it eventually. It's very sporadic, though. Happened on both my work and home Macs - but only like 2-3 times since the verification feature was rolled out. The actual archives always test out fine on a manual check, so it's not ACTUALLY failing. It's happened on much smaller ~5GB archives, as well.