API Bug: File download size and hash mis-match with iOS .heic files #2471

thomasfedb · 2023-08-15T11:16:24Z

Describe the bug

When synchronizing .heic files appear to have hash and file size mismatch errors similar to the following:

Downloading file Pictures/Camera Roll/2021/06/20210628_080438472_iOS.heic ... ERROR: File download size mis-match. Increase logging verbosity to determine why.
ERROR: File download hash mis-match. Increase logging verbosity to determine why.
INFO: Potentially add --disable-download-validation to work around this issue but downloaded data integrity cannot be guaranteed.

Operating System Details

➜ uname -a
Linux lakertya 6.4.9-200.fc38.x86_64 #1 SMP PREEMPT_DYNAMIC Tue Aug  8 21:21:11 UTC 2023 x86_64 GNU/Linux

~ 
➜ cat /etc/redhat-release
Fedora release 38 (Thirty Eight)

Client Installation Method

From Distribution Package

OneDrive Account Type

Personal

What is your OneDrive Application Version

2.4.25

What is your OneDrive Application Configuration

Configuration file successfully loaded
onedrive version                             = v2.4.25
Config path                                  = /home/thomasfedb/.config/onedrive
Config file found in config path             = true
Config option 'sync_dir'                     = /home/thomasfedb
Config option 'enable_logging'               = false
Config option 'log_dir'                      = /var/log/onedrive/
Config option 'disable_notifications'        = false
Config option 'min_notify_changes'           = 5
Config option 'skip_dir'                     = Documents/OneNote Notebooks
Config option 'skip_dir_strict_match'        = false
Config option 'skip_file'                    = ~*|.~*|*.tmp|thumbs.db*|Thumbs.db*
Config option 'skip_dotfiles'                = true
Config option 'skip_symlinks'                = true
Config option 'monitor_interval'             = 300
Config option 'monitor_log_frequency'        = 6
Config option 'monitor_fullscan_frequency'   = 12
Config option 'read_only_auth_scope'         = false
Config option 'dry_run'                      = false
Config option 'upload_only'                  = false
Config option 'download_only'                = false
Config option 'local_first'                  = false
Config option 'check_nosync'                 = false
Config option 'check_nomount'                = false
Config option 'resync'                       = false
Config option 'resync_auth'                  = false
Config option 'cleanup_local_files'          = false
Config option 'classify_as_big_delete'       = 1000
Config option 'disable_upload_validation'    = false
Config option 'bypass_data_preservation'     = false
Config option 'no_remote_delete'             = false
Config option 'remove_source_files'          = false
Config option 'sync_dir_permissions'         = 700
Config option 'sync_file_permissions'        = 600
Config option 'space_reservation'            = 52428800
Config option 'application_id'               = 
Config option 'azure_ad_endpoint'            = 
Config option 'azure_tenant_id'              = common
Config option 'user_agent'                   = 
Config option 'force_http_11'                = false
Config option 'debug_https'                  = false
Config option 'rate_limit'                   = 0
Config option 'operation_timeout'            = 3600
Config option 'dns_timeout'                  = 60
Config option 'connect_timeout'              = 10
Config option 'data_timeout'                 = 600
Config option 'ip_protocol_version'          = 0
Config option 'sync_root_files'              = false
Selective sync 'sync_list' configured        = true
sync_list contents:
Documents
Pictures
Config option 'sync_business_shared_folders' = false
Business Shared Folders configured           = false
Config option 'webhook_enabled'              = false

What is your 'curl' version

curl 8.0.1 (x86_64-redhat-linux-gnu) libcurl/8.0.1 OpenSSL/3.0.9 zlib/1.2.13 libidn2/2.3.4 nghttp2/1.52.0
Release-Date: 2023-03-20
Protocols: file ftp ftps http https
Features: alt-svc AsynchDNS GSS-API HSTS HTTP2 HTTPS-proxy IDN IPv6 Kerberos Largefile libz SPNEGO SSL threadsafe UnixSockets

Where is your 'sync_dir' located

Local

What are all your system 'mount points'

proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime,seclabel)
devtmpfs on /dev type devtmpfs (rw,nosuid,seclabel,size=4096k,nr_inodes=8185231,mode=755,inode64)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev,seclabel,inode64)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,seclabel,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,seclabel,size=13105444k,nr_inodes=819200,mode=755,inode64)
cgroup2 on /sys/fs/cgroup type cgroup2 (rw,nosuid,nodev,noexec,relatime,seclabel,nsdelegate,memory_recursiveprot)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime,seclabel)
efivarfs on /sys/firmware/efi/efivars type efivarfs (rw,nosuid,nodev,noexec,relatime)
bpf on /sys/fs/bpf type bpf (rw,nosuid,nodev,noexec,relatime,mode=700)
ramfs on /run/credentials/systemd-vconsole-setup.service type ramfs (ro,nosuid,nodev,noexec,relatime,seclabel,mode=700)
configfs on /sys/kernel/config type configfs (rw,nosuid,nodev,noexec,relatime)
/dev/nvme0n1p3 on / type btrfs (rw,relatime,seclabel,compress=zstd:1,ssd,discard=async,space_cache=v2,subvolid=257,subvol=/root)
selinuxfs on /sys/fs/selinux type selinuxfs (rw,nosuid,noexec,relatime)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=33,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=23620)
mqueue on /dev/mqueue type mqueue (rw,nosuid,nodev,noexec,relatime,seclabel)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime,seclabel,pagesize=2M)
debugfs on /sys/kernel/debug type debugfs (rw,nosuid,nodev,noexec,relatime,seclabel)
tracefs on /sys/kernel/tracing type tracefs (rw,nosuid,nodev,noexec,relatime,seclabel)
fusectl on /sys/fs/fuse/connections type fusectl (rw,nosuid,nodev,noexec,relatime)
ramfs on /run/credentials/systemd-sysctl.service type ramfs (ro,nosuid,nodev,noexec,relatime,seclabel,mode=700)
ramfs on /run/credentials/systemd-tmpfiles-setup-dev.service type ramfs (ro,nosuid,nodev,noexec,relatime,seclabel,mode=700)
/dev/nvme0n1p3 on /home type btrfs (rw,relatime,seclabel,compress=zstd:1,ssd,discard=async,space_cache=v2,subvolid=256,subvol=/home)
tmpfs on /tmp type tmpfs (rw,nosuid,nodev,seclabel,nr_inodes=1048576,inode64)
/dev/nvme0n1p2 on /boot type ext4 (rw,relatime,seclabel)
/dev/nvme0n1p1 on /boot/efi type vfat (rw,relatime,fmask=0077,dmask=0077,codepage=437,iocharset=ascii,shortname=winnt,errors=remount-ro)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,nosuid,nodev,noexec,relatime)
ramfs on /run/credentials/systemd-tmpfiles-setup.service type ramfs (ro,nosuid,nodev,noexec,relatime,seclabel,mode=700)
ramfs on /run/credentials/systemd-resolved.service type ramfs (ro,nosuid,nodev,noexec,relatime,seclabel,mode=700)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
tmpfs on /run/user/1000 type tmpfs (rw,nosuid,nodev,relatime,seclabel,size=6552720k,nr_inodes=1638180,mode=700,uid=1000,gid=1000,inode64)
gvfsd-fuse on /run/user/1000/gvfs type fuse.gvfsd-fuse (rw,nosuid,nodev,relatime,user_id=1000,group_id=1000)
portal on /run/user/1000/doc type fuse.portal (rw,nosuid,nodev,relatime,user_id=1000,group_id=1000)

What are all your local file system partition types

NAME        FSTYPE FSVER LABEL  UUID                                 FSAVAIL FSUSE% MOUNTPOINTS
zram0                                                                               [SWAP]
nvme0n1                                                                             
├─nvme0n1p1 vfat   FAT32        260D-4304                             580.6M     3% /boot/efi
├─nvme0n1p2 ext4   1.0          b3dae080-bb87-400b-9c9d-c3d60bf2499f  601.4M    31% /boot
└─nvme0n1p3 btrfs        fedora b9443b7d-a460-4fb0-86b5-c978ad726f6e  588.2G    37% /home
                                                                                    /

How do you use 'onedrive'

Local install without multiple access of any sort.

Steps to reproduce the behaviour

onedrive --synchronize

Complete Verbose Log Output

...
Downloading file Pictures/Camera Roll/2021/11/20211120_084805935_iOS.heic ... ERROR: File download size mis-match. Increase logging verbosity to determine why.
ERROR: File download hash mis-match. Increase logging verbosity to determine why.
INFO: Potentially add --disable-download-validation to work around this issue but downloaded data integrity cannot be guaranteed.
Downloading file Pictures/Camera Roll/2021/11/20211120_052353919_iOS.heic ... ERROR: File download size mis-match. Increase logging verbosity to determine why.
ERROR: File download hash mis-match. Increase logging verbosity to determine why.
INFO: Potentially add --disable-download-validation to work around this issue but downloaded data integrity cannot be guaranteed.
Downloading file Pictures/Camera Roll/2021/11/20211120_052341942_iOS.heic ... ERROR: File download size mis-match. Increase logging verbosity to determine why.
ERROR: File download hash mis-match. Increase logging verbosity to determine why.
INFO: Potentially add --disable-download-validation to work around this issue but downloaded data integrity cannot be guaranteed.
Downloading file Pictures/Camera Roll/2021/11/20211120_052337973_iOS.heic ... ERROR: File download size mis-match. Increase logging verbosity to determine why.
ERROR: File download hash mis-match. Increase logging verbosity to determine why.
INFO: Potentially add --disable-download-validation to work around this issue but downloaded data integrity cannot be guaranteed.
Downloading file Pictures/Camera Roll/2021/11/20211120_052258044_iOS.heic ... ERROR: File download size mis-match. Increase logging verbosity to determine why.
ERROR: File download hash mis-match. Increase logging verbosity to determine why.
INFO: Potentially add --disable-download-validation to work around this issue but downloaded data integrity cannot be guaranteed.
...

Screenshots

No response

Other Log Information or Details

No response

Additional context

No response

The text was updated successfully, but these errors were encountered:

abraunegg · 2023-08-15T19:21:08Z

@thomasfedb
Unfortunately you are missing information to assist further:

Downloading file Pictures/Camera Roll/2021/11/20211120_084805935_iOS.heic ... ERROR: File download size mis-match. Increase logging verbosity to determine why.
ERROR: File download hash mis-match. Increase logging verbosity to determine why.

The application is asking you to increase your logging output to help determine why you are getting this error message. To do this, you need to add --verbose for normal verbose logging or --verbose --verbose for debug logging. When using debug logging, please read: https://github.com/abraunegg/onedrive/wiki/Generate-debug-log-for-support

Once you do this, the reason as to why there is a size and hash mis-match will be visible and it can be determined what to do.

If this has only started occurring, 100% most likely there has been an API change, which would make this an API bug, not a bug with this application.

Please can you generate the required:

verbose log and update 'Complete Verbose Log Output' with this data
generate the debug log and send via email

abraunegg · 2023-08-15T19:25:45Z

Indeed this is a long standing OneDrive API bug: OneDrive/onedrive-api-docs#1532 - which has been over-zealously closed.

abraunegg · 2023-08-15T19:32:52Z

@thomasfedb
I have re-published a new API bug here: OneDrive/onedrive-api-docs#1723

Please can you provide the debug log with priority so that this can be used in the API bug report.

abraunegg · 2023-08-15T21:42:07Z

@thomasfedb
Additionally I am 100% unable to reproduce this issue:

Using 'user' Config Dir: /home/alex/.config/onedrive-2.4.25/
Configuration file successfully loaded
Deleting the saved application sync status ...
Checking Application Version ...
Initializing the OneDrive API ...
Configuring Global Azure AD Endpoints
Using Curl defaults for all HTTP operations
Opening the item database ...
All operations will be performed in: /home/alex/OneDrive
Application version: v2.4.25-4-g50d80d3
Account Type: personal
Default Drive ID: 66d53be8a5056eca
Default Root ID: 66D53BE8A5056ECA!101
Remaining Free Space: 2327838720
Fetching details for OneDrive Root
OneDrive Root does not exist in the database. We need to add it.
Added OneDrive Root to the local database
Initializing the Synchronization Engine ...
Syncing changes and items from OneDrive ...
Applying changes of Path ID: 66D53BE8A5056ECA!101
Updated Remaining Free Space: 2317764190
Processing 273 OneDrive items to ensure consistent local state due to sync_list being used
Creating local directory: ./heic_image_files
Downloading file heic_image_files/dwsample-heic-4k.heic ... 
Downloading 100% |oooooooooooooooooooooooooooooooooooooooo| DONE IN 00:00:03                                                                                                                    
done.
Downloading file heic_image_files/image4.heic ... done.
Downloading file heic_image_files/image1.heic ... done.
Downloading file heic_image_files/image3.heic ... done.
Downloading file heic_image_files/image2.heic ... done.
Downloading file heic_image_files/dwsample-heic-1920.heic ... done.
Downloading file heic_image_files/dwsample-heic-1280.heic ... done.
Downloading file heic_image_files/dwsample-heic-640.heic ... done.

This was done several times:

Upload the files to OneDrive via the OneDrive Website
Upload the files from Linux --> OneDrive via this client, then delete the files locally and issue a --resync to download files.

Comparing the actual data, using the sample heic files I have found, pre-upload and what was downloaded shows zero difference:

[alex@onedrive-client-dev OneDrive]$ diff heic_image_files/image1.heic downloaded_heic/image1.heic 
[alex@onedrive-client-dev OneDrive]$

So my outstanding question here for you to answer is this:

How are your .heic files getting to OneDrive in the first place? Are they being uploaded directly from your iOS device to the same account you are using to access OneDrive with, with this client - thus bypassing any API or website interaction?

I do not own an iOS device - so I cannot test this.

The only other item I can think of here is that a local file system is the issue. You are using btrfs ... which is your choice to use - but potentially don't rule out that your local filesystem is the cause here. I use either XFS (development) or ZFS (systems with data I care about).

thomasfedb · 2023-08-17T09:43:15Z

@abraunegg

I did generate the log and email it through at the time of reporting the issue. If you haven't received it yet then let me know and I will send it again.

Regarding how the .heic files are getting onto OneDrive - they were uploaded to OneDrive by the iOS OneDrive client from the iOS camera roll.

abraunegg · 2023-08-17T10:01:35Z

@thomasfedb

I did generate the log and email it through at the time of reporting the issue. If you haven't received it yet then let me know and I will send it again.

Nothing has been received and/or blocked or in any spam folder. Please can you resend.

You may also need to validate all the received data by also adding --debug-https and see what the actual API service is sending you data wise. Please refer to https://github.com/abraunegg/onedrive/wiki/Generate-https-debug-log-for-support for this.

Regarding how the .heic files are getting onto OneDrive - they were uploaded to OneDrive by the iOS OneDrive client from the iOS camera roll.

OK .. so this is something that I cannot test - however as of now, when .heic files are on OneDrive in my testing, they are downloaded without issue (size online = size downloaded = size on-disk & file hashes all match), and the 'live' preview video appears to remain intact.

thomasfedb · 2023-08-17T10:14:22Z

@abraunegg Can you confirm the correct email address?

abraunegg · 2023-08-17T10:29:11Z

@thomasfedb

@abraunegg Can you confirm the correct email address?

It is as per: https://github.com/abraunegg/onedrive#reporting-an-issue-or-bug

thomasfedb · 2023-08-20T13:05:59Z

@thomasfedb

@abraunegg Can you confirm the correct email address?

It is as per: https://github.com/abraunegg/onedrive#reporting-an-issue-or-bug

I've resent the email just now.

abraunegg · 2023-08-20T22:53:22Z

There are a couple of things going on here based on the debug log:

The OneDrive API is reporting one file size, then sending only a reduced file size. This means you are experiencing data loss on your .heic files and this is a Microsoft API Bug.
The file comparison that is being done on these .heic files, is comparing the incorrect hash values (quickxor vs sha256) - so this will always be different. This could be just a logging bug or an actual application bug being exposed by the OneDrive API issue that I need to potentially look into further. This however is not detrimental and is not the cause of your issue.

Evidence:
When the application is receiving the JSON data about a file to download, the application receives the following:

{
	"@odata.type": "#microsoft.graph.driveItem",
	"cTag": "aYzpEMTNBM0VEMzRDQzZENDgyITIxMTA0LjI1OA",
	"eTag": "aRDEzQTNFRDM0Q0M2RDQ4MiEyMTEwNC4y",
	"file": {
		"hashes": {
			"quickXorHash": "9r/UaOPdoW68czpEsknBOlP3xAI=",
			"sha1Hash": "5056DEEC39DF0AE1DE67875282359F3A79C74057",
			"sha256Hash": "FC39014A2A3BF1E3CD823BAE18CE757C2F501F4877B7C4807348F622EACD9762"
		},
		"mimeType": "image/heic"
	},
	"fileSystemInfo": {
		"createdDateTime": "2022-05-31T13:56:53.22Z",
		"lastModifiedDateTime": "2022-05-31T13:56:53.22Z"
	},
	"id": "D13A3ED34CC6D482!21104",
	"name": "XXXX.heic",
	"parentReference": {
		"driveId": "redacted",
		"driveType": "personal",
		"id": "redacted",
		"name": "05",
		"path": "redacted"
	},
	"size": 3039814
}

The reported size online is 3039814 bytes.

When the application is processing this JSON we get the following:

[DEBUG] Local Disk Space Actual: 631629778944
[DEBUG] Free Space Reservation:  52428800
[DEBUG] File Size to Download:   3039814
[DEBUG] Setting file permissions for: redacted/path/to/file/XXXX.heic
[DEBUG] File size on disk:          682474
[DEBUG] OneDrive API reported size: 3039814
ERROR: File download size mis-match. Increase logging verbosity to determine why.
[DEBUG] Actual file hash:           FC39014A2A3BF1E3CD823BAE18CE757C2F501F4877B7C4807348F622EACD9762
[DEBUG] OneDrive API reported hash: 9r/UaOPdoW68czpEsknBOlP3xAI=
ERROR: File download hash mis-match. Increase logging verbosity to determine why.

The file size as downloaded by the application via the API is 682474 bytes - a dramatic difference - thus this is why the trigger for file size & hash mis-match is being hit.

Now one could say where is the evidence that the application is not at fault here. Without looking deeper into the application by using --debug-https the application debug logs wont show what is being delivered by the OneDrive API at the HTTP Transport Layer.

However, in your debug log there are some .heic files that are >4Mb in size, and, when doing a session download, the application writes out all of the chunked bytes that the application is receiving. When the log is analysed for these types of files we get the following JSON:

{
	"@odata.type": "#microsoft.graph.driveItem",
	"cTag": "aYzpEMTNBM0VEMzRDQzZENDgyITIxMTQ4LjI1OA",
	"eTag": "aRDEzQTNFRDM0Q0M2RDQ4MiEyMTE0OC4y",
	"file": {
		"hashes": {
			"quickXorHash": "TrNg2ZqtJPADBVEt7WhzVta4GlA=",
			"sha1Hash": "F97A129F915F23C5E23A2258260B2EA96CFD0C55",
			"sha256Hash": "22E0226F71C7C3D375030C61586350284AA7B3B9E31DD1F0B6A966D34F14C53A"
		},
		"mimeType": "image/heic"
	},
	"fileSystemInfo": {
		"createdDateTime": "2022-06-01T05:56:29.67Z",
		"lastModifiedDateTime": "2022-06-01T05:56:29.67Z"
	},
	"id": "D13A3ED34CC6D482!21148",
	"name": "YYYYYYY.heic",
	"parentReference": {
		"driveId": "redacted",
		"driveType": "personal",
		"id": "redacted",
		"name": "06",
		"path": "redacted"
	},
	"size": 6887427
}

When this is processed:

Downloading file Pictures/Camera Roll/2022/06/20220601_043439324_iOS.heic ... 
[DEBUG] Local Disk Space Actual: 631628136448
[DEBUG] Free Space Reservation:  52428800
[DEBUG] File Size to Download:   6887427


                                                                                
Downloading   0% |                                        |   ETA   --:--:--:
[DEBUG] Data Received    = 50697
[DEBUG] Expected Total   = 4583414
[DEBUG] Percent Complete = 1

[DEBUG] Data Received    = 50697
[DEBUG] Expected Total   = 4583414
[DEBUG] Percent Complete = 1

We can see here that the application is expecting, based on the JSON, to download a file size of 6887427, however when the OneDrive API session is initiated, the data from the OneDrive API changes 6887427 to 4583414

When the session download chunks have completed, we get the expected size difference detected application response:

[DEBUG] Data Received    = 4583414
[DEBUG] Expected Total   = 4583414
[DEBUG] Percent Complete = 100
[DEBUG] Incrementing Progress Bar using fmod match

                                                                                
Downloading  95% |oooooooooooooooooooooooooooooooooooooo  |   ETA   00:00:02 
[DEBUG] Data Received    = 4583414
[DEBUG] Expected Total   = 4583414
[DEBUG] Percent Complete = 100

[DEBUG] Data Received    = 4583414
[DEBUG] Expected Total   = 4583414
[DEBUG] Percent Complete = 100

[DEBUG] Data Received    = 4583414
[DEBUG] Expected Total   = 4583414
[DEBUG] Percent Complete = 100

[DEBUG] Setting file permissions for: redacted/path/to/file/YYYYYYY.heic
[DEBUG] File size on disk:          4583414
[DEBUG] OneDrive API reported size: 6887427
ERROR: File download size mis-match. Increase logging verbosity to determine why.
[DEBUG] Actual file hash:           22E0226F71C7C3D375030C61586350284AA7B3B9E31DD1F0B6A966D34F14C53A
[DEBUG] OneDrive API reported hash: TrNg2ZqtJPADBVEt7WhzVta4GlA=
ERROR: File download hash mis-match. Increase logging verbosity to determine why.
INFO: Potentially add --disable-download-validation to work around this issue but downloaded data integrity cannot be guaranteed.
[DEBUG] Download or creation of local directory failed
[DEBUG] ------------------------------------------------------------------

So the application is receiving all the API data correctly. The issue here is that the OneDrive API, for .heic files, is stripping data out of the actual file (causing data loss on your data) - and telling this application to download the smaller file that is being provided to the application.

abraunegg · 2023-08-21T00:17:23Z

@thomasfedb
I have looked at the code investigating '2' above, and yes, there is a debug logging output issue in terms of what hash to compare

Please can you test the following PR:

git clone https://github.com/abraunegg/onedrive.git
cd onedrive
git fetch origin pull/2473/head:pr2473
git checkout pr2473

To run the PR, you need to run the client from the PR build directory:

./onedrive <any options needed>

Now - this PR only impacts debug logging output, it does not change any application fundamentals / comparison as to what is being received via JSON from the OneDrive API and what the OneDrive API sends via the HTTP Transport.

Also, this PR, the logging changes are 100% going to be changed with v2.5.0 alpha-0 in the next few weeks, so this is your call here to test the PR | use the PR, however the hash comparison piece is a valid bug in v2.4.25 and will exist until v2.5.0 is released.

The bigger aspect here is this is 100% a @microsoft bug with the OneDrive API that is causing data loss and there is nothing I can do to fix this unfortunately.

per-oestergaard · 2023-10-25T16:08:54Z

Hi. I have been struggling with this issue for some months. I cannot force it to happen, but it happens all the time with files in my camera roll.

I did an interesting thing to troubleshoot this. I created a Power Automate flow that copies new files being uploaded to my camera roll and into a /CameraRollCopy folder. This has been running for some weeks now and the interesting thing is that I do not see any errors on that folder. So maybe this is related to how the iOS OneDrive app backups/uploads photos? Or maybe it is related to the fact that this is in this special folder?

My folder is called SkyDrive Camera Roll, so you might guess I have been using this for a while (since 2009). And maybe that is the reason? Some compatibility issue on old folders? I will probably try to change the folder some day when I have time for it. However, it could be interesting to learn if others having this issue also have "old" folder names.

per-oestergaard · 2023-10-26T18:52:33Z

An update: I have now renamed "SkyDrive camera roll" to "Kamerarulle" (Danish) so let's see if that changes anything (I doubt).

per-oestergaard · 2023-11-09T16:34:17Z

An update: The rename did not change anything. My /CameraRollCopy (see two comments up) syncs fine.

Anyone who has ideas on how we can move this forward? Hmm. I think I will capture the traffic on Windows some day to see if the requests are different.

centomila · 2023-12-07T15:06:04Z

I have the same problem.

My images are uploaded from an iPhone SE 2nd generation to OneDrive using the iOS OneDrive backup feature.

In the same folder, JPG and MOV files are downloaded correctly, while some HEIC files cause a mismatch error.

I'm not sure if the mismatched files are the ones edited with the iPhone Photo app or if it's random.

This issue occurs during the initial synchronization and installation after authenticating with OneDrive version 2.4.25-1+np1 with the command onedrive --monitor

Downloading file Immagini/Rullino/2021/02/20210220_134310003_iOS.heic ... 
Downloading 100% |oooooooooooooooooooooooooooooooooooooooo| DONE IN 00:00:01                                                                                              
ERROR: File download size mis-match. Increase logging verbosity to determine why.
ERROR: File download hash mis-match. Increase logging verbosity to determine why.
INFO: Potentially add --disable-download-validation to work around this issue but downloaded data integrity cannot be guaranteed.

I'm waiting the completion of the first sync for additional info

abraunegg · 2023-12-07T18:26:06Z

@centomila
You must complain to Microsoft, This is a bug with their platform and there is nothing this client can do to remediate the situation.

abraunegg · 2023-12-07T18:26:41Z

@per-oestergaard

Anyone who has ideas on how we can move this forward? Hmm. I think I will capture the traffic on Windows some day to see if the requests are different.

You must complain to Microsoft, This is a bug with their platform and there is nothing this client can do to remediate the situation.

abraunegg · 2023-12-07T18:29:40Z

@per-oestergaard , @centomila

Please raise your issue with this @microsoft bug here: OneDrive/onedrive-api-docs#1723

If this fails - if you have a business account with support, raise a support case with @microsoft - it is a OneDrive Platform Bug

thomasfedb added the Bug Something isn't working label Aug 15, 2023

abraunegg added More Information Needed Waiting on Response labels Aug 15, 2023

abraunegg changed the title ~~Bug: File download size and hash mis-match.~~ Bug: File download size and hash mis-match with iOS heic files Aug 15, 2023

abraunegg added the OneDrive API Limitation OneDrive API Limitation label Aug 15, 2023

abraunegg changed the title ~~Bug: File download size and hash mis-match with iOS heic files~~ Bug: File download size and hash mis-match with iOS .heic files Aug 15, 2023

abraunegg added OneDrive API Bug OneDrive API Bug and removed OneDrive API Limitation OneDrive API Limitation labels Aug 15, 2023

abraunegg added the Unable to reproduce label Aug 15, 2023

abraunegg removed More Information Needed Waiting on Response labels Aug 20, 2023

abraunegg removed the Bug Something isn't working label Sep 2, 2023

abraunegg changed the title ~~Bug: File download size and hash mis-match with iOS .heic files~~ API Bug: File download size and hash mis-match with iOS .heic files Sep 2, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

API Bug: File download size and hash mis-match with iOS .heic files #2471

API Bug: File download size and hash mis-match with iOS .heic files #2471

thomasfedb commented Aug 15, 2023 •

edited

abraunegg commented Aug 15, 2023 •

edited

abraunegg commented Aug 15, 2023

abraunegg commented Aug 15, 2023

abraunegg commented Aug 15, 2023

thomasfedb commented Aug 17, 2023

abraunegg commented Aug 17, 2023

thomasfedb commented Aug 17, 2023

abraunegg commented Aug 17, 2023

thomasfedb commented Aug 20, 2023

abraunegg commented Aug 20, 2023 •

edited

abraunegg commented Aug 21, 2023

This comment was marked as off-topic.

This comment was marked as off-topic.

This comment was marked as off-topic.

This comment was marked as off-topic.

per-oestergaard commented Oct 25, 2023

per-oestergaard commented Oct 26, 2023

per-oestergaard commented Nov 9, 2023 •

edited

centomila commented Dec 7, 2023 •

edited by abraunegg

abraunegg commented Dec 7, 2023

abraunegg commented Dec 7, 2023

abraunegg commented Dec 7, 2023

API Bug: File download size and hash mis-match with iOS .heic files #2471

API Bug: File download size and hash mis-match with iOS .heic files #2471

Comments

thomasfedb commented Aug 15, 2023 • edited

Describe the bug

Operating System Details

Client Installation Method

OneDrive Account Type

What is your OneDrive Application Version

What is your OneDrive Application Configuration

What is your 'curl' version

Where is your 'sync_dir' located

What are all your system 'mount points'

What are all your local file system partition types

How do you use 'onedrive'

Steps to reproduce the behaviour

Complete Verbose Log Output

Screenshots

Other Log Information or Details

Additional context

abraunegg commented Aug 15, 2023 • edited

abraunegg commented Aug 15, 2023

abraunegg commented Aug 15, 2023

abraunegg commented Aug 15, 2023

thomasfedb commented Aug 17, 2023

abraunegg commented Aug 17, 2023

thomasfedb commented Aug 17, 2023

abraunegg commented Aug 17, 2023

thomasfedb commented Aug 20, 2023

abraunegg commented Aug 20, 2023 • edited

abraunegg commented Aug 21, 2023

This comment was marked as off-topic.

This comment was marked as off-topic.

This comment was marked as off-topic.

This comment was marked as off-topic.

per-oestergaard commented Oct 25, 2023

per-oestergaard commented Oct 26, 2023

per-oestergaard commented Nov 9, 2023 • edited

centomila commented Dec 7, 2023 • edited by abraunegg

abraunegg commented Dec 7, 2023

abraunegg commented Dec 7, 2023

abraunegg commented Dec 7, 2023

thomasfedb commented Aug 15, 2023 •

edited

abraunegg commented Aug 15, 2023 •

edited

abraunegg commented Aug 20, 2023 •

edited

per-oestergaard commented Nov 9, 2023 •

edited

centomila commented Dec 7, 2023 •

edited by abraunegg