Cannot move file error: invalid cross-device link

[camr0]

Contributing guidelines

• I've read the contributing guidelines and wholeheartedly agree

I've found a bug and checked that ...

• ... the documentation does not mention anything about my problem
• ... there are no open or closed issues that are related to my problem

Description

I've come across this error when using the tempFirst option. The issue seems to be with the os.Rename function being used between different filesystems/disks. I'm working on a fix and am planning on submitting a PR soon 👍

Expected behaviour

Files are copied successfully and operation is completed without error.

Actual behaviour

File downloads to the tmp directory just fine. Error occurs when the program attempts to move the file from the tmp directory to the destination.

Steps to reproduce

1. Run ftpgrab w/ tempFirst enabled
2. Destination on a different drive than tmp directory

Version

• FTPGrab version (5.0.0):
• Platform (windows/linux/freebsd/macos): Linux (TrueNAS Scale & Arch Linux)
• System info (type uname -a): Linux Ali-Asahi 6.1.0-asahi-2-1-edge-ARCH error with awk #2 SMP PREEMPT_DYNAMIC Wed, 14 Dec 2022 14:33:17 +0000 aarch64 GNU/Linux
• Remote FTP (Pure-FTPd) or SFTP (OpenSSH) server: FTP

Configuration

# ./ftpgrab.yml

db:
  path: ftpgrab.db

server:
  ftp:
    host: {hostname}
    port: {#}
    username: {username}
    password: {password}
    sources:
      - /path/to/completed
    timeout: 5s

download:
  output: /home/ali/Documents/ftpgrab/downloads
  retry: 3
  hideSkipped: false
  tempFirst: true
  createBaseDir: false

Logs

Wed, 22 Feb 2023 01:37:54 EST WRN No notifier available
Wed, 22 Feb 2023 01:37:54 EST DBG 4 entries found in database
Wed, 22 Feb 2023 01:37:54 EST DBG Listing files source=path/to/completed
Wed, 22 Feb 2023 01:37:55 EST INF 1 file(s) found
Wed, 22 Feb 2023 01:37:55 EST DBG Move temp file destfile=/home/ali/Documents/ftpgrab/downloads/testfile.txt tempfile=/tmp/.ftpgrab.2063516771/testfile.txt544694374
Wed, 22 Feb 2023 01:37:55 EST ERR Cannot move file error="rename /tmp/.ftpgrab.2063516771/testfile.txt544694374 /home/ali/Documents/ftpgrab/downloads/testfile.txt: invalid cross-device link" dest=/home/ali/Documents/ftpgrab/downloads size=0B src=/path/to/completed/testfile.txt
Wed, 22 Feb 2023 01:37:55 EST INF Finished duration=1.359s

Additional info

No response

[screamjojo]

Same issue. @camr0 did you find a solution ?

[camr0]

@screamjojo Yes, turns out the issue was that the destination was on a different drive than the default tmp directory. ftpgrab uses the "os.rename" function internally to move the files from the tmp directory to the destination, which doesn't support renaming between filesystems. Initially I tried replacing the os.rename function with something else but ran into issues.

Ultimately, I decided to add an option for a custom temporary directory, that way I could make a tmp folder on the same drive. The flag is called "tempDir" and is working on my personal fork here. The code is a little rough so I'm not sure if it's ready for a PR yet lol. Give it a try and let me know if you have any issues!

[screamjojo]

Thank you for reply, how can I add it with docker ?

[crazy-max]

@camr0 Thanks for your investigation

is working on my personal fork here.

I'm not sure what are the actual changes looking at your fork. Can you point me to the commit?

[nothing2obvi]

I run into the same issue on v7.9.0 if I enable tempFirst. If I disable it, there is no problem. However, I would like to enable the tempFirst option.