Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

After updating from 4.6.3 (2.0.9) to 4.6.3 (2.0.10) or 4.6.4 (2.0.10), all completed tasks disappeared. #20772

Open
ivan-yu opened this issue Apr 28, 2024 · 19 comments
Labels
Invalid Issues that fail to comply with the contributing guidelines and/or issue template requirements

Comments

@ivan-yu
Copy link

ivan-yu commented Apr 28, 2024

4 6 3 AllTasks
4 6 4 NoTasks

qBittorrent & operating system versions

qBittorrent: 4.6.4 aarch64
Operating system: Linux version 4.9.119
Qt: 6
libtorrent-rasterbar: 2.0.10

What is the problem?

I found below errors inside log file, after I update from 4.6.3 (2.0.9) to 4.6.4 (2.0.10)

(N) 2024-04-28T16:29:05 - qBittorrent v4.6.4 started
(N) 2024-04-28T16:29:05 - Using config directory: /usr/local/AppCentral/qbittorrent-native/home/.config/qBittorrent
(N) 2024-04-28T16:29:05 - qBittorrent termination initiated
(N) 2024-04-28T16:29:05 - qBittorrent is now ready to exit
(N) 2024-04-28T16:29:05 - qBittorrent v4.6.4 started
(N) 2024-04-28T16:29:05 - Using config directory: /usr/local/AppCentral/qbittorrent-native/home/.config/qBittorrent
(N) 2024-04-28T16:29:05 - Trying to listen on the following list of IP addresses: "0.0.0.0:49162,[::]:49162"
(I) 2024-04-28T16:29:05 - Peer ID: "-qB4640-"
(I) 2024-04-28T16:29:05 - HTTP User-Agent: "qBittorrent/4.6.4"
(I) 2024-04-28T16:29:05 - Distributed Hash Table (DHT) support: ON
(I) 2024-04-28T16:29:05 - Local Peer Discovery support: ON
(I) 2024-04-28T16:29:05 - Peer Exchange (PeX) support: ON
(I) 2024-04-28T16:29:05 - Anonymous mode: ON
(I) 2024-04-28T16:29:05 - Encryption support: FORCED
(W) 2024-04-28T16:29:05 - Failed to load Categories. File size exceeds limit. File: "/usr/local/AppCentral/qbittorrent-native/home/.
config/qBittorrent/categories.json". File size: 548320867016. Size limit: 1048576
(I) 2024-04-28T16:29:05 - Successfully listening on IP. IP: "127.0.0.1". Port: "TCP/49162"
(I) 2024-04-28T16:29:05 - Successfully listening on IP. IP: "127.0.0.1". Port: "UTP/49162"
(I) 2024-04-28T16:29:05 - Successfully listening on IP. IP: "192.168.1.4". Port: "TCP/49162"
(I) 2024-04-28T16:29:05 - Successfully listening on IP. IP: "192.168.1.4". Port: "UTP/49162"
(C) 2024-04-28T16:29:05 - Failed to resume torrent. Torrent: "14ee1e4c0bd612f786393376e94ff872b6e87cfa". Reason: "File size exceeds
limit. File: "/usr/local/AppCentral/qbittorrent-native/home/.local/share/qBittorrent/BT_backup/14ee1e4c0bd612f786393376e94ff872b6e87
cfa.fastresume". File size: 548318599328. Size limit: 104857600"
(C) 2024-04-28T16:29:05 - Failed to resume torrent. Torrent: "2df9db306f21320a986eb695cb9f63044b618140". Reason: "File size exceeds
limit. File: "/usr/local/AppCentral/qbittorrent-native/home/.local/share/qBittorrent/BT_backup/2df9db306f21320a986eb695cb9f63044b618
140.fastresume". File size: 548318599328. Size limit: 104857600"
(C) 2024-04-28T16:29:05 - Failed to resume torrent. Torrent: "89014345d130b47c1c0a870083fdd73206b09288". Reason: "File size exceeds
limit. File: "/usr/local/AppCentral/qbittorrent-native/home/.local/share/qBittorrent/BT_backup/89014345d130b47c1c0a870083fdd73206b09
288.fastresume". File size: 548318599328. Size limit: 104857600"
(C) 2024-04-28T16:29:05 - Failed to resume torrent. Torrent: "5c1429081a586f813b32ce5dcba9d646500b7d61". Reason: "File size exceeds
limit. File: "/usr/local/AppCentral/qbittorrent-native/home/.local/share/qBittorrent/BT_backup/5c1429081a586f813b32ce5dcba9d646500b7
d61.fastresume". File size: 548318599328. Size limit: 104857600"
(C) 2024-04-28T16:29:05 - Failed to resume torrent. Torrent: "0725c0afca67e94f7bca31636a95caa691391ada". Reason: "File size exceeds
limit. File: "/usr/local/AppCentral/qbittorrent-native/home/.local/share/qBittorrent/BT_backup/0725c0afca67e94f7bca31636a95caa691391
ada.fastresume". File size: 548318599328. Size limit: 104857600"
(C) 2024-04-28T16:29:05 - Failed to resume torrent. Torrent: "195ff0aab50e90bde03cde6c608206e7c21e273e". Reason: "File size exceeds
limit. File: "/usr/local/AppCentral/qbittorrent-native/home/.local/share/qBittorrent/BT_backup/195ff0aab50e90bde03cde6c608206e7c21e2
73e.fastresume". File size: 548318599328. Size limit: 104857600"

My categories.json is just 4 bytes, not what the log file says.

-rw-r--r--    1 root     root             4 Mar 23 20:52 categories.json
srwx------    1 root     root             0 Apr 28 16:32 ipc-socket=
-rw-rw-rw-    1 root     root             0 Apr 28 16:32 lockfile
-rwxrw-rw-    1 root     root           192 Apr 28 15:59 qBittorrent-data.conf*
-rwxrw-rw-    1 root     root          2823 Apr 28 16:28 qBittorrent.conf*
drwxr-xr-x    2 root     root          4096 Mar 23 20:52 rss/
-rw-r--r--    1 root     root             4 Mar 23 20:52 watched_folders.json

Steps to reproduce

No response

Additional context

No response

Log(s) & preferences file(s)

No response

@glassez
Copy link
Member

glassez commented Apr 29, 2024

I found below errors inside log file, after I update from 4.6.3 (2.0.9) to 4.6.4 (2.0.10)

It would be more interesting to see the differences of Qt between these two builds. I believe the current one causes the problem.

@Chocobo1
What do you think?

@ivan-yu
Copy link
Author

ivan-yu commented Apr 29, 2024

I just tried upgrading from 4.6.3 (2.0.9) to same binary 4.6.3 (with lib 2.0.10), and the same errors show up in the log file.

So I am pretty sure, it is Lib 2.0.10 and not the main binary.

For some reason, Lib 2.0.10 fails to read categories.json and all the fastresume files, same for 4.6.3 and 4.6.4

@glassez
Copy link
Member

glassez commented Apr 29, 2024

So I am pretty sure, it is Lib 2.0.10 and not the main binary.

For some reason, Lib 2.0.10 fails to read categories.json and all the fastresume files, same for 4.6.3 and 4.6.4

This error is not produced by libtorrent.

QFile file {path.data()};
if (!file.open(QIODevice::ReadOnly | additionalMode))
{
const QString message = QCoreApplication::translate("Utils::IO", "File open error. File: \"%1\". Error: \"%2\"")
.arg(file.fileName(), file.errorString());
return nonstd::make_unexpected(ReadError {ReadError::NotExist, message});
}
const qint64 fileSize = file.size();
if ((maxSize >= 0) && (fileSize > maxSize))
{
const QString message = QCoreApplication::translate("Utils::IO", "File size exceeds limit. File: \"%1\". File size: %2. Size limit: %3")
.arg(file.fileName(), QString::number(fileSize), QString::number(maxSize));
return nonstd::make_unexpected(ReadError {ReadError::ExceedSize, message});
}

And categories.json has nothing to do with libtorrent at all.

But if you still insist, I won't stop you from dealing with this problem as you know.

@ivan-yu
Copy link
Author

ivan-yu commented Apr 29, 2024

I do see your point, but 4.6.3 (2.0.9) produces no error, however same 4.6.3 (with 2.0.10) produces all the errors.

That's how I deduced my result.

Do you have any theories on why only changing the lib leads to these errors?

What you have given is the function that prints this error, but who is calling this function() ?

It is like you see a knife with blood, but who is using the knife?

@ivan-yu ivan-yu closed this as completed Apr 29, 2024
@ivan-yu ivan-yu reopened this Apr 29, 2024
@ivan-yu ivan-yu closed this as completed Apr 30, 2024
@ivan-yu ivan-yu reopened this Apr 30, 2024
@ivan-yu
Copy link
Author

ivan-yu commented Apr 30, 2024

@ivan-yu ivan-yu changed the title After updating 4.6.3 (2.0.9) to 4.6.4 (2.0.10), all completed tasks disappeared. After updating from 4.6.3 (2.0.9) to 4.6.3 (2.0.10) or 4.6.4 (2.0.10), all completed tasks disappeared. Apr 30, 2024
@glassez
Copy link
Member

glassez commented Apr 30, 2024

Cross checking all the components for QT6, the only change in the nox is Libtorrent

https://github.com/userdocs/qbittorrent-nox-static/releases/tag/release-4.6.3_v2.0.9 vs https://github.com/userdocs/qbittorrent-nox-static/releases/tag/release-4.6.3_v2.0.10

@ivan-yu
You do use 3rd-party builds. You should report their problems to their author (unless it is confirmed that this is an upstream problem).
@userdocs, ping!

@userdocs
Copy link

userdocs commented Apr 30, 2024

I'm not setting a filesize limit.

File size: 548318599328. Size limit: 104857600"

Unless that issue is specific to Qt 6.7.0 since I will assume this is the error path

I found below errors inside log file, after I update from 4.6.3 (2.0.9) to 4.6.4 (2.0.10)

Which will be Qt 6.6.2 to 6.7.0

I'd consider that an upstream issue as I just build the app, my build is not causing the issue it would be exposing the it,

@glassez
Copy link
Member

glassez commented Apr 30, 2024

I'm not setting a filesize limit.

I didn't say that you set any limit. It's just that one of your builds behaves incorrectly.

I'd consider that an upstream issue

Nothing that could cause this problem has changed in the qBittorrent code between version 4.6.3 and 4.6.4 (even up to the current master).
As you might have noticed in the code snippet above, qBittorrent just gets the file size using Qt and compare it to limit value. So this is either a bug in used Qt version (as I originally assumed), or a build bug (for example, due to incorrect/inconsistent compilation flags, incorrect/inconsistent dependencies, uncleaned build directory, etc.). Even if this is a Qt 6.7 problem, it clearly depends on certain factors (e.g. compiler, dependencies, hardware architecture), since at least in my builds on Windows and Linux I have not encountered this bug.

@ivan-yu
Copy link
Author

ivan-yu commented Apr 30, 2024

Please see below two log files.
First one (no error) is using 4.6.3_v2.0.9/aarch64-qbittorrent-nox
Second one (lots of file open errors) is using 4.6.3_v2.0.10/aarch64-qbittorrent-nox
Everything else are the same, i.e. my OS, qBittorrent config, fast resume files, save location, etc.

(N) 2024-04-30T10:50:31 - qBittorrent v4.6.3 started
(N) 2024-04-30T10:50:31 - Using config directory: /usr/local/AppCentral/qbittorrent-native/home/.config/qBittorrent
(N) 2024-04-30T10:50:31 - Trying to listen on the following list of IP addresses: "0.0.0.0:49162,[::]:49162"
(I) 2024-04-30T10:50:31 - Peer ID: "-qB4630-"
(I) 2024-04-30T10:50:31 - HTTP User-Agent: "qBittorrent/4.6.3"
(I) 2024-04-30T10:50:31 - Distributed Hash Table (DHT) support: ON
(I) 2024-04-30T10:50:31 - Local Peer Discovery support: ON
(I) 2024-04-30T10:50:31 - Peer Exchange (PeX) support: ON
(I) 2024-04-30T10:50:31 - Anonymous mode: ON
(I) 2024-04-30T10:50:31 - Encryption support: FORCED
(I) 2024-04-30T10:50:31 - Successfully listening on IP. IP: "127.0.0.1". Port: "TCP/49162"
(I) 2024-04-30T10:50:31 - Successfully listening on IP. IP: "127.0.0.1". Port: "UTP/49162"
(I) 2024-04-30T10:50:31 - Successfully listening on IP. IP: "192.168.1.4". Port: "TCP/49162"
(I) 2024-04-30T10:50:31 - Successfully listening on IP. IP: "192.168.1.4". Port: "UTP/49162"
(N) 2024-04-30T10:50:31 - Restored torrent. Torrent: "TVBOXNOW 聲夢1+2"
(N) 2024-04-30T10:50:31 - Restored torrent. Torrent: "TVBXONOW 怪宿宿"
(N) 2024-04-30T10:50:31 - Restored torrent. Torrent: "TVBOXNOW 香港婚後事"
(N) 2024-04-30T10:50:31 - Restored torrent. Torrent: "TVBOXNOW 逆天奇案2 H265"
(N) 2024-04-30T10:58:45 - qBittorrent v4.6.3 started
(N) 2024-04-30T10:58:45 - Using config directory: /usr/local/AppCentral/qbittorrent-native/home/.config/qBittorrent
(N) 2024-04-30T10:58:45 - Trying to listen on the following list of IP addresses: "0.0.0.0:49162,[::]:49162"
(I) 2024-04-30T10:58:45 - Peer ID: "-qB4630-"
(I) 2024-04-30T10:58:45 - HTTP User-Agent: "qBittorrent/4.6.3"
(I) 2024-04-30T10:58:45 - Distributed Hash Table (DHT) support: ON
(I) 2024-04-30T10:58:45 - Local Peer Discovery support: ON
(I) 2024-04-30T10:58:45 - Peer Exchange (PeX) support: ON
(I) 2024-04-30T10:58:45 - Anonymous mode: ON
(I) 2024-04-30T10:58:45 - Encryption support: FORCED
(W) 2024-04-30T10:58:45 - Failed to load Categories. File size exceeds limit. File: "/usr/local/AppCentral/qbittorrent-native/home/.
config/qBittorrent/categories.json". File size: 547866034680. Size limit: 1048576
(I) 2024-04-30T10:58:45 - Successfully listening on IP. IP: "127.0.0.1". Port: "TCP/49162"
(I) 2024-04-30T10:58:45 - Successfully listening on IP. IP: "127.0.0.1". Port: "UTP/49162"
(I) 2024-04-30T10:58:45 - Successfully listening on IP. IP: "192.168.1.4". Port: "TCP/49162"
(I) 2024-04-30T10:58:45 - Successfully listening on IP. IP: "192.168.1.4". Port: "UTP/49162"
(C) 2024-04-30T10:58:45 - Failed to resume torrent. Torrent: "4c9bba37fa279a4b4e740fe04701034723668af0". Reason: "File size exceeds
limit. File: "/usr/local/AppCentral/qbittorrent-native/home/.local/share/qBittorrent/BT_backup/4c9bba37fa279a4b4e740fe04701034723668
af0.fastresume". File size: 547863746816. Size limit: 104857600"
(C) 2024-04-30T10:58:45 - Failed to resume torrent. Torrent: "1ba472f6c47d6b5b65a2e39cdb959d48cecd6e3d". Reason: "File size exceeds
limit. File: "/usr/local/AppCentral/qbittorrent-native/home/.local/share/qBittorrent/BT_backup/1ba472f6c47d6b5b65a2e39cdb959d48cecd6
e3d.fastresume". File size: 547863746816. Size limit: 104857600"
(C) 2024-04-30T10:58:45 - Failed to resume torrent. Torrent: "fa129e88907d10914273178cabba8afc014c316d". Reason: "File size exceeds
limit. File: "/usr/local/AppCentral/qbittorrent-native/home/.local/share/qBittorrent/BT_backup/fa129e88907d10914273178cabba8afc014c3
16d.fastresume". File size: 547863746816. Size limit: 104857600"
(C) 2024-04-30T10:58:45 - Failed to resume torrent. Torrent: "fcfd4e7a4b1b820bb8f3e8ab87cee7156ff5ccde". Reason: "File size exceeds
limit. File: "/usr/local/AppCentral/qbittorrent-native/home/.local/share/qBittorrent/BT_backup/fcfd4e7a4b1b820bb8f3e8ab87cee7156ff5c
cde.fastresume". File size: 547863746816. Size limit: 104857600"

@ivan-yu
Copy link
Author

ivan-yu commented Apr 30, 2024

You can see all the file open function calls failed. They all say File size: 547866034680

547866034680 is obviously not the file size at all. For example, categories.json is just 4 bytes in size.

-rw-r--r-- 1 root root 4 Mar 23 20:52 categories.json

@ivan-yu
Copy link
Author

ivan-yu commented Apr 30, 2024

May I suggest you create a path the same as mine, to replicate this file open error?
Maybe with focus on how File size: 547866034680 (such a large number) can be returned?

@ivan-yu
Copy link
Author

ivan-yu commented Apr 30, 2024

Closing this issue since it seems to be more a build issue, following up with the builder.

@ivan-yu ivan-yu closed this as completed Apr 30, 2024
@glassez glassez added the Invalid Issues that fail to comply with the contributing guidelines and/or issue template requirements label May 1, 2024
@ivan-yu
Copy link
Author

ivan-yu commented May 6, 2024

It seems the root cause is statx() failing, when trying to get file size.

statx() seems to only exist from Linux Kernel 4.11

If anyone is running earlier version (e.g. Linux Kernel 4.9), this call will fail

@ivan-yu ivan-yu reopened this May 6, 2024
@ivan-yu
Copy link
Author

ivan-yu commented May 7, 2024

Due to my limited Linux knowledge, I cannot tell whether this issue is within the source, or due to how the aarch64 binaries are built by the builder.

strace of non daemon qbittorrent

openat(AT_FDCWD, "/usr/local/AppCentral/qbittorrent-native/home/.config/qBittorrent/watched_folders.json", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = 74
fcntl(74, F_SETFD, FD_CLOEXEC)          = 0
statx(74, "", AT_STATX_SYNC_AS_STAT|AT_NO_AUTOMOUNT|AT_EMPTY_PATH, STATX_ALL, {stx_mask=STATX_GID|STATX_INO|STATX_SIZE|0xdb06c000, stx_attributes=STATX_ATTR_IMMUTABLE|STATX_ATTR_ENCRYPTED|STATX_ATTR_AUTOMOUNT|STATX_ATTR_MOUNT_ROOT|STATX_ATTR_VERITY|0x7f7b068080, stx_size=549135500224, ...}) = 0
statx(74, "", AT_STATX_SYNC_AS_STAT|AT_NO_AUTOMOUNT|AT_EMPTY_PATH, STATX_ALL, {stx_mask=STATX_TYPE|STATX_ATIME, stx_attributes=STATX_ATTR_IMMUTABLE|STATX_ATTR_APPEND|STATX_ATTR_NODUMP|STATX_ATTR_ENCRYPTED|STATX_ATTR_AUTOMOUNT|STATX_ATTR_MOUNT_ROOT|STATX_ATTR_VERITY|0x7f7b068100, stx_mode=0177, ...}) = 0
close(74)                               = 0

@Chocobo1
Copy link
Member

Chocobo1 commented May 7, 2024

statx() seems to only exist from Linux Kernel 4.11
If anyone is running earlier version (e.g. Linux Kernel 4.9), this call will fail

Your strace shows that statx() calls return 0 which means success. Your platform may have backported statx to 4.9 kernel and might have bugs with it.
At this point it should be possible to write a simple C program to demonstrate the problem and provide it to your platform provider/manufacturer.

@ivan-yu
Copy link
Author

ivan-yu commented May 7, 2024

RIght, the statx() call returned non sense file size (stx_size=549135500224). Then QT treats this size as failure when compared to a user defined file limit. Finally, file open fails with QT, for all files including fastresume files. As a result, all completed torrents cannot be recovered.

However, I do not understand why QT4.6.3-LT2.0.9 does not have any issue. Does LT 2.0.9 also use statx()? Or does it use older stat()? My NAS (Linux environment) has not changed.

 const qint64 fileSize = file.size(); 
 if ((maxSize >= 0) && (fileSize > maxSize)) 
 { 
     const QString message = QCoreApplication::translate("Utils::IO", "File size exceeds limit. File: \"%1\". File size: %2. Size limit: %3") 
         .arg(file.fileName(), QString::number(fileSize), QString::number(maxSize)); 
     return nonstd::make_unexpected(ReadError {ReadError::ExceedSize, message}); 
 } 

@ivan-yu
Copy link
Author

ivan-yu commented May 9, 2024

I think the root cause has been identified.
The 3rd party aarch64 qBittorrent build is using musl, and latest musl 1.2.5 just started using statx().
https://musl.libc.org/releases.html

This release adds extension functions statx and preadv2/pwritev2, with fallback implementations for older kernels

The fallback did not work

@userdocs
Copy link

userdocs commented May 9, 2024

@glassez if you are interested in what is the cause of this is should it crop up again.

userdocs/qbittorrent-nox-static#141 (comment)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Invalid Issues that fail to comply with the contributing guidelines and/or issue template requirements
Projects
None yet
Development

No branches or pull requests

4 participants