SQLiteConstraintException: UNIQUE constraint failed
#1447
Comments
|
i get this exact issue... even tried multiple versions including the play store and f-droid versions. |
|
Versions prior to Please wait until |
|
Cannot pinpoint it to just one board. One suggestion if possible is have the user be able to select only the boards they want synced on mobile. |
|
Oh I forgot to say this is all on 1.22.1 |
|
This comment was marked as resolved.
This comment was marked as resolved.
This comment was marked as resolved.
This comment was marked as resolved.
|
Thanks for the information!
@desperateCoder should be able to track down the root cause 😉 |
|
I'll look into it these days, but this weekend is quite packed already. I'll keep you guys updated! |
|
Thank you so much. If I find anything I'll post it. |
|
@Gobytego can you please test the following Build and report back if this one works reliably for you: This is a debug build, which will be installed besides your regular Deck app, but it has a "DEV" label on its icon. Nothing will happen to your actual App. This build has a ton of changes regarding the sync, so it might just work or just die... Depending on the amount of data on your server the sync may take a while, but it should be more stable from what i can tell. |
|
I get this. |
|
Did it at least survive longer? Anyways, the cut-off logs are quite crappy, I'll see if I can make it more verbose. I'll check on this and report back as soon as I have a fix. |
|
It seems to be the same. I'll try again. |
|
Yah same thing. |
|
Ok, so I enhanced the error messages for failed calls like in your case - I should now see where it fails for the client side (you already provided server logs, thanks for that! I still need to make sure this is the issue on our side as well.) so the following APK should be more verbose regarding the failed request: Please uninstall the DEV app if you didn't do so already and install the APK above. It most probably will die at the exact same moment as before, but will hopefully provide a ton of more context to work with. |
|
Now its hanging with no errors. I tried wiping data from all three apps and started again but still just hanging but on a different board. |
|
Wait I got something. |
|
|
|
|
I'm slightly confused:
This is your actual app, not the one I posted above, right? The dev build should be like Edit: In case something won't work (e.g. no error at all, but no sync either) please try to delete app data before uninstalling. Android behaves quite randomly when it comes to uninstalling apps .. |
|
Oh maybe I accedently hit the regular app. Hold I'll do it again. |
|
So tried it again and it just hangs here with no traffic or anything. |
|
Ok so I uninstalled both the nextcloud app. And the deck app plus the deck Dev you sent. Just reinstalled both the nextcloud app and the deck Dev. Its now just sitting there looks like its stuck on importing either board 5,8,9, or 11 but no error. Where as before with the non Dev version it wouldn't get past board 4 with out giving me a bunch of the error. |
|
Okay this time it got stuck on 6. Very inconsistent. But still no error. |
|
how long did you let it do its thing? as mentioned, the time it runs highly depends of a boards data. Can you just let it do whatever it does for a longer period of time? I have a quite big test-account with like hundreds of cards and it took like 20 minutes to finally finish... if you have a log viewer (e.g. logcat) installed, you could check out if the app is still logging stuff, i bet it does. I didn't experience a single case of "it just stopped doing stuff", it either dies or just takes its time. |
|
Okay I'll keep it going. |
|
Here are two screen shots. One from a little after I started and one from now. It seems to just be sitting there but I'll keep it going. |
|
|
|
holy crap.. cleared the app data one more time... let it try to sync again and this time it seemed to work... some missing cards but no errors... going to play and see what else its missing... |
|
ok going to try on my production server... |
|
yeah no still getting errors.... but here is the server's log... /var/snap/nextcloud/39212/nextcloud/extra-apps/deck/lib/Validators/BaseValidator.php - line 85: OCA\Deck\Validators\BaseValidator->validate() /var/snap/nextcloud/39212/nextcloud/extra-apps/deck/lib/Service/AssignmentService.php - line 108: OCA\Deck\Validators\BaseValidator->check() <> OCA\Deck\Service\AssignmentService->__construct() /snap/nextcloud/39212/htdocs/lib/private/AppFramework/Utility/SimpleContainer.php - line 116: ReflectionClass->newInstanceArgs() /snap/nextcloud/39212/htdocs/lib/private/AppFramework/Utility/SimpleContainer.php - line 124: OC\AppFramework\Utility\SimpleContainer->buildClass() /snap/nextcloud/39212/htdocs/lib/private/AppFramework/Utility/SimpleContainer.php - line 142: OC\AppFramework\Utility\SimpleContainer->resolve() /snap/nextcloud/39212/htdocs/lib/private/AppFramework/DependencyInjection/DIContainer.php - line 494: OC\AppFramework\Utility\SimpleContainer->query() /snap/nextcloud/39212/htdocs/lib/private/AppFramework/DependencyInjection/DIContainer.php - line 466: OC\AppFramework\DependencyInjection\DIContainer->queryNoFallback() /snap/nextcloud/39212/htdocs/lib/private/AppFramework/Utility/SimpleContainer.php - line 97: OC\AppFramework\DependencyInjection\DIContainer->query() <> OC\AppFramework\Utility\SimpleContainer->OC\AppFramework\Utility{closure}("*** sensiti ... *") /snap/nextcloud/39212/htdocs/lib/private/AppFramework/Utility/SimpleContainer.php - line 116: array_map() /snap/nextcloud/39212/htdocs/lib/private/AppFramework/Utility/SimpleContainer.php - line 124: OC\AppFramework\Utility\SimpleContainer->buildClass() /snap/nextcloud/39212/htdocs/lib/private/AppFramework/Utility/SimpleContainer.php - line 142: OC\AppFramework\Utility\SimpleContainer->resolve() /snap/nextcloud/39212/htdocs/lib/private/AppFramework/DependencyInjection/DIContainer.php - line 494: OC\AppFramework\Utility\SimpleContainer->query() /snap/nextcloud/39212/htdocs/lib/private/AppFramework/DependencyInjection/DIContainer.php - line 466: OC\AppFramework\DependencyInjection\DIContainer->queryNoFallback() /snap/nextcloud/39212/htdocs/lib/private/AppFramework/App.php - line 162: OC\AppFramework\DependencyInjection\DIContainer->query() /snap/nextcloud/39212/htdocs/lib/private/Route/Router.php - line 315: OC\AppFramework\App::main() /snap/nextcloud/39212/htdocs/lib/base.php - line 1068: OC\Route\Router->match() /snap/nextcloud/39212/htdocs/index.php - line 36: OC::handleRequest() |
|
Tried with a new user with no boards associated. Worked fine. Added one board and got this...
|
|
then same user the board shows up and if i refresh it works but will only show new cards... all old cards are not showing |
|
ok with the new user if i set only one board at a time after they're signed in it will sync then the deck is useable but can only set one board at a time... also kind of sucks cause i would have to take that user ou of the group to get then to only be on one board at a time. once all the boards are synced up then i can add then to the group again then go into one board at a time and remove their name... kind of a pain in the ass but its a crude work around for now... really sucks when you have over 50 users and multiple groups especially when some users are in multiple groups and some of those groups only see one or two boards etc... yeah big pain in the ass. |
|
The last one seems to be UI-related (@stefan-niedermann). Holy cow! Is your setup kind of special or something? I don't have ever seen anything like these errors yet, only the ConstraintViolation thingy...
Regarding this one: Please tell me this didn't happen while syncing... The sync-process itself doesn't assign anything. At least it shouldn't and I'm quite sure it doesn't. Maybe someone else caused the error? Or was it you playing around with a half-broken dataset? Regarding
Totally agree! Thats not the goal and I'm doing my best to fix this. But its really hard to code in this highly fragmented environment while dealing with multithreaded hierarchical sync mechanisms. I'll see what i can do. Please see if the following behaves any different: (ಥ﹏ಥ) APK |
|
ok will try now.... |
|
|
Well, thats quite interesting. We're requesting Can you see anything in the logs with the following Request-ID (mentioned on the http error page)?
|
|
yeah nothing in the logs with that request id |
|
As you can see here the URL is constructed correctly and there is nothing i can or should change to this request. Something is behaving really weird in your setup. |
|
maybe im looking in the wrong place for the logs? i'm just using the built in nextcloud logging in admin settings... |
|
maybe it has something to do with the snap version of nextcloud? |
Here's what I found regarding this: https://help.nextcloud.com/t/getting-500-with-request-id-but-cannot-find-logs/146536 Regarding snap: I can't really tell, never tried it. If I remember correctly we had issues with the snap version in the past, but I'm not really sure it is related to deck or a different project. If you have a non-snap version running somewhere, feel free to test it there |
|
ok will spin up a standard install load my data and try there... but if i recall i do believe this is a non issue on the docker and regular installs... been awhile. oh also just to clear this... i do not have these issues with ios or the other android client on this one... |
I think you're referring to the nextcloud client itself? If yes: sure you don't. They are highly optimated and get their data with a handful of calls to an API that serves for this purpose only. Plus: No offline capabilities. We fetch our data at the initial sync by submitting a sh!tload of calls to the server, which result in even more calls (give me all boards -> ok, now give me all stacks -> now all cards -> now all card-related data and so on). To be able to get all the data in a reasonable time, we need to parallelize things, which leads to an insane amount of complexity, getting all the threads together. The load for the client AND the server are unlikely higher in our case. If I can't fix it this way, I'm actually thinking about contributing an export-feature to the server side, that can output the whole data at once. This would solve many problems at once. But this also could lead to performance-issues on the server side, also because my knowledge about PHP is quite antique. But I'll first try to fix this in the "standard" way, didn't see any constraint violation in the last DEV-version so far. |
Hi there and Happy New Year! :) |
|
Hi @TtuxX, thanks for testing and also happy new year! ❤️ That's great news already! Regarding the stalling: no matter how long you wait, nothing is moving forward? Did the screen turn off while importing? This could be caused by an Android feature, we can address that in a different issue. For now I'd be happy if the initial sync of 39 boards keeps working consistently. Can you try the initial sync a couple of times with the screen on and see if you run into constraint violations? In case it works consistently, this means we're at least perform better than before |
|
also if i recall i was getting the same issues on one of your test servers last april... i can dig up in my emails to see which one.... |
|
@desperateCoder ok found the email chain from april 26th 2023 i can fwd to you if that helps get back to where we left off... |
Thanks for your help and sorry for dragging you once again into the Android nightmare haha I tried import (& app data reset) like 6 times and this is what I got:
I have not looked at the code, but for debugging purpose, could a solution be to set a timeout+retry for each deck import, with a test of 2 max failed imports that would then throw an error? |
@Gobytego yep, also found it. I'll have a look these days
@TtuxX At least something i guess 😅
sounds like the thread pool had a deadlock then, I'll have a look.
Thats similar to what i already did for this issue, but not for the whole sync, just for the failing parts. A full retry would most probably run into similar problems. |
|
I think i found something on the server side regarding this error message:
According to this issue comment (#1560 (comment)), we get the following error (and stack): {
"reqId": "aLbVml1E6DfFz8GeNzwV",
"level": 3,
"time": "2024-01-08T20:37:52+00:00",
"remoteAddr": "82.101.240.101",
"user": "--",
"app": "index",
"method": "GET",
"url": "/index.php/apps/deck/api/v1.1/boards/9/stacks/30/cards/263?",
"message": "userId must be provided and must be not empty",
"userAgent": "Mozilla/5.0 (Android) Nextcloud-android/3.26.0",
"version": "28.0.1.1",
"exception": {
"Exception": "OCA\\Deck\\BadRequestException",
"Message": "userId must be provided and must be not empty",
"Code": 0,
"Trace": [
{
"file": "/var/www/html/custom_apps/deck/lib/Validators/BaseValidator.php",
"line": 85,
"function": "validate",
"class": "OCA\\Deck\\Validators\\BaseValidator",
"type": "->",
"args": [
[
null
]
]
},
{
"file": "/var/www/html/custom_apps/deck/lib/Service/AssignmentService.php",
"line": 108,
"function": "check",
"class": "OCA\\Deck\\Validators\\BaseValidator",
"type": "->",
"args": [
[
null
]
]
},
{
"function": "__construct",
"class": "OCA\\Deck\\Service\\AssignmentService",
"type": "->",
"args": [
[
"OCA\\Deck\\Service\\PermissionService"
],
[
"OCA\\Deck\\Db\\CardMapper"
],
[
"OCA\\Deck\\Db\\AssignmentMapper"
],
[
"OCA\\Deck\\Db\\AclMapper"
],
[
"OCA\\Deck\\Notification\\NotificationHelper"
],
[
"OCA\\Deck\\Activity\\ActivityManager"
],
[
"OCA\\Deck\\Db\\ChangeHelper"
],
[
"OC\\EventDispatcher\\EventDispatcher"
],
[
"OCA\\Deck\\Validators\\AssignmentServiceValidator"
],
null
]
},
{
"file": "/var/www/html/lib/private/AppFramework/Utility/SimpleContainer.php",
"line": 83,
"function": "newInstanceArgs",
"class": "ReflectionClass",
"type": "->",
"args": [
[
[
"OCA\\Deck\\Service\\PermissionService"
],
[
"OCA\\Deck\\Db\\CardMapper"
],
[
"OCA\\Deck\\Db\\AssignmentMapper"
],
[
"OCA\\Deck\\Db\\AclMapper"
],
[
"OCA\\Deck\\Notification\\NotificationHelper"
],
[
"OCA\\Deck\\Activity\\ActivityManager"
],
[
"OCA\\Deck\\Db\\ChangeHelper"
],
[
"OC\\EventDispatcher\\EventDispatcher"
],
[
"OCA\\Deck\\Validators\\AssignmentServiceValidator"
],
null
]
]
},
{
"file": "/var/www/html/lib/private/AppFramework/Utility/SimpleContainer.php",
"line": 128,
"function": "buildClass",
"class": "OC\\AppFramework\\Utility\\SimpleContainer",
"type": "->",
"args": [
[
"ReflectionClass",
"OCA\\Deck\\Service\\AssignmentService"
]
]
},
{
"file": "/var/www/html/lib/private/AppFramework/Utility/SimpleContainer.php",
"line": 146,
"function": "resolve",
"class": "OC\\AppFramework\\Utility\\SimpleContainer",
"type": "->",
"args": [
"OCA\\Deck\\Service\\AssignmentService"
]
},
{
"file": "/var/www/html/lib/private/AppFramework/DependencyInjection/DIContainer.php",
"line": 468,
"function": "query",
"class": "OC\\AppFramework\\Utility\\SimpleContainer",
"type": "->",
"args": [
"OCA\\Deck\\Service\\AssignmentService"
]
},
{
"file": "/var/www/html/lib/private/AppFramework/DependencyInjection/DIContainer.php",
"line": 440,
"function": "queryNoFallback",
"class": "OC\\AppFramework\\DependencyInjection\\DIContainer",
"type": "->",
"args": [
"OCA\\Deck\\Service\\AssignmentService"
]
},
{
"file": "/var/www/html/lib/private/AppFramework/Utility/SimpleContainer.php",
"line": 96,
"function": "query",
"class": "OC\\AppFramework\\DependencyInjection\\DIContainer",
"type": "->",
"args": [
"OCA\\Deck\\Service\\AssignmentService",
true
]
},
{
"function": "OC\\AppFramework\\Utility\\{closure}",
"class": "OC\\AppFramework\\Utility\\SimpleContainer",
"type": "->",
"args": [
"*** sensitive parameters replaced ***"
]
},
{
"file": "/var/www/html/lib/private/AppFramework/Utility/SimpleContainer.php",
"line": 83,
"function": "array_map",
"args": [
[
"Closure"
],
[
[
"ReflectionParameter",
"appName"
],
[
"ReflectionParameter",
"request"
],
[
"ReflectionParameter",
"cardService"
],
"*** sensitive parameters replaced ***",
[
"ReflectionParameter",
"userId"
]
]
]
},
{
"file": "/var/www/html/lib/private/AppFramework/Utility/SimpleContainer.php",
"line": 128,
"function": "buildClass",
"class": "OC\\AppFramework\\Utility\\SimpleContainer",
"type": "->",
"args": [
[
"ReflectionClass",
"OCA\\Deck\\Controller\\CardApiController"
]
]
},
{
"file": "/var/www/html/lib/private/AppFramework/Utility/SimpleContainer.php",
"line": 146,
"function": "resolve",
"class": "OC\\AppFramework\\Utility\\SimpleContainer",
"type": "->",
"args": [
"OCA\\Deck\\Controller\\CardApiController"
]
},
{
"file": "/var/www/html/lib/private/AppFramework/DependencyInjection/DIContainer.php",
"line": 468,
"function": "query",
"class": "OC\\AppFramework\\Utility\\SimpleContainer",
"type": "->",
"args": [
"OCA\\Deck\\Controller\\CardApiController"
]
},
{
"file": "/var/www/html/lib/private/AppFramework/DependencyInjection/DIContainer.php",
"line": 440,
"function": "queryNoFallback",
"class": "OC\\AppFramework\\DependencyInjection\\DIContainer",
"type": "->",
"args": [
"OCA\\Deck\\Controller\\CardApiController"
]
},
{
"file": "/var/www/html/lib/private/AppFramework/App.php",
"line": 163,
"function": "query",
"class": "OC\\AppFramework\\DependencyInjection\\DIContainer",
"type": "->",
"args": [
"OCA\\Deck\\Controller\\CardApiController"
]
},
{
"file": "/var/www/html/lib/private/Route/Router.php",
"line": 315,
"function": "main",
"class": "OC\\AppFramework\\App",
"type": "::",
"args": [
"OCA\\Deck\\Controller\\CardApiController",
"get",
[
"OC\\AppFramework\\DependencyInjection\\DIContainer"
],
[
"1.1",
"9",
"30",
"263",
"deck.card_api.get"
]
]
},
{
"file": "/var/www/html/lib/base.php",
"line": 1069,
"function": "match",
"class": "OC\\Route\\Router",
"type": "->",
"args": [
"/apps/deck/api/v1.1/boards/9/stacks/30/cards/263"
]
},
{
"file": "/var/www/html/index.php",
"line": 39,
"function": "handleRequest",
"class": "OC",
"type": "::",
"args": []
}
],
"File": "/var/www/html/custom_apps/deck/lib/Validators/BaseValidator.php",
"Line": 66,
"message": "userId must be provided and must be not empty",
"exception": {},
"CustomMessage": "userId must be provided and must be not empty"
}
}It seems that the dependency injection at the server side is kind of broken. The request is fine, but Injecting the (for this usecase unrelated)
@juliushaertl can you check on this please? I got the feeling that I don't do anything wrong here (see the called URL in JSON above, especially the |
|
@Gobytego could you also post a list of enabled Nextcloud server apps? Especially looking for TOTP and authentication related stuff. I rembember the "HTML page as response" in combination with authentication issues caused by some super duper groovy security snake oil apps on the corresponding instances 😄 |
|
I don't have any of the auth stuff installed |
|
@desperateCoder I'm not sure how that could happen yet. It would be good to see if a properly crafted manual request shows the same issue (of course with adjusting the url and ids): At least I cannot replicate any issue with valid credentials, with invalid ones or by leaving out I get the same error, but that would be expected from my point of view. |
I've tried to run this as a test since I'm also affected by the issue. I have 2FA though, so if I use my main password I get an "Internal Server Error" which is probably expected. If I'm using a generated app password as a test, I get a seemingly normal JSON reply: It could be that only certain boards/cards/data causes the error? Although I managed to get at least one other card from my boards: Is there a way to log into the Deck app using an app password? I could only see auth methods that interacted with the main Nextcloud apps (normal and dev versions), which makes it hard for me to test whether it's caused by authentication shenanigans (the Nextcloud app doesn't seem to allow me to log in with an App Password). |
Steps to reproduce the behavior:
Choose AccountExpected behavior
Import available boards
Screenshots
Versions
Smartphone (please complete the following information):
Stacktrace
The text was updated successfully, but these errors were encountered: