Tool "forgets" login status and rescans whole blog #404
Comments
…ogged out - Tumblr started to replace cookies far before their expiration date which lead to logouts, because the local cookie store wasn't constantly updated. - Now every cookie update request in a response is persisted.
- The normal crawler didn't recognize pinned posts.
|
|
I have the same 'forgot login status' issue for the last 3 days now. Is there an update somewhere? Update check says that i'm up to date. version 2.9.2.368 |
|
I installed v2.9.3 but the app still forgot the authentication status. I closed the app and deleted cookies.json. Same result (except that the authentication window stayed white; I tried again and this time the window showed my tumblr blog). I closed the app and rebooted. Same result. Thank you for your efforts, but it must be something else going on. |
|
Version: 2.9.3.376 Some plain observations of this behaviour, as it's happening to me. I can replicate it with the blogs I have in my queue (any number will suffice, as it's all based on time rather than quantity). Clearing the EBWebView folder + cookies.json, and starting afresh, I'll be logged out again in under 30 mins. Things to note: cookies.json doesn't get updated when I get logged out, but more weirdly it also doesn't get updated when I log in again after being logged out. (it obviously gets created if I delete and log in afresh, but after that no changes are made to it) The logs do also complain about an an uncaught exception in :However, since these are all hidden blogs, I'm unsure how much of that is cause and how much of that is effect. The only clue in the logs that you've been logged out is this combo: Which is reflected in the errors at the top of the UI, and isn't a precursor to being logged out. I'm happy to run a version with any kind of extended debug logging enabled since I'm in a replicable case (or is there any options for that already? Verbose logging doesn't tell me much more than |
|
Thanks for your observations. Are you leaving the settings dialog by using the save button or without? The cookies.json should be updated, because after a new login the cookies should be different. In the error message I see Do you download only hidden blogs or mixed together with normal blogs? Can you confirm that it works without being logged out if you download only newly added hidden blogs as written in the linked discussion #396? |
|
Not sure what you changed between 2.9.3 and 2.10 towards this bug, but last night it completed a full ~250 blogs download without being logged out after a period of time. |
|
You can always compare the tagged versions.
I wish it would be solved, but I just don't believe it yet. |
|
If I recall (I'm not presently at the computer on which I use the app), the user agent in my config is Firefox. Should I change it to Chrome (and if so, what should that field say)? |
|
It might help, maybe not. Currently it's "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/109.0.0.0 Safari/537.36" (no quotes). |
|
I checked and my user agent is exactly what you listed for the current user agent setting. So I guess that means we can rule that out as the culprit. |
|
So.. it's returned. Updating to 2.10.0.384 gave a respite of about 2 weeks, and now it can no longer retain logged in status properly. |
|
I haven't done any programming since the dark ages (late '80s and early '90s) but the more I think about it, the more convinced I am that one of the keys to figuring out what's going on is the fact that even after getting a 501 error, when I click the Authenticate button, I do NOT need to authenticate again. The Tumblr window that pops up goes straight to my blog as if I've already logged in. I wonder whether it would be possible for the app to simply mimic the steps of re-authenticating when it encounters a 501 error. |
|
Something to note, when I get Error 9 ("You need to login to tumblr.com for downloading I ponder if the relation here is that:
Thus this is why it doesn't affect new blogs, if they don't have many posts, they get a proper Things that need to happen when a blog is 501'ed:
|
|
Please use this test version, it doesn't reset the |
|
I installed and used the test version last night. The bad news is that I wasn't able to devote much time to it. The good news is that it never generated the 501 error. I downloaded a bunch of hidden blogs (about 20) and stayed logged in the whole time. I closed the program and restarted. Before doing anything else, I checked and I was still logged in (this hasn't been the case since the problem started). I then downloaded a bunch of non-hidden blogs and everything appeared fine. I'll try to test more over the next few days. |
|
The good news: I let the app download a large batch of blogs (both hidden and not) and it seems to working just fine. No 501 errors. The bad news: The app does appear to be rescanning each blog in full (or at least processing each blog more slowly than I remember. |
|
That's good news. |
|
So what was the problem and what did you change to fix the problem? |
|
Ok, so sadly the logout still happens with the new version, but it now is more clear in the logs. and then (the error is async, but logs after the cookie change): and thus: It should be noted that 'main' still thinks it's logged in, but 'blog' isn't. To test this I threw in 109 blogs, and it errored after getting the queue down to 73 (which is roughly better than before?) The LastId was thankfully unchanged, so there's a bit of success there? |
The problem, that some are being logged out, is known, the exact cause remains unclear. Actually I didn't fix too much yet. I changed that in case of errors the last position is no longer reset.
Is there also a section "Cookies response" in your log? The especially interesting time is when the exception occurs. |
Here's a whole section from the logs directly one after the other: Lightly edited to ensure that privacy is maintained. Though the |
|
I guess you started the blog download with already "broken" cookies. I think we need to look at the time when it changes from successful to failing requests, so the last working and the first failing request. |
|
If you can make a build that stops the queue instantly on a 401 then I can get you that log with ease; right now it's trying to sift through a lot of cooke reports that look identical (and trying to post them in such a way that doesn't give away any kind of privacy). Currently when you hit a 401, pausing the queue manually takes a while (and isn't instant), which then spits a lot of errors into the logs. |
|
It should be possible with a text editor to search from the end for "(401)". |
|
I had to postpone downloading any additional files via the app for a few weeks while I attended to some other matters. Anyway, when I came back to the app it was once again logging me out regularly. I updated to the new test version mentioned in the most recent Discussion post. Same problem. However, I did note two things that may be meaningful: First, the program now shows a 401 error where it previously showed a 501 error. Second, in the dialog with the Authenticate button, even after clicking Authenticate and then closing the popup Tumblr window, a logout button does not appear and the "not logged in" message continues to be displayed. |
…tinue - Occasionally something happens to the cookies and Tumblr returns an auth error on the next request. - In case of an auth error the cookies are fetched from the WebView instance again and the request is repeated. - It's not totally clear yet why this happens.
|
The latest update (2.11.2.421) doesn't seem to behave correctly; when it hits an auth error, it doesn't halt the worker threads, so while it's trying to reauth, the rest of the queue gets marked up as an error. Worst still, this seems to introduce a strange behaviour where the Concurrent Blogs limit is ignored, it then starts up multiple blogs past this limit (I have it set to 4, it'll hit 10 during this bug), and then download absolutely nothing because it's past the limit. Finally, everytime I start the app in the latest version, it thinks I'm not logged into Tumblr. As an aside, can we persist the 'Newly downloaded items' number until the blog gets a new crawl? Part of what makes this most frustrating is that this number is ephemeral, and is lost if you need to relaunch the app for any reason (eg, it's gotten stuck and you have to kill it in task manager). This means that while it's gotten the files, and while I know which blogs were crawled, I can't be sure which ones actually got an update (for context, there's ~540 blogs in the index, and roughly 40 have new content when I crawl once per day) |
|
I cannot reproduce your first three points. But as I said before, I don't get this error often anyway. If someone can support debugging here, help is welcome. But it should be possible to persist this number. |
|
By default the new behavior is disabled now. It can be enabled by an option in the advanced settings. |
Discussed in #396
Originally posted by stuwm November 21, 2022
When extracting files from hidden tumblrs, the tool is "forgetting" that I have already logged into Tumblr. When clicking on "Authenticate", the tool is not asking for credentials, but switches to "logged in" status immediately. This typically happens after 5-10 mins.
Along with this behaviour the tool is recognizing "latest post" days from years back, although newer posts exist. Then the tool is starting a rescan for that tumblr.
The text was updated successfully, but these errors were encountered: