upspinserver spending months in some sort of a read loop
#624
Comments
|
Did it stop doing this after you restarted it? Or is it reproducible? You might want to run the server with the environment variable |
|
The stack trace that I got from sending |
|
I see this behavior upon each restart. The server keeps on moving data around and attempts to do something useful in the meantime are starved. Some would eventually be fulfilled but it is a far cry from being able to do something useful. |
|
Good to see the tree traversal of log recovery here; that's what I
imagined from seeing your earlier Get trace.
But I'm not expert enough in that part of the code. Maybe presotto@ or adg@
can help?
I don't see any actual key material revealed in this dump, but makes me
nervous to dump registers during p256PointAddAsm.
…On Mon, Sep 23, 2019 at 9:19 AM Filip Filmar ***@***.***> wrote:
I see this behavior upon each restart. The server keeps on moving data
around and attempts to do something useful in the meantime are starved.
Some would eventually be fulfilled but it is a far cry from being able to
do something useful.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
<#624?email_source=notifications&email_token=ACADPOT3B5DIUR2PPCR2CCTQLDUBRA5CNFSM4IX2PBQ2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD7LNQPY#issuecomment-534173759>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/ACADPOWIP5QE5B4KKAWRUATQLDUBRANCNFSM4IX2PBQQ>
.
|
|
Oh, I see it now. This is because of the note from the stack trace above: https://github.com/upspin/upspin/blob/master/dir/server/tree/tree.go#L818 and here: https://github.com/upspin/upspin/blob/master/dir/server/tree/tree.go#L854 It does not seem practical to traverse the entire tree just to do some printf debugging, at least not in a running server. |
|
FWIW, restarting the server with |
|
I think the root issue is that the `upspin.io/dir/server/tree.Tree` type
has a String method that does a nontrivial amount of work to stringify the
Tree. This involves taking a lock, and possibly printing more log messages
that invoke the Stringer again, causing infinite recursion. The
straightforward fix is to simplify the Tree's Stringer method to just
return a string representation of the Tree's state without mutating it at
all. That was a case of bad design in the first place.
|
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
My upspinserver has been busy for months in the following loop. Sadly I had to restart the machine, so only have recent server logs to show. But, this is what transpires. Once
upspinserveris started,it prints the messages below, then goes on to a many-weeks-long read spree. Where the tail end of the log excerpt above is repeated ad infinitum (except, different hashes every time).
In the past, I kept the machine up for months with upspinserver running, and it just kept going.
I wouldn't mind it since it's not a cloud machine; except that the read spree seems to starve completely any other clients that attempt to connect. So
upspinutility andupspinfsbecome unusable as they wait indefinitely for something to happen. What to do?The text was updated successfully, but these errors were encountered: