panic: concurrent write to websocket connection

[ckhall]

output:

goroutine 54 [running]:
github.com/gorilla/websocket.(*messageWriter).flushFrame(0x14000ac6900, 0x1, {0x0?, 0x0?, 0x0?})
github.com/gorilla/websocket@v1.5.0/conn.go:617 +0x460
github.com/gorilla/websocket.(*messageWriter).Close(0x14000ac6930?)
github.com/gorilla/websocket@v1.5.0/conn.go:731 +0x48
github.com/gorilla/websocket.(*Conn).beginMessage(0x140005c09a0, 0x14000ac6930, 0x9)
github.com/gorilla/websocket@v1.5.0/conn.go:480 +0x3c
github.com/gorilla/websocket.(*Conn).NextWriter(0x140005c09a0, 0x9)
github.com/gorilla/websocket@v1.5.0/conn.go:520 +0x40
github.com/gorilla/websocket.(*Conn).WriteMessage(0x14000169f70?, 0x14000169f48?, {0x102fee7c0, 0x0, 0x0})
github.com/gorilla/websocket@v1.5.0/conn.go:773 +0x110
github.com/convox/stdsdk.keepalive({0x1022e00f8, 0x102fee7c0}, 0x0?)
github.com/convox/stdsdk@v0.0.0-20201005151143-fb7f05286eea/client.go:291 +0x88
created by github.com/convox/stdsdk.(*Client).Websocket in goroutine 1
github.com/convox/stdsdk@v0.0.0-20201005151143-fb7f05286eea/client.go:223 +0x70c

convox cli 3.13.8
installed via homebrew
running on Apple M1 max
os: ventura 13.6

[nightfury1204]

Hey It's hard to understand. Can share what kind operation lead you to this error?

[ckhall]

I'm running:

convox run web bundle exec rails console

at a random time, sometimes several times a day, sometimes once a week, sometimes 1 minute after opening the connection, sometimes 20 minutes after.

The connection suddenly closes with the error I posted above.

[nightfury1204]

what is your convox cli and rack version?

[ckhall]

staging rack: 20231108154121
prod rack: 20211012210133

cli: 3.14.0

this problem happens regardless of the rack i am connected to.

[nightfury1204]

You are using old version of rack i can see.
@ntner can you check this error?

[ckhall]

curious why the rack version would be related to this?

[nightfury1204]

curious why the rack version would be related to this?

because of the underlying library that's been used might be old.

[nickfishman]

I ran into this issue today on a freshly-installed rack on AWS. I was running convox instances ssh. Here's the stacktrace:

panic: concurrent write to websocket connection
goroutine 16 [running]:
github.com/gorilla/websocket.(*messageWriter).flushFrame(0xc0006a6150, 0x1, {0x0?, 0x0?, 0x0?})
github.com/gorilla/websocket.(*messageWriter).Close(0x0?)
/source/vendor/github.com/gorilla/websocket/conn.go:731 +0x45
github.com/gorilla/websocket.(*Conn).beginMessage(0xc000432420, 0xc000b82120, 0x9)
github.com/gorilla/websocket.(*Conn).NextWriter(0xc000432420, 0x9)
/source/vendor/github.com/gorilla/websocket/conn.go:520 +0x45
github.com/gorilla/websocket.(*Conn).WriteMessage(0xc00016cf80?, 0xc00016cf58?, {0x10284f6e8, 0x0, 0x0})
/source/vendor/github.com/gorilla/websocket/conn.go:773 +0x152
github.com/convox/stdsdk.keepalive({0x101abd8a8, 0xc0000460b8}, 0x0?)
/source/vendor/github.com/convox/stdsdk/client.go:291 +0xa9
created by github.com/convox/stdsdk.(*Client).Websocket
/source/vendor/github.com/convox/stdsdk/client.go:223 +0x86e

Here are the versions:

% convox version
client: 3.14.4
server: 3.14.4

I've been using Convox v3 racks for a while and have occasionally seen similar behavior when using proxy, run, or exec. I remember seeing convox/issues#9 which may be a similar issue, though the underlying cause may be different.

[nightfury1204]

@nickfishman thanks for reporting. We added ticket for this in our internal board. We will look into it.