unable to run codespace due to Failed to start VS Code Remote server

[moshederri]

Hey,

we are constantly getting the following error when launching codespace:

An unexpected error occurred that requires a reload of this page.
The workbench failed to connect to the server (Error: )

And after reload, it still doesn't work and there is not indication of a downtime in the status page

Any ideas what could be the problem?

[david-engelmann]

Do you have a lot of browser extensions installed?

[moshederri]

No, i dont.

Some additional context i gathered:
Most of the time the error returns empty (Error: ) and sometimes (Error: Failed to start VS Code Remote server.).

Looking at the console output while the codespace runs, i get:

vscs:web-client:efeb:codespaces-component:connection-hub:liveshare:connection:9895F2E1A03F7DE818C222ED008555FA1083:liveshare:WorkspaceClient Creating proxy for sourceEvent service
async.ts:30 Uncaught (in promise) Canceled: Canceled
...

however, few lines after, it seems like codespace is available:
vscs:web-client:efeb:codespace-state-watcher codespace state -> Available

Then, im getting the following Errors:

ERR Error while registering log channel vscode-log:/20221106T235607/remoteTunnel.log Timed out while waiting for file to be created
log.ts:324   
ERR Error while registering log channel file:///20221106T235607/ptyhost.log Timed out while waiting for file to be created

vscs:web-client:efeb:codespaces-component:connection-hub:liveshare:connection:........:liveshare:WorkspaceClient > IVSCodeServerHostService.startRemoteServerResult() error: 
vscs:web-client:efeb:codespaces-component:connection-hub:vscode:connection:.......:vscode Failed to complete start-remote-server with Live Share RPC Failed to start VS Code Remote server. code:-32098
Error: Failed to start VS Code Remote server.

the extensions we are using in the devcontainer are:
"extensions": [
"ms-python.python",
"ms-python.vscode-pylance",
"MarkLarah.pre-commit-vscode",
"mhutchie.git-graph",
"PKief.material-icon-theme",
"sgoley.lookml-syntax-highlighter",
"ms-vscode.test-adapter-converter"
]

Thanks for all the helpers!

[moshederri]

Some more context:

• we using an image which is based on mcr.microsoft.com/vscode/devcontainers/python:0-3.9
• we install the following package - debugpy pre-commit tox poetry dbt-core dbt-spark dbt-spark[PyHive] dbt-spark[ODBC]
• the codespace started failing since yesterday, but worked well before. No changes were made to .devcontainer or our image
• When running locally, we are getting

[Client:extension:1d29:liveshare:connection:....:liveshare:WorkspaceClient V] < streamManager.getStream()
[Client:extension:1d29:liveshare:connection:....:liveshare:WorkspaceClient V] > streamManager.getStream() succeeded t:89.0
[Client:extension:1d29:liveshare:connection:....:liveshare:WorkspaceClient I] openStreamingChannel -> streamId:ServerSharing_1
[Client:extension:1d29:resolver:tunnel:16635:127.0.0.1:liveshare-tunnel[16635]:local-port-forwarder[16635] I] Forwarding connection on channel #2.
[Client:extension:1d29:liveshare:connection:....:liveshare:WorkspaceClient V] > IVSCodeServerHostService.devContainerConfigChanged: 
[Client:extension:1d29:liveshare:connection:....:liveshare:WorkspaceClient V] > IVSCodeServerHostService.devContainerConfigChanged: 
[Client:extension:1d29:liveshare:connection:....:liveshare:WorkspaceClient V] > IVSCodeServerHostService.devContainerConfigChanged: 
[Client:extension:1d29:liveshare:connection:....:liveshare:WorkspaceClient V] > IVSCodeServerHostService.devContainerConfigChanged: 
[Client:extension:1d29:resolver:resolve:1:connection:....-34b44a1fa326_stable_129500ee4c8ab7263461ffe327268ba56b9f210d:vscode E] Failed to complete start-remote-server with gRPC, attempting to use Live Share RPC> object #0: "Exception was thrown by handler. Exception: Failed to start VS Code Remote server.
Error: Exception was thrown by handler. Exception: Failed to start VS Code Remote server.
    at Object.onEnd (/Users/..../.vscode/extensions/grpc/dist/src/clients/GrpcClientBase.js:29:1)
    at /Users/..../.vscode/extensions/node_modules/@improbable-eng/grpc-web/dist/grpc-web-client.js:1:24231
    at /Users/..../.vscode/extensions/node_modules/@improbable-eng/grpc-web/dist/grpc-web-client.js:1:11490
    at Array.forEach (<anonymous>)
    at e.rawOnError (/Users/..../.vscode/extensions/node_modules/@improbable-eng/grpc-web/dist/grpc-web-client.js:1:11322)
    at e.onTransportHeaders (/Users/..../.vscode/extensions/node_modules/@improbable-eng/grpc-web/dist/grpc-web-client.js:1:9020)
    at NodeHttp.responseCallback (/Users/..../.vscode/extensions/node_modules/@improbable-eng/grpc-web-node-http-transport/lib/index.js:31:1)
    at Object.onceWrapper (node:events:646:26)
    at ClientRequest.emit (node:events:526:28)
    at HTTPParser.parserOnIncomingClient (node:_http_client:618:27)
    at HTTPParser.parserOnHeadersComplete (node:_http_common:128:17)
    at Socket.socketOnData (node:_http_client:482:22)
    at Socket.emit (node:events:526:28)
    at addChunk (node:internal/streams/readable:315:12)
    at readableAddChunk (node:internal/streams/readable:289:9)
    at Socket.Readable.push (node:internal/streams/readable:228:10)
    at TCP.onStreamRead (node:internal/stream_base_commons:190:23)"
[Client:extension:1d29:liveshare:connection:....:liveshare:WorkspaceClient V] < IVSCodeServerHostService.startRemoteServerResult()
[Client:extension:1d29:resolver:tunnel:16635:127.0.0.1:liveshare-tunnel[16635]:local-port-forwarder[16635] I] Port-forward channel #2 closed.
[Client:extension:1d29:liveshare:connection:....:liveshare:WorkspaceClient V] > IVSCodeServerHostService.devContainerConfigChanged: 
[Client:extension:1d29:liveshare:connection:....:liveshare:WorkspaceClient V] > IVSCodeServerHostService.devContainerConfigChanged: 
[Client:extension:1d29:liveshare:connection:....:liveshare:WorkspaceClient V] > IVSCodeServerHostService.devContainerConfigChanged: 
[Client:extension:1d29:liveshare:connection:....:liveshare:WorkspaceClient V] > IVSCodeServerHostService.devContainerConfigChanged: 
[Client:extension:1d29:liveshare:connection:....:liveshare:WorkspaceClient E] > IVSCodeServerHostService.startRemoteServerResult() error: 
[Client:extension:1d29:resolver:resolve:1:connection:....-34b44a1fa326_stable_129500ee4c8ab7263461ffe327268ba56b9f210d:vscode E] Failed to complete start-remote-server with Live Share RPC> object #0: "Failed to start VS Code Remote server. code:-32098
Error: Failed to start VS Code Remote server.
    at handleResponse (/Users/..../.vscode/extensions/node_modules/vscode-jsonrpc/lib/main.js:436:1)
    at processMessageQueue (/Users/..../.vscode/extensions/node_modules/vscode-jsonrpc/lib/main.js:263:1)
    at Immediate.<anonymous> (/Users/..../.vscode/extensions/node_modules/vscode-jsonrpc/lib/main.js:247:1)
    at processImmediate (node:internal/timers:466:21)"
[Client:extension:1d29:resolver:resolve:1:connection:....-34b44a1fa326_stable_129500ee4c8ab7263461ffe327268ba56b9f210d:vscode I] Failed to execute start-remote-server: undefined
[perf V] name:start vscode server duration:22308.085333824158
[Client:extension:1d29:resolver:resolve:1:connection:....-34b44a1fa326_stable_129500ee4c8ab7263461ffe327268ba56b9f210d:vscode E] connection failed: > object #0: Failed to start VS Code Remote server. code:-32098
  at handleResponse (/Users/..../.vscode/extensions/node_modules/vscode-jsonrpc/lib/main.js:436:1)
  at processMessageQueue (/Users/..../.vscode/extensions/node_modules/vscode-jsonrpc/lib/main.js:263:1)
  at Immediate.<anonymous> (/Users/..../.vscode/extensions/node_modules/vscode-jsonrpc/lib/main.js:247:1)
  at processImmediate (node:internal/timers:466:21)
[Client:extension:1d29:resolver:resolve:1:connection:....-34b44a1fa326_stable_129500ee4c8ab7263461ffe327268ba56b9f210d:vscode I] connection status changed: connecting -> disconnected, error:> object #0: Failed to start VS Code Remote server. code:-32098
  at handleResponse (/Users/..../.vscode/extensions/node_modules/vscode-jsonrpc/lib/main.js:436:1)
  at processMessageQueue (/Users/..../.vscode/extensions/node_modules/vscode-jsonrpc/lib/main.js:263:1)
  at Immediate.<anonymous> (/Users/..../.vscode/extensions/node_modules/vscode-jsonrpc/lib/main.js:247:1)
  at processImmediate (node:internal/timers:466:21)
[Client:extension:1d29:resolver:resolve:1:connection:....-34b44a1fa326_stable_129500ee4c8ab7263461ffe327268ba56b9f210d:vscode I] firing change event: > object #0: {
  "status": "disconnected",
  "previousStatus": "connecting"
}
[Client:extension:1d29:vscode:onConnectionChanged:....-34b44a1fa326_stable_129500ee4c8ab7263461ffe327268ba56b9f210d I] connecting -> disconnected

Any help would be appreciated :)

[jkeech]

A couple questions/things to try:

1. Does it work on a new codespace or if you rebuild your current codespace? You could try rebuilding through the GH CLI gh codespace rebuild.
2. Can you share more of your devcontainer.json? In particular, what remoteUser are you using?

[jkeech]

I think I understand what is going on here. The Python image has remoteUser set to vscode by default, and the devcontainer CLI is now honoring config set in the underlying image if it is present. My guess is that something in your custom Dockerfile is trying to use the root user, such as hardcoding $HOME to /root or something similar, but your devcontainer config likely doesn't have remoteUser set to root. So in the end, the VS Code server is trying to start using the vscode user in the container, but it is pointing to the /root home directory, which it doesn't have permissions to install the remote server there.

If you want the devcontainer to run as root, could you try adding "remoteUser": "root" to your devcontainer config to override what is in the base image? Or alternatively, to run as vscode, remove any config in the Dockerfile which is forcing things back to paths owned by root.

[moshederri]

hey,

In our Dockerfile we had ENV HOME pointing to /root (without overriding remoteUser). We didn't understand why after we removed it, codespaces started working again, and after reading your comment, it all makes sense now.

thank you @jkeech !

[jkeech]

Glad it's working!

[kmp9928]

@moshederri I am having the same error, but can't launch the Codespace so I cannot remove Dockerfiles or anything to try to solve it. How did you do?

[david-engelmann]

@moshederri Are you currently hosting it on docker? If you are, it seems like it could be related to docker/wsl being down. What are the results from running the wsl -l -v command?