Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Azure App Service Linux Container - Web Socket Handshake Error 503 #49245

Closed
jlorek opened this issue Mar 1, 2020 · 55 comments
Closed

Azure App Service Linux Container - Web Socket Handshake Error 503 #49245

jlorek opened this issue Mar 1, 2020 · 55 comments
Assignees

Comments

@jlorek
Copy link

jlorek commented Mar 1, 2020

Environment

  • Azure App Service Linux Container, Default settings, WebSockets enabled
  • AspNet Core 3.1 Server Side Blazor Default Project

Problem

When using Web Sockets (eg. Blazor) the socket connection fails on the client side with following error:

Error during WebSocket handshake: Unexpected response code: 503

The Problem is somehow well known:
#19578
#31771
dotnet/aspnetcore#10370

Sadly, all issues are closed without any solution.

According to the issues listed above, source of the problem may be the perMessageDefalate setting on the WebSocket (which seemingly cannot be configured using Blazor Server Side). This limitation with WebSockets using App Service Linux Contains is also documented here:
https://docs.microsoft.com/en-us/azure/app-service/containers/app-service-linux-faq#language-support

Other people report, that the problem is related to CORS and Authentication settings:
https://stackoverflow.com/a/58240309

It would be very interesting to know if this is a bug that will be fixed, or works as designed.


Document Details

Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

@PRMerger7 PRMerger7 added the Pri3 label Mar 1, 2020
@BharathNimmala-MSFT
Copy link
Contributor

@jlorek Thank you for your query and feedback, our team will further look into it and get back to you at the earliest.

@LANSoftwareentwicklung
Copy link

We are experiencing the same issue.

@ameyakarve
Copy link

Any updates on this one? Getting the same issue

@AjayKumar-MSFT
Copy link
Contributor

AjayKumar-MSFT commented Mar 12, 2020

Apologies for the delay! I completely understand it is a frustrating experience with this issue. I'm checking on this with our product engineering team and I will post back with an update soon.

Just as a side note, I have seen similar issues (WebSocket | error 503) fixed, by scaling-up to a higher App Service Plan.

@JennyLawrance
Copy link
Contributor

@jlorek If you can share your app name where you are seeing the 503, I can help get to the bottom of this. Technically, Web sockets is supported for App Service Linux and the WebSocket-Extensions issue also should no longer be happening.

Is there anything more complex to try to repro this problem after deploying the default Blazor project app? I just tried to deploy a sample, and it loads for me fine. (https://blazerlinux.tuxedoase.p.azurewebsites.net/weatherforecast) But I admit I am not a app expert, so I might be missing something.

@AjayKumar-MSFT
Copy link
Contributor

Thanks @JennyLawrance for your help and comments.
@LANSoftwareentwicklung, @ameyakarve, as Jenny requested, could you please share one of the WebApp name directly here or indirectly?

Default Blazor app works for us, if there are any additional/complex steps that you're configuring and want to share the repo and exact repro steps, please send an email to AzCommunity[at]Microsoft[dot]com referencing this thread, the WebApp name, and the Azure subscription ID, we will follow-up with you further.

Thanks for your patience and co-operation!

@jlorek
Copy link
Author

jlorek commented Mar 22, 2020

@JennyLawrance - The /weatherforecast API Endpoint is Part of the Blazor Client Side Default Project. It's not included in the Server Side Default Project, since it's not required due to the Server Side Application Architecture. Would you mind trying to reproduce with the Server Side Project Template dotnet new blazorserver?

@AjayKumar-MSFT - Sure, just head to https://peakseason.azurewebsites.net
As mentioned at the beginning, it's the Blazor Server Side Default Project Template deployed to an Azure App Service Linux Container (with WebSocket enabled) on a F1 App Service Plan.

@jlorek
Copy link
Author

jlorek commented Mar 22, 2020

I also created another sample without Containers, using App Service with Web App on Linux on a Net Core 3.1 Stack:
https://peakseason-code.azurewebsites.net

I tried the F1 and B1 Service Plan. Both resulting in the same issue - Web Socket Handshake Error 503.

@LANSoftwareentwicklung
Copy link

@AjayKumar-MSFT - I sent you an e-mail with the app name and subscription id.

@AjayKumar-MSFT
Copy link
Contributor

@jlorek, thanks for sharing the requested details.
@LANSoftwareentwicklung, Ack. We have received your email.

We're investigating and will get back to you soon.

@JennyLawrance
Copy link
Contributor

@jlorek, can you clarify which URL is seeing the 503? I am able to access the app.

@classifieds-dev
Copy link

I was running into this same issue on a multi-container app service. When I encountered this error I was using an app service plan with the F1 tier (free). After reading through some of the recommendations I tried to switching to a b1 tier. After redeploying the multi-container app on the b1 tier the web socket is successfully connecting without the handshake error.

@jlorek
Copy link
Author

jlorek commented Mar 24, 2020

@JennyLawrance - Yes, you can access the app. But the underlying WebSocket connection cannot be established, because of a 503 error. To see this error, just open your browsers DevTools. Using Blazor there is a fallback mode implemented, when no WebSocket connection can be established. When not using Blazor and relying on WebSockets, your whole app may be broken due to this error.

@classifieds-dev - Tried the B1 without any success. But using the P1V2 service plan, the WebSocket connection works just fine for my project.

@JennyLawrance
Copy link
Contributor

@jlorek , @classifieds-dev , Yes I have identified a code issue on our end which completely blocks web sockets on free SKU. I have opened a work item on our side to fix this.

@AjayKumar-MSFT
Copy link
Contributor

As this issue is identified and a fix would be deployed. Also, the documentation will be refreshed to reflect the changes. We will now proceed to close this thread. If there are further questions regarding this matter, please tag us in your reply. We will gladly continue the discussion.

Thanks everyone for bringing this to our attention. We really appreciate your valuable feedback.

@Spongman
Copy link

Spongman commented Apr 1, 2020

I'm seeing a 503 with a single-container F1 free tier app service. is this the same issue that's going to be fixed, or should i file a separate bug? oh, it looks like the free tier is going to be fixed...

@lopejum
Copy link

lopejum commented Apr 8, 2020

Hello

All of those solutions did not work for me. Let me explain what I have:

I'm developing a solution with Twilio + Azure Cognitive Services (telephone based conversation, realtime, and using websockets)

From Twilio, when a call comes in, I send an initial POST request (Webhook) to my WebApp for Containers like this:
https://my_azure_web_app.azurewebsites.net/twiml

Inside <my_azure_web_app> server I have an end-point like this:

class TwiML(tornado.web.RequestHandler):
def post(self):
response_str = './greetings.mp3'
response_str += ''
response_str += ''
self.finish(str(response_str))
def set_default_headers(self):
print("setting headers!!!")
self.set_header("Access-Control-Allow-Origin", "*")
self.set_header("Access-Control-Allow-Headers", "x-requested-with")
self.set_header("Sec-WebSocket-Extensions", "permessage-deflate")

I can launch my app on localhost and into a container on localhost (i.e. I can open websockets from/to Twilio with no problem and get the websocket channel opened)

When I deploy the image into WebApp for Containers, and this container is running, I can receive the first part of the audio message (i.e. greetings). But, after that, I can not open the wss websocket connection (wss://my_azure_web_app.azurewebsites.net.azurewebsites.net/socket) and I always receive an error from Twilio like this:
Stream - WebSocket - Connection Broken Pipe

(I used the "EXPOSE 8080" into my Dockerfile and "WEBSITES_PORT = 8080" into my Configuration Settings WebApp)

The container is running:
2020-04-08 14:13:06.366 INFO - Logging is not enabled for this container.
Please use https://aka.ms/linux-diagnostics to enable logging to see container logs here.
2020-04-08 14:13:06.882 INFO - Initiating warmup request to container my_azure_web_app_8_91af842a for site my_azure_web_app
2020-04-08 14:13:10.919 INFO - Container my_azure_web_app_8_91af842a for site my_azure_web_app initialized successfully and is ready to serve requests.

I can access to its url:
https://my_azure_web_app.azurewebsites.net/

I can connect to my websocket channel with Advanced Rest Client (Google)
"Connected to wss://my_azure_web_app.azurewebsites.net/socket"

I think everything is fine BUT when I try to open the "wss://my_azure_web_app.azurewebsites.net/socket" from Twilio, it is not possible

I tried everything with no success (both, tornado server and Azure ). So:

  1. Web App for Containers
    1.1. CORS: empty (and also: Allowed Origins "*"). Both, without success.
    1.2. WebSockets enabled ("On")
    1.3. Scale Up Service Plan (P1V2)
  2. Tornado server:
    2.1. Set headers "Access-Control-Allow-Origin", "*"
    2.2. Set header "Sec-WebSocket-Extensions", "permessage-deflate" (is it necessary yet?)

Finally, I do not have my app running deployed into Web App for Containers
I run out my ideas.

Anyone can tell new ways to try?

Thanks a lot in advance!

@hibearpanda
Copy link

hibearpanda commented May 25, 2020

We started getting this issue last Friday randomly. All our App Service Plans are Linux.

Our clients were unable to establish a websocket connection to one of our services on a B3 App Service Plan. This service has been deployed for several months, and we never had any websocket connectivity issues (or any connectivity issues for that matter) with it.

I tried restarting the app service several times which did not work. Finally, after seeing this thread, I moved the app service from its original B3 App Service Plan to a different B1 App Service Plan we were using. This fixed the issue for us.

We then tried moving the app service back to the original B3 App Service Plan but the problem occurred again.

This is very confusing to us and we have sent a Support Ticket to Azure.

I don't think we have made any changes to the B3 App Service Plan or the affected App Service for a while.

@BlackRider97
Copy link
Contributor

BlackRider97 commented May 26, 2020

Same error

failed: Error during WebSocket handshake: Unexpected response code: 503

App service plan: LinuxContainerAppServicePlan (P1v2: 1)

@Corstiaan84
Copy link

Same issue on Linux B1

@mkvalluri
Copy link

Same issue using Linux B1, but it works when switched to Windows.

@Nitive
Copy link

Nitive commented Jul 6, 2020

Switching Linux B1 → Linux B2 and then switching back helped

@AjayKumar-MSFT
Copy link
Contributor

AjayKumar-MSFT commented Jul 21, 2020

Apologies everyone for the late response and frustration with this issue. We had been discussion on this internally.
For anyone still experiencing this issue, could you please send an email with subject line 'Attn:Ajay' to AzCommunity[at]Microsoft[dot]com referencing this thread, WebApp name, and the Azure subscription ID, for a deeper investigation, we will follow-up on this further.

Thanks for your patience and co-operation!

@AjayKumar-MSFT AjayKumar-MSFT added the escalated-product-team used when escalating an issue to product team label Jul 21, 2020
@Sijoma
Copy link

Sijoma commented Aug 11, 2020

Switch from free F1 plan to B2 worked. However, when switching back to F1 the error comes back. I'm not sure if websockets are not supported for the F1 plan.

My error: Error during WebSocket handshake: Unexpected response code: 503

@AjayKumar-MSFT
Copy link
Contributor

Switch from free F1 plan to B2 worked. However, when switching back to F1 the error comes back. I'm not sure if websockets are not supported for the F1 plan.

My error: Error during WebSocket handshake: Unexpected response code: 503

@Sijoma, Thanks for the update. Yes, as mentioned in this document -“Web Sockets are not currently supported for Linux apps on Free App Service Plans. We are working on removing this limitation and plan to support up to 5 web socket connections on Free App Service plans.” - Our product team is working on it, but we do not have an ETA to share at this time.

@Sijoma
Copy link

Sijoma commented Aug 12, 2020

They also do not work for paid plans (B1).

@AjayKumar-MSFT
Copy link
Contributor

They also do not work for paid plans (B1).

Users with a similar issue, fixed it by switching the plan -B1 -> B2 -> B1, kindly see if this works for you. Having mentioned that, I'm discussing on this internally and will post back with more update.

@remo87
Copy link

remo87 commented Aug 21, 2020

Does this works on the Windows Free instance? Or are websockets out of scope for free instances?

@AjayKumar-MSFT
Copy link
Contributor

Does this works on the Windows Free instance? Or are websockets out of scope for free instances?

We allow 5 websocket connections in FREE SKU in Windows.

@jona0497
Copy link

jona0497 commented Oct 16, 2020

I just want to add my experience in here. I had similar issues with web sockets not working on linux web apps.
For me it worked to upgrade to B2. Websockets did not work on F1 and B1 at all.

@mobyjames
Copy link

Had to upgrade to Linux B2 for it to work.

@rusel1989
Copy link

Same here, no luck with F1 or B1, when upgraded to B2 it has started to work immediately.

@kalkusa
Copy link

kalkusa commented Dec 1, 2020

To me, Linux B2 worked as well. B1 and F1 does not.

@ldeluigi
Copy link

ldeluigi commented Dec 6, 2020

On my multi-container web app websocket works but the error is still thrown. Using nginx, nodejs

@JonSilver
Copy link

JonSilver commented May 1, 2021

Is there any update to this issue? I see the issue was closed, but is it fixed? Its been affecting B1 plans for a long time now. I raised this as a premium support issue back in 2018/9. Alas the conclusion then was "that's just the way it is, we don't support websockets on Linux B1".

@claudiucristeagab
Copy link

I ran into the same problem on my B1 app, and switching to B2 and back to B1 has fixed the problem.

They also do not work for paid plans (B1).

Users with a similar issue, fixed it by switching the plan -B1 -> B2 -> B1, kindly see if this works for you. Having mentioned that, I'm discussing on this internally and will post back with more update.

@Crafoord
Copy link

This still seems to be an issue, got our socket working by switching from "F1" tier to "B2".

@Triazic
Copy link

Triazic commented Jun 22, 2021

Confirmed. Magically fixed by upgrading to B2. Will test downgrading tomorrow.

@DJango316
Copy link

Have enabled websockets under Configuration but also experiencing the same 503 issue with an app service running on Linux with an F1 plan.

@DJango316
Copy link

DJango316 commented Jul 6, 2021

I have also ended up going for the B2 plan which works fine. Such a shame that it doesn't work for F1

@FaustoMendezcom
Copy link

It worked on B1 for me. Websocket on (settings ) default values on socket.io

@yescine
Copy link

yescine commented Jan 3, 2022

it worked only after upgrading to B2, (containerized nodejs app)
should we do that for all our webapp ?!

@rsteimen
Copy link

rsteimen commented Apr 9, 2022

Just had the same issue with my node.js websocket server software. B1 did not work. Scaled up to B2 and it worked. Scaled down to B1 2mins later and it's still working. Seems like an internal azure issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests