Problem with sending subsequent responses (Bad headers)

[szalkerous]

So in reviewing a persistent issue, I've come across a unique scenario on the latest version of Watson (not Lite).

It seems an auto-generated defaultheader is causing a failure to send the response, but only after the first request/response is completed.

I think the header in question is this one:

DEF HEADERS: Key:"localhost:26080", Value:"localhost:8000"

The project runs in VS2022, written in C#, and targets .NET Framework 4.8.1. I'm using version 6.1.9 of Watson and Watson.Core. I'm setting Watson to use port 26080.

Initialization settings

            _Settings = new WebserverSettings
            {
                Hostname = "localhost",
                Port = 26080,
                IO = new WebserverSettings.IOSettings() 
                { 
                    EnableKeepAlive = true
                },
                AccessControl = new AccessControlManager()
                {
                    Mode = AccessControlMode.DefaultPermit
                }
            };

Response section of debug data during a routed event (after assigning relevant data):

"Response": {
"Timestamp": {
"Start": "2024-05-10T04:38:09.189941Z",
"TotalMs": 132.91999999999999,
"Messages": {}
},
"StatusCode": 200,
"StatusDescription": "OK",
"Headers": {
"Connection": "close",
"Content-Language": "en"
},
"ContentType": "text/html",
"ChunkedTransfer": false,
"ResponseSent": false
}

Exception after HttpResponseBase.Send():

System.AggregateException: One or more errors occurred. ---> System.ArgumentException: Specified value has invalid HTTP Header characters.
Parameter name: name
at System.Net.WebHeaderCollection.CheckBadChars(String name, Boolean isHeaderValue)
at System.Net.WebHeaderCollection.SetInternal(String name, String value)
at System.Net.HttpListenerResponse.AddHeader(String name, String value)
at WatsonWebserver.HttpResponse.SendHeaders()
at WatsonWebserver.HttpResponse.d__29.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at System.Runtime.CompilerServices.ConfiguredTaskAwaitable1.ConfiguredTaskAwaiter.GetResult() at WatsonWebserver.HttpResponse.<Send>d__21.MoveNext() --- End of inner exception stack trace --- at System.Threading.Tasks.Task.ThrowIfExceptional(Boolean includeTaskCanceledExceptions) at System.Threading.Tasks.Task1.GetResultCore(Boolean waitCompletionNotification)
at System.Threading.Tasks.Task1.get_Result() at ProgramNamePlaceholder.Route_Response(HttpResponseBase rsp) ---> (Inner Exception #0) System.ArgumentException: Specified value has invalid HTTP Header characters. Parameter name: name at System.Net.WebHeaderCollection.CheckBadChars(String name, Boolean isHeaderValue) at System.Net.WebHeaderCollection.SetInternal(String name, String value) at System.Net.HttpListenerResponse.AddHeader(String name, String value) at WatsonWebserver.HttpResponse.SendHeaders() at WatsonWebserver.HttpResponse.<SendInternalAsync>d__29.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task) at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at System.Runtime.CompilerServices.ConfiguredTaskAwaitable1.ConfiguredTaskAwaiter.GetResult()
at WatsonWebserver.HttpResponse.d__21.MoveNext()<---

Debug logging dump of headers active at the time of exception:

HttpResponseBase.Headers:

DEBUG COMMON - HEADERS: Key:"Connection", Value:"close"
DEBUG COMMON - HEADERS: Key:"Content-Language", Value:"en"

WebServerBase.Settings.Headers.DefaultHeaders: (Note: None of these are customized or modified directly by client code)

DEBUG COMMON - DEF HEADERS: Key:"Access-Control-Allow-Origin", Value:""
DEBUG COMMON - DEF HEADERS: Key:"Access-Control-Allow-Methods", Value:"OPTIONS, HEAD, GET, PUT, POST, DELETE, PATCH"
DEBUG COMMON - DEF HEADERS: Key:"Access-Control-Allow-Headers", Value:""
DEBUG COMMON - DEF HEADERS: Key:"Access-Control-Expose-Headers", Value:""
DEBUG COMMON - DEF HEADERS: Key:"Accept", Value:"/"
DEBUG COMMON - DEF HEADERS: Key:"Accept-Language", Value:"en-US, en"
DEBUG COMMON - DEF HEADERS: Key:"Accept-Charset", Value:"ISO-8859-1, utf-8"
DEBUG COMMON - DEF HEADERS: Key:"Cache-Control", Value:"no-cache"
DEBUG COMMON - DEF HEADERS: Key:"Connection", Value:"close"
DEBUG COMMON - DEF HEADERS: Key:"localhost:26080", Value:"localhost:8000"

Reference System.Net code (decompiled by VS2022):

System.Net.WebHeaderCollection ------------------------------------------

internal static string CheckBadChars(string name, bool isHeaderValue)
{
    if (name == null || name.Length == 0)
    {
        if (!isHeaderValue)
        {
            throw (name == null) ? new ArgumentNullException("name") : new ArgumentException(SR.GetString("net_emptystringcall", "name"), "name");
        }

        return string.Empty;
    }

    if (isHeaderValue)
    {
        name = name.Trim(HttpTrimCharacters);
        int num = 0;
        for (int i = 0; i < name.Length; i++)
        {
            char c = (char)(0xFFu & name[i]);
            switch (num)
            {
                case 0:
                    if (c == '\r')
                    {
                        num = 1;
                    }
                    else if (c == '\n')
                    {
                        num = 2;
                    }
                    else if (c == '\u007f' || (c < ' ' && c != '\t'))
                    {
                        throw new ArgumentException(SR.GetString("net_WebHeaderInvalidControlChars"), "value");
                    }

                    break;
                case 1:
                    if (c == '\n')
                    {
                        num = 2;
                        break;
                    }

                    throw new ArgumentException(SR.GetString("net_WebHeaderInvalidCRLFChars"), "value");
                case 2:
                    if (c == ' ' || c == '\t')
                    {
                        num = 0;
                        break;
                    }

                    throw new ArgumentException(SR.GetString("net_WebHeaderInvalidCRLFChars"), "value");
            }
        }

        if (num != 0)
        {
            throw new ArgumentException(SR.GetString("net_WebHeaderInvalidCRLFChars"), "value");
        }
    }
    else
    {
        if (name.IndexOfAny(ValidationHelper.InvalidParamChars) != -1)
        {
            throw new ArgumentException(SR.GetString("net_WebHeaderInvalidHeaderChars"), "name");
        }

        if (ContainsNonAsciiChars(name))
        {
            throw new ArgumentException(SR.GetString("net_WebHeaderInvalidNonAsciiChars"), "name");
        }
    }

    return name;
}

System.Net.ValidationHelper ------------------------------

internal static readonly char[] InvalidParamChars = new char[22]
{
    '(', ')', '<', '>', '@', ',', ';', ':', '\\', '"',
    '\'', '/', '[', ']', '?', '=', '{', '}', ' ', '\t',
    '\r', '\n'
};

[szalkerous]

The current workaround I've put in place as a stopgap measure is to iterate through all the default headers and parse out all illegitimate characters (using similar code to what's in System.Net.WebHeaderCollection) to replace all bad characters with an underscore.

Using this method, the responses are working fine until a proper fix or explanation is available.

[jchristn]

I'm curious, how did the header key localhost:26080 get set? I've never seen this behavior before and it doesn't appear to be set as the Key in the default headers.

[bjerregaardp]

I see something similar, but only after I have stopped and restarted the webserver within my application, ie. application not restarted. If I restart the whole application there is no problem.

                WebserverSettings settings = new WebserverSettings();
                settings.Port = portNo;
                settings.Hostname = myIp;
                server = new WatsonWebserver.Webserver(settings, DefaultRouteHandler);
                server.Routes.PreAuthentication.Static.Add(WatsonWebserver.Core.HttpMethod.GET, "/", IndexHtmlHandler);
                server.Start();

    async Task IndexHtmlHandler(HttpContextBase ctx)
    {
        HttpRequestBase r = ctx.Request;

        try
        {
            string result = "Hello";
            await ctx.Response.Send(result);
        }
        catch (Exception ex)
        {

        }
    }

            server.Stop();
            server.Dispose();

Creating a new webserver here will show the problem. The ctx.Response.Send(result) will throw the exception saying "Specified value has invalid HTTP characters".