Filtering out any property from a direct property of ExceptionDetails

[jaslam94]

Describe the feature

Problem

From the following log, I want to remove the field "HelpLink.ProdName", which is a property of "Data" and "Data" itself is a direct property of ExceptionDetails.

{
    "Timestamp": "2021-09-17T15:27:58.7179246+05:00",
    "Level": "Error",
    "MessageTemplate": "Error",
    "Exception": "System.Data.SqlClient.SqlException (0x80131904): Incorrect syntax near '+='.\r\n   at System.Data.SqlClient.SqlConnection.OnError(SqlException exception, Boolean breakConnection, Action`1 wrapCloseInAction)\r\n   at System.Data.SqlClient.SqlInternalConnection.OnError(SqlException exception, Boolean breakConnection, Action`1 wrapCloseInAction)\r\n   at System.Data.SqlClient.TdsParser.ThrowExceptionAndWarning(TdsParserStateObject stateObj, Boolean callerHasConnectionLock, Boolean asyncClose)\r\n   at System.Data.SqlClient.TdsParser.TryRun(RunBehavior runBehavior, SqlCommand cmdHandler, SqlDataReader dataStream, BulkCopySimpleResultSet bulkCopyHandler, TdsParserStateObject stateObj, Boolean& dataReady)\r\n   at System.Data.SqlClient.SqlDataReader.TryConsumeMetaData()\r\n   at System.Data.SqlClient.SqlDataReader.get_MetaData()\r\n   at System.Data.SqlClient.SqlCommand.FinishExecuteReader(SqlDataReader ds, RunBehavior runBehavior, String resetOptionsString, Boolean isInternal, Boolean forDescribeParameterEncryption, Boolean shouldCacheForAlwaysEncrypted)\r\n   at System.Data.SqlClient.SqlCommand.RunExecuteReaderTds(CommandBehavior cmdBehavior, RunBehavior runBehavior, Boolean returnStream, Boolean async, Int32 timeout, Task& task, Boolean asyncWrite, Boolean inRetry, SqlDataReader ds, Boolean describeParameterEncryptionRequest)\r\n   at System.Data.SqlClient.SqlCommand.RunExecuteReader(CommandBehavior cmdBehavior, RunBehavior runBehavior, Boolean returnStream, String method, TaskCompletionSource`1 completion, Int32 timeout, Task& task, Boolean& usedCache, Boolean asyncWrite, Boolean inRetry)\r\n   at System.Data.SqlClient.SqlCommand.RunExecuteReader(CommandBehavior cmdBehavior, RunBehavior runBehavior, Boolean returnStream, String method)\r\n   at System.Data.SqlClient.SqlCommand.ExecuteReader(CommandBehavior behavior, String method)\r\n   at System.Data.SqlClient.SqlCommand.ExecuteReader()\r\n   at CargoMatchv2.Common.Database.GetUser(String userid)\r\n   at CargoMatchv2.Controllers.AccountController.Login(LoginViewModel model, String returnUrl) in E:\\Workspace\\cargomatchv2\\CargoMatchv2\\Controllers\\AccountController.cs:line 110\r\nClientConnectionId:24bb9191-828c-431b-8793-b86707b0abd0\r\nError Number:102,State:1,Class:15",
    "Properties": {
        "UserName": "Not Authenticated",
        "ThreadId": 11,
        "MachineName": "DESKTOP-GHV3V41",
        "HttpRequestId": "a4fba8fb-05c0-4621-be83-a12def77c5f6",
        "HttpRequestNumber": 4,
        "HttpRequestClientHostIP": "::1",
        "HttpRequestType": "POST",
        "HttpRequestRawUrl": "/account/login/",
        "MvcRouteData": {
            "controller": "Account",
            "action": "Login"
        },
        "ExceptionDetail": {
            "Data": {
                "HelpLink.ProdName": "Microsoft SQL Server",
                "HelpLink.ProdVer": "15.00.2000",
                "HelpLink.EvtSrc": "MSSQLServer",
                "HelpLink.EvtID": "102",
                "HelpLink.BaseHelpUrl": "http://go.microsoft.com/fwlink",
                "HelpLink.LinkId": "20476"
            },
            "HResult": -2146232060,
            "Message": "Incorrect syntax near '+='.",
            "Source": ".Net SqlClient Data Provider",
            "StackTrace": "   at System.Data.SqlClient.SqlConnection.OnError(SqlException exception, Boolean breakConnection, Action`1 wrapCloseInAction)\r\n   at System.Data.SqlClient.SqlInternalConnection.OnError(SqlException exception, Boolean breakConnection, Action`1 wrapCloseInAction)\r\n   at System.Data.SqlClient.TdsParser.ThrowExceptionAndWarning(TdsParserStateObject stateObj, Boolean callerHasConnectionLock, Boolean asyncClose)\r\n   at System.Data.SqlClient.TdsParser.TryRun(RunBehavior runBehavior, SqlCommand cmdHandler, SqlDataReader dataStream, BulkCopySimpleResultSet bulkCopyHandler, TdsParserStateObject stateObj, Boolean& dataReady)\r\n   at System.Data.SqlClient.SqlDataReader.TryConsumeMetaData()\r\n   at System.Data.SqlClient.SqlDataReader.get_MetaData()\r\n   at System.Data.SqlClient.SqlCommand.FinishExecuteReader(SqlDataReader ds, RunBehavior runBehavior, String resetOptionsString, Boolean isInternal, Boolean forDescribeParameterEncryption, Boolean shouldCacheForAlwaysEncrypted)\r\n   at System.Data.SqlClient.SqlCommand.RunExecuteReaderTds(CommandBehavior cmdBehavior, RunBehavior runBehavior, Boolean returnStream, Boolean async, Int32 timeout, Task& task, Boolean asyncWrite, Boolean inRetry, SqlDataReader ds, Boolean describeParameterEncryptionRequest)\r\n   at System.Data.SqlClient.SqlCommand.RunExecuteReader(CommandBehavior cmdBehavior, RunBehavior runBehavior, Boolean returnStream, String method, TaskCompletionSource`1 completion, Int32 timeout, Task& task, Boolean& usedCache, Boolean asyncWrite, Boolean inRetry)\r\n   at System.Data.SqlClient.SqlCommand.RunExecuteReader(CommandBehavior cmdBehavior, RunBehavior runBehavior, Boolean returnStream, String method)\r\n   at System.Data.SqlClient.SqlCommand.ExecuteReader(CommandBehavior behavior, String method)\r\n   at System.Data.SqlClient.SqlCommand.ExecuteReader()\r\n   at CargoMatchv2.Common.Database.GetUser(String userid)\r\n   at CargoMatchv2.Controllers.AccountController.Login(LoginViewModel model, String returnUrl) in E:\\Workspace\\cargomatchv2\\CargoMatchv2\\Controllers\\AccountController.cs:line 110",
            "TargetSite": "Void OnError(System.Data.SqlClient.SqlException, Boolean, System.Action`1[System.Action])",
            "Errors": [
                {
                    "Source": ".Net SqlClient Data Provider",
                    "Number": 102,
                    "State": 1,
                    "Class": 15,
                    "Server": "localhost",
                    "Message": "Incorrect syntax near '+='.",
                    "Procedure": "",
                    "LineNumber": 1
                }
            ],
            "ClientConnectionId": "24bb9191-828c-431b-8793-b86707b0abd0",
            "Class": 15,
            "LineNumber": 1,
            "Number": 102,
            "Procedure": "",
            "Server": "localhost",
            "State": 1,
            "ErrorCode": -2146232060,
            "Type": "System.Data.SqlClient.SqlException"
        }
    }
}

What I did

I used the filter new DestructuringOptionsBuilder().WithFilter(new IgnorePropertyByNameExceptionFilter("HelpLink.ProdName")). It did not work.

In the above example, the filed "Data" can be removed using IgnorePropertyByNameExceptionFilter because "Data" is a direct property of "ExceptionDetails". However, any property under the direct property of "ExceptionDetails" cannot be removed.

Worse scenarios

In another scenario, an exception of type DBConcurrencyException was logged. It had an inner exception of type "OptimisticConcurrencyException". The "StateEntries" property of inner exception, contained over 12000 lines in the log record.

Such lengthy properties are not needed to be logged. We could easily get more than enough information from the other details of the exception.

Feature Request

There should be an option to filter out any property that's the direct property of "ExceptionDetails". e.g. "HelpLink.ProdName" in the above example.

[RehanSaeed]

I believe you can use Serilog.Expressions to filter out long properties yourself.

https://github.com/serilog/serilog-expressions

[arisliang]

It seems Serilog.Expression either include or exclude the whole property. Could it be possible to truncate the long property? Since we may still want some information from it.

[RehanSaeed]

Perhaps we could add a filter type that could allow inspection of properties before they are added:

public interface IPropertyFilter
{
    bool CanAddProperty(Exception exception, string key, string value);
}

We can take a PR for this work.