Unable to easily suppress dependency exception using java agent

[petergphillips]

Expected behavior

I was hoping to suppress an exception using exception.message or exception.stacktrace that is flooding our logs - 900,000 times a day.

Actual behavior

I only appear to be able to suppress the exception using exception.type.

To Reproduce

I started off followed the steps detailed in Span Attributes Available for Sampling to look for the exception. Whilst I can see all the dependencies exported that way, it appears that the exceptions are exported differently and don't appear with text exporting span. In my case I can see the exception as json in the log, for example

{
  "ver": 1,
  "name": "Exception",
  "time": "2023-06-19T11:41:42.152Z",
  "iKey": "b44af8fc-1647-443d-8cea-55c5cfbd4518",
  "tags": {
    "ai.internal.sdkVersion": "java:3.4.13",
  },
  "data": {
    "baseType": "ExceptionData",
    "baseData": {
      "ver": 2,
      "exceptions": [
        {
          "typeName": "java.sql.SQLException",
          "message": "ORA-06550: line 1, column 28:",
          "stack": "java.sql.SQLException: ORA-06550: line 1, column 28:\nPLS-00302: component 'GETMAXIDENTIFIERLENGTH' must be declared..."
        }
      ]
    }
  }

I can get the suppression working with configuration such as

          "telemetryType": "exception",
          "attributes": [
            {
              "key": "exception.type",
              "value": "java.sql.SQLException",
              "matchType": "strict"
            }
          ],
          "percentage": 0

but would ideally like to match on the message or stacktrace instead. According to the opentelemetry exception documentation I think I should be matching on exception.message or exception.stacktrace, but can't get either to work. For example I think:

          "telemetryType": "exception",
          "attributes": [
            {
              "key": "exception.stacktrace",
              "value": ".*GETDBCOMPATMAJORVERSION.*",
              "matchType": "regexp"
            }
          ],
          "percentage": 0

should do the trick, but the exception still appears in the logs. I've also tried suppressing the dependency, which works fine with the above regexp but doesn't suppress the corresponding exception.

Please could you let me know the key that I need therefore to match on the stacktrace and / or message to suppress the exception. Looking at the Oracle code that throws the exception it appears to be caught and doesn't then affect the behaviour of the application, hence the reason for the suppression.

System information

Please provide the following information:

• SDK Version: java:3.4.13
• OS type and version: Based off docker image eclipse-temurin:19-jre-jammy so Java 19
• Application Server type and version (if applicable): Spring Boot 3.1.0
• Using spring-boot? Yes

[heyams]

Hi @petergphillips did you follow this doc: https://learn.microsoft.com/en-us/azure/azure-monitor/app/java-standalone-sampling-overrides#span-attributes-available-for-sampling

in particular

[petergphillips]

hi @heyams yes, that is the same document that I followed. I saw all the dependencies that were exported with the exporting span, but couldn't see any exceptions with that text. I have attached some of the log for you to confirm
events.log

[heyams]

@petergphillips i have checked your log. there were 4 exceptions and none of them had attributes.
that's why sampling override didn't work.

can you submit a repro app here?
I want to understand why those attributes weren't generated.

{
  "ver": 1,
  "name": "Exception",
  "time": "2023-06-19T11:41:42.152Z",
  "iKey": "<REDACTED>",
  "tags": {
    "ai.internal.sdkVersion": "java:3.4.13",
  },
  "data": {
    "baseType": "ExceptionData",
    "baseData": {
      "ver": 2,
      "exceptions": [
        {
          "typeName": "java.sql.SQLException",
          "message": "ORA-06550: line 1, column 28:",
          "stack": "java.sql.SQLException: ORA-06550: line 1, column 28:\nPLS-00302: component 'GETMAXIDENTIFIERLENGTH' must be declared..."
        }
      ]
    }
  }

this is the payload we send to Application Insights after the sampling overrides have been applied.

[petergphillips]

Sorry, I won't be able to create a repo that reproduces the issue. It requires an Oracle 11g database with Oracle adque installed. We are connecting to the queue using an newer version of the Oracle adque library which causes an incompatibility and thus a stacktrace is generated deep within the Oracle library.

What I find odd here is that for the exception we don't get any exporting span events at all. The exporting span events only seem to be generated for dependencies in this example.

Note that we are able to suppress the exception by targeting exception.type so the suppressions are working, just not in the way that we would like as suppressing all sql exceptions is a bit too broad.