Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Azure Data Explorer query validator does not understand table name from variable, even though query itself works fine #559

Open
sandersaares opened this issue Feb 15, 2023 · 1 comment
Labels
datasource/ADX enhancement New feature or request

Comments

@sandersaares
Copy link

sandersaares commented Feb 15, 2023

What happened:
I am trying to use a $table variable (textbox type) to specify the table name using the Azure Data Explorer data source. For example:

$table
| where $__timeFilter(timestamp)
| where name == 'simkit_time_elapsed_ticks_total'

This works fine in terms of executing the query, but the query validation logic seems to freak out and think none of the columns are valid anymore (it is perhaps not resolving the variable?):

image

The query itself works fine when executed, only the validation messages are broken:

image

What you expected to happen:
Everything should work fine, as long as my variable has a value that is a legitimate table name.

How to reproduce it (as minimally and precisely as possible):

Take a working Azure Data Explorer query and replace the table name with a variable (e.g. plain textbox for manual input).

Anything else we need to know?:

Environment:

  • Grafana version: 9.3.2.2 (67a213dc85)
  • Data source type & version: Azure Data Explorer 4.2.0
  • OS Grafana is installed on: Azure
  • User OS & Browser: Windows 11 + Edge
  • Grafana plugins:
  • Others:
Key Value
Panel timeseries @ 9.3.2.2 (67a213dc85)
Grafana 9.3.2.2 (67a213dc85) // Pro
Panel debug snapshot dashboard
{
  "panels": [
    {
      "id": 2,
      "gridPos": {
        "h": 13,
        "w": 15,
        "x": 0,
        "y": 0
      },
      "type": "timeseries",
      "title": "Reproduced with embedded data",
      "datasource": {
        "type": "grafana",
        "uid": "grafana"
      },
      "fieldConfig": {
        "defaults": {
          "custom": {
            "drawStyle": "line",
            "lineInterpolation": "linear",
            "barAlignment": 0,
            "lineWidth": 1,
            "fillOpacity": 0,
            "gradientMode": "none",
            "spanNulls": false,
            "showPoints": "auto",
            "pointSize": 5,
            "stacking": {
              "mode": "none",
              "group": "A"
            },
            "axisPlacement": "auto",
            "axisLabel": "",
            "axisColorMode": "text",
            "scaleDistribution": {
              "type": "linear"
            },
            "axisCenteredZero": false,
            "hideFrom": {
              "tooltip": false,
              "viz": false,
              "legend": false
            },
            "thresholdsStyle": {
              "mode": "off"
            }
          },
          "color": {
            "mode": "palette-classic"
          },
          "mappings": [],
          "thresholds": {
            "mode": "absolute",
            "steps": [
              {
                "color": "green",
                "value": null
              },
              {
                "color": "red",
                "value": 80
              }
            ]
          }
        },
        "overrides": []
      },
      "options": {
        "tooltip": {
          "mode": "single",
          "sort": "none"
        },
        "legend": {
          "showLegend": true,
          "displayMode": "list",
          "placement": "bottom",
          "calcs": []
        }
      },
      "targets": [
        {
          "refId": "A",
          "datasource": {
            "type": "grafana",
            "uid": "grafana"
          },
          "queryType": "snapshot",
          "snapshot": [
            {
              "schema": {
                "refId": "A",
                "meta": {
                  "custom": {
                    "ColumnTypes": [
                      "datetime",
                      "real"
                    ]
                  },
                  "executedQueryString": "demo1\n| where timestamp >= datetime(2000-01-01T00:00:00Z) and timestamp <= datetime(2000-01-01T01:00:00Z)\n| where name == 'simkit_time_elapsed_ticks_total'\n| where simulation_id == '1676445720-6e04fdc7-e3e7-45e6-8558-109f2804d1fd'\n| where value > 0\n| join (demo1\n  | where name == 'simkit_time_elapsed_seconds_total'\n  | project simulation_id, timestamp, seconds_value = value\n  ) on simulation_id, timestamp\n| where seconds_value > 0\n| project tick_duration_seconds = seconds_value / value, timestamp\n| summarize tick_duration_seconds = avg(tick_duration_seconds) by bin(timestamp, 5000ms)\n| order by timestamp asc"
                },
                "fields": [
                  {
                    "name": "timestamp",
                    "type": "time",
                    "typeInfo": {
                      "frame": "time.Time",
                      "nullable": true
                    },
                    "config": {}
                  },
                  {
                    "name": "tick_duration_seconds",
                    "type": "number",
                    "typeInfo": {
                      "frame": "float64",
                      "nullable": true
                    },
                    "config": {}
                  }
                ]
              },
              "data": {
                "values": [
                  [
                    946684800000,
                    946684805000,
                    946684810000,
                    946684815000,
                    946684820000,
                    946684825000,
                    946684830000,
                    946684835000,
                    946684840000,
                    946684845000,
                    946684850000,
                    946684855000,
                    946684860000,
                    946684865000,
                    946684870000,
                    946684875000,
                    946684880000,
                    946684885000,
                    946684890000,
                    946684895000,
                    946684900000,
                    946684905000,
                    946684910000,
                    946684915000,
                    946684920000,
                    946684925000,
                    946684930000,
                    946684935000,
                    946684940000,
                    946684945000,
                    946684950000,
                    946684955000,
                    946684960000,
                    946684965000,
                    946684970000,
                    946684975000,
                    946684980000,
                    946684985000,
                    946684990000,
                    946684995000,
                    946685000000,
                    946685005000,
                    946685010000,
                    946685015000,
                    946685020000,
                    946685025000,
                    946685030000,
                    946685035000,
                    946685040000,
                    946685045000,
                    946685050000,
                    946685055000,
                    946685060000,
                    946685065000,
                    946685070000,
                    946685075000,
                    946685080000,
                    946685085000,
                    946685090000,
                    946685095000,
                    946685100000,
                    946685105000,
                    946685110000,
                    946685115000,
                    946685120000,
                    946685125000,
                    946685130000,
                    946685135000,
                    946685140000,
                    946685145000,
                    946685150000,
                    946685155000,
                    946685160000,
                    946685165000,
                    946685170000,
                    946685175000,
                    946685180000,
                    946685185000,
                    946685190000,
                    946685195000,
                    946685200000,
                    946685205000,
                    946685210000,
                    946685215000,
                    946685220000,
                    946685225000,
                    946685230000,
                    946685235000,
                    946685240000,
                    946685245000,
                    946685250000,
                    946685255000,
                    946685260000,
                    946685265000,
                    946685270000,
                    946685275000,
                    946685280000,
                    946685285000,
                    946685290000,
                    946685295000,
                    946685300000,
                    946685305000,
                    946685310000,
                    946685315000,
                    946685320000,
                    946685325000,
                    946685330000,
                    946685335000,
                    946685340000,
                    946685345000,
                    946685350000,
                    946685355000,
                    946685360000,
                    946685365000,
                    946685370000,
                    946685375000,
                    946685380000,
                    946685385000,
                    946685390000,
                    946685395000,
                    946685400000
                  ],
                  [
                    0.016525131855587315,
                    0.016625389949138578,
                    0.01664318954040204,
                    0.01665017842158281,
                    0.01665394058493172,
                    0.016656296408640556,
                    0.016657911554485712,
                    0.016659088314497826,
                    0.016659984019509022,
                    0.016660688698957284,
                    0.016661257614650753,
                    0.016661726575610478,
                    0.016662119809287383,
                    0.01666245429543418,
                    0.0166627422886908,
                    0.01666299285581629,
                    0.01666321284928424,
                    0.01666340754452045,
                    0.016663581069003862,
                    0.0166637366984647,
                    0.016663877065854937,
                    0.016664004311662796,
                    0.016664120193924298,
                    0.016664226170001024,
                    0.016664323458232686,
                    0.016664413085015293,
                    0.01666449592117274,
                    0.01666457271035897,
                    0.016664644091456553,
                    0.016664710616401707,
                    0.01666477276448965,
                    0.016664830953945352,
                    0.016664885551350702,
                    0.016664936879377545,
                    0.016664985223171537,
                    0.0166650308356536,
                    0.01666507394194742,
                    0.01666511474309621,
                    0.016665153419198886,
                    0.0166651901320683,
                    0.016665225027494755,
                    0.016665258237181322,
                    0.01666528988040531,
                    0.016665320065449983,
                    0.01666534889084275,
                    0.01666537644642967,
                    0.016665402814311073,
                    0.016665428069658716,
                    0.01666545228143149,
                    0.016665475513004433,
                    0.016665497822722624,
                    0.016665519264390548,
                    0.016665539887705406,
                    0.01666555973864162,
                    0.01666557885979314,
                    0.016665597290678475,
                    0.016665615068013364,
                    0.01666563222595494,
                    0.016665648796320775,
                    0.016665664808785844,
                    0.016665680291059904,
                    0.016665695269047463,
                    0.016665709766992547,
                    0.016665723807609542,
                    0.016665737412202026,
                    0.01666575060077064,
                    0.01666576339211115,
                    0.01666577580390382,
                    0.01666578785279508,
                    0.016665799554471825,
                    0.01666581092372972,
                    0.01666582197453555,
                    0.016665832720084428,
                    0.01666584317285242,
                    0.016665853344644863,
                    0.01666586324664079,
                    0.016665872889434,
                    0.016665882283070758,
                    0.016665891437084832,
                    0.016665900360529673,
                    0.016665909062008336,
                    0.016665917549701124,
                    0.016665925831391253,
                    0.01666593391448863,
                    0.016665941806052,
                    0.016665949512809544,
                    0.016665957041177862,
                    0.016665964397279985,
                    0.016665971586961835,
                    0.01666597861580779,
                    0.016665985489155166,
                    0.01666599221210767,
                    0.016665998789548064,
                    0.01666600522615008,
                    0.01666601152638929,
                    0.016666017694553746,
                    0.016666023734753496,
                    0.016666029650929816,
                    0.016666035446863778,
                    0.016666041126184326,
                    0.01666604669237579,
                    0.016666052148785047,
                    0.016666057498628307,
                    0.01666606274499728,
                    0.01666606789086531,
                    0.016666072939092818,
                    0.016666077892432754,
                    0.01666608275353549,
                    0.01666608752495367,
                    0.016666092209146537,
                    0.01666609680848429,
                    0.016666101325251974,
                    0.016666105761653352,
                    0.01666611011981445,
                    0.016666114401786943,
                    0.016666118609551413,
                    0.0166661227450204,
                    0.016666126810041194,
                    0.0166661308063988,
                    0.016666134735818346,
                    0.016666137064607518
                  ]
                ]
              }
            }
          ]
        }
      ]
    },
    {
      "gridPos": {
        "h": 7,
        "w": 9,
        "x": 15,
        "y": 0
      },
      "id": 5,
      "options": {
        "content": "<table width=\"100%\">\n    <tr>\n      <th width=\"2%\">Panel</th>\n      <td >timeseries @ 9.3.2.2 (67a213dc85)</td>\n    </tr>\n    <tr>\n      <th>Queries</th>\n      <td>A[grafana-azure-data-explorer-datasource]</td>\n    </tr>\n    \n    <tr><th>Data</th><td> 1 frames, 2 fields, 121 rows </td></tr>\n    \n    <tr>\n      <th>Grafana</th>\n      <td>9.3.2.2 (67a213dc85) // Pro</td>\n    </tr>\n  </table>",
        "mode": "html"
      },
      "title": "Debug info",
      "type": "text"
    },
    {
      "id": 6,
      "title": "Original Panel JSON",
      "type": "text",
      "gridPos": {
        "h": 13,
        "w": 9,
        "x": 15,
        "y": 7
      },
      "options": {
        "content": "{\n  \"id\": 2,\n  \"gridPos\": {\n    \"h\": 9,\n    \"w\": 12,\n    \"x\": 0,\n    \"y\": 0\n  },\n  \"type\": \"timeseries\",\n  \"title\": \"Tick duration\",\n  \"datasource\": {\n    \"uid\": \"u4kH7I14k\",\n    \"type\": \"grafana-azure-data-explorer-datasource\"\n  },\n  \"fieldConfig\": {\n    \"defaults\": {\n      \"custom\": {\n        \"drawStyle\": \"line\",\n        \"lineInterpolation\": \"linear\",\n        \"barAlignment\": 0,\n        \"lineWidth\": 1,\n        \"fillOpacity\": 0,\n        \"gradientMode\": \"none\",\n        \"spanNulls\": false,\n        \"showPoints\": \"auto\",\n        \"pointSize\": 5,\n        \"stacking\": {\n          \"mode\": \"none\",\n          \"group\": \"A\"\n        },\n        \"axisPlacement\": \"auto\",\n        \"axisLabel\": \"\",\n        \"axisColorMode\": \"text\",\n        \"scaleDistribution\": {\n          \"type\": \"linear\"\n        },\n        \"axisCenteredZero\": false,\n        \"hideFrom\": {\n          \"tooltip\": false,\n          \"viz\": false,\n          \"legend\": false\n        },\n        \"thresholdsStyle\": {\n          \"mode\": \"off\"\n        }\n      },\n      \"color\": {\n        \"mode\": \"palette-classic\"\n      },\n      \"mappings\": [],\n      \"thresholds\": {\n        \"mode\": \"absolute\",\n        \"steps\": [\n          {\n            \"color\": \"green\",\n            \"value\": null\n          },\n          {\n            \"color\": \"red\",\n            \"value\": 80\n          }\n        ]\n      }\n    },\n    \"overrides\": []\n  },\n  \"options\": {\n    \"tooltip\": {\n      \"mode\": \"single\",\n      \"sort\": \"none\"\n    },\n    \"legend\": {\n      \"showLegend\": true,\n      \"displayMode\": \"list\",\n      \"placement\": \"bottom\",\n      \"calcs\": []\n    }\n  },\n  \"targets\": [\n    {\n      \"datasource\": {\n        \"type\": \"grafana-azure-data-explorer-datasource\",\n        \"uid\": \"u4kH7I14k\"\n      },\n      \"database\": \"feb23\",\n      \"expression\": {\n        \"from\": {\n          \"property\": {\n            \"name\": \"demo1\",\n            \"type\": \"string\"\n          },\n          \"type\": \"property\"\n        },\n        \"groupBy\": {\n          \"expressions\": [],\n          \"type\": \"and\"\n        },\n        \"reduce\": {\n          \"expressions\": [],\n          \"type\": \"and\"\n        },\n        \"where\": {\n          \"expressions\": [],\n          \"type\": \"and\"\n        }\n      },\n      \"pluginVersion\": \"4.2.0\",\n      \"query\": \"$table\\n| where $__timeFilter(timestamp)\\n| where name == 'simkit_time_elapsed_ticks_total'\\n| where simulation_id == '$simulation_id'\\n| where value > 0\\n| join ($table\\n  | where name == 'simkit_time_elapsed_seconds_total'\\n  | project simulation_id, timestamp, seconds_value = value\\n  ) on simulation_id, timestamp\\n| where seconds_value > 0\\n| project tick_duration_seconds = seconds_value / value, timestamp\\n| summarize tick_duration_seconds = avg(tick_duration_seconds) by bin(timestamp, $__timeInterval)\\n| order by timestamp asc\",\n      \"querySource\": \"raw\",\n      \"rawMode\": true,\n      \"refId\": \"A\",\n      \"resultFormat\": \"table\"\n    }\n  ]\n}",
        "mode": "code",
        "code": {
          "language": "json",
          "showLineNumbers": true,
          "showMiniMap": true
        }
      }
    },
    {
      "id": 3,
      "title": "Data from panel above",
      "type": "table",
      "datasource": {
        "type": "datasource",
        "uid": "-- Dashboard --"
      },
      "gridPos": {
        "h": 7,
        "w": 15,
        "x": 0,
        "y": 13
      },
      "options": {
        "showTypeIcons": true
      },
      "targets": [
        {
          "datasource": {
            "type": "datasource",
            "uid": "-- Dashboard --"
          },
          "panelId": 2,
          "withTransforms": true,
          "refId": "A"
        }
      ]
    }
  ],
  "schemaVersion": 37,
  "title": "Debug: Tick duration // 2023-02-15 07:55:51",
  "tags": [
    "debug",
    "debug-timeseries"
  ],
  "time": {
    "from": "2000-01-01T00:00:00.000Z",
    "to": "2000-01-01T01:00:00.000Z"
  }
}
@alyssabull alyssabull transferred this issue from grafana/grafana Mar 1, 2023
@alyssabull
Copy link
Contributor

Hi @sandersaares - I can replicate this issue, thanks for bringing it to our attention. Currently, this is the expected behavior (due to when the template variable is interpolated) but we are going to be having a discussion on how we can handle this better.

@aangelisc aangelisc added the enhancement New feature or request label Mar 21, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
datasource/ADX enhancement New feature or request
Projects
Status: Backlog
Development

No branches or pull requests

3 participants