Tracing Tag Usage #873
Replies: 3 comments 15 replies
-
|
Thanks for the super detailed explanation @jgpruitt! This is a great example of how we could write the documentation for this feature. The solution looks very flexible. Tags will usually be text which means most users will only have to deal with the ? and the -> / ->> operators. Things get a lot more complicated if you have numerical values or dates in the tags (which will likely not be very common). A few questions:
|
Beta Was this translation helpful? Give feedback.
-
|
Agreed that this is awesome @jgpruitt! I agree on 1, I think it's more intuitive if it's a column rather than the whole row. |
Beta Was this translation helpful? Give feedback.
-
Here are the options as I see them:
|
Beta Was this translation helpful? Give feedback.
-
I have not tried it with Grafana yet. |
Beta Was this translation helpful? Give feedback.
-
Yeah, the jsonpath stuff can be complex, but super powerful. I like the idea of having it explained in a separate "advanced" section. We might be able to do some more advanced stuff with a simpler syntax, however we'll need to introduce more types to make it happen. For instance, currently there is a |
Beta Was this translation helpful? Give feedback.
-
Agreed 100%. |
Beta Was this translation helpful? Give feedback.
-
|
Having a tags column that doesn't look super strange on a SELECT * is hard at this point. I want to punt on it. I just want to point out that is equivalent to which in many ways is more "Standard SQL" and is optimized in the same way. From my point of view the choices we have is either
My preference is between 1 and 2 and NOT 3. probably leaning towards 2 but not strongly. |
Beta Was this translation helpful? Give feedback.
-
|
I lean pretty heavily towards option 2 at the moment |
Beta Was this translation helpful? Give feedback.
-
|
Why do you prefer 2? |
Beta Was this translation helpful? Give feedback.
-
|
I can expound more in the morning, but it's quite a bit less code to maintain, feels more natural, and is more composable/flexible |
Beta Was this translation helpful? Give feedback.
-
|
If I focus on the user experience, querying for tags becomes super tedious in my opinion with a lot of repeating code that looks anything but simple. Maybe heavy users of SQL databases would be ok with it but I doubt the large majority of users will prefer it. We should definitely test it and I am with us adding it later assuming this would just be building on top of what we would have versus requiring data migration. |
Beta Was this translation helpful? Give feedback.
-
|
My instinct is actually to lean towards option I will say though, option |
Beta Was this translation helpful? Give feedback.
-
|
Ok one more option. Along the lines of the view option I had earlier where each tag key becomes it's own column. What if we had one view per service/operation combo that had all keys as columns. That way we'd get:
The only thing is people would need to know which service/operation they are looking at. We'd need to track which keys occur with which operation but that should be easy and cheap. We'd also track that for event and log entries too but that should be similarly easy. This would allow views for event/logs on specific operations too. |
Beta Was this translation helpful? Give feedback.
-
|
so it would look like (for the checkout service, addcart operation): |
Beta Was this translation helpful? Give feedback.
-
|
1 potential downside: @ramonguiu you didn't like previous view ideas like this because you felt people wouldn't know which one to look at. Do you thin people would know if it was serviice/operation pair? |
Beta Was this translation helpful? Give feedback.
-
|
This would have to be in addition to the span table interface described at the top-level since some analysis needs to cross operation boundaries (e.g. which services talk to each other, their rate etc.) On the other hand I'd assume tags are a lot more important for individual operations and cross-operation analysis uses tags less (because not all things will have the same tags) |
Beta Was this translation helpful? Give feedback.
-
|
This idea for views makes more sense to me. There are usually two types of queries:
The challenge is that queries for 1 and 2 would look different and so users would need to learn multiple ways to query the data and they will still need the "non-pure SQL" queries for 1, so they need to learn those anyway. |
Beta Was this translation helpful? Give feedback.
-
|
@cevian I definitely like the example of having a I do also think that being able to query for all of the operations per service is important as well (maybe also just for discoverability), as there have been times on the Cloud team where we change the name of an operation, and not everyone will recall exactly the name of the operations. So being able to query first, and then drill into a specific |
Beta Was this translation helpful? Give feedback.
-
OTLP describes attributes which are key value pairs used to decorate various entities in the schema including span, resources, events, and links. In promscale, we chose to refer to these as “tags” rather than attributes as this is a more familiar and easily typed term.
Tags are managed in the database via several tables, types, and operators.
_ps_trace.tag_keytable maintains a distinct list of all the tag keys and provides a surrogate id._ps_trace.tagtable contains a row for every tag key value pair ingested. The pairs are given a surrogate id.ps_trace.tag_mapdomain defines a jsonb object in which keys are ids pointing to the_ps_trace.tag_keytable and values are ids pointing to the_ps_trace.tagtable. This type should be considered opaque for general usage; operators are defined to make interacting with values of this type easy. The span, link, and event tables have columns of this type to capture the relationship between these entities and the tags they are decorated with.Operators and functions in the
ps_traceandps_tagschemas are provided to make interacting with tags easier.In the above query, the
spanview is aliased ass. In theWHEREclause, we can refer tosand use the?operator to filter spans by the tags attached to the spans.s ?will consider both span tags and resource tags. The('text7' == 'ee')specifies the filtering criterion, and the parentheses are required.'text7'specifies the tag key to consider.==is a custom operator for use with tags that tests equality.'ee'is the the tag value. Thus, this query only returns tags for which there is a tag named "text7" with a value of "ee" either in the span tags or the resource tags associated with the span.Suppose you wish to view the value of the
text7tag in the results. The->operator can be applied to the table aliassand the tag keytext7to return the jsonb value of the tag. Note that the double quotes surrounding theees are due to the fact the the values are of jsonb type rather than text. To return the value as a text type, use the->>operator as below. These operators mimic the built-in jsonb operators.We noted that the
?operator applied to the span view considers both the span tags and resource tags. The same is true of the->and->>operators; they return the value of the tag if it exists either in the span tags or resource tags (span tags get precedence if the tag exists in both). In many cases, one likely will not care whether a tag is specifically a span tag or resource tag -- just that the tag is associated with the span in some way. However, in the case when the distinction matters, one can be more specific as shown in the queries below. Thevalandval_textfunctions can be used on the individual columns and work like the->and->>operators respectively. Similarly, the?operator can be used to filter against the individual columns too.Tag values support more than just text. Tag values can be booleans, numbers, text, etc. Because they are stored as jsonb values in the database, any value supported by jsonb can be used, however in practice one should limit values to those also supported by the OLTP specification for Attributes.
Furthermore, in some cases one does not want to filter by tag values but by the existence of a tag -- i.e. one wants to return any spans that have a
date13tag associated with them regardless of the value of thedate13tag. The?operator supports this usage pattern as well.The below query illustrates how one would find all spans with a
date13tag association and "rendering" the tag value as adatedatatype.The following query similarly finds spans with an
int42tag and renders the tag as aninttype.To be clear, you may use the capabilities on multiple tag keys at once in the same query. This is fully supported. The below query returns only those spans which have both an
int42tag AND abool5tag.The
==operator "understands" many data types that can be used on the right hand side and can apply them correctly to the tag value (which is stored as jsonb). In the below query, spans which have anint42tag AND abool5tag with afalsevalue are returned.The following operators are defined for use in filtering tag values:
==equals!==not equals#<less than#<=less than or equal#>greater than#>=greater than or equal==~regexp matches!=~regexp not matches@?jsonb path existsThe query below filters integer values using the
#>and#<=operators. It also demonstrates that the filter is considering both the span tags and resource tags.This query uses the
==~operator to find tag values that match a regular expression. The query also uses the#operator to return the tag id corresponding the value of thedate24tag.You can even use the
@?operator to filter tag values by a jsonpath query to do more powerful evaluations/computations.Beta Was this translation helpful? Give feedback.
All reactions