Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

HTTP Metrics Path Matching #7746

Open
wants to merge 34 commits into
base: master
Choose a base branch
from

Conversation

nelson-parente
Copy link

@nelson-parente nelson-parente commented May 20, 2024

Description

This PR implements the proposal dapr/proposals#58.

spec:
  metric:
    enabled: true
    http:
      increasedCardinality: true
      pathMatching:
        ingress:
        - /orders/{orderID}/items/{itemID}
        - /users/{userID}
        egress:
        - /orders
        - /orders/{orderID}/items/{itemID}

PathMatching configuration is opt-in and ensures consistent representation of HTTP request paths in server metrics.

Issue reference

HTTP Metrics Path Matching Proposal: dapr/proposals#58
Dapr Low Cardinality Issue with initial motivation for the proposal: #7719

Please reference the issue this PR will close: #[issue number]

Checklist

Please make sure you've completed the relevant tasks for this PR, out of the following list:

Signed-off-by: nelson.parente <nelson_parente@live.com.pt>
@nelson-parente nelson-parente force-pushed the feat/http-metrics-path-normalization branch from 80e852f to 561b9e8 Compare May 20, 2024 10:31
Signed-off-by: nelson.parente <nelson_parente@live.com.pt>
Signed-off-by: nelson.parente <nelson_parente@live.com.pt>
Signed-off-by: nelson.parente <nelson_parente@live.com.pt>
Signed-off-by: nelson.parente <nelson_parente@live.com.pt>
Signed-off-by: nelson.parente <nelson_parente@live.com.pt>
Signed-off-by: nelson.parente <nelson_parente@live.com.pt>
Signed-off-by: nelson.parente <nelson_parente@live.com.pt>
@nelson-parente nelson-parente changed the title [WIP] HTTP Metrics Path Normalization HTTP Metrics Path Normalization May 21, 2024
@nelson-parente nelson-parente marked this pull request as ready for review May 21, 2024 12:35
@nelson-parente nelson-parente requested review from a team as code owners May 21, 2024 12:35
Signed-off-by: nelson.parente <nelson_parente@live.com.pt>
@nelson-parente nelson-parente force-pushed the feat/http-metrics-path-normalization branch from 38e25af to 8dc24c6 Compare May 21, 2024 14:05
Copy link
Contributor

@ItalyPaleAle ItalyPaleAle left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Normally the mainainters would tell you it's a bit premature to do the work before the proposal is even reviewed...!

pkg/apis/configuration/v1alpha1/types.go Outdated Show resolved Hide resolved
pkg/config/configuration.go Outdated Show resolved Hide resolved
pkg/diagnostics/http_monitoring.go Outdated Show resolved Hide resolved
pkg/diagnostics/http_monitoring.go Outdated Show resolved Hide resolved
pkg/diagnostics/http_monitoring.go Outdated Show resolved Hide resolved
@artursouza
Copy link
Member

artursouza commented May 21, 2024

This proposal was reviewed among some runtime maintainers and people got invited to participate in the discussion too. While there is space for discussion, multiple maintainers thought this proposal to be reasonable and low risk to start implementation.

@artursouza
Copy link
Member

@nelson-parente Please, address the optimization suggestions as this is a hot path for Dapr and can cause a performance regression.

Thanks, @ItalyPaleAle

@ItalyPaleAle
Copy link
Contributor

This proposal was reviewed among some runtime maintainers and people got invited to participate in the discussion too. While there is space for discussion, multiple maintainers thought this proposal to be reasonable and low risk to start implementation.

This is interesting as every time I was in @nelson-parente's shoes, the response I got from maintainers was very different. I have not seen voting happening (to my understanding, you will need the majority of 3 votes - 1 person for each company), and only 1 person commented.

@yaron2
Copy link
Member

yaron2 commented May 21, 2024

The PR is still pending proposal approval

…turn

Signed-off-by: nelson.parente <nelson_parente@live.com.pt>
@nelson-parente nelson-parente force-pushed the feat/http-metrics-path-normalization branch from 0e20ef3 to a8f8a91 Compare May 21, 2024 16:50
Signed-off-by: nelson.parente <nelson_parente@live.com.pt>
@nelson-parente nelson-parente force-pushed the feat/http-metrics-path-normalization branch from e1a4830 to f9bfd3a Compare May 21, 2024 17:15
Signed-off-by: nelson.parente <nelson_parente@live.com.pt>
Signed-off-by: nelson.parente <nelson_parente@live.com.pt>
Signed-off-by: nelson.parente <nelson_parente@live.com.pt>
Signed-off-by: nelson.parente <nelson_parente@live.com.pt>
@nelson-parente nelson-parente force-pushed the feat/http-metrics-path-normalization branch from e750b0f to 7ef2c20 Compare May 24, 2024 18:18
Signed-off-by: nelson.parente <nelson_parente@live.com.pt>
Signed-off-by: nelson.parente <nelson_parente@live.com.pt>
Signed-off-by: nelson.parente <nelson_parente@live.com.pt>
@nelson-parente nelson-parente changed the title HTTP Metrics Path Normalization HTTP Metrics Path Matching May 28, 2024
@mikeee mikeee mentioned this pull request May 28, 2024
46 tasks
pkg/apis/configuration/v1alpha1/types.go Outdated Show resolved Hide resolved
pkg/config/configuration_test.go Outdated Show resolved Hide resolved
pkg/diagnostics/http_monitoring.go Outdated Show resolved Hide resolved
pkg/diagnostics/http_monitoring.go Outdated Show resolved Hide resolved
pkg/diagnostics/http_monitoring.go Outdated Show resolved Hide resolved
pkg/diagnostics/http_monitoring.go Outdated Show resolved Hide resolved
pkg/diagnostics/http_monitoring.go Outdated Show resolved Hide resolved
@nelson-parente nelson-parente force-pushed the feat/http-metrics-path-normalization branch from 7e36a31 to 7bbdafd Compare May 29, 2024 15:37
Signed-off-by: nelson.parente <nelson_parente@live.com.pt>
@nelson-parente nelson-parente force-pushed the feat/http-metrics-path-normalization branch from 7bbdafd to 2a2b4ef Compare May 29, 2024 15:40
Signed-off-by: nelson.parente <nelson_parente@live.com.pt>
Signed-off-by: nelson.parente <nelson_parente@live.com.pt>
Copy link
Contributor

@JoshVanL JoshVanL left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @nelson-parente, couple more from me 🙂

pkg/diagnostics/http_monitoring_path_matching.go Outdated Show resolved Hide resolved
pkg/diagnostics/http_monitoring_path_matching.go Outdated Show resolved Hide resolved
Comment on lines 53 to 64
if !catchAllRegistered {
mux.Handle("/", http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
if !legacy {
rw, ok := w.(*pathMatchingRW)
if !ok {
log.Errorf("Failed to cast to PathMatchingRW")
return
}
rw.MatchedPath = unmatchedPathPlaceholder
}
}))
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rather than have 2 HandlerFunc registrations, we can optionally append "/" if in non-legacy mode and remove dupes in the paths slice.

if !legacy {
  paths = append(paths, "/")
  slices.Sort(paths)
  paths = slices.Compact(paths)
}
Suggested change
if !catchAllRegistered {
mux.Handle("/", http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
if !legacy {
rw, ok := w.(*pathMatchingRW)
if !ok {
log.Errorf("Failed to cast to PathMatchingRW")
return
}
rw.MatchedPath = unmatchedPathPlaceholder
}
}))
}

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we need the 2 handlers registration because we want to distinguish when the user explicitly sets the "/" path or not.

When the user sets the "/" path we match it and use it otherwise if the user does not set a "/" we fallback to the unmatched placeholder "_" so we need to have different handler funcs for those cases.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done! we always need to register the catch all otherwise serveHTTP will throw an error when it doesn't match the path.

pkg/diagnostics/http_monitoring_path_matching.go Outdated Show resolved Hide resolved
Comment on lines +84 to +85
if !strings.HasPrefix(path, "/") {
path = "/" + path
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is this required?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Paths not starting / are invalid and the request will return an error on the handle function.

I noticed this when testing and we had errors with dapr/configpath.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Update: all invoke method paths are coming without the initial slash. I fixed the dapr/config but all these others are using the *invokev1.InvokeMethodRequest and I don't really know the why these don't have the initial slash.

pkg/diagnostics/http_monitoring_path_matching.go Outdated Show resolved Hide resolved
pkg/diagnostics/http_monitoring_path_matching.go Outdated Show resolved Hide resolved
pkg/diagnostics/http_monitoring.go Outdated Show resolved Hide resolved
Signed-off-by: nelson.parente <nelson_parente@live.com.pt>
Signed-off-by: nelson.parente <nelson_parente@live.com.pt>
Signed-off-by: nelson.parente <nelson_parente@live.com.pt>
Signed-off-by: nelson.parente <nelson_parente@live.com.pt>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

6 participants