Skip to content

Commit

Permalink
add uninstrumented backend
Browse files Browse the repository at this point in the history
Signed-off-by: Benedikt Bongartz <bongartz@klimlive.de>
  • Loading branch information
frzifus committed Mar 17, 2024
1 parent bbc9000 commit b99a9ef
Show file tree
Hide file tree
Showing 6 changed files with 191 additions and 3 deletions.
4 changes: 2 additions & 2 deletions 04-manual-instrumentation.md
Expand Up @@ -2,7 +2,7 @@

This tutorial section covers the manual instrumentation of a go application with the opentelemetry-sdk.

As a basis for the instrumentation we use [backend4](./app/backend4/main.go). To compile the application you need [go 1.22 or newer](https://go.dev/doc/install).
As a basis for the instrumentation we use [backend4-uninstrumented](./app/backend4-no-instrumentation/main.go) ([backend4-instrumented](./app/backend4/main.go)). To compile the application you need [go 1.22 or newer](https://go.dev/doc/install).

# Initialize OpenTelemetry-go-sdk

Expand Down Expand Up @@ -132,7 +132,7 @@ spec:
containers:
- name: backend2
- image: ghcr.io/pavolloffay/kubecon-eu-2024-opentelemetry-kubernetes-tracing-tutorial-backend2:latest
+ image: ghcr.io/frzifus/kubecon-eu-2024-opentelemetry-kubernetes-tracing-tutorial-backend4:with-instr
+ image: ghcr.io/pavolloffay/kubecon-eu-2024-opentelemetry-kubernetes-tracing-tutorial-backend4:latest
env:
+ - name: RATE_ERROR
+ value: 20
Expand Down
24 changes: 23 additions & 1 deletion 05-sampling.md
Expand Up @@ -12,7 +12,29 @@ In chapter 3 we saw the [schematic structure of the dice game application](https

Sampling refers to the practice of selectively capturing and recording traces of requests flowing through a distributed system, rather than capturing every single request. It is crucial in distributed tracing systems because modern distributed applications often generate a massive volume of requests and transactions, which can overwhelm the tracing infrastructure or lead to excessive storage costs if every request is traced in detail.

For example, a medium sized setup producing ~1M traces per minute can result in a cost of approximately $250,000 per month. (Note that this depends on your infrastructure costs, the SaaS provider you choose, the amount of metadata, etc.)
For example, a medium sized setup producing ~1M traces per minute can result in a cost of approximately $250,000 per month. (Note that this depends on your infrastructure costs, the SaaS provider you choose, the amount of metadata, etc.) You may want to check some service costs to get a better idea.

Pricing:
- AWS Xray ([calculator](https://aws.amazon.com/xray/pricing/))
- GCP Cloud Trace [pricing](https://cloud.google.com/stackdriver/pricing#trace-costs)

```
GCP
Feature Price Free allotment per month Effective date
Trace ingestion $0.20/million spans First 2.5 million spans November 1, 2018
---
X-Ray Tracing
Traces recorded cost $5.00 per 1 million traces recorded ($0.000005 per trace).
Traces retrieved cost $0.50 per 1 million traces retrieved ($0.0000005 per trace).
Traces scanned cost $0.50 per 1 million traces scanned ($0.0000005 per trace).
X-Ray Insights traces stored costs $1.00 per million traces recorded ($0.000001 per trace).
```

For more details, check the [offical documentation](https://opentelemetry.io/docs/concepts/sampling/).

Expand Down
15 changes: 15 additions & 0 deletions app/backend4-no-instrumentation/Dockerfile
@@ -0,0 +1,15 @@
FROM golang:1.22 as builder

WORKDIR /src

COPY . /src

RUN CGO_ENABLED=0 go build -v -o /app

FROM scratch

COPY --from=0 /app /app

EXPOSE 8080

CMD ["/app"]
15 changes: 15 additions & 0 deletions app/backend4-no-instrumentation/go.mod
@@ -0,0 +1,15 @@
module github.com/kubecon-eu-2024/backend

go 1.22.0

require github.com/prometheus/client_golang v1.19.0

require (
github.com/beorn7/perks v1.0.1 // indirect
github.com/cespare/xxhash/v2 v2.2.0 // indirect
github.com/prometheus/client_model v0.5.0 // indirect
github.com/prometheus/common v0.48.0 // indirect
github.com/prometheus/procfs v0.12.0 // indirect
golang.org/x/sys v0.17.0 // indirect
google.golang.org/protobuf v1.32.0 // indirect
)
20 changes: 20 additions & 0 deletions app/backend4-no-instrumentation/go.sum
@@ -0,0 +1,20 @@
github.com/beorn7/perks v1.0.1 h1:VlbKKnNfV8bJzeqoa4cOKqO6bYr3WgKZxO8Z16+hsOM=
github.com/beorn7/perks v1.0.1/go.mod h1:G2ZrVWU2WbWT9wwq4/hrbKbnv/1ERSJQ0ibhJ6rlkpw=
github.com/cespare/xxhash/v2 v2.2.0 h1:DC2CZ1Ep5Y4k3ZQ899DldepgrayRUGE6BBZ/cd9Cj44=
github.com/cespare/xxhash/v2 v2.2.0/go.mod h1:VGX0DQ3Q6kWi7AoAeZDth3/j3BFtOZR5XLFGgcrjCOs=
github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/google/go-cmp v0.6.0 h1:ofyhxvXcZhMsU5ulbFiLKl/XBFqE1GSq7atu8tAmTRI=
github.com/google/go-cmp v0.6.0/go.mod h1:17dUlkBOakJ0+DkrSSNjCkIjxS6bF9zb3elmeNGIjoY=
github.com/prometheus/client_golang v1.19.0 h1:ygXvpU1AoN1MhdzckN+PyD9QJOSD4x7kmXYlnfbA6JU=
github.com/prometheus/client_golang v1.19.0/go.mod h1:ZRM9uEAypZakd+q/x7+gmsvXdURP+DABIEIjnmDdp+k=
github.com/prometheus/client_model v0.5.0 h1:VQw1hfvPvk3Uv6Qf29VrPF32JB6rtbgI6cYPYQjL0Qw=
github.com/prometheus/client_model v0.5.0/go.mod h1:dTiFglRmd66nLR9Pv9f0mZi7B7fk5Pm3gvsjB5tr+kI=
github.com/prometheus/common v0.48.0 h1:QO8U2CdOzSn1BBsmXJXduaaW+dY/5QLjfB8svtSzKKE=
github.com/prometheus/common v0.48.0/go.mod h1:0/KsvlIEfPQCQ5I2iNSAWKPZziNCvRs5EC6ILDTlAPc=
github.com/prometheus/procfs v0.12.0 h1:jluTpSng7V9hY0O2R9DzzJHYb2xULk9VTR1V1R/k6Bo=
github.com/prometheus/procfs v0.12.0/go.mod h1:pcuDEFsWDnvcgNzo4EEweacyhjeA9Zk3cnaOZAZEfOo=
golang.org/x/sys v0.17.0 h1:25cE3gD+tdBA7lp7QfhuV+rJiE9YXTcS3VG1SqssI/Y=
golang.org/x/sys v0.17.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA=
google.golang.org/protobuf v1.32.0 h1:pPC6BG5ex8PDFnkbrGU3EixyhKcQ2aDuBS36lqK/C7I=
google.golang.org/protobuf v1.32.0/go.mod h1:c6P6GXX6sHbq/GpV6MGZEdwhWPcYBgnhAHhKbcUYpos=
116 changes: 116 additions & 0 deletions app/backend4-no-instrumentation/main.go
@@ -0,0 +1,116 @@
package main

import (
"context"
"crypto/sha256"
"fmt"
"math/rand"
"net/http"
"os"
"strconv"
"time"

"github.com/prometheus/client_golang/prometheus"
"github.com/prometheus/client_golang/prometheus/promhttp"
)

var (
rollCounter = prometheus.NewCounter(
prometheus.CounterOpts{
Name: "dice_roll_count",
Help: "How often the dice was rolled",
},
)

numbersCounter = prometheus.NewCounterVec(
prometheus.CounterOpts{
Name: "dice_numbers_count",
Help: "How often each number of the dice was rolled",
},
[]string{"number"},
)
)

func init() {
prometheus.MustRegister(rollCounter)
prometheus.MustRegister(numbersCounter)
}

func main() {
v, ok := os.LookupEnv("RATE_ERROR")
if !ok {
v = "0"
}
rateError, err := strconv.Atoi(v)
if err != nil {
panic(err)
}

v, ok = os.LookupEnv("RATE_HIGH_DELAY")
if !ok {
v = "0"
}
rateDelay, err := strconv.Atoi(v)
if err != nil {
panic(err)
}

mux := http.NewServeMux()

mux.HandleFunc("GET /rolldice", func(w http.ResponseWriter, r *http.Request) {
player := "Anonymous player"
if p := r.URL.Query().Get("player"); p != "" {
player = p
}

max := 8
if fmt.Sprintf("%x", sha256.Sum256([]byte(player))) == "f4b7c19317c929d2a34297d6229defe5262fa556ef654b600fc98f02c6d87fdc" {
max = 8
} else {
max = 6
}
result := doRoll(r.Context(), max)
causeDelay(r.Context(), rateDelay)
if err := causeError(r.Context(), rateError); err != nil {
w.WriteHeader(http.StatusInternalServerError)
return
}
resStr := strconv.Itoa(result)
rollCounter.Inc()
numbersCounter.WithLabelValues(resStr).Inc()
if _, err := w.Write([]byte(resStr)); err != nil {
w.WriteHeader(http.StatusInternalServerError)
}

})

mux.HandleFunc("GET /metrics", promhttp.Handler().ServeHTTP)
srv := &http.Server{
Addr: "0.0.0.0:5165",
Handler: mux,
}

if err := srv.ListenAndServe(); err != nil {
panic(err)
}
}

func causeError(ctx context.Context, rate int) error {
randomNumber := rand.Intn(100)
if randomNumber < rate {
err := fmt.Errorf("number(%d)) < rate(%d)", randomNumber, rate)
return err
}
return nil
}

func causeDelay(ctx context.Context, rate int) {
randomNumber := rand.Intn(100)
if randomNumber < rate {
time.Sleep(time.Duration(2+rand.Intn(3)) * time.Second)
}
}

func doRoll(_ context.Context, max int) int {
return rand.Intn(max) + 1
}

0 comments on commit b99a9ef

Please sign in to comment.