EnvoyProxy Rate Limit Not Working in Istio 1.21.1

[NamanGajjar26]

Is this the right place to submit this?

• This is not a security vulnerability or a crashing bug
• This is not a question about how to use Istio

Bug Description

I am using istio 1.21.1 version and use minikube to deploy my project , so i try to set rate limit on asp .net core api using Enabling Rate Limits using Envoy (Global rate limit)
this is below api yaml

`apiVersion: apps/v1
kind: Deployment
metadata:
  name: demoapidock-deployment
  namespace: istioapi
  labels:
    app: demoapidock
    version: v1
spec:
  selector:
    matchLabels:
      app: demoapidock
  template:
    metadata:
      labels:
        app: demoapidock
    spec:
      containers:
        - name: demoapidock
          image: demoapidock:latest
          imagePullPolicy: Never
          ports:
            - containerPort: 5028      

---
apiVersion: v1
kind: Service
metadata:
  name: demoapidock-service
  namespace: istioapi
  labels:
    app: demoapidock
    version: v1
spec:
  selector:
    app: demoapidock
  ports:
    - protocol: TCP
      port: 5028
      targetPort: 5028
  type: NodePort
---
apiVersion: v1
kind: ConfigMap
metadata:
  name: ratelimit-config
  namespace: istio-system
data:
  config.yaml: |
    domain: api-ratelimit
    descriptors:
    - key: ratelimitheader
        descriptors:
        - key: PATH
            value: "/api/CustomerControllers"
            rate_limit:
          unit: minute
          requests_per_unit: 2
---
apiVersion: networking.istio.io/v1alpha3
kind: EnvoyFilter
metadata:
  name: httpbin-global-ratelimit
  namespace: istioapi
spec:
  workloadSelector:
    # select by label in the same namespace
    labels:
      app: demoapidock
  configPatches:
    # The Envoy config you want to modify
    - applyTo: HTTP_FILTER
      match:
        context: SIDECAR_INBOUND
        listener:
          filterChain:
            filter:
              name: "envoy.filters.network.http_connection_manager"
              subFilter:
                name: "envoy.filters.http.router"
      patch:
        operation: INSERT_BEFORE
        # Adds the Envoy Rate Limit Filter in HTTP filter chain.
        value:
          name: envoy.filters.http.ratelimit
          typed_config:
            "@type": type.googleapis.com/envoy.extensions.filters.http.ratelimit.v3.RateLimit
            # domain can be anything! Match it to the ratelimter service config
            domain: api-ratelimit
            failure_mode_deny: true
            timeout: 10s
            rate_limit_service:
              grpc_service:
                envoy_grpc:
                  cluster_name: outbound|8081||ratelimit.istio-system.svc.cluster.local
                  authority: ratelimit.istio-system.svc.cluster.local
              transport_api_version: V3
---
apiVersion: networking.istio.io/v1alpha3
kind: EnvoyFilter
metadata:
  name: filter-ratelimit-svc
  namespace: istioapi
spec:
  workloadSelector:
    labels:
      app: httpbin
  configPatches:
    - applyTo: VIRTUAL_HOST
      match:
        context: SIDECAR_INBOUND
        routeConfiguration:
          vhost:
            name: ""
            route:
              action: ANY
      patch:
        operation: MERGE
        # Applies the rate limit rules.
        value:
          rate_limits:
            - actions: # any actions in here
              - request_headers:
                  header_name: "x-rate-limit-please"
                  descriptor_key: "ratelimitheader"
                  skip_if_absent: true
              - request_headers:
                  header_name: ":path"
                  descriptor_key: "PATH"
`    

upper yaml use and set istio rate limit but this not set one minute in more than 2 time get data and api call from asp .net core mvc

Version

istio version 1.21.1 in ubuntu
minikube version: v1.32.0
Client Version: v1.28.3
Kustomize Version: v5.0.4-0.20230601165947-6ce0bf390ce3
Server Version: v1.28.3

Additional Information

No response

Affected product area

• Ambient
• Docs
• Dual Stack
• Installation
• Networking
• Performance and Scalability
• Extensions and Telemetry
• Security
• Test and Release
• User Experience
• Developer Infrastructure
• Upgrade
• Multi Cluster
• Virtual Machine
• Control Plane Revisions

[zirain]

what's the real app you want limit? httpbin or demoapidock?

[NamanGajjar26]

demoapidock

and alos change here
app: httpbin // demoapidock

but stil not working

[zirain]

demoapidock

and alos change here app: httpbin // demoapidock

the filter-ratelimit-svc envoyfilter seems apply to httpbin?
I recall I have added some log for ratelimit filter, you're tring to enable it by running istioctl pc log <pod_name>.<pod_namespace> --level filter:debug

[NamanGajjar26]

pod/demoapidock-deployment-576d455d5c-jmw7z 2/2 Running 2 (124m ago) 141m

NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/demoapidock-service NodePort 10.99.47.35 5028:31135/TCP 141m

NAME READY UP-TO-DATE AVAILABLE AGE
deployment.apps/demoapi-d 1/1 1 1 3d17h
deployment.apps/demoapidock-deployment 1/1 1 1 141m

NAME DESIRED CURRENT READY AGE
replicaset.apps/demoapidock-deployment-576d455d5c 1 1 1 141m

this is my upper api pod and call mvc in like this

      `client.BaseAddress = new Uri("http://demoapidock-service.istioapi.svc.cluster.local:5028/api/");
        //HTTP GET
        HttpResponseMessage res = await client.GetAsync("CustomerControllers");
        
       if(res.IsSuccessStatusCode)
       {
            var data = res.Content.ReadAsStringAsync().Result;
            cust= JsonConvert.DeserializeObject<List<Customer>>(data);
       }`

i want to set when mvc method in call api to set api on rate limit using istio

[NamanGajjar26]

i am also change yaml

`apiVersion: v1
kind: ConfigMap
metadata:
  name: ratelimit-config
  namespace: istio-system
data:
  config.yaml: |
    domain: api-ratelimit
    descriptors:
    - key: ratelimitheader
        descriptors:
        - key: PATH
            value: "/api/CustomerControllers"
            rate_limit:
          unit: minute
          requests_per_unit: 2
---
apiVersion: networking.istio.io/v1alpha3
kind: EnvoyFilter
metadata:
  name: demoapidock-global-ratelimit
  namespace: istioapi
spec:
  workloadSelector:
    # select by label in the same namespace
    labels:
      app: demoapidock
  configPatches:
    # The Envoy config you want to modify
    - applyTo: HTTP_FILTER
      match:
        context: SIDECAR_INBOUND
        listener:
          filterChain:
            filter:
              name: "envoy.filters.network.http_connection_manager"
              subFilter:
                name: "envoy.filters.http.router"
      patch:
        operation: INSERT_BEFORE
        # Adds the Envoy Rate Limit Filter in HTTP filter chain.
        value:
          name: envoy.filters.http.ratelimit
          typed_config:
            "@type": type.googleapis.com/envoy.extensions.filters.http.ratelimit.v3.RateLimit
            # domain can be anything! Match it to the ratelimter service config
            domain: api-ratelimit
            failure_mode_deny: true
            timeout: 10s
            rate_limit_service:
              grpc_service:
                envoy_grpc:
                  cluster_name: outbound|8081||ratelimit.istio-system.svc.cluster.local
                  authority: ratelimit.istio-system.svc.cluster.local
              transport_api_version: V3
---
apiVersion: networking.istio.io/v1alpha3
kind: EnvoyFilter
metadata:
  name: filter-ratelimit-svc
  namespace: istioapi
spec:
  workloadSelector:
    labels:
      app: demoapidock
  configPatches:
    - applyTo: VIRTUAL_HOST
      match:
        context: SIDECAR_INBOUND
        routeConfiguration:
          vhost:
            name: ""
            route:
              action: ANY
      patch:
        operation: MERGE
        # Applies the rate limit rules.
        value:
          rate_limits:
            - actions: # any actions in here
              - request_headers:
                  header_name: "x-rate-limit-please"
                  descriptor_key: "ratelimitheader"
                  skip_if_absent: true
              - request_headers:
                  header_name: ":path"
                  descriptor_key: "PATH"`

[NamanGajjar26]

any update for me!

[zirain]

I do believe the following configuration is not right:

apiVersion: v1
kind: ConfigMap
metadata:
  name: ratelimit-config
  namespace: istio-system
data:
  config.yaml: |
    domain: api-ratelimit
    descriptors:
    - key: ratelimitheader
        descriptors:
        - key: PATH
            value: "/api/CustomerControllers"
            rate_limit:
          unit: minute
          requests_per_unit: 2

please check your ratelimit service log: kubectl logs -n istio-system -l app=ratelimit --tail=-1

[NamanGajjar26]

No resources found in istio-system namespace.

[zirain]

you point to a service in istio-system namespace(outbound|8081||ratelimit.istio-system.svc.cluster.local), and you cannot find pod/logs?

[NamanGajjar26]

there are not show in all pod in this config
minikube kubectl -- get all -n istio-system
pod/grafana-6f68dfd8f4-j4p4j 1/1 Running 16 (9m3s ago) 9d
pod/istio-egressgateway-6dc45c59fb-g4vjk 1/1 Running 16 (9m3s ago) 9d
pod/istio-ingressgateway-6875bb754b-vwrdr 1/1 Running 16 (9m3s ago) 9d
pod/istiod-87c4f5665-vjlfr 1/1 Running 16 (9m3s ago) 9d
pod/jaeger-7d7d59b9d-wfwsc 1/1 Running 20 (9m3s ago) 9d
pod/kiali-588bc98cd-dxz24 1/1 Running 18 (9m3s ago) 9d
pod/prometheus-7545dd48db-zpwll 2/2 Running 235 (5m18s ago) 9d

there are show only this pods

[NamanGajjar26]

there are solved this problem
get the log
minikube kubectl -- logs -l app=ratelimit --tail=-1 -n istioapi <-- command use

time="2024-04-22T06:39:49Z" level=warning msg="statsd is not in use"
time="2024-04-22T06:39:49Z" level=info msg="Tracing disabled"
time="2024-04-22T06:39:49Z" level=debug msg="runtime changed. loading new snapshot at /data/ratelimit/config"
time="2024-04-22T06:39:49Z" level=debug msg="runtime: processing /data/ratelimit/config"
time="2024-04-22T06:39:49Z" level=debug msg="runtime: processing /data/ratelimit/config/..2024_04_22_06_39_30.2113015249"
time="2024-04-22T06:39:49Z" level=debug msg="runtime: processing /data/ratelimit/config/..data"
time="2024-04-22T06:39:49Z" level=debug msg="runtime: processing /data/ratelimit/config/config.yaml"
time="2024-04-22T06:39:49Z" level=debug msg="runtime: adding key=config.yaml value=domain: api-ratelimit\ndescriptors:\n- key: ratelimitheader\n descriptors:\n - key: PATH\n value: "/api/CustomerControllers"\n rate_limit:\n unit: minute\n requests_per_unit: 1\n uint=false"
time="2024-04-22T06:39:49Z" level=warning msg="connecting to redis on redis:6379 with pool size 10"
time="2024-04-22T06:39:49Z" level=debug msg="Implicit pipelining enabled: false"
time="2024-04-22T06:39:49Z" level=debug msg="error loading config file: yaml: line 4: mapping values are not allowed in this context"
time="2024-04-22T06:39:49Z" level=error msg="error loading new configuration from runtime: config.yaml: error loading config file: yaml: line 4: mapping values are not allowed in this context"
time="2024-04-22T06:39:49Z" level=warning msg="Listening for HTTP on '[::]:8080'"
time="2024-04-22T06:39:49Z" level=debug msg="waiting for runtime update"
time="2024-04-22T06:39:49Z" level=warning msg="Listening for debug on '0.0.0.0:6070'"
time="2024-04-22T06:39:49Z" level=warning msg="Listening for gRPC on '[::]:8081'"

[zirain]

I don't which your tutorial you're following, for what I can see it's a mess what you show in the issue.

• ratelimit installed in namespace istioapi, but the envoyfilter still point to outbound|8081||ratelimit.istio-system.svc.cluster.local
• form the logs show above, error loading config file: yaml: line 4: mapping values are not allowed in this context the configuration is invalid.

this's unlike an real issue.