envoy fails to load as system daemon trying to access /dev/stdout

[coolaj86]

The example config file specifies /dev/stdout for logging, which is a pseudo-device that only exists when run interactively.

Expected Behavior

The example config file should use a more generic output specifier for logs that works for interactive and non-interactive processes.

Current Behavior

Mar 07 05:56:07 evo3 envoy[1721]: unable to open file '/dev/stdout': No such device or address
Mar 07 05:56:07 evo3 systemd[1]: envoy.service: Main process exited, code=exited, status=1/FAILURE

Possible Solution

I don't know the envoy config, but I imagine that there's a more generic way to specify the access log that works both interactively and with init systems.

      access_log:
        - name: envoy.access_loggers.file
          typed_config:
            "@type": type.googleapis.com/envoy.extensions.access_loggers.file.v3.FileAccessLog
            path: /dev/stdout

Update: Well, if I'm reading the github issues correctly, the envoy team are just being total donks about only supporting file pathnames and not supporting stdout.
(It's hard to believe that that's really the case but then again, the refusal of project maintainers to work with standards that have existed before they were born never ceases to surprise me...)

If that's really the case, then the only solution is to run it in a subshell with the pipefail option:

- ExecStart=/home/app/.local/bin/envoy --config-path /home/app/.config/envoy/config.yaml
+ ExecStart=bash -o pipefail -c '/home/app/.local/bin/envoy --config-path /home/app/.config/envoy/config.yaml 2>&1 | tee'

It seems that this is because /dev/stdout is a socket, not a file, but using a shell redirect opens it in such a way that it can be used by the child process.

Steps to Reproduce (for bugs)

~/.config/envoy/config.yaml:

!ignore filters: &filters
  - name: envoy.http_connection_manager
    typed_config:
      "@type": type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager
      stat_prefix: ingress_http
      codec_type: auto
      access_log:
        - name: envoy.access_loggers.file
          typed_config:
            "@type": type.googleapis.com/envoy.extensions.access_loggers.file.v3.FileAccessLog
            path: /dev/stdout
            log_format:
              json_format:
                timestamp: "%START_TIME%"
                client: "%DOWNSTREAM_REMOTE_ADDRESS_WITHOUT_PORT%"
                protocol: "%PROTOCOL%"
                method: "%REQ(:METHOD)%"
                uri: "%REQ(X-ENVOY-ORIGINAL-PATH?:PATH)%"
                upstream: "%UPSTREAM_HOST%"
                "http-status": "%RESPONSE_CODE%"
                "grpc-status": "%GRPC_STATUS%"
                "rx-bytes": "%BYTES_RECEIVED%"
                "tx-bytes": "%BYTES_SENT%"
                "response-flags": "%RESPONSE_FLAGS%"
                duration: "%DURATION%"
                authority: "%REQ(:AUTHORITY)%"
      http_filters:
        - name: envoy.filters.http.local_ratelimit
          typed_config:
            "@type": type.googleapis.com/envoy.extensions.filters.http.local_ratelimit.v3.LocalRateLimit
            stat_prefix: http_local_rate_limiter
            # see documentation https://www.envoyproxy.io/docs/envoy/latest/api-v3/type/v3/token_bucket.proto#envoy-v3-api-msg-type-v3-tokenbucket
            token_bucket:
              max_tokens: 300
              tokens_per_fill: 150
              fill_interval: 60s
            filter_enabled:
              runtime_key: local_rate_limit_enabled
              default_value:
                numerator: 100
                denominator: HUNDRED
            filter_enforced:
              runtime_key: local_rate_limit_enforced
              default_value:
                numerator: 100
                denominator: HUNDRED
            response_headers_to_add:
              - append: false
                header:
                  key: x-local-rate-limit
                  value: "true"
        - name: envoy.filters.http.grpc_web
          typed_config:
            "@type": type.googleapis.com/envoy.extensions.filters.http.grpc_web.v3.GrpcWeb
        - name: envoy.filters.http.cors
          typed_config:
            "@type": type.googleapis.com/envoy.extensions.filters.http.cors.v3.Cors
        - name: envoy.filters.http.router
          typed_config:
            "@type": type.googleapis.com/envoy.extensions.filters.http.router.v3.Router
      route_config:
        name: local_route
        virtual_hosts:
          - name: dapi_services
            domains: ["*"]
            routes:
              # tx subscription streaming endpoint configuration
              - match:
                  prefix: "/org.dash.platform.dapi.v0.Core/subscribeToTransactionsWithProofs"
                route:
                  cluster: tx_filter_stream
                  timeout: 660s
                  max_stream_duration:
                    grpc_timeout_header_max: 600s
              # block headers subscription streaming endpoint
              - match:
                  prefix: "/org.dash.platform.dapi.v0.Core/subscribeToBlockHeadersWithChainLocks"
                route:
                  cluster: tx_filter_stream
                  timeout: 660s
                  max_stream_duration:
                    grpc_timeout_header_max: 600s
              # set timeout for waitForStateTransitionResult
              - match:
                  prefix: "/org.dash.platform.dapi.v0.Platform/waitForStateTransitionResult"
                route:
                  cluster: core_and_platform
                  timeout: 80s
              # core unary endpoints
              - match:
                  prefix: "/org.dash.platform.dapi.v0.Core"
                route:
                  cluster: core_and_platform
                  timeout: 15s
              # platform unary endpoints
              - match:
                  prefix: "/org.dash.platform.dapi.v0.Platform"
                route:
                  cluster: core_and_platform
                  timeout: 15s
              # configuration of the static responses of unsupported api versions
              # core static response
              - match:
                  safe_regex:
                    google_re2: {}
                    regex: "\/org\\.dash\\.platform\\.dapi\\.v[1-9]+\\.Core"
                response_headers_to_add:
                  - header:
                      key: "Content-Type"
                      value: "application/grpc-web+proto"
                  - header:
                      key: "grpc-status"
                      value: "12"
                  - header:
                      key: "grpc-message"
                      value: "Specified service version is not supported"
                direct_response:
                  status: 204
              # platform static response
              - match:
                  safe_regex:
                    google_re2: {}
                    regex: "\/org\\.dash\\.platform\\.dapi\\.v[1-9]+\\.Platform"
                response_headers_to_add:
                  - header:
                      key: "Content-Type"
                      value: "application/grpc-web+proto"
                  - header:
                      key: "grpc-status"
                      value: "12"
                  - header:
                      key: "grpc-message"
                      value: "Specified service version is not supported"
                direct_response:
                  status: 204
              # JSON RPC endpoints
              - match:
                  path: "/"
                route:
                  cluster: json_rpc
            cors:
              allow_origin_string_match:
                - prefix: "*"
              allow_methods: GET, PUT, DELETE, POST, OPTIONS
              allow_headers: keep-alive,user-agent,cache-control,content-type,content-transfer-encoding,custom-header-1,x-accept-content-transfer-encoding,x-accept-response-streaming,x-user-agent,x-grpc-web,grpc-timeout
              max_age: "1728000"
              expose_headers: custom-header-1,grpc-status,grpc-message

static_resources:
  listeners:
    - name: grpc_and_json_rpc
      address:
        socket_address:
          address: 10.11.5.111
          #port_value: 10000
          port_value: 3080
      filter_chains:
        - filters: *filters
  clusters:
    - name: core_and_platform
      connect_timeout: 5s
      type: logical_dns
      lb_policy: round_robin
      typed_extension_protocol_options:
        envoy.extensions.upstreams.http.v3.HttpProtocolOptions:
          "@type": type.googleapis.com/envoy.extensions.upstreams.http.v3.HttpProtocolOptions
          explicit_http_config:
            http2_protocol_options:
              connection_keepalive:
                interval: 30s
                timeout: 5s
      load_assignment:
        cluster_name: core_and_platform
        endpoints:
          - lb_endpoints:
              - endpoint:
                  address:
                    socket_address:
                      address: 127.0.0.1
                      #address: dapi_api
                      port_value: 2501
    - name: tx_filter_stream
      connect_timeout: 5s
      type: logical_dns
      lb_policy: round_robin
      typed_extension_protocol_options:
        envoy.extensions.upstreams.http.v3.HttpProtocolOptions:
          "@type": type.googleapis.com/envoy.extensions.upstreams.http.v3.HttpProtocolOptions
          explicit_http_config:
            http2_protocol_options:
              connection_keepalive:
                interval: 30s
                timeout: 5s
      load_assignment:
        cluster_name: tx_filter_stream
        endpoints:
          - lb_endpoints:
              - endpoint:
                  address:
                    socket_address:
                      address: 127.0.0.1
                      #address: dapi_tx_filter_stream
                      port_value: 2510
    - name: json_rpc
      connect_timeout: 5s
      type: logical_dns
      lb_policy: round_robin
      load_assignment:
        cluster_name: json_rpc
        endpoints:
          - lb_endpoints:
              - endpoint:
                  address:
                    socket_address:
                      address: 127.0.0.1
                      #address: dapi_api
                      port_value: 2501

admin:
  address:
    socket_address:
      address: 10.11.5.111
      port_value: 8081

/etc/systemd/system/envoy.service:

# Generated for serviceman. Edit as you wish, but leave this line.
# sudo systemctl daemon-reload
# sudo systemctl restart envoy.service
# sudo journalctl -xefu envoy

[Unit]
Description=envoy
After=network-online.target
Wants=network-online.target systemd-networkd-wait-online.service

[Service]
# Restart on crash (bad signal), but not on 'clean' failure (error exit code)
# Allow up to 3 restarts within 10 seconds
# (it's unlikely that a user or properly-running script will do this)
Restart=always
StartLimitInterval=10
StartLimitBurst=3

# User and group the process will run as
User=app
Group=app

WorkingDirectory=/home/app/.config/envoy
ExecStart=/home/app/.local/bin/envoy --config-path /home/app/.config/envoy/config.yaml
ExecReload=/bin/kill -USR1 $MAINPID

[Install]
WantedBy=multi-user.target

Context

Trying to install envoy in a way that in runs on boot without becoming an envoy expert.

The FileAccessLog documentation doesn't mention the path config:
https://www.envoyproxy.io/docs/envoy/latest/configuration/observability/access_log/usage#configuration

See also:

• Access logging to stdout envoyproxy/envoy#1291 (comment)
• How do I make envoy log to the systemd journal other than using /dev/stdout? envoyproxy/envoy#8297 (comment)

Your Environment

Ubuntu 22.04 LTS

[shumkov]

Hello! Thank you for the report. This config is not meant to be working with systemd. It's a part of dashmate that uses docker to run services.