Edge-stack-agent is unable to get Ambassador Cloud token when AGENT_CONFIG_RESOURCE_NAME points to a secret.

[aosoriodw]

AES Version: 3.0.0

When the AGENT_CONFIG_RESOURCE_NAME environment variable points to a ConfigMap, edge-stack-agent is able to get the token to connect to Ambassadro Cloud. However, when AGENT_CONFIG_RESOURCE_NAME points to a Secret, AES agent does not get the token, which will break the integration with Ambassador Cloud.

The following log is printed by AES agent when the token is stored in a ConfigMap:

time="2022-07-12 20:47:53.1228" level=info msg="/usr/bin/python3 /ambassador/kubewatch.py --debug failed with exit status 1\n\n" func=github.com/emissary-ingress/emissary/v3/pkg/environment.EnvironmentSetupEntrypoint file="/go/pkg/environment/helper.go:45" CMD=agent PID=1
time="2022-07-12 20:47:53.1234" level=info msg="metrics service listening on :8080" func=github.com/emissary-ingress/emissary/v3/cmd/agent.run file="/go/cmd/agent/main.go:59" CMD=agent PID=1
time="2022-07-12 20:47:53.1247" level=info msg="Agent is running..." func="github.com/emissary-ingress/emissary/v3/pkg/agent.(*Agent).Watch" file="/go/pkg/agent/agent.go:290" CMD=agent PID=1 THREAD=/watch
time="2022-07-12 20:47:53.4365" level=info msg="Setting cloud connect token from configmap" func="github.com/emissary-ingress/emissary/v3/pkg/agent.(*Agent).handleAPIKeyConfigChange" file="/go/pkg/agent/agent.go:264" CMD=agent PID=1 THREAD=/watch
time="2022-07-12 20:47:53.5745" level=info msg="WatchGeneric: Listening for events from resouce \"argoproj.io/v1alpha1, Resource=rollouts\"" func="github.com/emissary-ingress/emissary/v3/pkg/agent.(*DynamicClient).WatchGeneric" file="/go/pkg/agent/k8s.go:138" CMD=agent PID=1 THREAD=/watch
time="2022-07-12 20:47:53.5748" level=info msg="WatchGeneric: Listening for events from resouce \"argoproj.io/v1alpha1, Resource=applications\"" func="github.com/emissary-ingress/emissary/v3/pkg/agent.(*DynamicClient).WatchGeneric" file="/go/pkg/agent/k8s.go:138" CMD=agent PID=1 THREAD=/watch
time="2022-07-12 20:47:53.5749" level=info msg="Beginning to watch and report resources to ambassador cloud" func="github.com/emissary-ingress/emissary/v3/pkg/agent.(*Agent).watch" file="/go/pkg/agent/agent.go:408" CMD=agent PID=1 THREAD=/watch
time="2022-07-12 20:47:53.8178" level=error msg="failed to validate OpenAPI spec: invalid components: unsupported 'format' value \"uuid\"" func=github.com/emissary-ingress/emissary/v3/pkg/agent.newOpenAPI file="/go/pkg/agent/api_docs.go:252" CMD=agent PID=1 THREAD=/watch
time="2022-07-12 20:47:54.0161" level=info msg="Connected to the CEPC Director" func="github.com/emissary-ingress/emissary/v3/pkg/agent.(*BasicDirectiveHandler).HandleDirective" file="/go/pkg/agent/directive_handler.go:47" CMD=agent PID=1 THREAD=/watch directive=1657658874-e974421e-8d4a-4792-827e-4ff0a1c807ce
time="2022-07-12 20:47:55.7512" level=info msg="Received 1307 metric(s)" func="github.com/emissary-ingress/emissary/v3/pkg/agent.(*Agent).MetricsRelayHandler" file="/go/pkg/agent/agent.go:655" CMD=agent PID=1 THREAD="/metrics-server/conn=10.56.8.107:8080"
time="2022-07-12 20:47:55.7514" level=info msg="Relaying 80 metric(s)" func="github.com/emissary-ingress/emissary/v3/pkg/agent.(*Agent).MetricsRelayHandler" file="/go/pkg/agent/agent.go:679" CMD=agent PID=1 THREAD="/metrics-server/conn=10.56.8.107:8080"
time="2022-07-12 20:47:55.7530" level=info msg="Next metrics relay scheduled for 2022-07-12 20:48:25.753060963 +0000 UTC" func="github.com/emissary-ingress/emissary/v3/pkg/agent.(*Agent).MetricsRelayHandler" file="/go/pkg/agent/agent.go:687" CMD=agent PID=1 THREAD="/metrics-server/conn=10.56.8.107:8080"

These are the agent logs when token is in a secret:

time="2022-07-12 20:49:32.2591" level=info msg="/usr/bin/python3 /ambassador/kubewatch.py --debug failed with exit status 1\n\n" func=github.com/emissary-ingress/emissary/v3/pkg/environment.EnvironmentSetupEntrypoint file="/go/pkg/environment/helper.go:45" CMD=agent PID=1
time="2022-07-12 20:49:32.2598" level=info msg="metrics service listening on :8080" func=github.com/emissary-ingress/emissary/v3/cmd/agent.run file="/go/cmd/agent/main.go:59" CMD=agent PID=1
time="2022-07-12 20:49:32.2608" level=info msg="Agent is running..." func="github.com/emissary-ingress/emissary/v3/pkg/agent.(*Agent).Watch" file="/go/pkg/agent/agent.go:290" CMD=agent PID=1 THREAD=/watch
time="2022-07-12 20:49:32.5251" level=info msg="Setting cloud connect token from environment" func="github.com/emissary-ingress/emissary/v3/pkg/agent.(*Agent).handleAPIKeyConfigChange" file="/go/pkg/agent/agent.go:275" CMD=agent PID=1 THREAD=/watch

This is how the egde-stack-agent environment configuration looks like:

  {
    "name": "AGENT_CONFIG_RESOURCE_NAME",
    "value": "edge-stack-agent-cloud-token"
  },

Note: When token is stored in a secret, it's base64 encoded.