How to connect Fargate Instances from AWS and EKS to coroot？

[franramb]

We have multiple applications hosted on AWS EKS and we are monitoring them using coroot. We have seen that automatically the coroot agents are installed on the EC2 instances but not on the Fargate instances:

The installation of coroot has been done using the helm chart.

Could you explain how to deploy the agents in the fargates or if you don't have it implemented, could you consider it?

Thanks a lot!

[bmike78]

I use AWS EKS for Coroot in some of my clusters and the deployment of node-agent is done as a DaemonSet, so your ensured that any new nodes get the agent installed on them.

With Fargate, daemonsets are not supported: https://docs.aws.amazon.com/eks/latest/userguide/fargate.html, so this is why node-agent is not being installed or collecting any data.

node-agent also uses eBPF to gather data. I'm not sure if the AWS Fargate service would give you this level of introspection at the kernel level.

Maybe @def or @apetruhin can answer more about this.

[franramb]

Thanks @bmike78 for the reply!

According to the AWS documentation, if we want to replace the daemonset, we would have to deploy the agent pod as a sidecar container -> Daemonsets aren't supported on Fargate. If your application requires a daemon, reconfigure that daemon to run as a sidecar container in your Pods.

A sidecar container is: "A sidecar container is a design pattern that allows you to run an additional container alongside your main container in the same pod. The sidecar container can perform tasks that complement the main container, such as synchronising data from a remote source, collecting and sending logs, providing health checks and metrics, directing network traffic, encrypting or decrypting data, or injecting bugs for testing purposes."

Would it be possible for you to consider integrating this agent installation option if there are fargates instances even if you can't collect all the metrics that the daemon itself installed on an ec2 can?

[def]

@franramb we've implemented basic support for AWS Fargate in Coroot. It's based on metrics collected by Cadvisor integrated into Fargate "nodes".
Unfortunately, all eBPF-based capabilities are not working for applications deployed to Fargate due to AWS limitations.

Please update Coroot to the latest Helm chart version, as it also contains some changes in the Prometheus config.

For more details, check out this Pull Request along with the associated changes in the chart

[franramb]

Hello @def ! I just updated coroot and we can already see that the fargates instances are being monitored. Great job!

Wouldn't it be possible to somehow capture the output logs (container logs) from the pods in the fargates (like kubernetes dashboard does) and store them in prometheus?

Thanks a lot for you work and effort, this application is amazing for us :D