Implementing Coroot for Large-Scale Environments with VMs and K8s

[ANSANJAY]

Hello Coroot Team,

I'm exploring observability solutions suitable for a large-scale environment that includes both Kubernetes clusters and virtual machines. I have a few questions about Coroot's capabilities and setup process:

1. Kubernetes Compatibility: Can Coroot be seamlessly integrated with Kubernetes environments, and what level of observability does it offer for K8s workloads?
2. VM Support: Is Coroot capable of monitoring applications on virtual machines, and if so, how does it gather metrics and logs from these environments?
3. Customization Options: For VM-based setups, are there customization options available to tailor Coroot's observability features to specific needs?
4. Deployment Guidance: Could you share any best practices or recommendations for deploying Coroot across a large fleet of VMs and Kubernetes clusters?
5. Use Cases and Success Stories: Are there any examples or case studies of similar large-scale deployments where Coroot has been successfully implemented?

I appreciate your guidance as I evaluate the best approach to enhance our observability infrastructure.

Thank you.

[def]

Before addressing your question, let's take a quick look at how Coroot works.
It collects metrics, logs, traces, and profiles, with each of these telemetry signals associated with containers. A container, in this context, is a group of processes running within a dedicated cgroup. Coroot categorizes Kubernetes pods, standalone Docker/Containerd/CRI-O containers, and Systemd units as containers.

Below is an overview of Coroot's architecture:

1. Yes, it can. Coroot's node-agent is deployed as a DaemonSet and runs on each node of the cluster. It discovers all application containers running on the node and gather comprehensive telemetry data including metrics, logs, traces and profiles. As a result, Coroot can monitor various aspects of application performance:

• SLO tracking (availability, latency)
• resource consumption (CPU usage/throttling/delay accounting, Memory, Storage)
• runtime metrics (JVM, .NET, Postgres, MongoDB, Redis,...)
• related nodes health and utilization
• communication with dependency services and databases (requests, errors, latency)
• network between the service and its dependency services (TCP connections, RTT, TCP Retransmissions)
• logs
• continuous profiling
• deployment tracking

To learn more about Coroot's capabilities, you can check out our demo.

2. Yes, it can. You just need to install coroot-node-agent on every VM in the cluster. Metrics, logs, traces and profiles can be collected for apps on dedicated VMs.
   Here, you can find a benchmark demonstrating that the performance impact of using the eBPF-based agent is negligible.

3. To suggest ways for customization, we need more info about the specific problem or task.

4. Firstly, it may require scaling and fine-tuning storage components such as ClickHouse and Prometheus/VictoriaMetrics. Additionally, some performance or even UI optimizations may be needed in Coroot. However, we would be happy to adjust/optimize Coroot to work with large-scale environments.

5. So far, we've only received positive feedback on using Coroot in an infrastructure with ~500 nodes and ~3000 apps. We're open to collaboration and creating a case study on the use of Coroot in larger fleets.