packetbeat,winlogbeat,x-pack/filebeat: add metrics documentation (#34889

) (#34901) This adds documentation for execution metrics collection by filebeat inputs packetbeat protocols, and winlogbeat event processing. (cherry picked from commit b8e0449) Co-authored-by: Dan Kortschak <90160302+efd6@users.noreply.github.com>
elastic · Mar 22, 2023 · a8dbc6c · a8dbc6c
1 parent b27f7eb
commit a8dbc6c
Show file tree

Hide file tree

Showing 10 changed files with 102 additions and 20 deletions.
diff --git a/CHANGELOG.next.asciidoc b/CHANGELOG.next.asciidoc
@@ -226,6 +226,7 @@ automatic splitting at root level, if root level element is an array. {pull}3415
 - Add Basic Authentication support on constructed requests to CEL input {issue}34609[34609] {pull}34689[34689]
 - Add string manipulation extensions to CEL input {issue}34610[34610] {pull}34689[34689]
 - Improve CEL input documentation {pull}34831[34831]
+- Add metrics documentation for CEL and AWS CloudWatch inputs. {issue}34887[34887] {pull}34889[34889]
 
 *Auditbeat*
 
@@ -261,6 +262,7 @@ automatic splitting at root level, if root level element is an array. {pull}3415
 - Reduce logging level for ENOENT to WARN when mapping sockets to processes. {issue}33793[33793] {pull}33854[33854]
 - Add metrics for TCP and UDP packet processing. {pull}33833[33833] {pull}34353[34353]
 - Allow user to prevent Npcap library installation on Windows. {issue}34420[34420] {pull}34428[34428]
+- Add metrics documentation for TCP and UDP protocols. {issue}34887[34887] {pull}34889[34889]
 
 *Packetbeat*
 
@@ -271,6 +273,7 @@ automatic splitting at root level, if root level element is an array. {pull}3415
 *Winlogbeat*
 
 - Add metrics for log event processing. {pull}33922[33922]
+- Add metrics documentation for event processing. {issue}34887[34887] {pull}34889[34889]
 
 *Elastic Log Driver*
 

diff --git a/filebeat/docs/inputs/input-tcp.asciidoc b/filebeat/docs/inputs/input-tcp.asciidoc
@@ -31,7 +31,7 @@ include::../inputs/input-common-tcp-options.asciidoc[]
 === Metrics
 
 This input exposes metrics under the <<http-endpoint, HTTP monitoring endpoint>>.
-These metrics are exposed under the `/dataset` path. They can be used to
+These metrics are exposed under the `/inputs` path. They can be used to
 observe the activity of the input.
 
 [options="header"]

diff --git a/filebeat/docs/inputs/input-udp.asciidoc b/filebeat/docs/inputs/input-udp.asciidoc
@@ -31,7 +31,7 @@ include::../inputs/input-common-udp-options.asciidoc[]
 === Metrics
 
 This input exposes metrics under the <<http-endpoint, HTTP monitoring endpoint>>.
-These metrics are exposed under the `/dataset` path. They can be used to
+These metrics are exposed under the `/inputs` path. They can be used to
 observe the activity of the input.
 
 [options="header"]

diff --git a/filebeat/docs/inputs/input-unix.asciidoc b/filebeat/docs/inputs/input-unix.asciidoc
@@ -33,7 +33,7 @@ include::../inputs/input-common-unix-options.asciidoc[]
 === Metrics
 
 This input exposes metrics under the <<http-endpoint, HTTP monitoring endpoint>>.
-These metrics are exposed under the `/dataset` path. They can be used to
+These metrics are exposed under the `/inputs` path. They can be used to
 observe the activity of the input.
 
 [options="header"]

diff --git a/packetbeat/docs/configuring-howto.asciidoc b/packetbeat/docs/configuring-howto.asciidoc
@@ -56,6 +56,8 @@ include::{libbeat-dir}/loggingconfig.asciidoc[]
 
 include::{libbeat-dir}/http-endpoint.asciidoc[]
 
+include::./protocol-metrics-packetbeat.asciidoc[]
+
 include::{libbeat-dir}/shared-instrumentation.asciidoc[]
 
 include::{libbeat-dir}/reference-yml.asciidoc[]
diff --git a/packetbeat/docs/protocol-metrics-packetbeat.asciidoc b/packetbeat/docs/protocol-metrics-packetbeat.asciidoc
@@ -0,0 +1,36 @@
+[[protocol-metrics-packetbeat]]
+=== Protocol-Specific Metrics
+
+Packetbeat exposes per-protocol metrics under the <<http-endpoint, HTTP monitoring endpoint>>.
+These metrics are exposed under the `/inputs` path. They can be used to
+observe the activity of Packetbeat for the monitored protocol.
+
+[float]
+==== TCP Metrics
+
+[options="header"]
+|=======
+| Metric                        | Description
+| `device`                      | Name of the device being monitored.
+| `received_events_total`       | Number of packets processed.
+| `received_bytes_total`        | Number of bytes processed.
+| `tcp_overlaps`                | Number of packets shrunk due to overlap.
+| `tcp.dropped_because_of_gaps` | Number of packets dropped because of gaps.
+| `arrival_period`              | Histogram of the elapsed time between packet arrivals.
+| `processing_time`             | Histogram of the elapsed time between packet receipt and publication.
+|=======
+
+
+[float]
+==== UDP Metrics
+
+[options="header"]
+|=======
+| Metric                  | Description
+| `device`                | Name of the device being monitored.
+| `received_events_total` | Number of packets processed.
+| `received_bytes_total`  | Number of bytes processed.
+| `arrival_period`        | Histogram of the elapsed time between packet arrivals.
+| `processing_time`       | Histogram of the elapsed time between packet receipt and publication.
+|=======
+
diff --git a/winlogbeat/docs/configuring-howto.asciidoc b/winlogbeat/docs/configuring-howto.asciidoc
@@ -53,23 +53,7 @@ include::{libbeat-dir}/loggingconfig.asciidoc[]
 
 include::{libbeat-dir}/http-endpoint.asciidoc[]
 
-[float]
-=== Metrics
-
-{beatname_uc} exposes the following additional metrics under the <<http-endpoint, HTTP monitoring endpoint>>.
-These metrics are exposed under the `/dataset` path.
-
-[options="header"]
-|=======
-| Metric                   | Description
-| `provider`               | The name of the provider being read.
-| `received_events_total`  | Total number of events read by the input.
-| `discarded_events_total` | Total number of events dropped by the input.
-| `errors_total`           | Total number of errors encountered by the input.
-| `batch_read_period`      | A histogram of intervals between non-empty event batch reads.
-| `received_events_count`  | A histogram of the number of events read in each batch.
-| `source_lag_time`        | The difference between the timestamp recorded in each event and the time when it was read.
-|=======
+include::./metrics-winlogbeat.asciidoc[]
 
 include::{libbeat-dir}/shared-instrumentation.asciidoc[]
 

diff --git a/winlogbeat/docs/metrics-winlogbeat.asciidoc b/winlogbeat/docs/metrics-winlogbeat.asciidoc
@@ -0,0 +1,21 @@
+[[metrics-winlogbeat]]
+=== Event Processing Metrics
+
+Winlogbeat exposes metrics under the <<http-endpoint, HTTP monitoring endpoint>>.
+These metrics are exposed under the `/inputs` path. They can be used to
+observe the event log processing activity of Winlogbeat.
+
+[float]
+==== Winlog Metrics
+
+[options="header"]
+|=======
+| Metric                   | Description
+| `provider`               | Name of the provider being read.
+| `received_events_total`  | Total number of events received.
+| `discarded_events_total` | Total number of discarded events.
+| `errors_total`           | Total number of errors.
+| `received_events_count`  | Histogram of the number of events in each non-zero batch.
+| `source_lag_time`        | Histogram of the difference between timestamped event's creation and reading.
+| `batch_read_period`      | Histogram of the elapsed time between non-zero batch reads.
+|=======
diff --git a/x-pack/filebeat/docs/inputs/input-aws-cloudwatch.asciidoc b/x-pack/filebeat/docs/inputs/input-aws-cloudwatch.asciidoc
@@ -130,6 +130,22 @@ cloudwatchlogs:DescribeLogGroups
 logs:FilterLogEvents
 ----
 
+[float]
+=== Metrics
+
+This input exposes metrics under the <<http-endpoint, HTTP monitoring endpoint>>.
+These metrics are exposed under the `/inputs` path. They can be used to
+observe the activity of the input.
+
+[options="header"]
+|=======
+| Metric                            | Description
+| `log_events_received_total`       | Number of CloudWatch log events received.
+| `log_groups_total`                | Logs collected from number of CloudWatch log groups.
+| `cloudwatch_events_created_total` | Number of events created from processing logs from CloudWatch.
+| `api_calls_total`                 | Number of API calls made total.
+|=======
+
 [id="{beatname_lc}-input-{type}-common-options"]
 include::../../../../filebeat/docs/inputs/input-common-options.asciidoc[]
 

diff --git a/x-pack/filebeat/docs/inputs/input-cel.asciidoc b/x-pack/filebeat/docs/inputs/input-cel.asciidoc
@@ -601,6 +601,26 @@ This specifies fields in the `state` to be redacted prior to debug logging. Fiel
 
 This specifies whether fields should be replaced with a `*` or deleted entirely from messages sent to debug logs. If delete is `true`, fields will be deleted rather than replaced.
 
+[float]
+=== Metrics
+
+This input exposes metrics under the <<http-endpoint, HTTP monitoring endpoint>>.
+These metrics are exposed under the `/inputs` path. They can be used to
+observe the activity of the input.
+
+[options="header"]
+|=======
+| Metric                    | Description
+| `resource`                | URL or path of the input resource.
+| `cel_executions`          | Number times the CEL program has been executed.
+| `batches_received_total`  | Number of event arrays received.
+| `events_received_total`   | Number of events received.
+| `batches_published_total` | Number of event arrays published.
+| `events_published_total`  | Number of events published.
+| `cel_processing_time`     | Histogram of the elapsed successful CEL program processing times in nanoseconds.
+| `batch_processing_time`   | Histogram of the elapsed successful batch processing times in nanoseconds (time of receipt to time of ACK for non-empty batches).
+|=======
+
 [id="{beatname_lc}-input-{type}-common-options"]
 include::../../../../filebeat/docs/inputs/input-common-options.asciidoc[]