[DOCS] Describe how to use Elastic Agent to monitor Logstash (#14959) (…

…#14971)

* [DOCS] Describe how to use Elastic Agent to monitor Logstash

* Apply suggestions from code review

Co-authored-by: Kevin Lacabane <klacabane@gmail.com>

* Remove reviewer questions

* Apply suggestions from code review

Co-authored-by: Karen Metts <35154725+karenzone@users.noreply.github.com>

* Fix security statement

---------

Co-authored-by: Kevin Lacabane <klacabane@gmail.com>
Co-authored-by: Karen Metts <35154725+karenzone@users.noreply.github.com>
(cherry picked from commit 58abffc)

Co-authored-by: DeDe Morton <dede.morton@elastic.co>

docs/static/monitoring/monitoring-ea.asciidoc

@@ -0,0 +1,85 @@
+[[monitoring-with-elastic-agent]]
+=== Collect {ls} monitoring data with {agent}
+[subs="attributes"]
+++++
+<titleabbrev>{agent} collection</titleabbrev>
+++++
+
+You can use {agent} to collect data about {ls} and ship it to the monitoring
+cluster. The benefit of {agent} collection is that the monitoring agent remains
+active even if the {ls} instance does not. Plus you can manage all your
+monitoring agents from a central location in {fleet}.
+
+To collect and ship monitoring data:
+
+. <<disable-default-include,Disable default collection of monitoring metrics>>
+. <<define-cluster__uuid-include,Specify the target `cluster_uuid` (optional)>>
+. <<configure-metricbeat,Install and configure {agent} to collect monitoring data>>
+
+[float]
+[[disable-default-include]]
+//include section about disabling default collection from the metricbeat topic
+include::monitoring-mb.asciidoc[tag=disable-default-collection]
+
+[float]
+[[define-cluster__uuid-include]]
+//include section about defining cluster_uuid from the metricbeat topic
+include::monitoring-mb.asciidoc[tag=define-cluster-uuid]
+
+[float]
+[[configure-elastic-agent]]
+==== Install and configure {agent}
+
+Prerequisites:
+
+* Set up {es} monitoring and optionally create a monitoring cluster as described
+in the {ref}/monitoring-production.html[{es} monitoring documentation].
+* Create a user on the production cluster that has the
+`remote_monitoring_collector` {ref}/built-in-roles.html[built-in role].
+
+To collect {ls} monitoring data, add a {ls} integration to an {agent} and
+deploy it to the host where {ls} is running.
+
+. Go to the {kib} home page and click **Add integrations**.
++
+NOTE: If you're using a monitoring cluster, use the {kib} instance connected to
+the monitoring cluster.
+
+. In the query bar, search for and select the **Logstash** integration for
+{agent}.
+. Read the overview to make sure you understand integration requirements and
+other considerations.
+. Click **Add Logstash**.
++
+TIP: If you're installing an integration for the first time, you may be prompted
+to install {agent}. Click **Add integration only (skip agent installation)**.
+
+. Configure the integration name and optionally add a description. Make sure you
+configure all required settings:
+* Under **Collect Logstash application and slowlog logs**, modify the log paths
+to match your {ls} environment.
+* Under **Collect Logstash node metrics and stats**, make sure the hosts setting
+points to your {ls} host URLs. By default, the integration collects {ls}
+monitoring metrics from `localhost:9600`. If that host and port number are not
+correct, update the `hosts` setting. If you configured {ls} to use encrypted
+communications, you must access it via HTTPS. For example, use a `hosts` setting
+like
+`https://localhost:9600`.
++
+**Elastic security.** The Elastic {security-features} are enabled by default. 
+Expand **Advanced options** and enter the username and password of a user that has
+the `remote_monitoring_collector` role.
+. Choose where to add the integration policy. Click **New hosts** to add it to
+new agent policy or **Existing hosts** to add it to an existing agent policy.
+. Click **Save and continue**. This step takes a minute or two to complete. When
+it's done, you'll have an agent policy that contains an integration for
+collecting monitoring data from {kib}.
+. If an {agent} is already assigned to the policy and deployed to the host where
+{kib} is running, you're done. Otherwise, you need to deploy an {agent}. To
+deploy an {agent}:
+.. Go to **{fleet} -> Agents**, then click **Add agent**.
+.. Follow the steps in the **Add agent** flyout to download, install,
+and enroll the {agent}. Make sure you choose the agent policy you created
+earlier.
+. Wait a minute or two until incoming data is confirmed.
+. {kibana-ref}/monitoring-data.html[View the monitoring data in {kib}].

docs/static/monitoring/monitoring-mb.asciidoc

@@ -16,8 +16,11 @@ To collect and ship monitoring data:
 . <<define-cluster__uuid,Specify the target `cluster_uuid` (optional)>>
 . <<configure-metricbeat,Install and configure {metricbeat} to collect monitoring data>>
 
+Want to use {agent} instead? Refer to <<monitoring-with-elastic-agent>>.
+
 [float]
 [[disable-default]]
+//tag::disable-default-collection[]
 ==== Disable default collection of {ls} monitoring metrics
 
 --
@@ -32,9 +35,11 @@ monitoring.enabled: false
 Remove the `#` at the beginning of the line to enable the setting.
 
 --
+//end::disable-default-collection[]
 
 [float]
 [[define-cluster__uuid]]
+//tag::define-cluster-uuid[]
 ==== Define `cluster_uuid` (Optional)
 To bind the metrics of {ls} to a specific cluster, optionally define the `monitoring.cluster_uuid`
 in the configuration file (logstash.yml):
@@ -43,6 +48,7 @@ in the configuration file (logstash.yml):
 ----------------------------------
 monitoring.cluster_uuid: PRODUCTION_ES_CLUSTER_UUID
 ----------------------------------
+//end::define-cluster-uuid[]
 
 [float]
 [[configure-metricbeat]]
@@ -114,9 +120,9 @@ To monitor multiple {ls} instances, specify a list of hosts, for example:
 hosts: ["http://localhost:9601","http://localhost:9602","http://localhost:9603"]
 ----------------------------------
 
-
-*Elastic security.* If the Elastic {security-features} are enabled, provide a user 
-ID and password so that {metricbeat} can collect metrics successfully: 
+**Elastic security.** The Elastic {security-features} are enabled by default. 
+You must provide a user ID and password so that {metricbeat} can collect metrics
+successfully:
 
 .. Create a user on the production cluster that has the 
 `remote_monitoring_collector` {ref}/built-in-roles.html[built-in role]. 
@@ -172,8 +178,8 @@ must access it via HTTPS. For example, use a `hosts` setting like
 IMPORTANT: The {es} {monitor-features} use ingest pipelines, therefore the
 cluster that stores the monitoring data must have at least one ingest node.
 
-If the {es} {security-features} are enabled on the monitoring cluster, you 
-must provide a valid user ID and password so that {metricbeat} can send metrics 
+**Elastic security.** The Elastic {security-features} are enabled by default.
+You must provide a user ID and password so that {metricbeat} can send metrics
 successfully: 
 
 .. Create a user on the monitoring cluster that has the 

docs/static/monitoring/monitoring-overview.asciidoc

@@ -13,9 +13,15 @@ For an introduction to monitoring your Elastic stack, see
 [[configuring-logstash-xpack]]
 === Configuring monitoring for {ls}
 
-Make sure monitoring is enabled on your {es} cluster. Then configure *one* of
+Make sure that monitoring is enabled on your {es} cluster. Then configure *one* of
 these methods to collect {ls} metrics:
 
+* <<monitoring-with-elastic-agent, {agent} collection>>. {agent} collects
+monitoring data from your {ls} instance and sends it directly to your monitoring
+cluster. The benefit of {agent} collection is that the monitoring agent remains
+active even if the {ls} instance does not. Plus you can mange all your
+monitoring agents from a central location in {fleet}.
+
 * <<monitoring-with-metricbeat, {metricbeat} collection>>. Metricbeat collects
 monitoring data from your {ls} instance and sends it directly to your monitoring
 cluster. The benefit of Metricbeat collection is that the monitoring
@@ -24,6 +30,7 @@ agent remains active even if the {ls} instance does not.
 * <<monitoring-internal-collection-legacy,Legacy collection (deprecated)>>. 
 Legacy collectors send monitoring data to your production cluster.
 
+include::monitoring-ea.asciidoc[]
 include::monitoring-mb.asciidoc[]
 include::monitoring-internal-legacy.asciidoc[]
 include::monitoring-ui.asciidoc[]

docs/static/monitoring/monitoring-ui.asciidoc

@@ -2,18 +2,17 @@
 [[logstash-monitoring-ui]]
 === Monitoring UI
 
-Use the https://www.elastic.co/products/x-pack/monitoring[monitoring feature in
-X-Pack] to view metrics and gain insight into how your Logstash deployment is
-running. In the overview dashboard, you can see all events received and sent by
-Logstash, plus info about memory usage and uptime:
+Use the {stack} {monitor-features} to view metrics and gain insight into how
+your {ls} deployment is running. In the overview dashboard, you can see all
+events received and sent by Logstash, plus info about memory usage and uptime:
 
 image::static/monitoring/images/overviewstats.png[Logstash monitoring overview dashboard in Kibana]
 
 Then you can drill down to see stats about a specific node:
 
 image::static/monitoring/images/nodestats.png[Logstash monitoring node stats dashboard in Kibana]
 
-NOTE: A Logstash node is considered unique based on its persistent UUID, which
+NOTE: A {ls} node is considered unique based on its persistent UUID, which
 is written to the <<logstash-settings-file,`path.data`>> directory when the node
 starts.