Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

docs: review edge processor documentation #2402

Merged
merged 6 commits into from May 12, 2024
Merged

docs: review edge processor documentation #2402

Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
7853220
docs: Edge Processor docs
ikheifets-splunk Apr 5, 2024
00159c9
Update edge_processor.md
jenworthington Apr 16, 2024
3215069
Update edge_processor.md
jenworthington Apr 18, 2024
06c1902
Update edge_processor.md
jenworthington Apr 19, 2024
9bb3c61
Update edge_processor.md
jenworthington Apr 23, 2024
3986bbf
docs: fix logical mistakes
ikheifets-splunk May 12, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
51 changes: 26 additions & 25 deletions docs/edge_processor.md
View file
Open in desktop
Expand Up @@ -2,13 +2,13 @@

## Intro

`Edge Processor` can be used on that usecases:
You can use the `Edge Processor` to:

* Enrich log message extra data (for example add some field or override index) using `SPL2`
* Filter log message using `SPL2`
* Send log messages to alternative destanations (like `AWS S3`, `Apache Kafka`, etc.)
* Enrich log messages with extra data, such as adding a new field or overriding an index using `SPL2`.
* Filter log messages using `SPL2`.
* Send log messages to alternative destinations, for example, `AWS S3` or `Apache Kafka`.
jenworthington marked this conversation as resolved.
Show resolved Hide resolved

## How it's working
## How it works

```mermaid
stateDiagram
Expand All @@ -33,52 +33,53 @@ stateDiagram
EP --> Dest
```

## Basic Setup
## Set up the Edge Processor for SC4S

### Docker / Podman
### Set up on Docker / Podman

Setup on your `env_file` HEC URL as IP of EP worker EC2 instance.
Token you can find in EP "global settings" page.
1. On the `env_file`, configure the HEC URL as IP of *managed instance*, that you registered on Edge Processor.
2. Add your HEC token. You can find your token in the Edge Processor "global settings" page.

```
SC4S_DEST_SPLUNK_HEC_{EP1}_URL=http://x.x.x.x:8088
SC4S_DEST_SPLUNK_HEC_{EP1}_TOKEN=secret
SC4S_DEST_SPLUNK_HEC_DEFAULT_URL=http://x.x.x.x:8088
SC4S_DEST_SPLUNK_HEC_DEFAULT_TOKEN=secret
```

### Kubernetes

Setup on your `values.yaml` HEC URL as IP of EP worker EC2 instance.
Token you can find in EP "global settings"page.
1. Set up the Edge Processor on your `values.yaml` HEC URL using the IP address of the Edge Processor worker's EC2 instance.

2. Provide the hec_token. You can find this token on the Edge Processor's "global settings" page.

```
splunk:
hec_url: "https://x.x.x.x:8088"
hec_url: "http://x.x.x.x:8088"
hec_token: "secret"
```

## mTLS encryption

### Preparing certs
### Prepare your certificates

Before setup you need to [generate mTLS certificates](https://docs.splunk.com/Documentation/SplunkCloud/9.1.2308/EdgeProcessor/SecureForwarders). Server mTLS certificates should be uploaded to `Edge Processor` and client certifcates should be used with `SC4S`.
Before setup, [generate mTLS certificates](https://docs.splunk.com/Documentation/SplunkCloud/9.1.2308/EdgeProcessor/SecureForwarders). Server mTLS certificates should be uploaded to `Edge Processor` and client certifcates should be used with `SC4S`.

Please rename your files, we expcting such filenames for client mTLS cerificates:
Rename the certificate files. SC4S requires the following names:

* `key.pem` - client certificate key
* `cert.pem` - client certificate
* `ca_cert.pem` - certificate authority

### Docker / Podman

1. Use HTTPS in HEC url: `SC4S_DEST_SPLUNK_HEC_DEFAULT_URL=https://x.x.x.x:8088`
2. Move your clients mTLS certificates to `/opt/sc4s/tls/hec`
1. Use HTTPS in HEC url: `SC4S_DEST_SPLUNK_HEC_DEFAULT_URL=https://x.x.x.x:8088`.
2. Move your clients mTLS certificates to `/opt/sc4s/tls/hec`.
jenworthington marked this conversation as resolved.
Show resolved Hide resolved
3. Mount `/opt/sc4s/tls/hec` to `/etc/syslog-ng/tls/hec` using docker/podman volumes.
4. Define mounting mTLS point for HEC: `SC4S_DEST_SPLUNK_HEC_DEFAULT_TLS_MOUNT=/etc/syslog-ng/tls/hec`
5. Start/Restart SC4S
4. Define mounting mTLS point for HEC: `SC4S_DEST_SPLUNK_HEC_DEFAULT_TLS_MOUNT=/etc/syslog-ng/tls/hec`.
5. Start or restart SC4S.

### Kubernetes

1. Add secret name of mTLS certs at `values.yaml`:
1. Add the secret name of the mTLS certificates to the `values.yaml` file:

```
splunk:
Expand All @@ -87,7 +88,7 @@ splunk:
hec_tls: "secret-name"
```

2. Add your mtls certs at `secrets.yaml`:
2. Add your mTLS certificates to the `secrets.yaml` file:

```
hec_tls:
Expand All @@ -106,5 +107,5 @@ hec_tls:
```

3. Encrypt your `secrets.yaml` using `ansible-vault`.
4. Add IP of cluster nodes to inventoey file `ansible/inventory/inventory_microk8s_ha.yaml`
5. Deploy ansible playbook `ansible/playbooks/microk8s_ha.yml`
4. Add the IP address for your cluster nodes to the inventory file `ansible/inventory/inventory_microk8s_ha.yaml`.
5. Deploy the Ansible playbook `ansible/playbooks/microk8s_ha.yml`