Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

docs: update docker-systemd-general.md #2420

Merged
merged 8 commits into from Apr 29, 2024
Merged

docs: update docker-systemd-general.md #2420

Changes from 1 commit
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
45a2400
Update docker-systemd-general.md
jenworthington Apr 24, 2024
7892a5e
Update docker-systemd-general.md
jenworthington Apr 26, 2024
36ad0f5
Merge branch 'main' into jenworthington-patch-5
mstopa-splunk Apr 29, 2024
0c10404
Update list markers
mstopa-splunk Apr 29, 2024
0a85691
Update list markers
mstopa-splunk Apr 29, 2024
de43a6c
Remove redundant command
mstopa-splunk Apr 29, 2024
c7e2dbd
Fix list markers
mstopa-splunk Apr 29, 2024
d7cf274
Try fixing the list for mkdocs
mstopa-splunk Apr 29, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
91 changes: 47 additions & 44 deletions docs/gettingstarted/docker-systemd-general.md
View file
Open in desktop
@@ -1,73 +1,82 @@
# Install Docker CE

Refer to relevant installation guides:
## Before you begin

Before you start, make sure you are familiar with [IPv4 forwarding](./getting-started-runtime-configuration.md#ipv4-forwarding) and refer to installation guides for your docker configuration:

* [CentOS](https://docs.docker.com/install/linux/docker-ce/centos/)
* [Ubuntu](https://docs.docker.com/install/linux/docker-ce/ubuntu/)
* [Debian](https://docs.docker.com/install/linux/docker-ce/debian/)

NOTE: [READ FIRST (IPv4 forwarding)](./getting-started-runtime-configuration.md#ipv4-forwarding)

# Initial Setup
## Initial Setup

* IMPORTANT: Always use the _latest_ unit file (below) with the current release. By default, the latest container is
automatically downloaded at each restart. Therefore, make it a habit to check back here regularly to be sure any changes
Use the latest unit file provided in this topic with the current release. By default, the latest container is
automatically downloaded at each restart. Make it a habit to check back here regularly to be sure any changes
that may have been made to the template unit file below (e.g. suggested mount points) are incorporated in production prior
to relaunching via systemd.
to relaunching with systemd.

1. Create the systemd unit file `/lib/systemd/system/sc4s.service` based on the following template:

* Create the systemd unit file `/lib/systemd/system/sc4s.service` based on the following template:
#### Unit file
```ini
--8<--- "docs/resources/docker/sc4s.service"
```

* Execute the following command to create a local volume that will contain the disk buffer files in the event of a communication
failure to the upstream destination(s). This will also be used to keep track of the state of syslog-ng between restarts, and in
particular the state of the disk buffer. This is a required step.
2. Execute the following command to create a local volume that contains the disk buffer files in the event of a communication
failure to the upstream destinations.

```
sudo docker volume create splunk-sc4s-var
```

* NOTE: Be sure to account for disk space requirements for the docker volume created above. This volume is located in
`/var/lib/docker/volumes/` and could grow significantly if there is an extended outage to the SC4S destinations
(typically HEC endpoints). See the "SC4S Disk Buffer Configuration" section on the Configuration page for more info.
3. Account for disk space requirements for the new docker volume, which could grow significantly if there is an extended outage to the SC4S destinations. This volume can be found at
`/var/lib/docker/volumes/`. See [SC4S Disk Buffer Configuration](https://splunk.github.io/splunk-connect-for-syslog/main/configuration/#sc4s-disk-buffer-configuration).
4. Create the following subdirectories:
* `/opt/sc4s/local`
* `/opt/sc4s/archive`
* `/opt/sc4s/tls`

* Create subdirectories `/opt/sc4s/local` `/opt/sc4s/archive` `/opt/sc4s/tls`

* Create a file named `/opt/sc4s/env_file` and add the following environment variables and values:
5. Create a file named `/opt/sc4s/env_file` and add the following environment variables and values:

```dotenv
--8<--- "docs/resources/env_file"
```

* Update `SC4S_DEST_SPLUNK_HEC_DEFAULT_URL` and `SC4S_DEST_SPLUNK_HEC_DEFAULT_TOKEN` to reflect the correct values for your environment. Do _not_ configure HEC
Acknowledgement when deploying the HEC token on the Splunk side; the underlying syslog-ng http destination does not support this
feature. Moreover, HEC Ack would significantly degrade performance for streaming data such as syslog.
6. Update `SC4S_DEST_SPLUNK_HEC_DEFAULT_URL` and `SC4S_DEST_SPLUNK_HEC_DEFAULT_TOKEN` to reflect the correct values for your environment. Do not configure HEC
Acknowledgement when deploying the HEC token on the Splunk side, the underlying syslog-ng http destination does not support this
feature.

* The default number of `SC4S_DEST_SPLUNK_HEC_WORKERS` is 10. Consult the community if you feel the number of workers (threads) should
7. The default number of `SC4S_DEST_SPLUNK_HEC_WORKERS` is 10. Consult the community if you feel the number of workers (threads) should
deviate from this.

* NOTE: Splunk Connect for Syslog defaults to secure configurations. If you are not using trusted SSL certificates, be sure to
uncomment the last line in the example above.
8. Splunk Connect for Syslog defaults to secure configurations. If you are not using trusted SSL certificates, be sure to
uncomment the last line in the example in step 5.

For more information about configuration refer to [Docker and Podman basic configurations](./getting-started-runtime-configuration.md#docker-and-podman-basic-configurations)
For more information see [Docker and Podman basic configurations](./getting-started-runtime-configuration.md#docker-and-podman-basic-configurations)
and [detailed configuration](../configuration.md).

# Configure SC4S for systemd and start SC4S
## Configure SC4S for systemd
To configure SC4S for systemd run the following commands:

```bash
sudo systemctl daemon-reload
sudo systemctl enable sc4s
sudo systemctl start sc4s
```
## Restart SC4S
To restart SC4S run the following command:

```bash
sudo systemctl restart sc4s
```

If changes were made to the configuration Unit file above (e.g. to configure with dedicated ports), you must first stop SC4S and re-run
## Implement unit file changes
If you made changes to the configuration unit file, for example to configure with dedicated ports, you must stop SC4S and re-run the systemd configuration commands to implement your changes.

1. Stop SC4S:
```bash
sudo systemctl stop sc4s
jenworthington marked this conversation as resolved.
Show resolved Hide resolved
```
2. Re-run
the systemd configuration commands:

```bash
Expand All @@ -77,17 +86,11 @@ sudo systemctl enable sc4s
sudo systemctl start sc4s
```

## Stop SC4S

```bash
sudo systemctl stop sc4s
```

# Verify Proper Operation
## Validate your configuration

SC4S has a number of "preflight" checks to ensure that the container starts properly and that the syntax of the underlying syslog-ng
configuration is correct. After this step completes, to verify SC4S is properly communicating with Splunk,
execute the following search in Splunk:
SC4S performs checks to ensure that the container starts properly and that the syntax of the underlying syslog-ng
configuration is correct. Once the checks are complete, verify that SC4S properly communicate with Splunk.
To do this, execute the following search in Splunk:

```ini
index=* sourcetype=sc4s:events "starting up"
Expand All @@ -99,13 +102,13 @@ This should yield an event similar to the following:
syslog-ng starting up; version='3.28.1'
```

When the startup process proceeds normally (without syntax errors). If you do not see this,
The startup process should proceed normally without syntax errors. If it does not,
follow the steps below before proceeding to deeper-level troubleshooting:

* Check to see that the URL, token, and TLS/SSL settings are correct, and that the appropriate firewall ports are open (8088 or 443).
* Check to see that the proper indexes are created in Splunk, and that the token has access to them.
* Ensure the proper operation of the load balancer if used.
* Lastly, execute the following command to check the sc4s startup process running in the container.
1. Verify that the URL, token, and TLS/SSL settings are correct, and that the appropriate firewall ports are open (8088 or 443).
2. Verify that your indexes are created in Splunk, and that your token has access to them.
3. If you are using a load balancer, verify that it is operating properly.
4. Execute the following command to check the sc4s startup process running in the container.

```bash
docker logs SC4S
Expand All @@ -120,5 +123,5 @@ starting goss
starting syslog-ng
```

If you do not see the output above, proceed to the ["Troubleshoot sc4s server"](../troubleshooting/troubleshoot_SC4S_server.md)
and ["Troubleshoot resources"](../troubleshooting/troubleshoot_resources.md) sections for more detailed information.
5. If you do not see this output, see ["Troubleshoot sc4s server"](../troubleshooting/troubleshoot_SC4S_server.md)
and ["Troubleshoot resources"](../troubleshooting/troubleshoot_resources.md) for more detailed information.