Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat: Edge Processor MVP #2346

Closed
wants to merge 5 commits into from
Closed

feat: Edge Processor MVP #2346

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
be52182
feat: Edge Processor MVP
ikheifets-splunk Feb 14, 2024
b67a908
feat: Support TLS for EP
ikheifets-splunk Feb 18, 2024
647dd31
feat: Kubernetes support for EP
ikheifets-splunk Feb 18, 2024
d6cf470
feat: use more generic names for certs and add docs
ikheifets-splunk Feb 19, 2024
7d872db
fix: timestamp problem
ikheifets-splunk Apr 3, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
1 change: 1 addition & 0 deletions package/etc/conf.d/conflib/_common/t_templates.conf
View file
Open in desktop
Expand Up @@ -151,6 +151,7 @@ template t_splunk_hec {

template t_edge_hec {
template('$(format-json
time=$(if ("${.netsource.sc4s_use_recv_time}" eq "yes") $R_UNIXTIME $S_UNIXTIME)
host=$(lowercase ${HOST})
source=${.splunk.source:-SC4S}
sourcetype=${.splunk.sourcetype:-sc4s:fallback}
Expand Down
5 changes: 5 additions & 0 deletions package/etc/conf.d/destinations/dest_hec/plugin.jinja
View file
Open in desktop
Expand Up @@ -34,6 +34,11 @@ destination d_hec{{ dest_mode }}{{ altname }}{
{%- endif %}
tls(
peer-verify({{ peer_verify }})
{% if tls_mount %}
key-file("{{ tls_mount }}/data_source_client_key.pem")
cert-file("{{ tls_mount }}/data_source_client_cert.pem")
ca-dir("{{ tls_mount }}")
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we explore the idea of Port based TLS authentication as well ? @ikheifets-splunk

{% endif %}
{%- if cipher_suite %}
cipher-suite("{{ cipher_suite }}")
{%- endif %}
Expand Down
1 change: 1 addition & 0 deletions package/etc/conf.d/destinations/dest_hec/plugin.py
View file
Open in desktop
Expand Up @@ -145,6 +145,7 @@ def hec_endpoint_collector(hec_path, url_hec):
f"SC4S_DEST_SPLUNK_HEC_{group}_DISKBUFF_DISKBUFSIZE",
int(disk_space / workers),
),
tls_mount=os.getenv("SC4S_DEST_TLS_MOUNT"),
peer_verify=os.getenv(f"SC4S_DEST_SPLUNK_HEC_{group}_TLS_VERIFY", "yes"),
cipher_suite=os.getenv(f"SC4S_DEST_SPLUNK_HEC_{group}_CIPHER_SUITE"),
ssl_version=os.getenv(f"SC4S_DEST_SPLUNK_HEC_{group}_SSL_VERSION"),
Expand Down
11 changes: 9 additions & 2 deletions package/sbin/entrypoint.sh
View file
Open in desktop
Expand Up @@ -170,18 +170,25 @@ if [ "$SC4S_DEST_SPLUNK_HEC_GLOBAL" != "no" ]
then
HEC=$(echo $SC4S_DEST_SPLUNK_HEC_DEFAULT_URL | cut -d' ' -f 1)
if [ "${SC4S_DEST_SPLUNK_HEC_DEFAULT_TLS_VERIFY}" == "no" ]; then export NO_VERIFY=-k ; fi

if [ -n "${SC4S_DEST_TLS_MOUNT}" ]; then
export HEC_TLS_OPTS="--cert ${SC4S_DEST_TLS_MOUNT}/data_source_client_cert.pem --key ${SC4S_DEST_TLS_MOUNT}/data_source_client_key.pem";
else
export HEC_TLS_OPTS="";
fi

SC4S_DEST_SPLUNK_HEC_FALLBACK_INDEX=$(grep -Po '(?<=^splunk_sc4s_fallback,index,).*' -m1 $SC4S_ETC/conf.d/local/context/splunk_metadata.csv )
export SC4S_DEST_SPLUNK_HEC_FALLBACK_INDEX=${SC4S_DEST_SPLUNK_HEC_FALLBACK_INDEX:=main}
SC4S_DEST_SPLUNK_HEC_EVENTS_INDEX=$(cat $SC4S_ETC/conf.d/local/context/splunk_metadata.csv | grep ',index,' | grep sc4s_events | cut -d, -f 3)
export SC4S_DEST_SPLUNK_HEC_EVENTS_INDEX=${SC4S_DEST_SPLUNK_HEC_EVENTS_INDEX:=main}

if curl -s -S ${NO_VERIFY} "${HEC}?/index=${SC4S_DEST_SPLUNK_HEC_FALLBACK_INDEX}" -H "Authorization: Splunk ${SC4S_DEST_SPLUNK_HEC_DEFAULT_TOKEN}" -d '{"event": "HEC TEST EVENT", "sourcetype": "sc4s:probe"}' 2>&1 | grep -v -e '{"text":"Success"' -e '{"text": "Success"'
if curl -s -S ${NO_VERIFY} "${HEC}?/index=${SC4S_DEST_SPLUNK_HEC_FALLBACK_INDEX}" -H "Authorization: Splunk ${SC4S_DEST_SPLUNK_HEC_DEFAULT_TOKEN}" -d '{"event": "HEC TEST EVENT", "sourcetype": "sc4s:probe"}' ${HEC_TLS_OPTS} 2>&1 | grep -v -e '{"text":"Success"' -e '{"text": "Success"'
then
echo -e "SC4S_ENV_CHECK_HEC: Invalid Splunk HEC URL, invalid token, or other HEC connectivity issue index=${SC4S_DEST_SPLUNK_HEC_FALLBACK_INDEX}. sourcetype=sc4s:fallback\nStartup will continue to prevent data loss if this is a transient failure."
echo ""
else
echo -e "SC4S_ENV_CHECK_HEC: Splunk HEC connection test successful to index=${SC4S_DEST_SPLUNK_HEC_FALLBACK_INDEX} for sourcetype=sc4s:fallback..."
if curl -s -S ${NO_VERIFY} "${HEC}?/index=${SC4S_DEST_SPLUNK_HEC_EVENTS_INDEX}" -H "Authorization: Splunk ${SC4S_DEST_SPLUNK_HEC_DEFAULT_TOKEN}" -d '{"event": "HEC TEST EVENT", "sourcetype": "sc4s:probe"}' 2>&1 | grep -v -e '{"text":"Success"' -e '{"text": "Success"'
if curl -s -S ${NO_VERIFY} "${HEC}?/index=${SC4S_DEST_SPLUNK_HEC_EVENTS_INDEX}" -H "Authorization: Splunk ${SC4S_DEST_SPLUNK_HEC_DEFAULT_TOKEN}" -d '{"event": "HEC TEST EVENT", "sourcetype": "sc4s:probe"}' ${HEC_TLS_OPTS} 2>&1 | grep -v -e '{"text":"Success"' -e '{"text": "Success"'
then
echo -e "SC4S_ENV_CHECK_HEC: Invalid Splunk HEC URL, invalid token, or other HEC connectivity issue for index=${SC4S_DEST_SPLUNK_HEC_EVENTS_INDEX}. sourcetype=sc4s:events \nStartup will continue to prevent data loss if this is a transient failure."
echo ""
Expand Down