Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

USE_KIBANA/ELASTICSEARCH=0 ignored, still tries to connect #269

Open
brsolomon-deloitte opened this issue Apr 21, 2022 · 2 comments
Open

USE_KIBANA/ELASTICSEARCH=0 ignored, still tries to connect #269

brsolomon-deloitte opened this issue Apr 21, 2022 · 2 comments

Comments

@brsolomon-deloitte
Copy link

Using local docker image scirius:test-1 built from scirius at commit 0b8fd6d fails in trying to connect to Elasticsearch, even though local_settings.py defaults to USE_ELASTICSEARCH=False and USE_KIBANA=False.

It appears that docker/scirius/bin/reset_dashboards.sh doesn't respect these variables whatsoever and calls python manage.py kibana_reset indiscriminately:

scirius/docker/scirius/bin/reset_dashboards.sh

Line 25 in 0b8fd6d

reset_dashboards() { 

$ docker container run -e SECRET_KEY=$(openssl rand -hex 16) --expose 8000 scirius:test-1
Migrations for 'accounts':
  accounts/migrations/0007_auto_20220421_1743.py
    - Alter field timezone on sciriususer
Operations to perform:
  Apply all migrations: accounts, auth, authtoken, contenttypes, rules, sessions, suricata
Running migrations:
  Applying rules.0001_initial... OK
  Applying rules.0002_auto_20141207_1824... OK
  Applying rules.0003_auto_20141210_1421... OK
  Applying rules.0004_auto_20141210_1525... OK
  Applying rules.0005_auto_20141210_1734... OK
  Applying rules.0006_auto_20141210_1846... OK
  Applying rules.0007_auto_20141210_2037... OK
  Applying rules.0008_auto_20141210_2057... OK
  Applying rules.0009_auto_20141214_1203... OK
  Applying rules.0010_auto_20141222_1209... OK
  Applying rules.0011_auto_20141222_1304... OK
  Applying rules.0012_auto_20141222_1306... OK
  Applying rules.0013_auto_20141229_1527... OK
  Applying rules.0014_auto_20141229_1528... OK
  Applying rules.0015_auto_20141229_1610... OK
  Applying rules.0016_auto_20141229_1629... OK
  Applying rules.0017_auto_20141229_1716... OK
  Applying rules.0018_auto_20141229_1716... OK
  Applying rules.0019_auto_20141229_1719... OK
  Applying rules.0020_auto_20141229_1852... OK
  Applying rules.0021_auto_20141229_1853... OK
  Applying rules.0022_auto_20141229_1858... OK
  Applying rules.0023_auto_20141229_1903... OK
  Applying rules.0024_auto_20141229_2204... OK
  Applying rules.0025_auto_20141230_0812... OK
  Applying rules.0026_auto_20141231_0948... OK
  Applying rules.0027_auto_20141231_0953... OK
  Applying rules.0028_auto_20150101_2305... OK
  Applying rules.0029_auto_20150102_1212... OK
  Applying rules.0030_auto_20150103_1136... OK
  Applying rules.0031_auto_20150103_1138... OK
  Applying rules.0032_auto_20150103_1255... OK
  Applying rules.0033_auto_20150109_2319... OK
  Applying rules.0034_auto_20150111_2200... OK
  Applying rules.0035_auto_20150202_0937... OK
  Applying rules.0036_auto_20150203_1421... OK
  Applying rules.0037_auto_20150407_2040... OK
  Applying rules.0038_auto_20150516_0912... OK
  Applying rules.0039_auto_20150805_1737... OK
  Applying rules.0040_ruleset_rules_count... OK
  Applying rules.0041_source_authkey... OK
  Applying rules.0042_rule_state_in_source... OK
  Applying rules.0043_threshold... OK
  Applying rules.0044_flowbit_type... OK
  Applying rules.0045_auto_20160405_1300... OK
  Applying rules.0046_source_cert_verif... OK
  Applying rules.0047_proxy_validation... OK
  Applying rules.0048_custom_es... OK
  Applying rules.0049_auto_20161121_2342... OK
  Applying contenttypes.0001_initial... OK
  Applying contenttypes.0002_remove_content_type_name... OK
  Applying rules.0050_auto_20161128_2110... OK
  Applying rules.0051_auto_20161207_0758... OK
  Applying auth.0001_initial... OK
  Applying rules.0052_useraction_user... OK
  Applying rules.0053_unique_none_rules... OK
  Applying rules.0054_login_action... OK
  Applying rules.0055_auto_20180213_1723... OK
  Applying rules.0056_auto_20180223_0823... OK
  Applying rules.0057_auto_20180302_1312... OK
  Applying rules.0058_source_public_source... OK
  Applying rules.0059_auto_20180309_2012... OK
  Applying rules.0060_auto_20180403_0921... OK
  Applying rules.0061_auto_20180507_1410... OK
  Applying rules.0062_useraction_username... OK
  Applying rules.0063_ruleprocessingfilter_ruleprocessingfilterdef... OK
  Applying rules.0064_ruleprocessingfilter_rulesets... OK
  Applying rules.0061_auto_20180503_2200... OK
  Applying rules.0063_merge_20180718_0118... OK
  Applying rules.0065_merge_20180719_1505... OK
  Applying rules.0066_auto_20180807_1428... OK
  Applying rules.0067_source_use_iprep... OK
  Applying rules.0068_auto_20180818_2204... OK
  Applying rules.0069_auto_20190220_1500... OK
  Applying rules.0070_ruleprocessingfilterdef_full_string... OK
  Applying rules.0071_filterset... OK
  Applying rules.0072_send_mail... OK
  Applying rules.0073_filterset_description... OK
  Applying rules.0074_redlights_useraction... OK
  Applying rules.0075_suppress_validator... OK
  Applying rules.0075_custom_es_no_empty... OK
  Applying rules.0076_merge_20190926_1233... OK
  Applying rules.0077_auto_20191002_0820... OK
  Applying rules.0078_auto_20200206_1648... OK
  Applying rules.0079_source_remove_choice... OK
  Applying rules.0080_source_version... OK
  Applying rules.0081_django-2... OK
  Applying rules.0082_source_use_sys_proxy... OK
  Applying rules.0083_multi_es_validation... OK
  Applying rules.0084_fakepermissionmodel... OK
  Applying auth.0002_alter_permission_name_max_length... OK
  Applying auth.0003_alter_user_email_max_length... OK
  Applying auth.0004_alter_user_username_opts... OK
  Applying auth.0005_alter_user_last_login_null... OK
  Applying auth.0006_require_contenttypes_0002... OK
  Applying auth.0007_alter_validators_add_error_messages... OK
  Applying auth.0008_alter_user_username_max_length... OK
  Applying auth.0009_alter_user_last_name_max_length... OK
  Applying auth.0010_alter_group_name_max_length... OK
  Applying auth.0011_update_proxy_permissions... OK
  Applying accounts.0001_initial... OK
  Applying accounts.0002_auto_20151110_1657... OK
  Applying accounts.0003_timezone... OK
  Applying accounts.0004_group... OK
  Applying rules.0085_roles_migrations... OK
  Applying accounts.0005_remove_user_flags... OK
  Applying accounts.0006_fix_role_default_priority... OK
  Applying accounts.0007_auto_20220421_1743... OK
  Applying authtoken.0001_initial... OK
  Applying authtoken.0002_auto_20160226_1747... OK
  Applying authtoken.0003_tokenproxy... OK
  Applying rules.0086_ruleset_suppressed_sids... OK
  Applying rules.0087_systemsettings_use_proxy_for_es... OK
  Applying rules.0088_ruleprocessingfilter_import_member... OK
  Applying rules.0089_ruleprocessingfilter_event_type... OK
  Applying rules.0090_useraction_ip... OK
  Applying rules.0091_useraction_missing_ruleset... OK
  Applying sessions.0001_initial... OK
  Applying suricata.0001_initial... OK
  Applying suricata.0002_auto_20151110_1657... OK
  Applying suricata.0003_suricata_yaml_file... OK
  Applying suricata.0004_auto_20160316_0844... OK
  Applying suricata.0005_django-2... OK
from django.contrib.auth.models import User; User.objects.create_superuser(***)
Successfully created source "ETOpen Ruleset"
Successfully updated source "ETOpen Ruleset"
Successfully created source "SSLBL abuse.ch"
Successfully updated source "SSLBL abuse.ch"
Successfully created source "PT Research Ruleset"
Successfully updated source "PT Research Ruleset"
Successfully created default ruleset "Default ruleset"
Successfully removed "stream-events" from ruleset "Default ruleset"
Successfully created suricata "suricata"
Successfully pushed ruleset to suricata "suricata"

236 static files copied to '/static'.
Kibana dashboards reset: Elasticsearch not ready, retrying in 10 seconds.
Kibana dashboards reset: Elasticsearch not ready, retrying in 10 seconds.
Kibana dashboards reset: Elasticsearch not ready, retrying in 10 seconds.
Kibana dashboards reset: Elasticsearch not ready, retrying in 10 seconds.
Kibana dashboards reset: Elasticsearch not ready, retrying in 10 seconds.
Kibana dashboards reset: Elasticsearch not ready, retrying in 10 seconds.
Kibana dashboards reset: Elasticsearch not ready, retrying in 10 seconds.
@brsolomon-deloitte
Copy link
Author

Same thing with https://github.com/StamusNetworks/scirius/blob/master/docker/scirius/bin/create_ILM_policy.sh which tries to contact Elasticsearch indiscriminately.

@brsolomon-deloitte
Copy link
Author

It would be easy for kibana_reset in rules/es_data.py to actually check for USE_KIBANA before trying to make a bunch of API calls to it.

@brsolomon-deloitte brsolomon-deloitte mentioned this issue Apr 21, 2022
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Skip scripts based on USE_ELASTICSEARCH and USE_KIBANA
1 participant
@brsolomon-deloitte