urllib3 certificate verification

[Cyb3rSn0rlax]

HELLO

• I use de docker image with the docker file that supports python3. (by @coandco)
• I use ssl certificates between my cluster nodes
• When I clone the YELP repo of elastalert it works fine and the indexes are created with my certificates and username and password but when i use the elastalert-server i got this problem with both docker and repo install version in both python 3.8 and 3.6

When i look it up i found some clues that it may be related to the python 3 version forcing certificate verification. I decided to go first here with this issue for your help.

ERROR :

When i run the docker container (npm start) i get this error and i can't create the indexes :

07:26:19.785Z ERROR elastalert-server:
    ProcessController:  WARNING:elasticsearch:GET https://localhost:9200/ [status:N/A request:0.008s]
    Traceback (most recent call last):
      File "/usr/lib/python3.8/site-packages/urllib3-1.25.9-py3.8.egg/urllib3/connection.py", line 159, in _new_conn
        conn = connection.create_connection(
      File "/usr/lib/python3.8/site-packages/urllib3-1.25.9-py3.8.egg/urllib3/util/connection.py", line 84, in create_connection
        raise err
      File "/usr/lib/python3.8/site-packages/urllib3-1.25.9-py3.8.egg/urllib3/util/connection.py", line 74, in create_connection
        sock.connect(sa)
    ConnectionRefusedError: [Errno 111] Connection refused

    During handling of the above exception, another exception occurred:

    Traceback (most recent call last):
      File "/usr/lib/python3.8/site-packages/urllib3-1.25.9-py3.8.egg/urllib3/connectionpool.py", line 670, in urlopen
        httplib_response = self._make_request(
      File "/usr/lib/python3.8/site-packages/urllib3-1.25.9-py3.8.egg/urllib3/connectionpool.py", line 381, in _make_request
        self._validate_conn(conn)
      File "/usr/lib/python3.8/site-packages/urllib3-1.25.9-py3.8.egg/urllib3/connectionpool.py", line 976, in _validate_conn
        conn.connect()
      File "/usr/lib/python3.8/site-packages/urllib3-1.25.9-py3.8.egg/urllib3/connection.py", line 308, in connect
        conn = self._new_conn()
      File "/usr/lib/python3.8/site-packages/urllib3-1.25.9-py3.8.egg/urllib3/connection.py", line 171, in _new_conn
        raise NewConnectionError(
    urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPSConnection object at 0x7f2121c585b0>: Failed to establish a new connection: [Errno 111] Connection refused

    During handling of the above exception, another exception occurred:

    Traceback (most recent call last):
      File "/usr/lib/python3.8/site-packages/requests-2.23.0-py3.8.egg/requests/adapters.py", line 439, in send
        resp = conn.urlopen(
      File "/usr/lib/python3.8/site-packages/urllib3-1.25.9-py3.8.egg/urllib3/connectionpool.py", line 724, in urlopen
        retries = retries.increment(
      File "/usr/lib/python3.8/site-packages/urllib3-1.25.9-py3.8.egg/urllib3/util/retry.py", line 439, in increment
        raise MaxRetryError(_pool, url, error or ResponseError(cause))
    urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='localhost', port=9200): Max retries exceeded with url: / (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f2121c585b0>: Failed to establish a new connection: [Errno 111] Connection refused'))

    During handling of the above exception, another exception occurred:

    Traceback (most recent call last):
      File "/usr/lib/python3.8/site-packages/elasticsearch-7.0.0-py3.8.egg/elasticsearch/connection/http_requests.py", line 77, in perform_request
        response = self.session.send(prepared_request, **send_kwargs)
      File "/usr/lib/python3.8/site-packages/requests-2.23.0-py3.8.egg/requests/sessions.py", line 643, in send
        r = adapter.send(request, **kwargs)
      File "/usr/lib/python3.8/site-packages/requests-2.23.0-py3.8.egg/requests/adapters.py", line 516, in send
        raise ConnectionError(e, request=request)
    requests.exceptions.ConnectionError: HTTPSConnectionPool(host='localhost', port=9200): Max retries exceeded with url: / (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f2121c585b0>: Failed to establish a new connection: [Errno 111] Connection refused'))

07:26:22.796Z ERROR elastalert-server:
    ProcessController:  WARNING:elasticsearch:GET https://localhost:9200/ [status:N/A request:0.008s]
    Traceback (most recent call last):
      File "/usr/lib/python3.8/site-packages/urllib3-1.25.9-py3.8.egg/urllib3/connection.py", line 159, in _new_conn
        conn = connection.create_connection(
      File "/usr/lib/python3.8/site-packages/urllib3-1.25.9-py3.8.egg/urllib3/util/connection.py", line 84, in create_connection
        raise err
      File "/usr/lib/python3.8/site-packages/urllib3-1.25.9-py3.8.egg/urllib3/util/connection.py", line 74, in create_connection
        sock.connect(sa)
    ConnectionRefusedError: [Errno 111] Connection refused

    During handling of the above exception, another exception occurred:

    Traceback (most recent call last):
      File "/usr/lib/python3.8/site-packages/urllib3-1.25.9-py3.8.egg/urllib3/connectionpool.py", line 670, in urlopen
        httplib_response = self._make_request(
      File "/usr/lib/python3.8/site-packages/urllib3-1.25.9-py3.8.egg/urllib3/connectionpool.py", line 381, in _make_request
        self._validate_conn(conn)
      File "/usr/lib/python3.8/site-packages/urllib3-1.25.9-py3.8.egg/urllib3/connectionpool.py", line 976, in _validate_conn
        conn.connect()
      File "/usr/lib/python3.8/site-packages/urllib3-1.25.9-py3.8.egg/urllib3/connection.py", line 308, in connect
        conn = self._new_conn()
      File "/usr/lib/python3.8/site-packages/urllib3-1.25.9-py3.8.egg/urllib3/connection.py", line 171, in _new_conn
        raise NewConnectionError(
    urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPSConnection object at 0x7f2121c58880>: Failed to establish a new connection: [Errno 111] Connection refused

    During handling of the above exception, another exception occurred:

    Traceback (most recent call last):
      File "/usr/lib/python3.8/site-packages/requests-2.23.0-py3.8.egg/requests/adapters.py", line 439, in send
        resp = conn.urlopen(
      File "/usr/lib/python3.8/site-packages/urllib3-1.25.9-py3.8.egg/urllib3/connectionpool.py", line 724, in urlopen
        retries = retries.increment(
      File "/usr/lib/python3.8/site-packages/urllib3-1.25.9-py3.8.egg/urllib3/util/retry.py", line 439, in increment
        raise MaxRetryError(_pool, url, error or ResponseError(cause))
    urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='localhost', port=9200): Max retries exceeded with url: / (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f2121c58880>: Failed to establish a new connection: [Errno 111] Connection refused'))

    During handling of the above exception, another exception occurred:

    Traceback (most recent call last):
      File "/usr/lib/python3.8/site-packages/elasticsearch-7.0.0-py3.8.egg/elasticsearch/connection/http_requests.py", line 77, in perform_request
        response = self.session.send(prepared_request, **send_kwargs)
      File "/usr/lib/python3.8/site-packages/requests-2.23.0-py3.8.egg/requests/sessions.py", line 643, in send
        r = adapter.send(request, **kwargs)
      File "/usr/lib/python3.8/site-packages/requests-2.23.0-py3.8.egg/requests/adapters.py", line 516, in send
        raise ConnectionError(e, request=request)
    requests.exceptions.ConnectionError: HTTPSConnectionPool(host='localhost', port=9200): Max retries exceeded with url: / (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f2121c58880>: Failed to establish a new connection: [Errno 111] Connection refused'))

07:26:24.719Z ERROR elastalert-server:
    ProcessController:  WARNING:apscheduler.scheduler:Execution of job "ElastAlerter.handle_pending_alerts (trigger: interval[0:00:05], next run at: 2020-05-17 07:26:24 UTC)" skipped: maximum number of running instances reached (1)

My config.json file :

{
  "appName": "elastalert-server",
  "port": 3030,
  "wsport": 3333,
  "elastalertPath": "/opt/elastalert",
  "verbose": false,
  "es_debug": false,
  "debug": false,
  "rulesPath": {
    "relative": true,
    "path": "/rules"
  },
  "templatesPath": {
    "relative": true,
    "path": "/rule_templates"
  },
  "es_host": "localhost",
  "es_port": 9200,
  "es_username": "hidden",
  "es_password": "hidden",
  "es_ssl": true,
  "verify_certs": false,
  "writeback_index": "elastalert_status"
}

my elastalert.yaml file :

es_port: 9200

# This is the folder that contains the rule yaml files
# Any .yaml file will be loaded as a rule
rules_folder: rules

# How often ElastAlert will query elasticsearch
# The unit can be anything from weeks to seconds
run_every:
  seconds: 5

# ElastAlert will buffer results from the most recent
# period of time, in case some log sources are not in real time
buffer_time:
  minutes: 1

# Optional URL prefix for elasticsearch
#es_url_prefix: elasticsearch

# Connect with TLS to elasticsearch
use_ssl: True

# Verify TLS certificates
verify_certs: False

# GET request with body is the default option for Elasticsearch.
# If it fails for some reason, you can pass 'GET', 'POST' or 'source'.
# See http://elasticsearch-py.readthedocs.io/en/master/connection.html?highlight=send_get_body_as#transport
# for details
#es_send_get_body_as: GET

# Option basic-auth username and password for elasticsearch
es_username: hidden
es_password: hidden

# The index on es_host which is used for metadata storage
# This can be a unmapped index, but it is recommended that you run
# elastalert-create-index to set a mapping
writeback_index: elastalert_status

# If an alert fails for some reason, ElastAlert will retry
# sending the alert until this time period has elapsed
alert_time_limit:
  days: 2

my dockerfile (built successfully)

FROM alpine:latest as py-ea
ARG ELASTALERT_VERSION=1334b611fdd7adf39991a1b0b11689568d612690
ENV ELASTALERT_VERSION=${ELASTALERT_VERSION}
# URL from which to download Elastalert.
ARG ELASTALERT_URL=https://github.com/Yelp/elastalert/archive/$ELASTALERT_VERSION.zip
ENV ELASTALERT_URL=${ELASTALERT_URL}
# Elastalert home directory full path.
ENV ELASTALERT_HOME /opt/elastalert

WORKDIR /opt

RUN apk add --update --no-cache ca-certificates openssl-dev openssl python3-dev python3 py3-pip py3-yaml libffi-dev gcc musl-dev wget && \
# Download and unpack Elastalert.
    wget -O elastalert.zip "${ELASTALERT_URL}" && \
    unzip elastalert.zip && \
    rm elastalert.zip && \
    mv e* "${ELASTALERT_HOME}"

WORKDIR "${ELASTALERT_HOME}"

# Install Elastalert.
# With the latest hash we no longer need to monkey with package versions
RUN python3 setup.py install

FROM node:alpine
LABEL maintainer="BitSensor <dev@bitsensor.io>"
# Set timezone for this container
ENV TZ Etc/UTC

RUN apk add --update --no-cache curl tzdata python3 make libmagic && \
    ln -s /usr/bin/python3 /usr/bin/python

COPY --from=py-ea /usr/lib/python3.8/site-packages /usr/lib/python3.8/site-packages
COPY --from=py-ea /opt/elastalert /opt/elastalert
COPY --from=py-ea /usr/bin/elastalert* /usr/bin/

WORKDIR /opt/elastalert-server
COPY . /opt/elastalert-server

RUN npm install --production --quiet
COPY config/elastalert.yaml /opt/elastalert/config.yaml
COPY config/elastalert-test.yaml /opt/elastalert/config-test.yaml
COPY config/config.json config/config.json
COPY rule_templates/ /opt/elastalert/rule_templates
COPY elastalert_modules/ /opt/elastalert/elastalert_modules

# Add default rules directory
# Set permission as unpriviledged user (1000:1000), compatible with Kubernetes
RUN mkdir -p /opt/elastalert/rules/ /opt/elastalert/server_data/tests/ \
    && chown -R node:node /opt

USER node

EXPOSE 3030
ENTRYPOINT ["npm", "start"]

I am not a developper. I spent the whole night trying to understand this error. Help please !

[Cyb3rSn0rlax]

Hello @nsano-rururu and thank you for your response. I didn't follow this error any further I dropped the docker version and used the repo version by @ServerCentral