Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

unexpected error error_class=Errno::EADDRINUSE error="Address already in use - bind(2) for 0.0.0.0:24224" #23

Open
jithendra12 opened this issue Feb 2, 2021 · 2 comments
Open

unexpected error error_class=Errno::EADDRINUSE error="Address already in use - bind(2) for 0.0.0.0:24224" #23

jithendra12 opened this issue Feb 2, 2021 · 2 comments
Assignees
@peters95

Comments

@jithendra12
Copy link

I have used the same td-agent.conf file provided in this repo: fluent.conf.rt to collect application metrics of jfrog artifactory on a centos VM. but when I start td-agent, I keep seeing these below errors that 24224 port is already in use.

2021-02-01 23:58:18 +0000 [error]: #0 unexpected error error_class=Errno::EADDRINUSE error="Address already in use - bind(2) for 0.0.0.0:24224"
  2021-02-01 23:58:18 +0000 [error]: #0 /opt/td-agent/lib/ruby/2.7.0/socket.rb:201:in `bind'
  2021-02-01 23:58:18 +0000 [error]: #0 /opt/td-agent/lib/ruby/2.7.0/socket.rb:201:in `listen'
  2021-02-01 23:58:18 +0000 [error]: #0 /opt/td-agent/lib/ruby/gems/2.7.0/gems/fluentd-1.11.2/lib/fluent/plugin_helper/server.rb:357:in `server_create_tcp_socket'
  2021-02-01 23:58:18 +0000 [error]: #0 /opt/td-agent/lib/ruby/gems/2.7.0/gems/fluentd-1.11.2/lib/fluent/plugin_helper/server.rb:212:in `server_create_for_tcp_connection'
  2021-02-01 23:58:18 +0000 [error]: #0 /opt/td-agent/lib/ruby/gems/2.7.0/gems/fluentd-1.11.2/lib/fluent/plugin_helper/server.rb:92:in `server_create_connection'
  2021-02-01 23:58:18 +0000 [error]: #0 /opt/td-agent/lib/ruby/gems/2.7.0/gems/fluentd-1.11.2/lib/fluent/plugin/in_forward.rb:172:in `start'
  2021-02-01 23:58:18 +0000 [error]: #0 /opt/td-agent/lib/ruby/gems/2.7.0/gems/fluentd-1.11.2/lib/fluent/root_agent.rb:200:in `block in start'
  2021-02-01 23:58:18 +0000 [error]: #0 /opt/td-agent/lib/ruby/gems/2.7.0/gems/fluentd-1.11.2/lib/fluent/root_agent.rb:189:in `block (2 levels) in lifecycle'
  2021-02-01 23:58:18 +0000 [error]: #0 /opt/td-agent/lib/ruby/gems/2.7.0/gems/fluentd-1.11.2/lib/fluent/root_agent.rb:188:in `each'
  2021-02-01 23:58:18 +0000 [error]: #0 /opt/td-agent/lib/ruby/gems/2.7.0/gems/fluentd-1.11.2/lib/fluent/root_agent.rb:188:in `block in lifecycle'
  2021-02-01 23:58:18 +0000 [error]: #0 /opt/td-agent/lib/ruby/gems/2.7.0/gems/fluentd-1.11.2/lib/fluent/root_agent.rb:175:in `each'
  2021-02-01 23:58:18 +0000 [error]: #0 /opt/td-agent/lib/ruby/gems/2.7.0/gems/fluentd-1.11.2/lib/fluent/root_agent.rb:175:in `lifecycle'
  2021-02-01 23:58:18 +0000 [error]: #0 /opt/td-agent/lib/ruby/gems/2.7.0/gems/fluentd-1.11.2/lib/fluent/root_agent.rb:199:in `start'
  2021-02-01 23:58:18 +0000 [error]: #0 /opt/td-agent/lib/ruby/gems/2.7.0/gems/fluentd-1.11.2/lib/fluent/engine.rb:248:in `start'
  2021-02-01 23:58:18 +0000 [error]: #0 /opt/td-agent/lib/ruby/gems/2.7.0/gems/fluentd-1.11.2/lib/fluent/engine.rb:147:in `run'
  2021-02-01 23:58:18 +0000 [error]: #0 /opt/td-agent/lib/ruby/gems/2.7.0/gems/fluentd-1.11.2/lib/fluent/supervisor.rb:603:in `block in run_worker'
  2021-02-01 23:58:18 +0000 [error]: #0 /opt/td-agent/lib/ruby/gems/2.7.0/gems/fluentd-1.11.2/lib/fluent/supervisor.rb:840:in `main_process'
  2021-02-01 23:58:18 +0000 [error]: #0 /opt/td-agent/lib/ruby/gems/2.7.0/gems/fluentd-1.11.2/lib/fluent/supervisor.rb:594:in `run_worker'
  2021-02-01 23:58:18 +0000 [error]: #0 /opt/td-agent/lib/ruby/gems/2.7.0/gems/fluentd-1.11.2/lib/fluent/command/fluentd.rb:361:in `<top (required)>'
  2021-02-01 23:58:18 +0000 [error]: #0 /opt/td-agent/lib/ruby/2.7.0/rubygems/core_ext/kernel_require.rb:72:in `require'
  2021-02-01 23:58:18 +0000 [error]: #0 /opt/td-agent/lib/ruby/2.7.0/rubygems/core_ext/kernel_require.rb:72:in `require'
  2021-02-01 23:58:18 +0000 [error]: #0 /opt/td-agent/lib/ruby/gems/2.7.0/gems/fluentd-1.11.2/bin/fluentd:8:in `<top (required)>'
  2021-02-01 23:58:18 +0000 [error]: #0 /opt/td-agent/bin/fluentd:23:in `load'
  2021-02-01 23:58:18 +0000 [error]: #0 /opt/td-agent/bin/fluentd:23:in `<top (required)>'
  2021-02-01 23:58:18 +0000 [error]: #0 /sbin/td-agent:7:in `load'
  2021-02-01 23:58:18 +0000 [error]: #0 /sbin/td-agent:7:in `<main>'
2021-02-01 23:58:18 +0000 [error]: #0 unexpected error error_class=Errno::EADDRINUSE error="Address already in use - bind(2) for 0.0.0.0:24224"
  2021-02-01 23:58:18 +0000 [error]: #0 suppressed same stacktrace
2021-02-01 23:58:18 +0000 [info]: Worker 0 finished unexpectedly with status 1
2021-02-01 23:58:19 +0000 [info]: adding filter pattern="jfrog.rt.**.service" type="record_transformer"
2021-02-01 23:58:19 +0000 [info]: adding filter pattern="jfrog.rt.**.service" type="parser"
2021-02-01 23:58:19 +0000 [info]: adding filter pattern="jfrog.rt.artifactory.request" type="record_transformer"
2021-02-01 23:58:19 +0000 [info]: adding filter pattern="jfrog.rt.frontend.request" type="record_transformer"
2021-02-01 23:58:19 +0000 [info]: adding filter pattern="jfrog.rt.metadata.request" type="record_transformer"
2021-02-01 23:58:19 +0000 [info]: adding filter pattern="jfrog.**" type="record_transformer"
2021-02-01 23:58:19 +0000 [info]: adding filter pattern="jfrog.callhome" type="record_transformer"
2021-02-01 23:58:19 +0000 [info]: adding match pattern="jfrog.callhome" type="http"
2021-02-01 23:58:19 +0000 [warn]: #0 Status code 503 is going to be removed from default `retryable_response_codes` from fluentd v2. Please add it by yourself if you wish
2021-02-01 23:58:19 +0000 [info]: adding filter pattern="jfrog.rt.artifactory.request" type="prometheus"
2021-02-01 23:58:20 +0000 [info]: adding filter pattern="jfrog.rt.artifactory.service" type="prometheus"
2021-02-01 23:58:20 +0000 [info]: adding filter pattern="jfrog.rt.artifactory.access" type="prometheus"
2021-02-01 23:58:20 +0000 [info]: adding filter pattern="jfrog.rt.access.audit" type="prometheus"
2021-02-01 23:58:20 +0000 [info]: adding source type="prometheus"
2021-02-01 23:58:20 +0000 [info]: adding source type="monitor_agent"
2021-02-01 23:58:20 +0000 [info]: adding source type="forward"
2021-02-01 23:58:20 +0000 [info]: adding source type="prometheus_monitor"
2021-02-01 23:58:20 +0000 [info]: adding source type="prometheus_output_monitor"
2021-02-01 23:58:20 +0000 [info]: adding source type="prometheus_tail_monitor"
2021-02-01 23:58:20 +0000 [info]: adding source type="tail"
2021-02-01 23:58:20 +0000 [info]: adding source type="tail"
2021-02-01 23:58:20 +0000 [info]: adding source type="tail"
2021-02-01 23:58:20 +0000 [info]: adding source type="tail"
2021-02-01 23:58:20 +0000 [info]: adding source type="tail"
2021-02-01 23:58:20 +0000 [info]: adding source type="tail"
2021-02-01 23:58:20 +0000 [info]: adding source type="tail"
2021-02-01 23:58:20 +0000 [info]: adding source type="tail"
2021-02-01 23:58:20 +0000 [info]: adding source type="tail"
2021-02-01 23:58:20 +0000 [info]: adding source type="tail"
2021-02-01 23:58:20 +0000 [info]: adding source type="tail"
2021-02-01 23:58:20 +0000 [info]: adding source type="tail"
2021-02-01 23:58:20 +0000 [info]: adding source type="tail"
2021-02-01 23:58:20 +0000 [info]: adding source type="exec"
2021-02-01 23:58:20 +0000 [info]: adding source type="exec"
2021-02-01 23:58:20 +0000 [info]: #0 starting fluentd worker pid=25680 ppid=25342 worker=0
2021-02-01 23:58:20 +0000 [info]: #0 [access_security_audit_tail] following tail of /app/jfrog/artifactory/var/log/access-security-audit.log
2021-02-01 23:58:20 +0000 [info]: #0 [artifactory_access_tail] following tail of /app/jfrog/artifactory/var/log/artifactory-access.log
2021-02-01 23:58:20 +0000 [info]: #0 [router_request_tail] following tail of /app/jfrog/artifactory/var/log/router-request.log
2021-02-01 23:58:20 +0000 [info]: #0 [metadata_request_tail] following tail of /app/jfrog/artifactory/var/log/metadata-request.log
2021-02-01 23:58:20 +0000 [info]: #0 [frontend_request_tail] following tail of /app/jfrog/artifactory/var/log/frontend-request.log
2021-02-01 23:58:20 +0000 [info]: #0 [artifactory_request_tail] following tail of /app/jfrog/artifactory/var/log/artifactory-request.log
2021-02-01 23:58:20 +0000 [info]: #0 [access_request_tail] following tail of /app/jfrog/artifactory/var/log/access-request.log
2021-02-01 23:58:20 +0000 [info]: #0 [router_traefik_tail] following tail of /app/jfrog/artifactory/var/log/router-traefik.log
2021-02-01 23:58:20 +0000 [info]: #0 [router_service_tail] following tail of /app/jfrog/artifactory/var/log/router-service.log
2021-02-01 23:58:20 +0000 [info]: #0 [metadata_service_tail] following tail of /app/jfrog/artifactory/var/log/metadata-service.log
2021-02-01 23:58:20 +0000 [info]: #0 [frontend_service_tail] following tail of /app/jfrog/artifactory/var/log/frontend-service.log
2021-02-01 23:58:20 +0000 [info]: #0 [artifactory_service_tail] following tail of /app/jfrog/artifactory/var/log/artifactory-service.log
2021-02-01 23:58:20 +0000 [info]: #0 [access_service_tail] following tail of /app/jfrog/artifactory/var/log/access-service.log
2021-02-01 23:58:20 +0000 [info]: #0 listening port port=24224 bind="0.0.0.0"
2021-02-01 23:58:20 +0000 [error]: #0 unexpected error error_class=Errno::EADDRINUSE error="Address already in use - bind(2) for 0.0.0.0:24224"
  2021-02-01 23:58:20 +0000 [error]: #0 /opt/td-agent/lib/ruby/2.7.0/socket.rb:201:in `bind'
  2021-02-01 23:58:20 +0000 [error]: #0 /opt/td-agent/lib/ruby/2.7.0/socket.rb:201:in `listen'
  2021-02-01 23:58:20 +0000 [error]: #0 /opt/td-agent/lib/ruby/gems/2.7.0/gems/fluentd-1.11.2/lib/fluent/plugin_helper/server.rb:357:in `server_create_tcp_socket'
  2021-02-01 23:58:20 +0000 [error]: #0 /opt/td-agent/lib/ruby/gems/2.7.0/gems/fluentd-1.11.2/lib/fluent/plugin_helper/server.rb:212:in `server_create_for_tcp_connection'
  2021-02-01 23:58:20 +0000 [error]: #0 /opt/td-agent/lib/ruby/gems/2.7.0/gems/fluentd-1.11.2/lib/fluent/plugin_helper/server.rb:92:in `server_create_connection'
  2021-02-01 23:58:20 +0000 [error]: #0 /opt/td-agent/lib/ruby/gems/2.7.0/gems/fluentd-1.11.2/lib/fluent/plugin/in_forward.rb:172:in `start'
  2021-02-01 23:58:20 +0000 [error]: #0 /opt/td-agent/lib/ruby/gems/2.7.0/gems/fluentd-1.11.2/lib/fluent/root_agent.rb:200:in `block in start'
  2021-02-01 23:58:20 +0000 [error]: #0 /opt/td-agent/lib/ruby/gems/2.7.0/gems/fluentd-1.11.2/lib/fluent/root_agent.rb:189:in `block (2 levels) in lifecycle'
  2021-02-01 23:58:20 +0000 [error]: #0 /opt/td-agent/lib/ruby/gems/2.7.0/gems/fluentd-1.11.2/lib/fluent/root_agent.rb:188:in `each'
  2021-02-01 23:58:20 +0000 [error]: #0 /opt/td-agent/lib/ruby/gems/2.7.0/gems/fluentd-1.11.2/lib/fluent/root_agent.rb:188:in `block in lifecycle'
  2021-02-01 23:58:20 +0000 [error]: #0 /opt/td-agent/lib/ruby/gems/2.7.0/gems/fluentd-1.11.2/lib/fluent/root_agent.rb:175:in `each'
  2021-02-01 23:58:20 +0000 [error]: #0 /opt/td-agent/lib/ruby/gems/2.7.0/gems/fluentd-1.11.2/lib/fluent/root_agent.rb:175:in `lifecycle'
  2021-02-01 23:58:20 +0000 [error]: #0 /opt/td-agent/lib/ruby/gems/2.7.0/gems/fluentd-1.11.2/lib/fluent/root_agent.rb:199:in `start'
  2021-02-01 23:58:20 +0000 [error]: #0 /opt/td-agent/lib/ruby/gems/2.7.0/gems/fluentd-1.11.2/lib/fluent/engine.rb:248:in `start'
  2021-02-01 23:58:20 +0000 [error]: #0 /opt/td-agent/lib/ruby/gems/2.7.0/gems/fluentd-1.11.2/lib/fluent/engine.rb:147:in `run'
  2021-02-01 23:58:20 +0000 [error]: #0 /opt/td-agent/lib/ruby/gems/2.7.0/gems/fluentd-1.11.2/lib/fluent/supervisor.rb:603:in `block in run_worker'
  2021-02-01 23:58:20 +0000 [error]: #0 /opt/td-agent/lib/ruby/gems/2.7.0/gems/fluentd-1.11.2/lib/fluent/supervisor.rb:840:in `main_process'
  2021-02-01 23:58:20 +0000 [error]: #0 /opt/td-agent/lib/ruby/gems/2.7.0/gems/fluentd-1.11.2/lib/fluent/supervisor.rb:594:in `run_worker'
  2021-02-01 23:58:20 +0000 [error]: #0 /opt/td-agent/lib/ruby/gems/2.7.0/gems/fluentd-1.11.2/lib/fluent/command/fluentd.rb:361:in `<top (required)>'
  2021-02-01 23:58:20 +0000 [error]: #0 /opt/td-agent/lib/ruby/2.7.0/rubygems/core_ext/kernel_require.rb:72:in `require'
  2021-02-01 23:58:20 +0000 [error]: #0 /opt/td-agent/lib/ruby/2.7.0/rubygems/core_ext/kernel_require.rb:72:in `require'
  2021-02-01 23:58:20 +0000 [error]: #0 /opt/td-agent/lib/ruby/gems/2.7.0/gems/fluentd-1.11.2/bin/fluentd:8:in `<top (required)>'
  2021-02-01 23:58:20 +0000 [error]: #0 /opt/td-agent/bin/fluentd:23:in `load'
  2021-02-01 23:58:20 +0000 [error]: #0 /opt/td-agent/bin/fluentd:23:in `<top (required)>'
  2021-02-01 23:58:20 +0000 [error]: #0 /sbin/td-agent:7:in `load'
  2021-02-01 23:58:20 +0000 [error]: #0 /sbin/td-agent:7:in `<main>'

I checked for processes which are using this port to kill them. But couldn't find any. Also, tried opening range of ports 24210 to 24250. Still did not work. Any suggestions on how to resolve or troubleshoot this issue? Thanks!
@peters95 peters95 self-assigned this Feb 5, 2021
@peters95
Copy link
Contributor

peters95 commented Feb 5, 2021 •
edited

You have another process already listing on port 24224.

You can change the port that the metrics server is exposed inside the fluent.conf.rt to resolve this issue.

Example:

<source>
  @type prometheus
  port 22222
</source>

@jithendra12
Copy link
Author

Yes, I tried it already. On the http interface of the VM, it is working fine when I give a port and metrics_path to the prometheus source. I am able to see logs printing on http interface in the browser or through curl http://<IP of VM>:24231/metrics:

<source>
  @type prometheus
  bind 0.0.0.0
  port 24231
  metrics_path /metrics
</source>

but, when I start td-agent in the VM's shell, the earlier error still shows up:
2021-02-06 00:31:44 +0000 [error]: #0 unexpected error error_class=Errno::EADDRINUSE error="Address already in use - bind(2) for 0.0.0.0:24224"
I was wondering if that effects the logs that are being forwarded to http interface.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@peters95 peters95

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@peters95 @jithendra12