Need help with understanding why my ESP32 is disconnecting from caster

[mbackonja]

I often have disconnections between esp32 and the caster. These are in random intervals between 5 minutes and 20 minutes. In periods when there are no interruptions, the connection lasts for 7+ hours (a couple of days a week), while most days I have 30+ interruptions every 5-20 minutes...

WiFi is not a problem, I changed 2 3G routers, 1 4G router, and home WiFi... My RTK Base is right next to the router...
My ESP32 is on the UPS, I restarted it today, but it had 27 days of uptime previously...

Nothing helps...

I tried both the stable version 0.5.2 and the latest version 0.5.4...

When I open log.html, this is what I see:

...
15	18:40:26.731ms	INFO	MAIN	Synced time from SNTP
16	19:23:40.414ms	WARN	NTRIP_SERVER	Disconnected from rtk2go.com:2101/XXX
17	19:23:40.418ms	INFO	NTRIP_SERVER	Connecting to rtk2go.com:2101/XXX
18	19:23:41.678ms	INFO	NTRIP_SERVER	Successfully connected to rtk2go.com:2101/XXX
19	19:23:43.074ms	WARN	NTRIP_SERVER	Disconnected from rtk2go.com:2101/XXX
20	19:23:43.078ms	INFO	NTRIP_SERVER	Connecting to rtk2go.com:2101/XXX
21	19:23:43.827ms	INFO	NTRIP_SERVER	Successfully connected to rtk2go.com:2101/XXX
22	19:40:26.668ms	INFO	MAIN	Synced time from SNTP
23	19:46:50.913ms	WARN	NTRIP_SERVER	Disconnected from rtk2go.com:2101/XXX
24	19:46:50.917ms	INFO	NTRIP_SERVER	Connecting to rtk2go.com:2101/XXX
25	19:46:52.912ms	INFO	NTRIP_SERVER	Successfully connected to rtk2go.com:2101/XXX
26	19:46:55.103ms	WARN	NTRIP_SERVER	Disconnected from rtk2go.com:2101/XXX
27	19:46:55.106ms	INFO	NTRIP_SERVER	Connecting to rtk2go.com:2101/XXX
28	19:46:56.904ms	INFO	NTRIP_SERVER	Successfully connected to rtk2go.com:2101/XXX
29	19:56:31.913ms	WARN	NTRIP_SERVER	Disconnected from rtk2go.com:2101/XXX
30	19:56:31.917ms	INFO	NTRIP_SERVER	Connecting to rtk2go.com:2101/XXX
31	19:56:42.192ms	ERROR	NTRIP_SERVER	/home/nebkat/Work/esp32/esp32-xbee/main/interface/ntrip_server.c:146 (ntrip_server_task): Could not receive response from caster: 11 No more processes
32	19:56:44.194ms	INFO	NTRIP_SERVER	Connecting to rtk2go.com:2101/XXX
33	19:56:45.716ms	INFO	NTRIP_SERVER	Successfully connected to rtk2go.com:2101/XXX
...

And I can't understand what's the reason for those disconnections...

@nebkat can you help me understand what's causing it?

P.S. This message

31	19:56:42.192ms	ERROR	NTRIP_SERVER	/home/nebkat/Work/esp32/esp32-xbee/main/interface/ntrip_server.c:146 (ntrip_server_task): Could not receive response from caster: 11 No more processes

I never saw it on 0.5.2 version, this is the first time I see it now (on 0.5.4 version)

[mbackonja]

In the meantime (1 hour and 20 minutes since my first post):

3	20:10:04.423ms	WARN	NTRIP_SERVER	Disconnected from rtk2go.com:2101/XXX
4	20:10:04.427ms	INFO	NTRIP_SERVER	Connecting to rtk2go.com:2101/XXX
5	20:10:07.972ms	INFO	NTRIP_SERVER	Successfully connected to rtk2go.com:2101/XXX
6	20:10:12.133ms	WARN	NTRIP_SERVER	Disconnected from rtk2go.com:2101/XXX
7	20:10:12.136ms	INFO	NTRIP_SERVER	Connecting to rtk2go.com:2101/XXX
8	20:10:14.018ms	INFO	NTRIP_SERVER	Successfully connected to rtk2go.com:2101/XXX
9	20:19:23.423ms	WARN	NTRIP_SERVER	Disconnected from rtk2go.com:2101/XXX
10	20:19:23.427ms	INFO	NTRIP_SERVER	Connecting to rtk2go.com:2101/XXX
11	20:19:25.002ms	INFO	NTRIP_SERVER	Successfully connected to rtk2go.com:2101/XXX
12	20:19:28.143ms	WARN	NTRIP_SERVER	Disconnected from rtk2go.com:2101/XXX
13	20:19:28.146ms	INFO	NTRIP_SERVER	Connecting to rtk2go.com:2101/XXX
14	20:19:28.893ms	INFO	NTRIP_SERVER	Successfully connected to rtk2go.com:2101/XXX
15	20:40:26.840ms	INFO	MAIN	Synced time from SNTP
16	20:50:31.891ms	WARN	NTRIP_SERVER	Disconnected from rtk2g
17	20:50:31.894ms	INFO	NTRIP_SERVER	Connecting to rtk2go.com:2101/XXX
18	20:50:32.980ms	INFO	NTRIP_SERVER	Successfully connected to rtk2go.com:2101/XXX

Uptime 06:54:53

And rtk2go status of my base

Last Restart 04:03 M:S Up(1/30) (04:03 MIN:SEC 100.0% overall) is 30th connection

[mbackonja]

There was caster restart 1 Day 07:56 (HH:MM) ago

My status line on rtk2go says:

Last Restart 07:07 M:S Up(2/95) (43:47 MIN:SEC 99.1% overall) is 95th connection

And my log.html is still full of "Disconnected from rtk2go.com:2101/XXX" followed by "Connecting" and "Successfully connected" messages...

Any idea what is happening, how can I debug this?

[DavidKelleySCSC]

Can you mention the mountPt name used here so we (RTK2go staff folks) can check this out further for you? PM me if you wish.

Aside; The RTK2go.com host machine has been under attack for about a week now. It reached a huge level overnight, but has caused some caster restarts over the past few days as well. We are mitigating this by blocking groups of IPs but this can always disrupt a legitimate user as well. It working again now but we will be watching it closely for a while to be sure.

[mbackonja]

Hello @DavidKelleySCSC where can I PM you? Should I just send email to support@use-snip.com or?

Yes, I'm aware of the problems, today was the worse day, I'm glad you are back online! It was half the day (in my timezone)...

[DavidKelleySCSC]

Yes that email works well, and I get copies of the traffic.