MeshAgents went offline. Cannot re-add or reestablish communication outside server subnet.

[It-Fella]

Windows Mesh Agents no longer communicating with MC server

Not sure this can be reproduced as a bug. Apologies if this is the wrong area to ask.

Expected behavior
I expect to see network devices online and to be reached through MC web gui but they are not.

Screenshots

Server Software (please complete the following information):

• OS: Window 10 Pro x64 22H2
• Virtualization: Install with node.js through windows command line
• Network: Subnetted LAN, local
• Version: MC version 1.1.22
• Node: 18.14.2

Client Device (please complete the following information):

• Device: Windows Desktops/laptops
• OS: Windows 10 pro x64 22H2
• Network: [e.g. Local to Meshcentral
• Browser: Chrome, Edge, Firefox
• MeshCentralRouter Version: N/A

Remote Device (please complete the following information):

• Device: Windows Desktops/laptops
• OS: [e.g. Windows 10 22H2]
• Network: [e.g. Local to Meshcentral
• Current Core Version (if known): Current Core: Dec 9 2022, 3840084365
  Agent Time: 2024-04-04 11:55:03.835-05:00.
  User Rights: 0xffffffff.
  Platform: win32.
  Capabilities: 15.
  Server URL: wss://10.2.200.71:443/agent.ashx.
  OS: Microsoft Windows 10 Pro - 22H2/19045.
  Modules: amt-apfclient, amt-lme, amt-manage, amt-mei, computer-identifiers, monitor-border, smbios, sysinfo, util-agentlog, wifi-scanner-windows, wifi-scanner, win-console, win-deskutils, win-info, win-securitycenter, win-terminal, win-virtual-terminal, win-volumes.
  Server Connection: true, State: 1.
  Application Location: C:\Program Files\Mesh Agent\

this is the only device that remains online and is located on the same subnet as the server.

Additional context
Add any other context about the problem here.

json config file remains the same as it was at install. I made no changes and everything worked out of the box so to speak.

All AMT enabled devices are discovered and have an agent on them but I am unable to communicate with them.

Your config.json file

{
  "$schema": "https://raw.githubusercontent.com/Ylianst/MeshCentral/master/meshcentral-config-schema.json",
  "__comment1__": "This is a simple configuration file, all values and sections that start with underscore (_) are ignored. Edit a section and remove the _ in front of the name. Refer to the user's guide for details.",
  "__comment2__": "See node_modules/meshcentral/sample-config-advanced.json for a more advanced example.",
  "settings": {
    "_cert": "myserver.mydomain.com",
    "_WANonly": true,
    "_LANonly": true,
    "_sessionKey": "MyReallySecretPassword1",
    "_port": 443,
    "_aliasPort": 443,
    "_redirPort": 80,
    "_redirAliasPort": 80
  },
  "domains": {
    "": {
      "_title": "MyServer",
      "_title2": "Servername",
      "_minify": true,
      "_newAccounts": true,
      "_userNameIsEmail": true
    }
  },
  "_letsencrypt": {
    "__comment__": "Requires NodeJS 8.x or better, Go to https://letsdebug.net/ first before trying Let's Encrypt.",
    "email": "myemail@mydomain.com",
    "names": "myserver.mydomain.com",
    "production": false
  }
}

[si458]

right so meshcentral is running in LAN mode, because you havent set a cert name as its commented _cert
so the first thing to check is ur screenshot shows u have antivirus installed cynet antivirus, check if the antivirus has a firewall enabled, and make sure the meshagent is allowed through the firewall.
when meshcentral is run in LAN mode, each device sends a BROADCAST packet over the network which meshcentral will pick up and reply back

[It-Fella]

I removed CYNET from the machine meshcentral is hosted on as well as from a remote machine with the mesh agent on it and it did not seem to fix the issue. Is there a service name I should be looking for? C:\meshcentral\WinService\daemon\meshcentral.exe is listed on the host machine as the service.

I did further test this on a machine that is not and has never been protected by CYNET and was still unable to communicate with the machine via meshagent but can ping and RDP into it.

Thanks.

[si458]

yes thats the daemon to run the service,
you might need to add node.exe also into the allowed list on your meshcentral server
you can also add meshagent.exe into the firewall rules too on the local machines.
can any of the remote machines goto the web ui?
https://IPOFSERVER ?
also is the config.json above the same as yours?
(you can get urs by going into the web ui then my server then show server configuration

[It-Fella]

OK. I will check with our system manager on adding exceptions for node.exe and meshagent.exe

All of the unresponsive remote machines I have tested can reach the MC server address without issue.

Here is the current output from the show server configuration:

{
"$schema": "https://raw.githubusercontent.com/Ylianst/MeshCentral/master/meshcentral-config-schema.json",
"comment1": "This is a simple configuration file, all values and sections that start with underscore (_) are ignored. Edit a section and remove the _ in front of the name. Refer to the user's guide for details.",
"comment2": "See node_modules/meshcentral/sample-config-advanced.json for a more advanced example.",
"settings": {
"_cert": "myserver.mydomain.com",
"_WANonly": true,
"LANonly": true,
"_sessionKey": "MyReallySecretPassword1",
"_port": 443,
"_aliasPort": 443,
"_redirPort": 80,
"_redirAliasPort": 80
},
"domains": {
"": {
"_title": "MyServer",
"_title2": "Servername",
"_minify": true,
"_newAccounts": true,
"_userNameIsEmail": true
}
},
"_letsencrypt": {
"comment": "Requires NodeJS 8.x or better, Go to https://letsdebug.net/ first before trying Let's Encrypt.",
"email": "myemail@mydomain.com",
"names": "myserver.mydomain.com",
"production": false
}
}

Thanks

[si458]

If each node can see the web interface then it shouod be OK, so it must be something else.

If u run the meshagent.exe on a node, u should get a panel, if u click the info button it should show more information. Can u share a screenshot of it?

Also I would also check the command line console for the meshcentral and see if the is any errors being shown
Or
Click the show server errors in the Web ui, and see if any errors are being shown?

[It-Fella]

As a side note, I set up another instance of MC on another machine which is located on the same network as my host I am having issues with. I was able to add and connect machines across all subnets without issue. That seems to confirm that something is wrong with my MC instance an it is not related to antivirus, firewalls or security.

I really appreciate your time on this.

Thanks!

[si458]

dont run 2 meshcentral instances at the same time, this will confuse the agents!!
but yes if you where able to create another instance of meshcentral and install the agents on that server ok and they all connect and last reboots too, then i think the must be a weird issue/firewall issue with ur old server?

[It-Fella]

Ok. So, no further investigation at this time is needed it sounds like.

Is there any way to get my existing agents to respond to a new instance of MC?

Thanks.

[si458]

Sadly this is only possible if u have setup the the machines with a dns name.

As with a dns name, u would just point it to the new vm ip, copy over meshcentral-data and away u go.

But u could try doing that as ur only using local?

Copy the whole meshcentral-data from old to new.
Stop old, start new, see if they all come bk online?