Minions appears only randomly in UI

[manfredw]

Minions (several Linux or Windows OS) appears only randomly/partly in Alcali UI.
Saltstack itself works fine, all returns/events are written into DB and also shown as jobs/events in Alcali.

Keys are imported complete (all existing minions are visible).
Salt-API does not report any permissions issues, commands sent from Alcali UI are executed.

Steps to reproduce the behavior:
1: Execute Keys, Refresh
2: Keys for all existing minions are listed
3: Execute Minions, Refresh
4: Jobs for test.ping, grains.items and pillar.items executed
5: Jobs completed successfully, job details are OK
6: only some minions are shown (first 15-30)
7: additional runs of former commands or highstates does not change the behavior
8: truncating DB entries and restart the process also does not succeed.

I suspect that there is a problem with parsing the return data, but I'm not able to debug this process.
Many minions are set up identically, but even from these clients only parts are recognized, it seems
that mainly the older 3002.x are parsed correctly - but not all of them (with same minion version).

Is there a way to debug the parsing process of the returned data?
IMHO this is done in models.py and netapi.py, but there are no logging handlers.
Error log from gunicorn with loglevel debug does not show any issues.

Grains and Pillars are specific to devices - maybe problems with encoding or DB collation?
Of course we tried to use only ASCII characters within salt.

Used Software:
salt-master 3006.0 on openSUSE with python 3.6, mariadb 10.11
alcali 3006.3 on openSUSE with python 3.10
salt-minion from 3002.x up to 3006.3
Browser: latest Firefox or Chrome

[manfredw]

Update:
Did some research and found a way to debug django DB activities, but it seems that there were no abnormalities.
All executed SQL queries (selects, inserts, updates) had no errors - so I will eleminate DB issues.
I've also activated debugging in salt-api to see the alcali activities, but I don't know what should be seen there regarding minion refresh.

Now I try to understand how population and updates of the salt_minion DB table works.

When refreshing the minions, salt test.ping is sent to all minions - but what is the source of which minions to poll?

After triggering the minion refresh in UI, there are responses from ALL of my minions in the database, tables jids/salt_events/salt_returns. But these tables are populated by salt-master, not by alcali.

IMHO alcali takes the responses from salt-master, but are these taken from API responses or from DB entries?

Regardless of the data source of grains and pillars, what triggers alcali to populate/update the salt_minions table?
If there are parsing issues, how to debug them?

I found out that when populating salt_minions table manually with minon_id and empty grains/pillars ({}), then I'm able to gather grains/pillars from this specific minion by clicking the refresh button. But this is not an option with regular changing minions...