Unable to load Root Categories into ElasticSearch after a sub category was added

[brian-vdb]

Preconditions

While I was analyzing the interactions between ElasticSearch and ElasticSuite that are performed to handle data in the database, I came to the point where I wanted to know more about the category index mapping. After I created a single custom category in Magento 2, the reindexing and cache cleaning still worked fine, but when I tried loading the categories page, it gave an error. So I decided to query ElasticSearch for the category mappings and it returned only the new entry for the custom category. All of the root categories including "Default Category" were deleted from the Search Engine and not added back or updated.

Magento Version : 2.4.6

ElasticSuite Version : 2.11.5.1

Environment : Developer

Third party modules : markshust/magento2-module-disabletwofactorauth

Steps to reproduce

1. Install a new Magento 2.4.6 instance that uses markshust/magento2-module-disabletwofactorauth, ElasticSuite 2.11.5.1 and ElasticSearch 7.17.5. The PHP version on which I worked was 8.2-fpm. The installation steps can be viewed below.
2. reindex and clean cache.
3. On the admin panel, navigate to "CATALOG -> Categories"
4. Click on "Add Subcategory" and fill in the required fields. I named the category "Shirts".
5. If the page still renders, try performing a reindex and cache clean.

Expected result

1. A sub-category should be created to assign products to

Actual result

1. Error on Frontend:

2. Database Content:

3. Category mapping contents:

GET brian.local:9200/magento2_default_catalog_category/_search?scroll=1m
Content-Type: application/json
Accept: application/json

{
  "size": 1000,
  "query": {
    "match_all": {}
  }
}

HTTP/1.1 200 OK
X-elastic-product: Elasticsearch
content-type: application/json; charset=UTF-8
content-encoding: gzip
content-length: 503

{
    "_scroll_id": "FGluY2x1ZGVfY29udGV4dF91dWlkDXF1ZXJ5QW5kRmV0Y2gBFnZIcWo4LVJuUWx5Y1Z2WDBEc19vekEAAAAAAAAANBZyUFRuSkNTTFJseWFpRXI1eWJHN1RR",
    "took": 14,
    "timed_out": false,
    "_shards": {
        "total": 1,
        "successful": 1,
        "skipped": 0,
        "failed": 0
    },
    "hits": {
        "total": {
            "value": 1,
            "relation": "eq"
        },
        "max_score": 1.0,
        "hits": [
            {
                "_index": "magento2_default_catalog_category_20240221_133048",
                "_type": "_doc",
                "_id": "3",
                "_score": 1.0,
                "_source": {
                    "entity_id": "3",
                    "attribute_set_id": "3",
                    "parent_id": "2",
                    "created_at": "2024-02-21 13:40:14",
                    "updated_at": "2024-02-21 13:40:14",
                    "path": "1\/2\/3",
                    "position": "1",
                    "level": "2",
                    "children_count": "0",
                    "is_active": "1",
                    "is_displayed_in_autocomplete": [
                        true
                    ],
                    "option_text_is_displayed_in_autocomplete": [
                        "Display Category in Autocomplete"
                    ],
                    "name": [
                        "Shirts"
                    ],
                    "url_key": [
                        "shirts"
                    ],
                    "url_path": [
                        "shirts"
                    ]
                }
            }
        ]
    }
}

4. Default Category contents:

GET brian.local:9200/magento2_default_catalog_category/_doc/2

HTTP/1.1 404 Not Found
X-elastic-product: Elasticsearch
content-type: application/json; charset=UTF-8
content-encoding: gzip
content-length: 112

{
    "_index": "magento2_default_catalog_category_20240221_134453",
    "_type": "_doc",
    "_id": "2",
    "found": false
}

5. System Logs on Page Render

[2024-02-21T15:10:31.587437+00:00] main.INFO: Broken reference: the 'tracking' element cannot be added as child to 'header', because the latter doesn't exist [] []
[2024-02-21T15:10:31.588674+00:00] main.INFO: Broken reference: the 'notification_area' element cannot be added as child to 'notifications', because the latter doesn't exist [] []
[2024-02-21T15:10:31.589440+00:00] main.INFO: Broken reference: the 'unread_system_messages' element cannot be added as child to 'notifications', because the latter doesn't exist [] []
[2024-02-21T15:10:31.590290+00:00] main.INFO: Broken reference: the 'notification_window' element cannot be added as child to 'notifications', because the latter doesn't exist [] []
[2024-02-21T15:10:31.591085+00:00] main.INFO: Broken reference: the 'notification.messages' element cannot be added as child to 'header', because the latter doesn't exist [] []
[2024-02-21T15:10:31.591710+00:00] main.INFO: Broken reference: the 'global_notices' element cannot be added as child to 'global.notices', because the latter doesn't exist [] []
[2024-02-21T15:10:31.592528+00:00] main.INFO: Broken reference: the 'logo' element cannot be added as child to 'header', because the latter doesn't exist [] []
[2024-02-21T15:10:31.593287+00:00] main.INFO: Broken reference: the 'global.search' element cannot be added as child to 'header', because the latter doesn't exist [] []
[2024-02-21T15:10:31.593856+00:00] main.INFO: Broken reference: the 'user' element cannot be added as child to 'header', because the latter doesn't exist [] []
[2024-02-21T15:10:31.594456+00:00] main.INFO: Broken reference: the 'menu' element cannot be added as child to 'page.menu', because the latter doesn't exist [] []
[2024-02-21T15:10:31.595039+00:00] main.INFO: Broken reference: the 'breadcrumbs' element cannot be added as child to 'page.breadcrumbs', because the latter doesn't exist [] []
[2024-02-21T15:10:31.595586+00:00] main.INFO: Broken reference: the 'formkey' element cannot be added as child to 'page.formkey', because the latter doesn't exist [] []
[2024-02-21T15:10:31.596142+00:00] main.INFO: Broken reference: the 'page.title' element cannot be added as child to 'main.top', because the latter doesn't exist [] []
[2024-02-21T15:10:31.596791+00:00] main.INFO: Broken reference: the 'messages' element cannot be added as child to 'page.messages', because the latter doesn't exist [] []
[2024-02-21T15:10:31.597352+00:00] main.INFO: Broken reference: the 'page.actions.toolbar' element cannot be added as child to 'page.main.actions', because the latter doesn't exist [] []
[2024-02-21T15:10:31.597872+00:00] main.INFO: Broken reference: the 'legal' element cannot be added as child to 'footer', because the latter doesn't exist [] []
[2024-02-21T15:10:31.598590+00:00] main.INFO: Broken reference: the 'header.inner.left' element cannot be added as child to 'header', because the latter doesn't exist [] []
[2024-02-21T15:10:31.599165+00:00] main.INFO: Broken reference: the 'header.inner.right' element cannot be added as child to 'header', because the latter doesn't exist [] []
[2024-02-21T15:10:31.599855+00:00] main.INFO: Broken reference: the 'user' tries to reorder itself towards '', but their parents are different: 'header' and '' respectively. [] []
[2024-02-21T15:10:31.600418+00:00] main.INFO: Broken reference: the 'notification.messages' tries to reorder itself towards 'user', but their parents are different: 'header' and '' respectively. [] []
[2024-02-21T15:10:31.600985+00:00] main.INFO: Broken reference: the 'logo' tries to reorder itself towards '', but their parents are different: 'header' and '' respectively. [] []
[2024-02-21T15:10:31.601538+00:00] main.INFO: Broken reference: the 'global.search' tries to reorder itself towards 'logo', but their parents are different: 'header' and '' respectively. [] []
[2024-02-21T15:10:31.602087+00:00] main.INFO: Broken reference: the 'page.actions.toolbar' tries to reorder itself towards '', but their parents are different: 'page.main.actions' and '' respectively. [] []
[2024-02-21T15:10:31.603354+00:00] main.INFO: Broken reference: the 'header.inner.left' tries to reorder itself towards '', but their parents are different: 'header' and '' respectively. [] []
[2024-02-21T15:10:31.603981+00:00] main.INFO: Broken reference: the 'header.inner.right' tries to reorder itself towards 'header.inner.left', but their parents are different: 'header' and '' respectively. [] []
[2024-02-21T15:10:31.604790+00:00] main.INFO: Broken reference: the 'notification.messages' tries to reorder itself towards 'user', but their parents are different: 'header.inner.right' and '' respectively. [] []
[2024-02-21T15:10:36.981332+00:00] main.INFO: Broken reference: the 'notification.messages' tries to reorder itself towards 'user', but their parents are different: 'header.inner.right' and 'header' respectively. [] []
[2024-02-21T15:10:42.745754+00:00] main.INFO: Broken reference: the 'notification.messages' tries to reorder itself towards 'user', but their parents are different: 'header.inner.right' and 'header' respectively. [] []
[2024-02-21T15:10:46.366684+00:00] main.ERROR: The "is_virtual_category" attribute name is invalid. Reset the name and try again. [] []
[2024-02-21T15:10:46.487345+00:00] main.ERROR: The "is_virtual_category" attribute name is invalid. Reset the name and try again. [] []

[vahonc]

Hello @brian-vdb,

Have you run the bin/magento setup:upgrade command? Were there any errors during its execution?

Are you sure that you don't disable any ElasticSuite modules, especially the module-elasticsuite-virtual-category?

BR,
Vadym

[brian-vdb]

I have run setup:upgrade, but I can check if enabling the module manually would fix it. I have not tried that

…

On Tue, 27 Feb 2024, 12:26 Vadym Honcharuk, ***@***.***> wrote: Hello @brian-vdb <https://github.com/brian-vdb>, Have you run the bin/magento setup:upgrade command? Were there any errors during its execution? Are you sure that you don't disable any ElasticSuite modules, especially the module-elasticsuite-virtual-category? BR, Vadym — Reply to this email directly, view it on GitHub <#3192 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AWTFOT4PIZC3CI2HKB45MJDYVW7EXAVCNFSM6AAAAABDTHT4JSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSNRWGMZTSMRUGA> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[brian-vdb]

Hello Vadym, Due to my other responsibilities, I did not manage to test this up till now, but I have further looked into the matter now. I did notice that I forgot to enable the "Smile_ElasticsuiteCatalogGraphQL" and "Smile_ElasticsuiteIndices" modules in my installation, however when I did test the category creation with them enabled, the error persisted. To provide a wider context for my installation, I will add my installation steps after installing a bare Magento 2.4.6 backend with the "Markshust_DisableTwoDactorAuth" module: - rm -rf var/di/* var/generation/* - bin/magento deploy:mode:set developer - bin/magento indexer:reindex - bin/magento cache:clean - composer require smile/elasticsuite ~2.11.5.1 - bin/magento module:enable Smile_ElasticsuiteAdminNotification Smile_ElasticsuiteCore Smile_ElasticsuiteCatalog Smile_ElasticsuiteCatalogGraphQl Smile_ElasticsuiteCatalogRule Smile_ElasticsuiteCatalogOptimizer Smile_ElasticsuiteTracker Smile_ElasticsuiteThesaurus Smile_ElasticsuiteSwatches Smile_ElasticsuiteIndices Smile_ElasticsuiteAnalytics Smile_ElasticsuiteVirtualCategory - bin/magento setup:upgrade After this, I had to manually change the module versions in the "setup_module" database table as they were all null. I also had to change the Catalog search engine to Elasticsuite, but the communication with the Elasticsearch endpoint appeared to be fine as indexing and searching for products worked as expected. - bin/magento indexer:reindex - bin/magento cache:clean After this, I opened the admin panel and created a sub-category "Shirts" under the default root category. But after it deleted the default category again using this installation, I tried to do another refresh of the module like this: - bin/magento setup:upgrade - bin/magento indexer:reindex - bin/magento cache:clean And the problem persisted after this. I checked the database and the default root category is definitely still saved in the database, but the category is missing from the elasticsearch indexed documents. I would be happy if this is an error on my side and if this is the case, please let me know, but not having the ElasticSuite module installed does not necessarily hinder my research in any way. with kind regards, Brian van den Berg Op di 27 feb 2024 om 12:26 schreef Vadym Honcharuk ***@***.***

…

: Hello @brian-vdb <https://github.com/brian-vdb>, Have you run the bin/magento setup:upgrade command? Were there any errors during its execution? Are you sure that you don't disable any ElasticSuite modules, especially the module-elasticsuite-virtual-category? BR, Vadym — Reply to this email directly, view it on GitHub <#3192 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AWTFOT4PIZC3CI2HKB45MJDYVW7EXAVCNFSM6AAAAABDTHT4JSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSNRWGMZTSMRUGA> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[vahonc]

@brian-vdb,

I will try to reproduce your issue according to your steps on this week.

BR,
Vadym