Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Magento product search in admin product grid not working correctly. No OR functionality #38587

Open
1 of 5 tasks
cptX opened this issue Apr 6, 2024 · 19 comments · May be fixed by #38634
Open
1 of 5 tasks

Magento product search in admin product grid not working correctly. No OR functionality #38587

cptX opened this issue Apr 6, 2024 · 19 comments · May be fixed by #38634
Assignees
Labels
Area: Account Component: Search Issue: Confirmed Gate 3 Passed. Manual verification of the issue completed. Issue is confirmed Priority: P2 A defect with this priority could have functionality issues which are not to expectations. Progress: PR in progress Reported on 2.4.6 Indicates original Magento version for the Issue report. Reproduced on 2.4.x The issue has been reproduced on latest 2.4-develop branch

Comments

@cptX
Copy link

cptX commented Apr 6, 2024

Preconditions and environment

  • Magento version
    2.4.5 and 2.4.6
  • Anything else that would help a developer reproduce the bug
    In admin product list let's try the following:
    Let's say we have a product called "Erika Running Short".
    If we search with full name the product appears in the results.
    If we search for "Erika Running" the product appears.
    But if we search for "Erika Short" no product appears.

In my everyday administering I search for product names or titles and I cannot remember all the words in between. So I need to put 2 words included in the title and expecting to get the corresponding products. Now I get 0 products in the product list search results when I don't put all words in between the wanted keywords.

Steps to reproduce

  1. Open a magento instance (2.4.5 or 2.4.6) which has some products.
    Let's say it has the default sample products magento 2.4.5 and 2.4.6 have.

  2. Go to Catalog > Products

  3. Search for "Erika Running Short". There will be several results

  4. Search for "Erika Short". There will be no result!

Expected result

When searching for "Erika Short" I want the system to return all the products that include these 2 words in their title independently of the position these 2 words are placed in the products titles.
For example If a product has the title "Erika Running Short", it should appear when searching for "Erika Short"

Actual result

Now when searching for "Erika Short" no products appear, although there are many products available with the title "Erika Running Short"

Additional information

No response

Release note

No response

Triage and priority

  • Severity: S0 - Affects critical data or functionality and leaves users without workaround.
  • Severity: S1 - Affects critical data or functionality and forces users to employ a workaround.
  • Severity: S2 - Affects non-critical data or functionality and forces users to employ a workaround.
  • Severity: S3 - Affects non-critical data or functionality and does not force users to employ a workaround.
  • Severity: S4 - Affects aesthetics, professional look and feel, “quality” or “usability”.
Copy link

m2-assistant bot commented Apr 6, 2024

Hi @cptX. Thank you for your report.
To speed up processing of this issue, make sure that the issue is reproducible on the vanilla Magento instance following Steps to reproduce. To deploy vanilla Magento instance on our environment, Add a comment to the issue:


Join Magento Community Engineering Slack and ask your questions in #github channel.
⚠️ According to the Magento Contribution requirements, all issues must go through the Community Contributions Triage process. Community Contributions Triage is a public meeting.
🕙 You can find the schedule on the Magento Community Calendar page.
📞 The triage of issues happens in the queue order. If you want to speed up the delivery of your contribution, join the Community Contributions Triage session to discuss the appropriate ticket.

@m2-community-project m2-community-project bot added this to Ready for Confirmation in Issue Confirmation and Triage Board Apr 6, 2024
@cptX cptX changed the title Magento product search in admin product list not working correctly Magento product search in admin product list not working correctly. No OR functionality Apr 6, 2024
@cptX cptX changed the title Magento product search in admin product list not working correctly. No OR functionality Magento product search in admin product grid not working correctly. No OR functionality Apr 6, 2024
@engcom-Dash engcom-Dash self-assigned this Apr 7, 2024
Copy link

m2-assistant bot commented Apr 7, 2024

Hi @engcom-Dash. Thank you for working on this issue.
In order to make sure that issue has enough information and ready for development, please read and check the following instruction: 👇

    1. Verify that issue has all the required information. (Preconditions, Steps to reproduce, Expected result, Actual result).
    1. Verify that issue has a meaningful description and provides enough information to reproduce the issue.
    1. Add Area: XXXXX label to the ticket, indicating the functional areas it may be related to.
    1. Verify that the issue is reproducible on 2.4-develop branch
      Details- Add the comment @magento give me 2.4-develop instance to deploy test instance on Magento infrastructure.
      - If the issue is reproducible on 2.4-develop branch, please, add the label Reproduced on 2.4.x.
      - If the issue is not reproducible, add your comment that issue is not reproducible and close the issue and stop verification process here!

@drinkingsouls
Copy link

drinkingsouls commented Apr 9, 2024

I agree, the search functionality from the Admin Product Grid is annoyingly limited.
We've had the same issue since launching Magento 2.
The frontend search with Elasticsearch more helpful, I wonder why this was never integrated into the Admin side.

@engcom-Dash engcom-Dash added the Reported on 2.4.6 Indicates original Magento version for the Issue report. label Apr 9, 2024
@engcom-Dash
Copy link

Hi @cptX

Thanks for reproting and collaboration.

Verified the issue in magento 2.4 dev instance and the issue is reproducible.

Steps to reproduce

  1. Open a magento instance (2.4.5 or 2.4.6) which has some products.
  2. Let's say it has the default sample products magento 2.4.5 and 2.4.6 have.
  3. Go to Catalog > Products
  4. Search for "Erika Running Short". There will be several results
  5. Search for "Erika Short". There will be no result!

Magento product search in admin product grid not working correctly. We are not seeing results when searching with "Erika Short".

Hence, confirming the issue.

Please refer the attached screenshots.

Screenshot 2024-04-10 at 1 37 11 PM Screenshot 2024-04-10 at 6 45 20 PM

@engcom-Dash engcom-Dash added Component: Search Issue: Confirmed Gate 3 Passed. Manual verification of the issue completed. Issue is confirmed Reproduced on 2.4.x The issue has been reproduced on latest 2.4-develop branch Area: Account labels Apr 10, 2024
@m2-community-project m2-community-project bot moved this from Ready for Confirmation to Confirmed in Issue Confirmation and Triage Board Apr 10, 2024
@github-jira-sync-bot
Copy link

✅ Jira issue https://jira.corp.adobe.com/browse/AC-11757 is successfully created for this GitHub issue.

Copy link

m2-assistant bot commented Apr 10, 2024

✅ Confirmed by @engcom-Dash. Thank you for verifying the issue.
Issue Available: @engcom-Dash, You will be automatically unassigned. Contributors/Maintainers can claim this issue to continue. To reclaim and continue work, reassign the ticket to yourself.

@engcom-Dash engcom-Dash added the Priority: P2 A defect with this priority could have functionality issues which are not to expectations. label Apr 10, 2024
@m2-community-project m2-community-project bot added this to Ready for Development in High Priority Backlog Apr 10, 2024
@digitalrisedorset
Copy link

@magento I am working on this

@digitalrisedorset
Copy link

I have now committed some changes that makes the product 'Erika Running Short' appearing when using the full text search term 'Erika short'. I am yet to add tests, documentation. Although we are seeing the product as expected, I am still trying to understand how to handle the elasticsearch boost parameter to ensure the results are more sorted more intuitively

@cptX
Copy link
Author

cptX commented Apr 17, 2024

Hi @digitalrisedorset,
can you please share the code you modified? Also, How is elasticsearch related to this search? Are you trying to use elasticsearch for the admin instead of the embedded mysql search? This would be excellent if true!

@digitalrisedorset
Copy link

@cptX #38630
--> yes, there was a hook already in place but this hook was using Magento database, I swapped it to use elasticsearch now

@cptX
Copy link
Author

cptX commented May 2, 2024

Any progress on this? Are there any practical specific steps to follow in order to have some enhanced functionality, either by the standard mysql search or by using elasticsearch?

@digitalrisedorset
Copy link

@cptX is the question for me? Apologies, I have joined the project recently and I may not have performed the steps you'd expect by now.
The situation with this work is: I have committed some changes and made a PR from them. This PR is now awaiting to be reviewed. The changes do consist in taking over the backend product full text feature and make use of elasticsearch instead.

@digitalrisedorset
Copy link

the PR is #38634

@cptX
Copy link
Author

cptX commented May 2, 2024

Hi @digitalrisedorset, can there be any implications from your modification? Do you believe that this would be a better solution than just enabling some OR functionality? I'm afraid that with elasticsearch the results could be more broad than needed and products that are not needed could come to surface. If I want just to enable OR functionality in the existing mysql search what should I change?

@digitalrisedorset
Copy link

@cptX thanks for this feedback. Using elasticsearch has the benefits to reuse the elasticsearch built-in engine strategies that bring the products that are relevant to the keywords in the full text search: this is the same use experience as the frontend.
Using mysql and making an OR search would imply to start all over again with different changes.

  • I was hoping we should first deploy and test what we have right now to make a judgement of UX, relevance and performance

@digitalrisedorset
Copy link

digitalrisedorset commented May 6, 2024

@cptX I have had more thoughts on how to handle this task. I will put a summary of these thoughts so that we can decide how we can move on before making any additional changes. Ideally, I’d like a confirmation of our next move by more than 2 people so that we are assured we perform some changes for which we understand the solution before its implementation

- the admin uses Mysql to retrieve the products for the Catalog/Product admin filters:
This is what we currently have and I have made some comments below on this particular context:

  • the admin search currently uses mysql and no OR search is implemented
  • if we add an OR search, it may bring results that are less relevant than what the search results returns at this time unless we implement a relevance algorithm as well. Ideally, we'd need an OR search that also scores the matches with 3 words better than the matches with 2 words and so on (to illustrate with the example you provide, the search 'Erika Short' would need to score better the products that make use of both keywords than the other products that make use of either 'Erika' or 'Short' but not both

- the admin search uses Elasticsearch to enhance a search that has the relevance and the OR search built-in
This is the solution I have come up with and I add the justifications for it below:

  • this method was chosen because it resolves 2 of the problems that are identified in this issue: 1. Erika Short returns results (meaning we are doing something similar to an OR search), 2. ElasticSearch will score the matches better if the products make use of more of the search keywords
  • the problem that ElasticSearch has is that is it originally built for the frontend. As a consequence, Elasticsearch does not return any invisible products. Yet, the admin user may want to search for these products. To address this issue, I have currently created a post mysql process that pulls the simple products for the matches that are configurable
  • The ElasticSearch implementation does not take over the mysql implementation fully. Instead, what the ElasticSearch implementation performs is a search for matches for the fulltext keywords and adds the simple products if relevant. This search does not disabled the search using the other filters (sku, product name, store, visibility..). For this reason, this implementation can return the invisible products in the catalog as long as it is not done whilst a fulltext search occurs

Conclusion:
If we simply choose to add an OR search and do not have relevance, we will have some results with less relevance than now at the top of the product grid.
If we use ElasticSearch, we may have the concern that we use a solution that is not designed to be used for the backend.
Both solutions have pros and cons and we should decide on the next step once we have confirmed these pros and cons are all understood and once we have a final idea on how to finalise this solution.

@digitalrisedorset
Copy link

@magento run all tests

@cptX
Copy link
Author

cptX commented May 11, 2024

@digitalrisedorset thanks for spending time explaining the possibilities.

I'll describe my everyday needs as the eshop manager:
Let's say I have many chargers for electric vehicles (check here https://electrons.gr/en/batteries/chargers)
As you see the names are with this format:

  • Charger 54.6V 2A with 5.5x2.1 plug for lithium ion (Li-Ion) 13s 46.8V (48V) battery
  • Charger 54.6V 2A with GX16-3 plug for lithium ion (Li-Ion) 13s 46.8V (48V) battery
  • Charger 54.6V 2A with Τ-type / IEC13 plug for lithium ion (Li-Ion) 13s 46.8V (48V) battery

If we enable the OR functionality I can very easily type "charger 54.6V GX16" for example to find the second item. Without OR functionality I have to type ALL words up to GX16 until I have the correct item returned, in our case "Charger 54.6V 2A with GX16"! As you understand it's impossible to remember all the correct words that exist between the important keywords. So almost all my searches fail with no results. Also, if I know that the system works with OR I can use my search keywords wisely and put only the keywords that exist in the specific item I want to find, so I don't think any irrelevant items will come out. Anyway more irrelevant items are better than no returned items.

So in any case I'm pretty confident the OR functionality is absolutely necessary. Now regarding irrelevant items, of course taking advantage of the elasticsearch power would be great but then I'm afraid that this indeed will bring irrelevant items. Specifically I want you to clarify if the returned items will follow exactly the same search methodology as in frontend. For the frontend search I have in every product an extra field where I write keywords, and these keywords are searchable. If these keywords are also used in adminsearch then it will indeed return many more irrelevant products.

As a simpler and first step can you describe what can I do to enable just a simple OR functionality with the existing mysql implementation? At the moment I cannot do my required job. At least with the OR I will be able to do it.

@digitalrisedorset
Copy link

digitalrisedorset commented May 11, 2024

Hi @cptX , how about you give it a try to this solution that I have written. I suggest you download the file below, install it and enable it in your store and perform couple of tests. If it does not work, then we know right away that we can't go with this on Magento on a bigger scale.

If it works, we might have a temporary solution and we can mature the requirements for this issue so that it addresses your needs but in a way that is usable for all Magento owners and with an implementation that is not slowing down the backend either.

catalog-admin-search-main.zip

Disclaimer: the module is not ready for production. You may install it on a dev/test environment and perform a backup. If you believe the module is not performing well, you can disable the module, clear the cache and everything is back as normal: this module does not alter the database at all.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Area: Account Component: Search Issue: Confirmed Gate 3 Passed. Manual verification of the issue completed. Issue is confirmed Priority: P2 A defect with this priority could have functionality issues which are not to expectations. Progress: PR in progress Reported on 2.4.6 Indicates original Magento version for the Issue report. Reproduced on 2.4.x The issue has been reproduced on latest 2.4-develop branch
Projects
High Priority Backlog
  
Pull Request In Progress
5 participants