New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add in way to handle synonyms #37
base: main
Are you sure you want to change the base?
Conversation
b9e8d2e
to
b1cca0d
Compare
from training.synonym.synonym_expander import SynonymExpander | ||
|
||
|
||
class EnhanceData: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
"data" (in EnhanceData
) sounds quite generic,
what about
EnhanceDescriptions
or
EnrichDescriptions
?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yeah thats fair, will have a think
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I still need to think about this actually 🤔
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It's really good,
the expansion logic is not clear to me, to be honest,
(the code is clear, is why we do it that way is not clear)
I've Just added a few minor comments
e6e94ac
to
50c6a11
Compare
8a120bf
to
0e12b73
Compare
0e12b73
to
0d14afd
Compare
Jira link
https://transformuk.atlassian.net/browse/HOTT-4489
What?
This is a way of adding synonyms into the training data when generating the model. It will come after the search references and enrich the data so when we allow the model against the querying we have more data to match to commodity numbers