Search courses by description?

[hawkrives]

Followup from #2341 and #2383

@hawkrives:

I feel like searching description is going to generate false positives and be really slow.

I'd like some benchmarks on search speed with and without description searching.

Nothing fancy; just, like, log performance.now before and after the filter process happens.

[drewvolz]

I would ask that it would be implemented with non fuzzy search.

[hawkrives]

I think that the description is one of the places that would benefit from a fuzzy search, though. Maybe not the same lib, but it's going to have the longest words of any field (and thus the ones most likely to be mistyped).

I will ask, though; do we want this? Is this actually going to be useful? Why?

[drewvolz]

I'll still debate against using fuzzy here. The likeliness of running into misspellings is, IMHO, not significant enough to take the risk of returning an exorbitant amount of false matches. Descriptions really are one case where I see it as unhelpful to return fuzzied matches.

As for its usefulness, I can go either way on it. I've run into cases where it has been only possible to locate a course by a keyword in the description. I understand a large swath of keywords live in the course description. But I remain indifferent here.

[hawkrives]

keywords

what if we analyzed notes/description on the server and made a keywords array oout of them

removing things like "the" and "a"

sorted alphabetically

then we could maybe do a keywords.some(kw => fuzzy(query, kw))? or keywords.some(kw => query.startsWith(kw)) or something

[drewvolz]

startsWith would be much preferred, and you're going to run into issues with fuzzy even on keywords. I'm okay with your second suggestion.

But I really cannot justify fuzzy here when searching for Kent matches on Karen Cherewatuk

[hawkrives]

A keyword would be a single word, never a phrase (that would be a keyphrase instead :p)

And that's how fuzzy matching is supposed to work. Which is why I think it might work well on a keyword array.

[drewvolz]

I get keywords aren't phrases in this example. But startsWith sounds like the tool for the job.

[hannesmcman]

For what it's worth, when I was testing this I got the most reasonable results by doing description.includes(query). I like the startsWith on keywords idea best.

[drewvolz]

@hawkrives does startsWith on keywords sound good to you?

[drewvolz]

I like the ideas proposed by Hawken and Hannes, especially the keywords idea. I'm unsure why I was so opposed to fuzzy matching even after reading what I wrote before.

[hawkrives]

I think you were opposed to names matching fuzzily. I think the solution for that is to let the names match fuzzily, but fix the sorting to sore by closeness - exact matches first, then descending by score. I do think that such a sort is a requirement for this feature.