Fuzzy header search

[PratyushChauhan]

PR Description

Previously we were only checking if a pattern is present in a header string. As a result you would not get suggestions if you made a typo.
BEFORE:

This PR attempts to fix this issue using Fuzzy package.

AFTER:

Related Issues

• Related Issue Implement fuzzy string matching to get header suggestions #207
• Closes Implement fuzzy string matching to get header suggestions #207

Checklist

• I have gone through the contributing guide
• I have updated my branch and synced it with project main branch before making this PR
• I have run the tests (flutter test) and all tests are passing

Added/updated tests?

All previous tests except one have been added for the new function.

• Yes
• No, and this is why: please replace this line with details on why tests have not been included

[animator]

It should show recommendation if there is a typo but if an exact substring match exists, it should show only them.

[PratyushChauhan]

It should show recommendation if there is a typo but if an exact substring match exists, it should show only them.

@animator I have a suggestion to implement this.

in getFuzzyHeaderSuggestions():

List<String> getFuzzyHeaderSuggestions(String pattern) {
  final keys = headers.keys.toList();
  final fuse = Fuzzy(keys,
      options: FuzzyOptions(
        distance: 10,
        tokenize: true,
        matchAllTokens: true,
        tokenSeparator: "-",
        threshold: 0.2,
        shouldSort: true,
        minMatchCharLength: 1,
      ));
  final results = fuse.search(pattern);
  final List<String> suggestions =
      results.map((result) => result.item as String).toList();
  return suggestions;
}

when we obtain suggestions we can then iteratively sort them into 2 lists, "predictions" and "recommendations" {in case there is a typo}.

Example: every suggestion that exactly contains(pattern) will be a "prediction", else it will go to "recommendations".
Then if predictions.length == 0 in which case the user made a typo, we will display the fuzzy "recommendations" list otherwise we will only display "predictions" that will contain the pattern substring.

Let me know if this is desirable.

[PratyushChauhan]

Doing this also passes the one test case I had to edit. This should do better than the previous approach.

[PratyushChauhan]

Passes all test cases and does not show approximate suggestions if no typo exists. If typo exists then it shows fuzzy suggestions.

[ashitaprasad]

@PratyushChauhan For the case, when the user just types a the result should include headers beginning with a followed by other headers. This is not happening currently.

[PratyushChauhan]

@ashitaprasad worked on your suggestion, now the first character match is given preference in predictions.

[animator]

@PratyushChauhan Why is Accept lower than Accept-Charset in your implementation?

[PratyushChauhan]

@animator this most likely happens when we re-order the predictions {in case of typo} to include first character matching strings near the top.

// sort the predictions based on first character match
  for (int i = 0; i < predictions.length; i++) {
    if (predictions[i][0].toLowerCase() == pattern[0].toLowerCase()) {
      final String temp = predictions.removeAt(i);
      predictions.insert(0, temp); //push to front
    }
  }

In case of a typo each string that has a matching first character is pushed to position 0. Hence the order given by the fuzzy finder is reversed which makes Accept have a lower ranking than Accept-Charset.

A way to fix this would be to introduce a currentIndex variable to keep track of which index we need to push to. This will preserve the order given by the fuzzy finder package.

Fix:

  int currentIndex = 0;
  // sort the predictions based on first character match
  for (int i = 0; i < predictions.length; i++) {
    if (predictions[i][0].toLowerCase() == pattern[0].toLowerCase()) {
      final String temp = predictions.removeAt(i);
      predictions.insert(currentIndex, temp); //push to front
      currentIndex++;
    }
  }

[PratyushChauhan]

@animator @ashitaprasad is this implementation fine?