Sorting by text field with values with accents not returning correct order

[jorge-anova]

When sorting by text field with values with accents we are not getting the correct order in the results.

Creating index data:

1. FT.CREATE users SCHEMA nametext TEXT PHONETIC dm:es SORTABLE nametag TAG SORTABLE
2. FT.ADD users user1 1.0 FIELDS nametext "ÁREA DE SERVICIO" nametag "ÁREA DE SERVICIO"
3. FT.ADD users user2 1.0 FIELDS nametext "VIRIATO FERREIRA" nametag "VIRIATO FERREIRA"
4. FT.ADD users user3 1.0 FIELDS nametext "CONST MARTINEZ" nametag "CONST MARTINEZ"
5. FT.ADD users user4 1.0 FIELDS nametext "AREA DE SERVICIO" nametag "AREA DE SERVICIO"

Sorting by nametext field:
FT.AGGREGATE users * SORTBY 2 @NameText ASC LOAD 1 @NameText

1. "4"
2. 1. "nametext"
   2. "area de servicio"
3. 1. "nametext"
   2. "const martinez"
4. 1. "nametext"
   2. "viriato ferreira"
5. 1. "nametext"
   2. "\xc3\xa1rea de servicio"

Sorting by nametag field:
FT.AGGREGATE users * SORTBY 2 @NameTag ASC LOAD 1 @NameTag

1. "4"
2. 1. "nametag"
   2. "area de servicio"
3. 1. "nametag"
   2. "const martinez"
4. 1. "nametag"
   2. "viriato ferreira"
5. 1. "nametag"
   2. "\xc3\xa1rea de servicio"

Using FT.SEARCH, we get the same results.

So, "ÁREA DE SERVICIO" will be in the end. Should it be right after "AREA DE SERVICIO"?

Tested with RediSearch v.2.6.6.

[raz-mon]

Hi @jorge-anova,

Thanks for reaching out!

It should; this is a known limitation in our sorting mechanism with terms containing diacritics. Several developments are currently underway in this area, and we hope to address this soon as well.

[github-actions]

This issue is stale because it has been open for 60 days with no activity.