Auto-highlighting specific words in paginated text

[eton1234]

Attempting to build functionality off of the testApp where the content of the preloaded pages are analyzed against a dictionary of known words. Unknown words would be automatically highlighted. (Ideally in the context of language learning)

First of all, how might I access that specific text and then parse through it?

How could I then add highlights at locations where there are unknown words?

If I can accomplish these things, I would also like to create a custom overlay that is displayed when a word is tapped. Any ideas for how to do that?

Apologies, I'm a CS student so am pretty new to IOS dev, but I have spent a lot of time trying to make sense of the codebase and documentation.

[mickael-menu]

Pretty interesting use case!

You're in luck, we just merged something that will help with that. It's not yet released but available in develop. Take a look at the user guide for the Content iteration.

This will extract full paragraphs, but you can split them by words using a tokenizer. The default one handles splitting by words:

guard let content = publication.content() else {
    return
}

let wordTokenizer = makeTextContentTokenizer(
    defaultLanguage: publication.metadata.language,
    textTokenizerFactory: { language in
        makeDefaultTextTokenizer(unit: .word, language: language)
    }
)

let words: [TextContentElement.Segment] = try content
    .elements()
    .flatMap { try wordTokenizer($0) }
    .compactMap { $0 as? TextContentElement }
    .flatMap { $0.segments }

for word in words {
    print("Word '\(word.text)' found at \(word.locator)")
}

Output

Word 'The' found at {"href":"/OPS/title.xml","locations":{"cssSelector":"html > body > div.body > h1"},"text":{"after":" Divine Comedy","highlight":"The"},"type":"application/xhtml+xml"}
Word 'Divine' found at {"href":"/OPS/title.xml","locations":{"cssSelector":"html > body > div.body > h1"},"text":{"after":" Comedy","before":"The ","highlight":"Divine"},"type":"application/xhtml+xml"}
Word 'Comedy' found at {"href":"/OPS/title.xml","locations":{"cssSelector":"html > body > div.body > h1"},"text":{"before":"The Divine ","highlight":"Comedy"},"type":"application/xhtml+xml"}
Word 'Dante' found at {"href":"/OPS/title.xml","locations":{"cssSelector":"html > body > div.body > h2"},"text":{"after":" Alighieri","highlight":"Dante"},"type":"application/xhtml+xml"}
Word 'Alighieri' found at {"href":"/OPS/title.xml","locations":{"cssSelector":"html > body > div.body > h2"},"text":{"before":"Dante ","highlight":"Alighieri"},"type":"application/xhtml+xml"}
...

The locator object can be used to highlight each word using the Decorator API. Unfortunately there's no Swift user guide for it yet, but you can take a look at this other discussion #17

I'm not sure how well this would perform if you need to highlight every word in each resource though, probably slowly.

You can use the Decorator API to be notified when the user taps on a highlighted word. The associated event has a frame parameter you can use to position your overlay next to the word.

(navigator as? DecorableNavigator)?.apply(
    decorations: words.enumerated().map { (index, word) in
        Decoration(
            id: "word-\(index)",
            locator: word.locator,
            style: .highlight(tint: .red, isActive: false)
        )
    },
    in: "words"
)

[eton1234]

Thanks so much! Hmm would it be faster if highlights were loaded for the pages the user was interacting with then deleted afterwards?

[mickael-menu]

That would help but there's no easy way to know on which screen pages the words are.

But in the context of language learning, maybe you won't have performance issues. When starting to learn a language we use simpler books with short resources, so less words. When "graduating" to real books the most frequent words will already be known, so maybe only a smaller percentage of words will need to be highlighted.

[eton1234]

How about just loading a reasonable length of text at the users currentLoc? Is that possible with navigator.currentLocation?

[mickael-menu]

You could try with:

publication.firstVisibleElementLocator { locator in
    let content = publication.content(from: locator)
    ...
}

The content iterator is supposed to be used in a background thread, as it is synchronous. As far as I know, the slowest part is the parsing and browsing (with visitor pattern) of the HTML documents which cannot be done concurrently. However, parsing multiple resources in parallel could be done I guess. So far this was not needed so I kept the code simple.

[eton1234]

Wait why can't parsing/browsing of the html doc be done concurrently by the way?

[eton1234]

Sorry to bring this up again. Took a break as I felt I lacked some contextual knowledge ab databases, os, etc which I now have. I was thinking recently... is it possible the reason auto highlighting is slow due to db transactions? IIRC highlights are added to the SQLlite db somewhere. I would imagine since there could b thousands of words on a page or group of page... it might b hard to commit all of those at once to disk? Just a guess.

I was thinking maybe I could implement a different type of highlighting that is only in memory? And then saved to the db only when the word is marked by the user

[mickael-menu]

For this use case you certainly don't need to add any highlights in the database. If you look at the code I shared above, there's no database involved.