iOS Highlights / notes in the text

[openm1nd]

Could you please provide samples of how highlights / notes can be integrated in iOS (similar to how it is done in Android)

[mickael-menu]

(Right now, the following is only implemented on the EPUB navigator)

1. Add a "Highlight" selection menu item

You can add custom "editing actions" for the selection menu when initializing a navigator, if it supports it. For example, with EPUB:

init() {
    var config = EPUBNavigatorViewController.Configuration()
    config.editingActions.append(
        EditingAction(title: "Highlight", action: #selector(makeHighlight))
    )
    if publication.isProtected {
        config.editingActions = [.lookup, .translate]
    }
    
    let navigator = EPUBNavigatorViewController(publication: publication, initialLocation: location, resourcesServer: app.server, config: config)
}

@objc func makeHighlight(_ sender: Any) {
    // ...
}

2. Create a new highlight in your database

When creating a new highlight, you need to store at least two things in your database:

• The Locator for the current selection.
• Some tint color information.

You can get a Locator from the current selection if the navigator implements SelectableNavigator.

@objc func makeHighlight(_ sender: Any) {
    guard
        let navigator = navigator as? SelectableNavigator,
        let currentSelection = navigator.currentSelection
    else {
        return
    }

    navigator.clearSelection()

    // Save the highlight in your database.
    // HighlightRepository and the Highlight model are examples types which should be defined in your app.
    highlightRepository.add(Highlight(
        locator: currentSelection.locator,
        color: Color.yellow
    ))

    // Re-render the highlights.
    ...
}

3. Render the highlights in the publication

To render the highlights, you can use the Decorator API on any navigator implementing DecorableNavigator.

Every time the highlights stored in your database change (or when creating the navigator), you need to re-apply all your decorations:

func render(highlights: [Highlight]) {
    guard let navigator = navigator as? DecorableNavigator else {
        return
    }

    navigator.apply(
        decorations: highlights.map { $0.decoration },
        in: "highlights" // Unique identifier for this group of decorations.
    )
}

extension Highlight {
    /// Converts the highlight into a `Decoration`.
    var decoration: Decoration {
        Decoration(
            id: String(databaseID),
            locator: locator,
            style: .highlight(tint: color)
        )
    }
}

The Decorator API is similar to iOS's Diffable Data Sources, so re-applying all your highlights every time the DB changes is not a problem because only the highlights that really changed will be re-rendered by the navigator.

4. Displaying the highlight content in the UI

If you have a UI showing the list of highlights, you can extract the highlight content from the Locator.

let text = highlight.locator.text.highlight

You can also use the locator to jump to the page where the highlight is located.

navigator.go(to: highlight.locator)

[rohitvishwakarma-sf]

Hi @mickael-menu ,
I'd like to get the subchapter name of the current selected text. How can I ?
Please guide

[mickael-menu]

Readium can't do that, but you can add a custom JavaScript file to handle it using EPUBNavigatorViewController.evaluateJavaScript().

[rohitvishwakarma-sf]

Hi @mickael-menu
Thanks 🙏 🤩 , I'm able to get subchapter name for the selected text.
Now, I want to get html elements of the currently showing content in the pages, How can I ? Please help/guide.

[rohitvishwakarma-sf]

Is there any docs to understand how html content is paginated in WKWebview in readium ?

[mickael-menu]

The content is paginated using CSS columns by Readium CSS, here's the relevant documentation.

Now, I want to get html elements of the currently showing content in the pages, How can I ? Please help/guide.

This is not yet merged, but you can take a look at this JavaScript API which does some of the work:

swift-toolkit/Sources/Navigator/EPUB/Scripts/src/dom.js

Line 28 in f074a93

function findElement(rootElement) {

[Amit2192]

Hi @mickael-menu

I am also willing to implement this "Highlight/Note" feature,
Would you please do let me know what's the latest Readium SDK for swift does support this feature?

Also, Do I need to anything extra except the code you have suggested in above answer?

Thank you
Amit

[mickael-menu]

Hi @Amit2192, this was recently contributed by @gatamar already. You should be able to test it out from the develop branch. Let me know if you had more in mind.

[Amit2192]

Hi @mickael-menu, Thank you for your quick reply
Will check out the develop branch and find out how it works

Thank you
Amit

[Agnisikha93]

Hi @mickael-menu ,

can you please let me know how I can enable the note feature in iOS? Which latest Readium SDK supports this for swift?

Thanks!
Agnisikha

[AmitSP91]

Hi @mickael-menu

As you mentioned here that Note feature is available in both develop and main branch.
I have configured test app from both develop and main branches.

But it seems Note feature or it's sample code is not available in any branch, Only Highlight is available with some other annotations like copy, Loock up, Translate etc. See attached SS.

Can you please confirm if Note is implemented for iOS same as it is available for android?

Thank you
Amit

[gatamar]

@Amit2192 No, only the Highlight feature was added. The Note feature can be implemented identically to the Highlight one (using a GRDB repository). The main challenge is that the new UI should be added, that's why I omitted this step when implementing Highlights UI support.

[AmitSP91]

@gatamar Thank you for your quick response.

[mickael-menu]

To clarify, the Kotlin and Swift toolkits have the same features regarding annotations/decorations.

The toolkits don't know about "highlights", "notes", etc. These are concepts for user data in reading apps and not related to Readium. However, the toolkits are able to draw decorations on the publication content, which can be used to represent a highlight or a note on the book.

Each decoration has a "style" which determine how it looks like on the book. By default Readium ships with two decoration styles: highlight and underline.

We don't have a style for a side margin mark (which can be used for the feature "Note" that you want) because they are usually different in every app. However, apps are free to add custom decoration styles.

The Swift Test App doesn't showcase a note feature and doesn't have a custom decoration style, but the Kotlin Test App does. But you can implement it in your app with the Swift toolkit.

Take a look at this documentation for more details about Decorations: https://readium.org/architecture/proposals/008-decorator-api.html

---

Now for a concrete example in Swift. To add a new custom decoration style, you need to do the following:

1. Declare a new decoration style.

   extension Decoration.Style.Id {
       static let noteMark: Decoration.Style.Id = "note_mark"
   }
   
   extension Decoration.Style {
       static func noteMark(tint: UIColor? = nil) -> Decoration.Style {
           Decoration.Style(
               id: .noteMark,
               config: Decoration.Style.HighlightConfig(tint: tint)
           )
       }
   }

2. For each type of Navigator, declare a decoration style template to customize how it looks like. Right now only the EPUB navigator supports decorations, so you need to provide an HTML/CSS template for your note mark.

   var config = EPUBNavigatorViewController.Configuration()
   config.decorationTemplates[.noteMark] = annotationMarkTemplate()
   let navigator = EPUBNavigatorViewController(..., config: config)
   
   func annotationMarkTemplate(defaultTintColor: UIColor = .red) -> HTMLDecorationTemplate {
       let className = "cantook-annotation-mark"
   
       return HTMLDecorationTemplate(
           layout: .bounds,
           width: .page,
           element: { decoration in
               let tint = (decoration.style.config as? Decoration.Style.HighlightConfig)?.tint
                   ?? defaultTintColor
               
               return """
                   <div><div data-activable="1" class="\(className)" style="background-color:\(tint.cssValue()) !important"/></div>
               """
           },
           stylesheet: """
               .\(className) {
                   float: left;
                   margin-top: 4px;
                   margin-left: 4px;
                   width: 18px;
                   height: 18px;
                   border-radius: 50%;
                   background-size: 65%;
                   opacity: 0.8;
               }
           """
       )
   }

3. Now you can use your custom decoration style when creating Decoration objects:

   Decoration(
       id: "my note mark",
       locator: locator,
       style: .noteMark(tint: .yellow)
   )

---

Here's a patch you can apply in the current Swift Test App which will display this note mark for every highlight: https://gist.github.com/mickael-menu/1e4d98f04d5c559d6194c29f7d489e13

It will look like this:

[mickael-menu]

Here's the Kotlin template which is slightly different:

https://github.com/readium/kotlin-toolkit/blob/540c80fc1c57f04d09db45764f103724177aedf9/test-app/src/main/java/org/readium/r2/testapp/reader/EpubReaderFragment.kt#L307-L347

[enverygtlr]

Hello @mickael-menu,

I'd like to highlight the fragment with using its html id. Is it possible to create a locator with only an id and href and highlight the fragment?

Thanks

[mickael-menu]

It's not supported at the moment but it should be pretty easy to add. Could you open a dedicated feature request issue? Thanks.

[rohitvishwakarma-sf]

Hi @mickael-menu ,
I want to show highlights in a tableview with highlights sorted by their reading order.
I tried to sort them by totalProgression property this works fine except a case that is -

But it doesn't work for the highlights close to each other.

Any idea how can I fix this ?
My decision to sort by totalProgression is right ?

[mickael-menu]

totalProgression seems correct, but I think that highlights are taking the progression and totalProgression of the current page, so it's not working. Could you open a bug report issue on the repo for this? Thanks.

[rohitvishwakarma-sf]

Logged issue #270
@mickael-menu

Thanks!