[FEATURE] KOReader sync implementation

[avidal]

The KOReader¹ reader software is very popular. It supports a large number of e-readers, and is often used as an alternative reader on jailbroken Kindles. It can be natively installed on Kobo and Boox e-readers (or any e-reader that supports Android applications).

There are additionally other clients that can sync with a KOReader sync server, such as Obsidian², which allows you to annotate or highlight books in a KOReader client (such as your Kobo), and then pull those annotations into Obsidian.

This is all powered by a simple API. There is an official implementation³ as well as numerous other implementations, such as a single-file Go implementation⁴ and a Rust implementation⁵.

I'm assuming there are general plans to support progress sync in Stump, and I'm also assuming that (currently) the first implementation would be based on pages fetched over OPDS-PS. Considering that implementation will need to come with some datastore for tracking progress, a koreader-sync implementation would be able to leverage that same datastore.

For reference purposes, the actual API consists of five endpoints (register user, check auth, update progress, get progress, and healthcheck)⁶ and one JSON schema that is used both to update the progress on a document and to fetch the current progress given a document id. The JSON schema also includes a unique device id.

Footnotes

1. https://koreader.rocks ↩

2. https://github.com/Edo78/obsidian-koreader-sync ↩

3. https://github.com/koreader/koreader-sync-server ↩

4. https://git.sr.ht/~amk/go-kosync/tree/master/item/server.go ↩

5. https://github.com/pborzenkov/koreader-syncd ↩

6. https://github.com/koreader/koreader-sync-server/blob/master/config/routes.lua ↩

[aaronleopold]

Thanks for all of the references, it's really helpful and I appreciate it! This is really cool and I'm definitely interested in getting support for this at some point.

I'm assuming there are general plans to support progress sync in Stump, and I'm also assuming that (currently) the first implementation would be based on pages fetched over OPDS-PS. Considering that implementation will need to come with some datastore for tracking progress, a koreader-sync implementation would be able to leverage that same datastore.

There already is progress 'sync' support in Stump, both via OPDS-PS and then natively through the API. Whenever you fetch a page in ODPS-PS it will update the progress, as you suggested. Epubs are a bit different, though. Currently, epub progress is tracked via a combination of percentage completed and your current epubcfi string. Perhaps you strictly meant wrt koreader-sync, but I wanted to clarify just in case.

I'll try and review what would be involved on my end to support this sometime soon. Not sure how open you are to contributing, but I'd be happy to help get you going if that is of interest and you have the time.

[avidal]

re: progress support

Ah, I was looking for an implementation of that but couldn't quite find it. But now that I'm looking it's pretty obvious. I think I must have been looking at a different codebase (hard to juggle all of those open tabs).

As far as contribution goes, I'm pretty interested. I'm still getting my new setup going and will gone for the weekend but I may take a stab at it myself after that.

The general open question is: should the koreader sync API be built-in, or should there just be a stump-koreader-sync binary that communicates with the stump API? It's perhaps not a really important question to answer right now, but it may set the stage for how "integrations" are built in the future.

[aaronleopold]

As far as contribution goes, I'm pretty interested. I'm still getting my new setup going and will gone for the weekend but I may take a stab at it myself after that.

Sounds good, if you wind up giving it a shot feel free to give me a poke if you need anything and I'll try and help where I can

The general open question is: should the koreader sync API be built-in, or should there just be a stump-koreader-sync binary that communicates with the stump API?

Good question. It likely depends on how large the integration itself winds up being. A separate service just for this can maybe be more scalable, but not sure how much that really matters here. Regardless, I feel it should be constructed in a way to support isolated or integrated runs. A clean example of what I mean would be how the server can actually be bundled in and started from the desktop app.

[avidal]

A clean example of what I mean would be how the server can actually be bundled in and started from the desktop app.

Ah, right. My question wasn't really about scalability (at least from a resource perspective) but about maintainability. I forgot that this was a monorepo.

I think the most straightforward may be a crate that exposes endpoints, and there's a config switch that maps the endpoint implementations to the existing http muxer.

The separate crate could be both a library and a binary crate. The main stump crate could include the koreader-sync library by default. The koreader-sync binary crate could expose its own http interface.

Someone running a NAS may then choose to run stump without the sync and run the sync separately via its own binary.

Or something along those lines.

[aaronleopold]

Gotcha. Well, what you proposed sounds good to me! It would likely fit nicely in either the crates or apps directories of the monorepo, but it it makes sense to eventually break it out it isn't the end of the world (if anything, it would make isolated versioning and releasing less hassle down the road)