[Kavita] Better parsing support for PDF's

[DieselTech]

Idea Description

Posted by Nico
· Jan 2022
· 27 Upvotes
https://feats.kavitareader.com/posts/50/magazines-date-parsing

Description

I would like to add newspapers and magazines in my kavita library... I tried something but it's a mess. Could it be possible to create some kind of naming/parsing scheme similar to this for example ?
for a weekly/monthly magazine --> //magazine/new-yorker/2022/new-yorker-2022-01-10.pdf
and for daily papers = //newspapers/jan2022/nyt-2022-01-10.pdf

any other idea or suggestion are welcome ;-)

Duration of Using Kavita

42

Idea Category

Feature Enhancement

Agreement

• I agree that this is solely for submitting ideas, and I will search for existing ideas before posting.

[DieselTech]

themoot
· 12 months ago

Define a limited amount of valid formats that most magazines will adhere to. From my POV, these cover 95% of what I run into:

Date (ISO 8061) standard: YYYY-MM(-DD) (ie, 2023-01, 2023-01-19)
Date (Months): YYYY-Mon-[Mon] (ie, 2023-Jan-Feb; 2019-Jul-Aug-Sep)
Date (Season): YYYY-Spring/Summer/Fall/Winter (ie, 2023-Spring, 2021-Fall)
Vol/No = already handled?
Issue = already handled

Yes, you may have to normalize and rename files, but aren't we already doing that?

[majora2007]

This has been accepted. Kavita+ users can provide feedback to help me build out the initial version. I will be in contact with questions as necessary.

[majora2007]

If anyone has naming conventions and folder layouts, please share them here and I will try to build as much support as possible.

[majora2007]

Added a comment from https://feats.kavitareader.com/posts/450/minimal-interpretation-for-pdfs

I would also love to see some extra library types like "Magazine", "Article", "Leaflet", "Document", "Other", ... I think power users would appreciate a feature to specify one or multiple regular expressions that overwrite the default name parsing of Kavita and use the regex to get some metadata. e.g.

Microsoft - NET Microservices Architecture for Containerized NET Applications .NET 7.0 (2022).pdf
Insight Guides - South India (2017).epub
^(?.)\s-\s(?<Title>.)((?[0-9]{4}))

National Geographic 2021-01.pdf
^(?<Title>.*)\s(?[0-9]{4})-(?[0-9]{2})

Tolkien, J.R.R. - The Hobbit.epub
^(?.)\s-\s(?<Title>.)

[dstibbe]

Is there not a way to create a custom parser? For regular books (PDF), I like to group the books by collection (e.g. DnD 4e) without having to add 'DnD 4e' as a prefix to each file, or other dirty hacks (creating a dir for each pdf).
Just have a proper set of book rules instead of defaulting to 'manga rules'. When I deal with books the most common case is that there are no separate files for chapters.

The current scanner is just a mess

[therobbiedavis]

Is there not a way to create a custom parser? For regular books (PDF), I like to group the books by collection (e.g. DnD 4e) without having to add 'DnD 4e' as a prefix to each file, or other dirty hacks (creating a dir for each pdf).

Just have a proper set of book rules instead of defaulting to 'manga rules'. When I deal with books the most common case is that there are no separate files for chapters.

The current scanner is just a mess

This is an interesting idea, have an advanced option to let the admin write their own regex per pdf library.

[dstibbe]

Let's say someone wanted to edit the parser (for a PR), where would he start looking?

[majora2007]

Parser.cs but I would advise you to speak to me before you get in too deep. Doing a POC is fine, but just because there is a PR, doesn't mean I'll pull it in.

Can also look at the unit tests (MangaParserTests, etc)

[dstibbe]

Of course i don't expect anything to be 'just' accepted 😊

[majora2007]

For Magazines exclusively, the story to implement them is here:
#2578

If you have common filenames, please comment on that. Do not comment about non-Magazine related information. This is exclusively designed around Magazines.

[huntsfromshadow]

You requested other organizational schemes.
This is an example of how I order a very large library of RPG pdfs.

Basically, I end up categorizing by publisher, then Game then book type then books.
Sometimes within a game, I'll have versions (Eg. Chaosium\Runequest\ then 3 folder there - RQ1 then RQ2, then RQG)

- RPG Books\
  - Masiff Publishing\
    - Lancer
      - Core Rules
        - Lancer Core.pdf
        - Character Sheet.pdf
      - Long Edge Setting
        - Setting Maps.pdf
        - Long Edge.pdf
  - Modiphius
    - Star Trek Adventures
      - Core Rules
        - STA.pdf
        - Utopia.pdf
      - Quick Start Box Set
        - Maps.pdf
        - Help Sheets.pdf
        - Rules.pdf
        - Setting.pdf
        - Campaign.pdf
      - Adventures
        - Lower Decks Adv
          - Lower Decks File.pdf
          - Lower Decks Map.pdf
          - Lower Decks Characters.pdf

[XyberDAWG]

You requested other organizational schemes. This is an example of how I order a very large library of RPG pdfs.

Basically, I end up categorizing by publisher, then Game then book type then books. Sometimes within a game, I'll have versions (Eg. Chaosium\Runequest\ then 3 folder there - RQ1 then RQ2, then RQG)

- RPG Books\
  - Masiff Publishing\
    - Lancer
      - Core Rules
        - Lancer Core.pdf
        - Character Sheet.pdf
      - Long Edge Setting
        - Setting Maps.pdf
        - Long Edge.pdf
  - Modiphius
    - Star Trek Adventures
      - Core Rules
        - STA.pdf
        - Utopia.pdf
      - Quick Start Box Set
        - Maps.pdf
        - Help Sheets.pdf
        - Rules.pdf
        - Setting.pdf
        - Campaign.pdf
      - Adventures
        - Lower Decks Adv
          - Lower Decks File.pdf
          - Lower Decks Map.pdf
          - Lower Decks Characters.pdf

This is very similar to my Structure and Most likely the most common structure among TTRPG users.

Root
- Publisher
-- RPG (With Edition)
--- Core Books
--- Settings
--- Adventures
--- Supplements

So in these cases Each Folder below the "RPG" would in essence be treated as "Named Volumes"