Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Speeding up HPX with a lightweight "lite" option. #248

Open
Hexadecimalism opened this issue Mar 27, 2021 · 1 comment
Open

Speeding up HPX with a lightweight "lite" option. #248

Hexadecimalism opened this issue Mar 27, 2021 · 1 comment

Comments

@Hexadecimalism
Copy link

Hexadecimalism commented Mar 27, 2021

HPX is phenomenal for what it does but it can also be slow depending on the size of your library. SQLite is unusable at larger library sizes. Even PostgreSQL on an average machine is a bit tedious to use (my library is several thousand large). This issue lies at the heart of SQL in general, and HPX has uses a MASSIVE relational database. A lite mode could drop features like tags, circle, maybe even favorites etc, etc, and keep it barebones with something like artist, title and page number being the only queries needed.

In addition a multi DB mode would help if time (a ridiculous amount of it) was spent writing an ETL to convert the relational DB into star schema (if unfeasible for primary HPX usage, it would still be viable and even faster if the user switches the server to "lite" mode.

@404Fox
Copy link

404Fox commented Mar 28, 2021

There are probably other ways to improve the performance of HPX, star schema included.
We shouldn't need a lite mode if poor performance is not the intended state of the project.

Tags as a whole can probably be generalized such that artist/circle/parody/tag/etc. can use the same "tag" system and push parsing of tag groups onto the web client side.

Say for example this kind of structure:
Each tag belongs to a category. User can create categories, although standard categories would be artist, circle, parody, collection. Each category appears as its own tag group on the gallery page.

  • Categories exist as rows in a category table rather than separate tables for categories.
  • When reading a gallery, the tag has a category_id column that reads from category table for sorting.
    Thus reducing the number of different tables involved.

That being said, I don't know how galleries tags are currently stored that could be better or worse.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants