Library Statistics Graph Tab

[herby2212]

Description

Addition of a new graph tab called "Library Statistics" which features currently two new graphs.

The first one called "Daily addition by media type" shows the added movies, shows, seasons and episodes on a per day basis. The time range is set like for the other graphs.

The second one called "Addition count by media type" shows the total count of movies, shows, seasons and episodes added in the set time range split in two categories: Movies and TV which includes shows, seasons and episodes.

Update 01/08/2021: #1472 (comment)
Third graph added called "Addition count by source resolution" which shows the total count of movies, shows, seasons and episodes added in the set time range by the different resolutions your media have. (Currently supported: 4k, 1080p, 720p, 576p, 480p, SD)

Update 19/08/2021: #1472 (comment)
Fourth graph called "Library Growth" shows the library growth by the library stats on a per day basis. It is synced with the "Daily addition by media type" graph above so the influence of the additions in regards to the overall growth can be seen instantly.

This feature is work in progress and there are the following things planned before its finished:

• When clicking on a day show the corresponding movies/shows/seasons/episodes (depending on which media type clicked) added on that day. - Moving to a future update.
• Music category including artist, album and track count Library Statistics Graph Tab #1472 (comment)
• Graph improvements (removing not required options and maybe visual improvements)
• Optimize queries
• Data Connection (new table sourced from Plex) Library Statistics Graph Tab #1472 (comment)
• Moving the episode count in the "Daily addition by media type" graph on a second y-axis with bigger incremental steps (for better scaling)
• Library Growth/Stats Graph
• Check active libraries
• Handle in Tautulli deleted libraries f87fce6

I decided to open this request now so improvements, ideas or changes required (that come up through a review for example) can be implemented while in development.

This feature is based on the ideas mentioned in issue #1403.

Screenshot

Library Statistics Graph Tab

General Tab that might get expanded with additional graphs related to library statistics.

Addition count by media type

TV category combines Shows, Seasons and Episodes count.

Daily addition by media type

Updated version with Episode count moved to a second axis.

Broader view of Movies, Shows and Seasons added over time. Episodes filtered out here for a better overview.

Type of Change

• New feature (non-breaking change which adds functionality)

Checklist

• My code follows the style guidelines of this project
• I have performed a self-review of my own code
• I have commented my code, particularly in hard-to-understand areas
• I have added or updated the docstring for new or existing methods

[herby2212]

Addition count by resolution [Added 01.08.2021]

Shows the total count of movies, shows, seasons and episodes added in the set time range by the different resolutions your media have. (Currently supported: 4k, 1080p, 720p, 576p, 480p, SD)

[JonnyWong16]

The graphs are a good idea, but we really shouldn't be using the recently_added table for the data. That is only a temporary table that can get erased.

A new permanent table should be created to cache the libraries.

[herby2212]

The graphs are a good idea, but we really shouldn't be using the recently_added table for the data. That is only a temporary table that can get erased.

A new permanent table should be created to cache the libraries.

Oh. Under what circumstances does the table get erased? As the table is quite good to analyse the library growth. An alternative would be to create a new permanent table for this and get the initial data from the Plex server itself. Which in the end could also just be done by making a permanent table based on recently_added. Though I'm not sure if this table is still in tact for all users.

[herby2212]

Library Growth graph synced with Daily addition by media type graph [Added 19.08.2021]

This fourth graph called "Library Growth" shows the library growth by the library stats on a per day basis. It is synced with the "Daily addition by media type" graph above so the influence of the additions in regards to the overall growth can be seen instantly.

Enabling/disabling of a axis is also synced as well as the local storage (so both graphs get loaded with the same axis selected) and the Episode axis (including yAxis and label).

Demo:

If your library is a bit older like this one the more significant growth phase can be seen at a higher day count:

This graph also includes the year in the date which is helpful when inspecting longer time periods.

[JonnyWong16]

Oh. Under what circumstances does the table get erased?

Clicking this button

Or running the update_metadata.py script after moving media/libraries.

The recently_added table also only stores data from after Tautulli was installed, but I don't see anything in these graphs that would be limited by Tautulli. It's all data from the Plex server.

A new table should be created and sourced directly from Plex. Or a table might not even be required if the data can be loaded live from Plex, but that will have to be tested for performance to see if it's viable.

[herby2212]

Oh. Under what circumstances does the table get erased?

Clicking this button

Or running the update_metadata.py script after moving media/libraries.

The recently_added table also only stores data from after Tautulli was installed, but I don't see anything in these graphs that would be limited by Tautulli. It's all data from the Plex server.

A new table should be created and sourced directly from Plex. Or a table might not even be required if the data can be loaded live from Plex, but that will have to be tested for performance to see if it's viable.

Thanks for the information.
Agreed. I would go with a table in Tautulli as (besides the performance side) it would be bad if the user can't use the graphs, because there is no connection at this moment to Plex. As the rest of Tautulli can still be used even when there is no connection so we should stay coherent here.
I will add the point data connection to the ToDo's.

[herby2212]

Database Implementation [Added 20.08.2021]

Added the library_stats_item table with a corresponding refresh function and options for refresh on startup and interval + corresponding implementation in the settings (see picture below):

Through this table I was able to optimize my queries.

The initial creation of the table and the update takes quit long (which is why I set the min. hour interval to 6).
Times for my around 16000 items (movies + shows + seasons + episodes + tracks + albums + artists):

• Creation: 16min
• Update: 8min

The best would be to disable the refresh on startup by default and only trigger it once after the update (and have the interval running) with something like a special db upgrade function. Still unsure about that so feedback would be appreciated.

[JonnyWong16]

What is the change in database size before and after? How many rows in the table?

[herby2212]

What is the change in database size before and after? How many rows in the table?

16694 rows. Datasize before: 67152 KB after: 97764 KB = ~30 MB

[herby2212]

Music Integration [Added 22.08.2021]

Daily addition by media type and Library Growth:

All other categories disabled for a better visibility.

Additions by media type:

Done for visual elements/additions.

[herby2212]

Notice: It can happen that some media items have a timestamp that is in the future or in a time period before the server existed. This issue comes from the way how Plex handels the date added when scanning in files during the creation of a new library.

if you create a fresh library and in the course of this library creation you add a folder full of media files, Plex will read the datetime stamp of the media files and use this instead as ‘date added’.

Taken from https://forums.plex.tv/t/date-added-wrong/302575/12.

A way to solve this would be to set the added_at field for those media items in the library_stats_items table to the library section created_at timestamp upon initial creation of the table.
This field is currently not in the library_sections table of Tautulli present, only in Plex.

UPDATE: Fixed in commit herby2212@2386cab.
Media items with a timestamp older then their corresponding library instead receive the library creation timestamp.