Dictionary of dictionaries than a list of objects?

[davidetacchini]

Hi @TeKrop, nice API; can't wait to make the switch for my project and use it in production!

Wouldn't it be better using a dictionary of dictionaries than a list of dictionaries in the following section?

"career_stats": {
    "all-heroes": [
        {
          "category": "best",
          "label": "Best",
          "stats": [
            {
            "key": "eliminations_most_in_game",
            "label": "Eliminations - Most in Game",
            "value": 62
            },
          ]
        },
        {
          "category": "average",
          "label": "Average",
          "stats": [
            {
            "key": "objective_kills_avg_per_10_min",
            "label": "Objective Kills - Avg per 10 Min",
            "value": 5.9
            },
          ]
        }
    ]
}

Personally, I think it is better in case someone wants to get a specific category (e.g. "best") without having to iterate through the entire list each time, so the time complexity would be O(1) instead of the actual O(n).

"career_stats": {
    "all-heroes": {
        "best": {
            "eliminations_most_in_game": 62,
            "final_blows_most_in_game": 38,
            "all_damage_done_most_in_game": 21507,
            all the other stats...
        },
        "average": {
            "objective_kills_avg_per_10_min": 5.9,
            all the other stats...
        },
    }
}

I see that you can have multiple key/value pairs per object by using a list of objects, thus adding more information. However, in this specific case, it's pretty easy to get the label out of the key if a dictionary of dictionaries is used by simply formatting it. Thanks for the attention, keep up the good work :)

[TeKrop]

First, thank you for your comment, it's always a pleasure to know your project is used by others for their own projects 😄
You made a fair point regarding the format, I first wanted to do it the way you explained, but I changed my mind for two reasons :

• It provides the label for those who would want to use it, and I know some people are using it, even if it could be easily determined from the key as you said

• The keys are dynamically generated from the data retrieved on Blizzard pages, I don't have an exhaustive list of existing keys. Exposing a dict of dict by using FastAPI default behaviour would require a huge amount of work compared to now, as I would have to specify each possible key/value pair in my Pydantic models, which are used for automatic validation and documentation. It would also mean having to update it for every new hero update, or it would possibly break things as they come with specific statistics 🙈

It's my first project with FastAPI, and I didn't want to already do some specific tweaks. But now, I know there are some ways to bypass this default behaviour by using custom responses, and adding additional responses in the OpenAPI specs used for the documentation. The additionalProperties from JSONSchema specification seems to be what I'm searching for, for dynamic dicts without knowing the keys in advance. I'll take a look at it, check if it would fit the needs, and keep you updated 😉

Finally, if I do it, I'll probably define a new endpoint for it, in order to avoid breaking changes as I'm not exposing several versions of the API at the same time as of today 🙈

[davidetacchini]

I'm not familiar with FastAPI but isn't it possibile?

from /app/parsers/player_parser.py starting from line 377

career_stats[hero_key].append(
    {
        "category": string_to_snakecase(category_label),
        "label": category_label,
        "stats": {},
    }
)

for stat_row in content_div.find_all("div", class_="stat-item"):
    stat_name = stat_row.find("p", class_="name").get_text()
    value = {}
    value[stat_name] = get_computed_stat_value(stat_row.find("p", class_="value").get_text())
    career_stats[hero_key][-1]["stats"][string_to_snakecase(stat_name)] = value

Finally, if I do it, I'll probably define a new endpoint for it, in order to avoid breaking changes as I'm not exposing several versions of the API at the same time as of today 🙈

I understand, that sounds good, thanks!

[TeKrop]

Ah yes, the scraping / parsing part is not the issue, it's the exposition of the resulting object by the API. There is a validation process to check if it matches a given data format (with Pydantic objects) which is done automatically by default and handling the documentation generation.

Should be doable without having to explicitly list all possibilities of keys with what I found, I'll keep you updated 😉

[TeKrop]

@davidetacchini I did a quick search and I found that it's natively supported in Fast API, it will be easier than expected 🙈
I'm starting to work on this new endpoint 😄

[davidetacchini]

Amazing, thank you!

[TeKrop]

@davidetacchini The new endpoint is available ! Here is the associated documentation : https://overfast-api.tekrop.fr/#tag/Players/operation/get_player_career_stats_players__player_id__stats_career_get 😉
Let me know if everything is OK for you, so I can close this topic if it's the case 😄

[davidetacchini]

That's perfect! Thank you for your time