Some weird first forgetting curves

[L-M-Sherlock]

[Expertium]

These 2 are the weirdest. The other ones may simply be too noisy, but these two have a clear pattern: R goes up as time passes.

[L-M-Sherlock]

Hypothesis: The user has two presets with two graduate intervals. And the materials are very different in these two presets.

To validate the hypothesis, we need to know which presets those cards are from.

[Expertium]

A bit unrelated to the current issue, but I wonder if we should change this:

        for small_rating, big_rating in (
            (1, 2),
            (2, 3),
            (3, 4),
            (1, 3),
            (2, 4),
            (1, 4),
        ):
            if small_rating in rating_stability and big_rating in rating_stability:
                # if rating_count[small_rating] > 300 and rating_count[big_rating] > 300:
                #     continue
                if rating_stability[small_rating] > rating_stability[big_rating]:
                    if rating_count[small_rating] > rating_count[big_rating]:
                        rating_stability[big_rating] = rating_stability[small_rating]
                    else:
                        rating_stability[small_rating] = rating_stability[big_rating]

Specifically, these lines:

                    if rating_count[small_rating] > rating_count[big_rating]:
                        rating_stability[big_rating] = rating_stability[small_rating]
                    else:
                        rating_stability[small_rating] = rating_stability[big_rating]

I wonder if we should just simply switch rating_stability[big_rating] and rating_stability[small_rating].
rating_stability[big_rating], rating_stability[small_rating] = rating_stability[small_rating], rating_stability[big_rating]
I will benchmark that and report the results here.

[Expertium]

I tried this:

            if small_rating in rating_stability and big_rating in rating_stability:
                if rating_stability[small_rating] > rating_stability[big_rating]:
                    rating_stability[big_rating], rating_stability[small_rating] = rating_stability[small_rating], rating_stability[big_rating]

It made results worse.

[L-M-Sherlock]

@dae, is it possible to include the preset or deck info in the benchmark dataset? PresetName and DeckName will be really useful to validate my hypothesis (see #29 (comment)).

Besides, I'm also working on the question: should we use different presets (parameters) in different decks? If so, when? Those features are also helpful to research the question. Here is an initial analysis: https://github.com/open-spaced-repetition/fsrs-when-to-separate-presets

If sharing PresetName and DeckName violates the privacy of AnkiWeb, preset_id or deck_id is OK, too.

[dae]

It would need to be anonymised. I can do it, but it will be a while until I can find the time.

[Expertium]

@dae I also have a request: include both card ID and note ID. I have an idea how to account for reviews of siblings, though it will probably only work "on paper", in the benchmark, and won't be feasible in Anki itself (it requires inserting pseudo-reviews into a card's history). Still, I would appreciate it if you made a dataset with both nid and cid.