-
-
Notifications
You must be signed in to change notification settings - Fork 122
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Question] #643
Comments
It's possible when you switch from SM-2 to FSRS, or increase your desired retention. |
Ok, thank you for the quick response. I assume after the initial decrease, it should continue to increase with Good responses though right? I'll keep using it and report back here if I see any decreases after a 2nd response using this. |
So after a few days I'm getting the general sense that any intervals that were fairly long (e.g. > 3months) are reduced after turning on FSRS. What is the logic there? Why do these cards that were obviously easy for me to remember in the past, given such low initial stability and retrievability? e.g. the card below had an interval of 2.9 months for Good. |
I'm going to go ahead and reopen this to request some attention to this as I'm continuing to see the same trend, which is going to lead me to have a huge number of cards due within the next few months rather than spread through years. I was under the assumption using this would lead to less total review time, but unless the setting of the initial stability and retrievability can be better explained, this is still a no go for me. |
What's your parameters for this deck? And what's your previous true retention and current desired retention? Besides, if your decks have various difficulty (cramming vocabularies vs. immersed learning), I recommend using different presets and parameters. |
The current parameters and the ones used prior to enabling FSRS are shown in my initial comment to this issue. Where would I find the stats to answer the question about "true retention"? I left the desired retention at the default of 0.9.
I do this already, but this issue is focusing on my giant immersed learning collection. Here is the stats for that deck anki-stats-2024-05-07@14-55-38.pdf If you see anything in there that strikes you as odd/unexpected, please let me know know and I can try to help identify the cause. This is a collection of subdecks, where I used to have separate deck options for each, but gave up trying to manage them all independently after finding anki had really messed up a lot of my intervals and ease values due to a bug (long story). So I defaulted everything back to one deck option before moving to FSRS in hopes that it would avoid such hassles hereafter. Maybe this was a mistake and I should go back and try to retrain on each subdeck? The content of each sub deck is not significantly different in difficulty just sources (single books, textbooks, etc.). I don't really cram for anything, and keep a pretty consistent pace for reviews, and only add in new cards when my review count is well below what I can handle in a single day. Of course some content within each individual subdeck is much easier than others, but I think that is expected right? |
Also just curious, is there any documentation that better discusses how these initial stability and retrievability values are calculated based on past non-FSRS review history? I'm only seeing the calculations assuming we are starting from 0 here unless I'm misunderstanding something. |
I find that your initial stabilities are very different. The initial stability (good) is 5.9837 and the initial stability (again) is 0.8377. That explains why those cards' intervals are reduced. Because their first ratings are again. Here is an visualizer: https://open-spaced-repetition.github.io/anki_fsrs_visualizer/ |
Thank you for your reply and the link to the visualizer. After playing around with that a bit, I think I may understand the situation better than before. Can you please confirm my understanding? So to calculate the intervals after switching from the old scheduler to FSRS you take the answer history of values [1-4], and plug that into S'_r calculation and model what the interval would be had I given all the same answers under the FSRS scheduler. Did I get that right? Does this also imply that this does not account for the previous scheduler's time intervals and how those relate to the answers? For example, answering 3 after an interval difference from 6.23 months to 1.98 years should indicate I have a very firm grasp on the card, but in the FSRS calculation, this will not look very significant as it would only be modeled as me answering 3 between a 1 month and 2 month (very rough guess) interval. Is that more or less correct? If that is the case, I don't see any issue with the model itself when used from 0 answers, but this calculation for the initial intervals after switching to FSRS I would think needs to somehow account for the significance between answers and previously scheduled intervals. Or maybe this is somehow accounted for in the value of
Comparing this to your default values of 0.4 and 2.4, It looks like they are just scaled by a factor of 2, but I'm still not quite grasping the implications of this as I don't see a way to compare two plots from different parameter values. Without understand all the details of the paper, is there any generalization you can make as to why the training would result in such values compared to the default ones? With regards to the visualizer, the y-axis is stability right? What are the percentages above each point? What about the values you see when hovering over each point? Understanding these may give me a bit more insight into what you are trying to tell me 😅
[Edit] Looking at this again, I think what I wrote below and this quote are beside the point now. I'm not really confused as to how the algorithm calculates intervals, but more confused/concerned as to how these intervals are calculated in spite of previous interval history.
I guess a more simple and frank question would be regardless if this aligns with the math or not, is this the expected/desired outcome? Thanks again for your time and thoughts. |
FSRS also considered the previous scheduler's time intervals.
FSRS will output a longer interval if your previous interval is 1.98 years than 6.23 months. But the new interval is not linearly related to previous interval. The previous intervals affect the stability via the
The initial stabilities are optimized from your first forgetting curves for the four first ratings. Assuming your previous intervals of new cards for
The percentage is difficulty.
It's expected. But I wonder the evaluation results of your parameters. If it's too high, I recommend using different parameters for your decks. |
Maybe I'm misunderstanding what you mean, but FSRS output an initial interval of 2.9 months for Good after the 1.98 year interval (i.e. < 6.23 months).
I would like to return the parameters to their defaults to see what the initial interval after the 1.98 year interval would be according to FSRS to confirm what you are saying will reduce the interval or not. As I already answered that card with Easy (4), do you know of a way to delete this initial FSRS based review from that card so I can actually compare them? Or do you know of a way to easily compare the initial Good interval of a given card under a given set of parameters and answer history? I'm not seeing the actual intervals in the visualizer, only the Difficulty and Stability, right? |
Initial interval is the interval for new cards. It's impossible that a new cards has been reviewed 1.98 years ago.
This add-on could delete the review history: https://ankiweb.net/shared/info/1398071003 |
I see where we are having a misunderstanding. When I said "initial interval of 2.9 months for Good" I meant the first interval I saw generated from FSRS while reviewing (i.e. the interval I saw while reviewing after first switching to FSRS, but with my existing review history). I did not mean the initial interval used by FSRS. Sorry, that was confusing. So in the screenshot I had 14 reviews in the history, so if the first interval when the card is new is referred to as I_0, I was referring to I_14 as this was the first interval generated by FSRS for that card.
Thats not quite what I was looking for, I was looking for a way to just delete the last review. Looks like I can do this manually with an sqlite editor like sqlite3, so I'll play around with that and come back here after I check the difference in I_14 with my, and your default parameters to confirm whether or not the parameters are the cause of this very short I_14, or if the algorithm in general would lead to such a short I_14. The result will help me narrow down my issue better. |
OK. In the view of FSRS, it only considers the first review per day for the same card. So this card's If we ignore the https://huggingface.co/spaces/open-spaced-repetition/fsrs4anki_previewer |
Thank you. Nice to have that previewer. And thank you for taking the time to add the memory state history feature, though I'm not quite sure I understand where this delta_ts is being used or how it affects the output. Shouldn't we see the same values of delta_ts somewhere in the interval history calculations on the right?
Good to know.
So this does not include the most recent interval (1.98 years). i.e. the t_history between 2022-05-22 and now (the time I am reviewing now)? This seems important to me, as otherwise it is ignoring what I'd consider the most significant interval. To put it another way, if I press Good after the existing 1.98 year interval, I expect the next interval to be longer than 1.98 years, not longer than 186 days. Even worse, and maybe I'm misunderstanding the output in the "test sequences", it looks like the calculated interval in place of 1.98 years is 10 days? I'm still not seeing the logic as to why this is expected behavior. |
Ok, so rerunning that with the default weights given here I see a stability of around 295.8, which I suppose it less shocking than 92. I also played around with the retention value and it did not change this final stability at all. Are you sure it is being used in the back-end? Also other than resetting all parameters to the default ones, is there any way to set them manually in any intelligent way? (i.e do they have any non latent meanings for each?). As I mentioned before, I don't think the intervals are so far off what I'd expect for shorter intervals, so I'd think tuning one of the parameters that influences the slope/gradient of one of the exponentials directly would be suspect here (w_10 and w_14). Is that logical? Also I noticed the default weights in the visualizer: are different than those listed in here Is one set recommended over the other at this point? Also, I just wanted to also post the results of the Evaluate button in the FSRS Anki options for this deck: Log loss: 0.2537, RMSE(bins): 3.42%. Are either of these surprisingly high? I guess I'd expect them to be if the interval proposed by FSRS is 92 days and the one prior was >1.98 years.
|
The 295.8 is stability instead of interval, so the retention value doesn't affect it.
It's not recommend to set the parameters manually.
They are really small. The average log loss is 0.33 and the average RMSE(bins) is 0.053.
Here is the code to optimize the parameters related with initial stability: |
Any other questions? |
Question
Hello again,
Trying this out again after seeing a fair bit of progress on this and anki itself. Thanks for your hard work!
I had posted an issue in the past here, that may or may not have been fully resolved since, but just wanted to reference it as I had sent you my anki collection for testing in case you wanted to test against it again. (Though I'm happy to send again).
As I was afraid of a giant back log like before, I opted to use the default OFF setting for the "Reschedule cards on change". I started answering a few cards, and was watching the card info as I answered to see if the updated intervals were making sense. For the first few, it appeared normal, with somewhat longer intervals that previously (which I expected), but then I ran into one car where answering Good resulted in the interval decreasing. Is this expected? Here is the history for that card with the last answer being the only one performed after enabling FSRS.
Here are the deck options used after enabling FSRS:
Anki specs below:
Version 24.04.2 (82caffec)
Python 3.9.18 Qt 6.6.2 PyQt 6.6.1
The text was updated successfully, but these errors were encountered: