🀄 Review improvements, SRS V2 Beta

HanziHero Announcements
1

Hello, got some updates you’ve probably seen around the forums :slight_smile:

Daily Review Limit

There’s been issues of review accumulations. Sometimes you’ll have days with just a ton more reviews, making it inconsistent and unrewarding.

So there’s now a Review Limit which allows you to set a “goal” of the number of reviews you want to do each day. This allows you to make reviewing more consistent (it’s at most X amount of reviews each day) and also whittle down a backlog in case you’re coming back (it’s not fun to have 1k reviews presented to you as if you must complete them all today!)

You can read more in the Docs.

This change was motivated through feedback from @enicolle & @idrinkwhisky :slight_smile: Thanks!

Drill down on mistakes in Extra Study

Previously if you wanted to do some extra studying, when you finish the summary was a little lacking:

image
image2616×964 136 KB

Now when you finish some extra studying, you’ll see which ones you got right and wrong, as well as an ability to drill down further on the ones you missed:

drill-down

Pinyin Validation, More Lenient Typos, Card label colors

Recently we added the ability to validate your pinyin input. Anything that’s invalid pinyin will be denied – we only recommend flipping this setting on when you’re comfortable with pinyin :+1:

pinyin validation demonstration

In addition to pinyin validation, we’ve also added more leniency to typos. Previously for all answers if you were off by more than one edit-distance, it would be marked as wrong.

To account for longer answers, we gradually increase the leniency. So now you don’t have to worry as much about typing perfectly:

typo

I think @tuobiyasi suggested these changes, so thanks!

You may also notice that Meaning questions have a light background now. Hopefully this makes it easier to differentiate between pronunciation and meaning questions :slight_smile: Thanks for both this suggestion & the pinyin validation @lorentz!

New SRS Algorithm beta

The accumulation of reviews is something we’ve been hoping to solve, so we decided to take another look at our SRS algorithm and implement a new one :slight_smile:

The new SRS algorithm is a modified SM-2 algorithm. It has a few improvements over the last one:

  • You either get the item right or wrong. Additional incorrect answers are no longer considered.
  • If you get it wrong, the current interval is halved. e.g. If you see an item after 10 days and get it wrong, you’ll see it again in 5 days.
  • How long an item has been ready to review is factored into the next SRS interval when you get the item right
  • A bit more aggressive spacing to distribute the load and allow SRS to work its magic

We hope all of these changes helps tackle the mountain of reviews.

You can see roughly the new timings within the docs.

If you want to opt-in, just a heads up: because of the new SRS timings, previous stages no longer correspond. You may see “demotions” of the SRS stage of an item, but that’s only because we’ve bumped the intervals of all the stages :slight_smile:

To opt in, just navigate to your application settings and toggle the “Enable new SRS algorithm”:

image

I think that’s all for now. Happy studying!

4 Likes
2

“If you want to opt-in…”
So the new SRS algorithm isn’t automatically implemented? How would we opt-in then?

1 Like
3

I forgot that part :sweat_smile: thanks! I’ve updated the announcement on how to opt-in :+1:

1 Like
4

Is there any plan to allow user to tweak these values? I fully understand you’re not trying to write a general SRS app here so I’m expecting not, but just curious.

Relearn steps, how much the current interval is modified etc all have a pretty reasonable affect of my queue and how many leeches I end up with in Anki (for what it’s worth I ended up on going to 20% old interval with 2 and 10m relearn steps). If you’re moving to SM-2 for a better SRS algorithm, people have some very strong opinions on what good values are, and they differ quite significantly. So I got to wondering if you were going to allow tweaking to appease the hardcore masses? Or just pick some values and apply it to everyone?

1 Like
5

Nice, great changes all around. One thing about the SRS v2 algorithm though, I noticed that the longer intervals are 4M 8M 1Y. Is the 1Y actually year, or is it ~16 months and you were rounding to make things symmetric? It just seems a little unnecessary to have a 1 year interval after you’ve succeeded the 8 months interval.

Also in general I can’t tell if I like or dislike these longer tail intervals. The time to master a card used to be roughly 8 months. It’s now almost 3 years. For some cards, I could see the benefit. If you’re a real stickler and want to memorize which of 适合 or 合适 is the verb and which is the adjective, yeah an SRS will help with that. But OTOH if you’re on year two of consistent study I don’t think you could possibly forget the reading of 东西. I suppose, I wish there was an option for the expert level cards for you to tell the algorithm “Ok I got it”

1 Like
6

I already noticed that I have less reviews the next day cause cards I know better, vanish into the future quicker. Which is a huge improvement cause now I actually repeat the cards I’m bad at instead of spending time on words that where easier to remember.
But i do agree that it’s kind of discouraging/ becomes an impossible task in my head if a card takes 3 years to master. No matter what the science says on SRS to my brain specifically this just feels like learning to read Chinese is basically impossible. Cause i would need 2 years to get through all of them ones and then another 3 to master them so basically forever. I hate my brain for this but it often tries to convince itself “well if it seems impossible why even try”.
I am trying to convince myself that expert 3 is good enough and i don’t need to “master” them.
But imagine getting a word wrong that is in expert 3 i would probably cry.

1 Like
7

The high duration tail intervals are an artifact of our transition period between having static SRS stages and this newer more dynamic SRS that takes into account how “overdue” it is among other factors. In fact, the “stages” in V2 are simply visual approximations of familiarity than actually part of the algorithm logic itself. We didn’t want cards to be accidentally marked as “finished” earlier than people would like due to our more aggressive spacing during this transition period.

They are partially offset by the fact that the new algorithm doesn’t demote as aggressively as the old one, where two wrong answers could lead to an item going back multiple stages instead of one.

If the beta period is successful and we roll this out for everyone, we will adjust how stages look in the UI and additionally look into adding a setting for adjusting the “finish interval”. Some people may want cards that “never finish” (at least many people like this exist in the Anki community), while some may like them to be marked as “finished” once their next scheduled interval is e.g., >4 months. :+1:

edit: I’ll also file an issue to better document how the new algorithm behaves. We wanted to roll this out as early as possible, so some of the documentation has been a bit delayed. I’ll write up better docs sometime today.

1 Like
8

We updated the documentation, it now goes into more detail about how the V2 algorithm works under the hood.

2 Likes
9

Oh i get it now after looking at the code it’s just that there basically are no more stages. And there just there to give you an idea of progress ^^

1 Like
10

Yep :slight_smile: with our own dogfeeding it seems SRSv2 is far better thus far, so we hope to switch it as the default after this grace period and clean up the stages to reflect progress

It could be possible to expose additional SRSv2 values beyond the “finish interval” we just haven’t given it too much thought – for now we just want to make sure the current settings are a leg up from SRSv1 :+1:

11

How will the stages be cleaned up?

Also, will there be an option to un-master items for those of us who mastered items in SRSv1? There are a few cards that I think are worth keeping around because I haven’t seen them as much in immersion.

12

We’re currently thinking of getting rid of the roman numerals and just having five buckets

I don’t see why we can’t support this :slight_smile: how would an “unlearned” action sound? You would just start from the beginning. I wouldn’t be sure how to support an alternative “unfinished” action other than just resetting the item

1 Like
13

Ah I see. The only thing I caution here is Expert as is will just be the vast vast majority of cards. The breakdown wont look too interesting for long time users. Not really sure how to get around this other than suggesting making the large intervals their own label, and combining the smaller ones.

I personally would never do that as it would just add noise to my already 200ish reviews a day. I think this is the solution most people would be happy with:

  • set the interval to Expert II (~200 days or whatever it should have been), this is what my cards that would have been mastered go to today.
  • set the start date to be when the card was mastered.
  • so basically its like the card was never removed from the pool.

The only edge case here is if someone mastered cards 200 days before you implement unmaster. But I don’t think there should be many people in that situation. And even so, you could just mark as due for review. Remember adding X 100day interval cards to the pool is (amortized) the same as adding 100X new cards - they don’t contribute much review burden.

2 Likes
14

Agreed. The Expert bucket is pretty large, and the roman numerals at least help split it apart.

This makes sense :slight_smile: I wonder if there’s a way to generalize this to account for other scenarios. Would there ever be a desire to set a card to a certain interval/SRS stage beyond this use case?

15

I could see this being useful for when you see a new word that you are somewhat familiar with, but not enough to skip it.