Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Last position people are awarded positive rating changes. #23

Open
atiabjobayer opened this issue Nov 16, 2023 · 1 comment
Open

Last position people are awarded positive rating changes. #23

atiabjobayer opened this issue Nov 16, 2023 · 1 comment

Comments

@atiabjobayer
Copy link

Used Algorithm: mmr

Consider a situation where 3 user participated in a contest[user1, user2, user3]. Consider it's a programming contest. Let's say their scores are [300/500, 200/500, 0/500].

Now the basic sense suggest that since the user3 scored absolutely zero, he should get a negative rating change. But what happens is, if user1 and user2 have higher rating than user3 before the contest, then user3 receives a positive rating change, even if he scores zero.

Is there any way to prevent this behavior? Doesn't seem right at all.

@EbTech
Copy link
Owner

EbTech commented Nov 16, 2023

Hmm it's hard to tell what exactly happened without going through the math in detail, but this shouldn't usually happen. From the 0/500 player's perspective, the algorithm treats the result as 1 loss against the 300 player, 1 loss against the 200 player, and either 1 loss + 1 win or 0.5 loss + 0.5 win (depending on the split_ties parameter) against themself. Losses never help, so you might like to try removing one or both of the winners from the match to confirm that having fewer losses results in a higher rating for the 0/500 player.

That leaves the question of why a player's rating would increase from a neutral result, where they play only against themself. The answer is that probably mu didn't change, but sigma decreased; if we publish something like mu - 2*sigma, then the published rating will be higher. sigma changes a lot for new players, but stabilizes later so you'd no longer see this effect. Of course, a 1-player match is completely uninformative so sigma really shouldn't be updated, but Elo-MMR's measure of information gain is based on the limit as the number of participants goes to infinity.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants