Note: All of my knowledge of the ToS Elo system comes from this wiki page.
The Elo system is completely broken in my eyes. The main issue with it is that Elo inflation allows any competent player to reach Master Elo if they play enough games. Mathematically, a 2000 Elo Town is expected to beat a 1200 Elo Mafia 99 percent of the time. I do believe that the Town would win in 80 or even 90 percent of these games, but ToS is random enough that the weaker Mafia would win a lot more than 1 percent of the time. Anyway, here's how I think Elo should be fixed:
1. Different K-factors for Town and Mafia
Elo inflation exists in ToS because a of differences between the actual and expected winrates of each faction. The game calculates an offline winrate factor that is used to modify Elo changes. As far as I know, this isn't dependent of the Elo of players in the game. However, Town wins more often at higher levels than at lower levels, leading to Elo inflation at higher levels and deflation at lower levels (this also partially explains Elo hell). To see this, imagine a situation where 9 Townies face 6 scum. All players have the same Elo. The overall winrate is assumed to be 50/50, but Town really wins 60% of games at this Elo level. Each faction gets +10 for a win and -10 for a loss. If these players play five games with the same factions, it is expected that Town will win 3 of the games. This would result in +10 Elo overall for the 9 Townies and -10 for the 6 scum. The average Elo of the group will increase by 2 despite the fact that these players did not play against anyone outside the group. Worst case scenario, players could intentionally queue at an unpopular time, all end up in the same game, and purposely throw to let Town win. After hundreds of matches, all players in the group will reach master level.
My proposed solution for this is to change the K-factors for each faction. For example, Town could have a K-factor of 20 while Mafia has a K-factor of 45. I will address Neutrals later. In any setup where Mafia's K-factor is 9/4 that of Town's, there will be no Elo inflation for Town and Mafia members regardless of the accuracy of predicted winrates.
2. Change handling of base winrates
Right now, the amount that a player's Elo changes is multiplied by a factor based on their faction's baseline winrate. The wiki claims that this is equal to 1 minus the winrate, but this wouldn't make sense when the player loses because it would punish players more for losing as a harder faction. I am going to assume that the factor is changed to the factional winrate when the player loses. So a player who would normally gain +15 Elo for winning with a 20% winrate role would actually gain +12 for a win and -3 for a loss. For factions with moderate winrates, this shouldn't be problematic, but it could lead to disproportionately low weighting on games where a player has a role with a low winrate (such as Witch). Assume a K-factor of 30 and a 10% baseline winrate for Mafia. I know this is unrealistic but I'm going to discuss Neutrals later. Normally, a faction with a 382 Elo advantage would be expected to win 90% of the time. In this case, if the Mafia has a 382 Elo advantage, they would gain 30 * 0.1 * 0.9 = 2.7 Elo for a win and lose the same amount for a loss. Meanwhile, a faction with a 50% winrate and no Elo advantage would get 30 * 0.5 * 0.5 = 7.5 Elo for a win and lose that amount for a loss. Players who are good at difficult roles will have unfairly low Elo while players who are especially bad at those roles will have unfairly high Elo.
To fix this, incorporate the factional winrate in the original win probability calculation. Playing as an easy role should be treated like having higher Elo and vice-versa. The team Elo modification should be set so that the expected winrate for a faction is equal to its baseline winrate when the two factions have equal Elo. For instance, being part of a faction with a 10% winrate should be treated the same as having 382/2 = 191 fewer Elo while the opposing faction is treated as having 191 more Elo. If both teams really have 1200 Elo, the win probability calculation should be performed assuming a 1009 Elo for the 10% winrate faction and 1391 Elo for the 90% winrate faction. The actual Elo values don't matter, just that the difference is 382.
3. Rework placement games
This isn't as much of a problem, but placement games can provide a secondary source of Elo inflation or deflation even with the above fixes. With those changes, the average Elo would still be 1200 if everyone plays their placement games at the same time. However, players who start late will have some of their games count as placement games while everyone else has those games count normally. If the players who start late are more skilled and win more often, they will gain more Elo than the other players lose due to the higher K-factor for placement games. Eventually, this would lead to a higher average Elo across the playerbase.
The best way to fix this would be to remove placement games, but this would make it harder to grind. Other solutions such as averaging the K-factors across all players in a game or changing everyone's Elo by a few points within a game to counteract inflation/deflation would disproportionately weight games where there are a lot of players doing placement games. Another solution would be to decrease the K-factor for all players as the season proceeds, but this would unfairly punish good players who start later in the season. I'm not sure of the best way to go about this, but I would suggest a version of that last suggestion. The season should start with a "placement period" of about one week where the K-factor is doubled. Players should be informed that this is the best time to grind and that they may have a harder time increasing their Elo if they start later on.
On another note, the K-factor should be the same for all players regardless of Elo. Having a lower K-factor for higher ratings may work in a game like chess where you have one opponent who is likely to have a similar rating to you, but it could lead to inflation or deflation in ToS. If master Elo players have a lower K-factor and lose more often than expected (due to regression to the mean), they will lose less Elo than their opponents gain, leading to Elo inflation.
4. How to handle Neutrals
This is by far the hardest part of making an Elo system for a game like ToS, and I'm not exactly sure how the current system handles them. Treating Neutrals as separate from Town and Mafia would lead to inflation if they win more often than expected. And if you treat NEs as Mafia members and adjust the K-factors to prevent inflation, Mafia members will be incentivized to kill NEs once they are guaranteed to win. This is how the Town of Morons Elo system works, but it wouldn't work in ToS. However, having the Mafia K-factor stay constant while punishing the Town for NE wins could work. If an NE gets +18 Elo for winning and there are nine Townies, each one could get -2 in addition to the amount they would normally lose for losing the game. For Witch, this shouldn't cause any issues besides encouraging Townies to possibly push a Witch over a Mafia member. This would make it significantly harder for Executioners to side with Town and would encourage Townies to deny Executioners the win, which is arguably a good thing since Executioner is designed to be anti-Town. These influences would generally only result in Elo swings of 1-2 points, so they would likely not be a major problem.
The other difficulty in incorporating Neutrals into the Elo system is calculating their win probability. A Witch's win probability is likely influenced by their own Elo, the overall Mafia Elo, and the overall Town Elo. The Executioner's win probability likely involves the same factors as Witch but also its target's Elo. It may be difficult to calculate how much each of these factors influences the overall win probability calculation. One way to do it could be a logistic regression where the relative values of each coefficient could be interpreted as the relative influence of each variable on the overall win probability. I'm certainly no expert on machine learning, but maybe someone with more knowledge of the topic can provide better insight. If these values do not hold for all Elo levels, there would not be any overall inflation, but players may have a slightly positive or negative expected Elo change depending on their faction. For instance, it is likely that Mafia Elo is more important in determining a Witch's win probability at high Elo levels because Witches are more likely to coordinate with Mafia in those games. If that's the case, a Witch who is much better than the Mafia in a high-Elo game will win less often than expected, causing them to lose Elo as Witch in the long run. Since high-Elo players tend to play with players with lower ratings, they are likely to experience deflation while weaker players would experience inflation. This is not as severe of an issue as the current Elo inflation because it only affects Witch and Executioner compared to all 15 slots, and it does not affect the average Elo across the playerbase.
5. New team Elo calculation
Currently, the average factional Elo is used to determine the win probabilities for Town and Mafia. This assumes that all players contribute equally to the victory of a faction. However, it is very common for one skilled player to carry the rest of their team, especially among Town members. Intuitively, a Town with nine 1200 Elo members is less likely to win than a Town with eight 1100 Elo players and one 2000 Elo player even though both teams have the same average Elo. As a result, better players are likely to win more often than their average Elo suggests, causing them to experience Elo inflation while weaker players experience deflation.
This problem can be mitigated by using the procedure that I suggested for calculating Neutral win probabilities. While far from perfect, it would lessen the inflation that high Elo players experience by carrying weaker teammates. The win probability of Town is influenced by the top Elo of a Townie, average Town Elo, average Mafia Elo, and top Mafia Elo among other factors. The model could also include the Elo for other players and possibly certain roles like the Jailor or Godfather. The win probabilities will still be slightly off the true values, but they would probably be closer than the values obtained by just taking the average Elo for each faction.
