Details of upcoming patch

Announcements made here about the game and the company.

Re: Details of upcoming patch

Postby shapesifter13 » Wed Jul 22, 2020 1:47 pm

We just posted the patch notes!: viewtopic.php?f=11&t=110286
shapesifter13
Developer
Developer
 
Posts: 4681
Joined: Fri Jan 02, 2015 4:55 pm

Re: Details of upcoming patch

Postby Superalex11 » Sun Aug 02, 2020 8:58 am

Achilles wrote:Juggernaut will still be more rare than NKs and has the lowest chance of any role to spawn from an ANY slot, but it will be around 6 times more likely to spawn in every slot it is able to spawn in.

So uhhh... are you sure this is correct? I've just checked role frequency over all CAA reports before this patch and this is what I get: https://i.gyazo.com/7c6dc4b269a3462ac10 ... eb2e9c.png
The table shows the count of each role over all CAA reports, functionally representing their spawnrate. Though, since reports don't account for role conversions, these numbers don't accurately reflect spawnrates of all roles, just those that can't convert. But with that in mind we can look at some, including jugg.

The roles I've bolded are unique roles which can't be converted, all of which (except jugg) should have about the same spawnrate. From this table, the average spawnrate of non-jugg bolded roles is about 1.96%. This means that over 100 'any' slots (in CAA), we could expect, for example, about 2 ww's. The spawnrate of jugg, however, seems to be a much lower 0.36%.

The problem then, is that if we're to increase jugg's spawnrate by 6x, it... goes above the others...? Clearly something is wrong here, and as yet I can't see the fault being in my calculations.
Soon™
User avatar
Superalex11
Retributionist
Retributionist
 
Posts: 321
Joined: Fri Jan 08, 2016 10:11 pm

Re: Details of upcoming patch

Postby ScarfVendetta » Sun Aug 02, 2020 10:19 am

Superalex11 wrote:
Achilles wrote:Juggernaut will still be more rare than NKs and has the lowest chance of any role to spawn from an ANY slot, but it will be around 6 times more likely to spawn in every slot it is able to spawn in.

So uhhh... are you sure this is correct? I've just checked role frequency over all CAA reports before this patch and this is what I get: https://i.gyazo.com/7c6dc4b269a3462ac10 ... eb2e9c.png
The table shows the count of each role over all CAA reports, functionally representing their spawnrate. Though, since reports don't account for role conversions, these numbers don't accurately reflect spawnrates of all roles, just those that can't convert. But with that in mind we can look at some, including jugg.

The roles I've bolded are unique roles which can't be converted, all of which (except jugg) should have about the same spawnrate. From this table, the average spawnrate of non-jugg bolded roles is about 1.96%. This means that over 100 'any' slots (in CAA), we could expect, for example, about 2 ww's. The spawnrate of jugg, however, seems to be a much lower 0.36%.

The problem then, is that if we're to increase jugg's spawnrate by 6x, it... goes above the others...? Clearly something is wrong here, and as yet I can't see the fault being in my calculations.


I'm not an expert in mathematics, but I think the problem with your calculations might be due to the sample distorting the perceived spawnrate of unique roles, as each game will have 15 Any slots.

This effect would be more obvious if your sample was of 1000 Any slots per game instead of 15. There would be 1 Werewolf and 1 Juggernaut in almost every game, so the spawnrates would appear very similar, even though in reality this would not be the case.

It's trickier to visualise it with only 15 Any slots, but across your entire sample there will be many Werewolves and Juggernauts that did not spawn due to another role of that type already being present. As Werewolf has a higher spawnrate, this means that MORE Werewolves will have failed to spawn than Juggernauts. To simplify it: Werewolf's perceived spawnrate will be more of an underestimate than Juggernaut's perceived spawnrate.

Coven Ranked would produce more accurate data, as there is only 1 Neutral Killing slot. This means that Werewolf or Juggernaut being Unique would not influence the data.
Just some Sheriff who didn't claim until Day 5, and has no leads whatsoever

No need to be alarmed...
User avatar
ScarfVendetta
Summer 2020 Winner
Summer 2020 Winner
 
Posts: 286
Joined: Sat Jul 27, 2019 6:52 pm
Location: Essex, England

Re: Details of upcoming patch

Postby kyuss420 » Mon Aug 03, 2020 1:29 am

Superalex11 wrote:
Achilles wrote:Juggernaut will still be more rare than NKs and has the lowest chance of any role to spawn from an ANY slot, but it will be around 6 times more likely to spawn in every slot it is able to spawn in.

So uhhh... are you sure this is correct? I've just checked role frequency over all CAA reports before this patch and this is what I get: https://i.gyazo.com/7c6dc4b269a3462ac10 ... eb2e9c.png
The table shows the count of each role over all CAA reports, functionally representing their spawnrate. Though, since reports don't account for role conversions, these numbers don't accurately reflect spawnrates of all roles, just those that can't convert. But with that in mind we can look at some, including jugg.

The roles I've bolded are unique roles which can't be converted, all of which (except jugg) should have about the same spawnrate. From this table, the average spawnrate of non-jugg bolded roles is about 1.96%. This means that over 100 'any' slots (in CAA), we could expect, for example, about 2 ww's. The spawnrate of jugg, however, seems to be a much lower 0.36%.

The problem then, is that if we're to increase jugg's spawnrate by 6x, it... goes above the others...? Clearly something is wrong here, and as yet I can't see the fault being in my calculations.


While using the trial reports as a sample is a good idea, as theyre the only records of games... Not every game has someone reported, so the data just confirms that less reports are generated when theres a juggernaut in game.

I could probably confirm that is the case, because every 2nd/3rd Jugg game Im in, seems like everyone is throwing to either - ''let Jugg win, cos its a rare role'', ''ive never seen a jugg win before'', ''i dont have a jugg win acheivement yet'' and the old ''lets get Jugg to attack Pestilence, so we can get the acheivement''. So even though rules are blatantly broken, no one bothers to report. (Hell Ive gottten bagged out and harrassed for the next 5 games, because I executed Juggy as jailor, and people wanted the pest vs jugg acheivement - i think i even got reported)
goosegoosegoosegoosegoose
Spoiler: Image Spoiler: Image Spoiler: Image Spoiler: Image Spoiler: Image
User avatar
kyuss420
Serial Killer
Serial Killer
 
Posts: 2284
Joined: Sat Aug 19, 2017 1:33 am
Location: Im here

Re: Details of upcoming patch

Postby Superalex11 » Tue Aug 04, 2020 1:36 am

@Kyuss I think you've misunderstood. It's not that jugg spawnrate appears too low compared to those in its category (unique non-convertibles), but that it's too high. And plus, I'm alreading working with error margins of ~3% in some areas here, so even if we're speaking in the opposite direction I don't think sampling bias is a significant problem.

@Scarf I've been thinking about this since you posted, and I'm still not able to reconcile it with the data, or even connect it intuitively. Ww and jugg are both unique non-convertibles, so they should remain in proportion regardless.
Oddly, though, is that the CRP numbers do make a lot more sense. Here I've counted the number of matches which have at least one of each role: https://i.gyazo.com/aa3e082820b811fb936 ... 401ee0.png

But now, things get a lot more complicated, as it seems this discovery opened a massive can of worms. So first, some background. Many months back I began work on a project which could determine spawnrates based on theory rather than historical data. As far as I've known, it's been 100% accurate (with the exception of jugg), since it was based on a simple algorithm that made sense. It's not likely that there would be some minor mistake avalanching everything else into ruin.

Back to relevance, I started comparing this theoretical data to the historical date. Importantly this let me fix jugg in my code so it had a correctly lowered probability, verified when I simulated CRP matches: https://i.gyazo.com/712aaaa01b6edb448db ... 3396a1.png. But most importantly this let me see if anything else weird was going on, and oh boy.


There's a lot of raw data to look at here, so I'll post them full and explain brief parts.
Historical: https://i.gyazo.com/7ff89e98e3a919958ff ... 78e7ff.png (*This data is from 2019 onward)
Theory: https://i.gyazo.com/169173f1ed4a2af5c23 ... 1aa18e.png
% Diffs: https://i.gyazo.com/9fd7d55297b506362e3 ... ea55f9.png

The first thing to note, again, is that many of the differences can be accounted for by the fact that data from reports is post-game, while the theoretical data is pre-game, so most numbers involving amnes, vamps, and maf are guaranteed to be off. The second thing to note are the very few odd numbers (like <100% probability of a jailor or gf in historical ranked practice) - these are due to small bugs in my code, but ultimately are still within a safe error margin, and thus shouldn't be a problem.

But now, onto the interesting stuff...
Here, I've recolored the diff-table to show differences not accounted for by the above: https://i.gyazo.com/d9f006a2a6fbddc8778 ... 626a8c.png
The variance seen more on the right is due to low sample size, so that's no problem. But then we look to the left and see quite a few oddities:

RP/Ranked: High arso, low sk, high mayor, low trans
Classic: High mayor
AA: Very low vamp, all low nk's, low witch; also, while uncolored, surv is very low (that 30% diff is certainly not from conversions)
CAA: Insanely high jugg, all low other nk's
VIP: Very high hm, very low pois, very low trans
Lovers: Very high mayor, very low trans


So for now I'm going to leave it here. I'm currently working on a way to account for conversions, and I hope to get better data in future. And as a brief conclusion for now, I think it's undeniable that probabilities are being artificially inflated and deflated.
Soon™
User avatar
Superalex11
Retributionist
Retributionist
 
Posts: 321
Joined: Fri Jan 08, 2016 10:11 pm

Re: Details of upcoming patch

Postby Brilliand » Tue Aug 04, 2020 2:16 am

Superalex11 wrote:@Kyuss I think you've misunderstood. It's not that jugg spawnrate appears too low compared to those in its category (unique non-convertibles), but that it's too high. And plus, I'm alreading working with error margins of ~3% in some areas here, so even if we're speaking in the opposite direction I don't think sampling bias is a significant problem.

@Scarf I've been thinking about this since you posted, and I'm still not able to reconcile it with the data, or even connect it intuitively. Ww and jugg are both unique non-convertibles, so they should remain in proportion regardless.
Oddly, though, is that the CRP numbers do make a lot more sense. Here I've counted the number of matches which have at least one of each role: https://i.gyazo.com/aa3e082820b811fb936 ... 401ee0.png


The ratio between WW and Jugg spawnrates should depend both on their relative base spawnrates, and on the number of Any, Random Neutral and Neutral Killing slots in the game. The more slots that can spawn either a WW or a Jugg, the more the WW-to-Jugg ratio will skew toward 1:1. Did you compensate for that effect? Because at first glance it appears difficult to compensate for accurately, and the expected general effect would be for the Jugg spawnrate to be "too high" but still "less than the WW spawnrate".
User avatar
Brilliand
Godfather
Godfather
 
Posts: 1503
Joined: Sat Nov 29, 2014 8:34 pm

Re: Details of upcoming patch

Postby Superalex11 » Tue Aug 04, 2020 2:33 am

Brilliand wrote:The ratio between WW and Jugg spawnrates should depend both on their relative base spawnrates, and on the number of Any, Random Neutral and Neutral Killing slots in the game. The more slots that can spawn either a WW or a Jugg, the more the WW-to-Jugg ratio will skew toward 1:1. Did you compensate for that effect?

Assuming the algorithm BMG uses to determine roles simply reduces the odds of a jugg pick compared to other roles, then yes, as that's what my algorithm does in the simulation I ran for the theoretical probabilities.



Edit because I finished this so quickly, here's the updated difference chart: https://i.gyazo.com/644f7acb658fe1cf8f8 ... ce792f.png
This chart factors in vamp+amne conversions (NOT mafia, ga>surv, exe>jest, or vh>vig), and also doesn't contain bugs from my code (e.g. 1 amne included in a ranked match).
And yeah, looking at it now, it's way worse than first thought... but I cannot imagine sampling bias accounting for so much of this.
Soon™
User avatar
Superalex11
Retributionist
Retributionist
 
Posts: 321
Joined: Fri Jan 08, 2016 10:11 pm

Previous

Return to Announcements

Who is online

Users browsing this forum: No registered users and 5 guests

cron