Jump to content


[WN7] What is it and how does it work?

statistics stats noobmeter performance efficiency metric compare quality rating Garbad

  • Please log in to reply
2483 replies to this topic

Praetor77 #1 Posted Nov 29 2012 - 00:36

    Captain

  • Players
  • 29723 battles
  • 1,675
  • Member since:
    05-01-2011

*
POPULAR

WN7 formula:
(1240-1040/(MIN(TIER,6))^0.164)*FRAGS
+DAMAGE*530/(184*e^(0.24*TIER)+130)
+SPOT*125*MIN(TIER, 3)/3
+MIN(DEF,2.2)*100
+((185/(0.17+e^((WINRATE-35)*-0.134)))-500)*0.45
-[(5 - MIN(TIER,5))*125] / [1 + e^( ( TIER - (GAMESPLAYED/220)^(3/TIER) )*1.5 )]
Would like to highlight that MIN() means the number capped at that value, so MIN(TIER, 5) means avg tier capped at 5, (so player avg tier is used if it is lower than 5, otherwise 5 is used) and MIN(DEF,2.2) means defense is capped at 2.2.
WN7 SUMMARY
written by Neverwish, Crabeatoff and Praetor77
Work on WN7 is a community effort. I consider everyone who has posted in this forum to have contributed their two cents into making WN what it is today. However, I would like to highlight the contribution of the people who have dedicated the most time and effort into making WN:
Key contributors: Tpapp157, Neatoman, Syndicate, Maokai, Makaze2048, Guerdon, DracoArgentum, Crabeatoff, TheKilltech, etc. etc.
The WN rating was created using statistical analysis tools like correlation studies and evolutionary algorythms to create an accurate formula, using Win Rate as a proxy to accurately determine the weight each stat would have in the formula. The idea was to create a formula which actually tried to measure player skill in the most accurate way possible, using global account stats.
The Efficiency formula was the basis for an analysis to figure out what was wrong with it and create an improved formula with those problems fixed.
WN is short for Weighed and Normalized, and implements various ways to deal with statpadding, and in the end tried to develop a metric that could only be padded by actually being good at the game.
Key points of the WN Rating are:
  • Damage is scaled according to your average tier and is the most important stat in the formula. The points you get for damage are carefully tied to the avg tier played, so that players with avg tier played 6 or 9 could be accurately compared, despite having very different average damage. To do this, average damage for tiers were collected from vbaddict.net and the data was carefully fitted to a non-linear curve.
  • Players with a considerable number of battles who have an average tier lower than 4 are heavily penalized for sealclubbing. New players with few battles under their belts are not penalized until they achieve a big number of battles but remain with a low average tier played.
  • Cap points are not counted towards your rating, since despite HUGE efforts and statistical analysis mainly performed by Syndicate, there was no statistically sound way to include cap points into the formula. The data suggests that for the average player cap points which are actually useful in winning a game for your team are drowned out by the huge amount of useless cap points gathered at the end of already won games.
  • Winrate is used as a proxy to measure intangible stats which are not available on the player profile like spotted damage, cap used to lure the enemy out, stopping scouts from killing your arty, tracking enemies at crucial moments, keeping your teammates alive, map awareness and other crucial decisions not recorded in the stats. This term of the forumla counts for 0-10% of final WN rating.
  • Average Defense points is capped at 2.2 to prevent padding. Defense also proved to be highly correlated to winrate, suggesting players who have map awareness and return to base when needed to stop enemy capping win more often.
  • One of the most important characteristics of the WN Rating is the open development format, meaning any player can post in this thread and suggest modifications, which will be tested and, if successful, implemented. By having this open development model, it effectively eliminates biases which closed formulas such as Efficiency have.
  • After the rating was released, WoTLabs was the first website to implement it, keeping it up to date every time a new version is launched. Although many people dislike the fast evolution of the rating (having gone through several changes and versions in only 4 months), this means that the formula rapidly grows more accurate. It has spread to the point where the WN Rating is now the standard rating used on the XVM mod, although transformed into a 0-99 scale rating.

Common Misconceptions
Despite having been based on advanced algorithms, the WN Rating did not pass without heavy criticism, although most, if not all, of this criticism turned out to be misconceptions.
One common complaint was that, if it was made to correlate with win rate, then we could just use win rate. Unfortunately win rate can be easily padded by platooning with good players. The WN Rating separates those padded players by using their actual stats. A veteran player with a low WN Rating but a high win rate has been probably been heavily padded.
Another common criticism is that we should stop caring about statistics and just play the game, since statistics eventually lead to mockery. Unfortunately World of Tanks is a competitive game, and as in all competitive games, there must be a measuring stick in order to know if you are improving and how much you are improving, otherwise we might as well go play Farmville. Shooting tanks for the heck of it gets boring after a while. The idea behind WN is to use it as a tool to make sure you keep improving, and also as a wealth of information when used in XVM to help you make critical decisions based on the skill of your teammates and enemies.
Other players criticize the formula for not taking things like spotting damage into account. This can hardly be pinned on the WN Rating, since Wargaming has not released this information publicly. The WN Rating can only work with what it has available. To account for those invisible stats that help win games, Win Rate has been added to the formula.
Lastly, many players criticized using winrate in the formula, since the very same winrate was used as a proxy to weigh the other stats in the evolutionary formula. However, as posted above, the idea was not to reward winrate, but to use it as a proxy for intangible stats. Also, the reward for these "intangibles" are carefully tailored by Neatoman into an S-curve with diminishing returns for winrates above 60%, where correlation to stats drop significantly, suggesting winrates above this value are purely due to platooning and companies. Also supporting this data is the fact that Zakaladas (quite possibly NA server´s best player) almost always plays solo and averages 64% wins.
Limitations and problems
Formulas can only be created from stats that are made available via the official WoT website. Efficiency depends on those same stats. Everything WoT-news computes is off those same stats. More information on YOUR history is available from the cached dossier file, but unless everyone starts mass uploading those (which will never happen), then the official website stats are THE source for data.
What isn't in WoT website stats?
  • Normalized Experience (XP) - theoretically WG could keep track of experience based on whether a user had a premium account or not, and then either remove the premium bonus OR give all standard account users the bonus (for stats purposes) to normalize XP across users.
  • Damage Upon Detection - Damage done to targets you are spotting yourself, by tanks who are not spotting them themselves. The latter is the bread and butter of light tanks (LTs) and of front line fighters. The other bonuses are relatively small compared to the latter. But it is the most noticeably missing in all rating calculations, and it particularly screws over LTs.
  • Per tank anything - the website cannot tell you damage per tank, spots per tank, etc. This information lives in your dossier and somewhere on the WG servers. If you use a dossier parsing tool (there are several web based and one local), you can obtain this information on your per tank performances. There is something called an API which gives you this kind of information, but currently the NA server API does not work correctly, like for example it says I have 15.4 spots per game on my IS-4.

Implications of these limitations
Light tanks
Due to the lack of DUD (damage upon spotting) on the WG website, light tanks are unfairly measured by WN7. They normally get lower WN7 scores than heavy, med or TDs of the same tier. That being said, WN7 is actually the metric that gives one of NA server´s best scouts (Redparadize) the highest rating...
PR: 1842
Eff2.0: 1742
WN7: 1943
SPGs
SPGs also cause issues, as their tiers are not lined up with the rest of tanks! They do much more damage than their tier value would indicate for a HT, MT or TD. This is a known limitation of the formulas. Extensive programming (parsing the website stats for SPG counts and adjusting their tier) COULD fix this, but the problem will go away when the SPGs tiers are stretched to match (per the latest ASAP with SerB). For now...we deal with it. Who care about SPG players stats anyways, amirite?!?!?
Some statistical limitations
When measuring a population, its not going to be possible to put every single person on the scale and have the scale make sense. Again, returning to a notable outlier, Tazilon and his 20k+ VK2801 games. This massive number of games means his average tier played is 5.35, which is lower than is "generally expected" for someone with 28k total games. It takes longer to move through higher tier tanks, and so you end up with more weight at 6+. Because WN7 is designed to measure the population relative to each other, some assumptions have to be made about the habits of the general population. Most players don't play 20k games in any single class or tier below 8, let alone 20k in a single tier 5. If someone plays 10k games in the MS-1....outlier! Takeaway: population ratings cannot account for every outlier.
Back to the formula
A detailed explanation of each portion of the WN7 formula by Crabeatoff and Praetor77.
Includes the answer to questions like "Why is cap not included in the formula?", "How much does winrate contribute to the equation" and "How does the low tier penalty work?":
Spoiler

WN7 Scale
This is the current scale for the WN7 Rating:
WN7 Rating Key
Under 500 Very Bad
500 to 699 Bad
700 to 899 Below Average
900 to 1099 Average
1100 to 1349 Good
1350 to 1549 Very Good
1550 to 1849 Great
1849 to 2049 Unicum
2050 and Above Super Unicum
This scale is different from the one used in XVM, since their analysis of russian player database gave different results. The scale is currently on schedule to be readjusted to an analysis of the wotlabs player database being performed by Neverwish.
And this is what goes on inside the WN Rating. I hope I could clear some doubts regarding this formula! Feel free to post on this thread if you have any suggestions or questions.
EXTRAS
Boom_Box's script which modifies Player profile page and shows a lot of extra information, including WN7.
Excel sheet by Folterknecht for offline WN7 calculations
Wotlabs website for awesome signatures and other stats
Mywotstats website for signatures, player stats, and some neat features like customized server rankings for 30 or 60 day stats
Noobmeter website for noobmeter metric, WN7 stats, a neat history of your wN7, NM and WR stats, etc.
Old information for previous versions of WN:
Spoiler

Edited by Praetor77, Feb 22 2014 - 22:24.


__Endo__ #2 Posted Nov 29 2012 - 00:41

    First lieutenant

  • Players
  • 7679 battles
  • 723
  • [JHAM] JHAM
  • Member since:
    10-24-2011
Wow, the math that went into this is quite amazing. Glad you did this and +1 to you!

justinboer #3 Posted Nov 29 2012 - 00:47

    Sergeant

  • Players
  • 9551 battles
  • 103
  • Member since:
    11-27-2011
5 min ago i thought i could crunch numbers rather well.  Thank you for ruining it for me.

well done +1..... I think

SuicidalPigeon #4 Posted Nov 29 2012 - 00:49

    Captain

  • Players
  • 12902 battles
  • 1,428
  • Member since:
    07-12-2012
I didn't read anything but I upvoted you anyway because I can't math.

AMartin223 #5 Posted Nov 29 2012 - 00:51

    Major

  • Players
  • 12351 battles
  • 2,102
  • [T1C] T1C
  • Member since:
    01-18-2012
One major concern.  You are using W/L as your benchmark.  While I agree W/L is a good indicator, isn't finding a metric that best mimics it kind of pointless?  Wouldn't it be better to say, get players to keep track of their non-platooned public matches only and see how the various metrics relate to winrate in those games, then base your metric off of those ratios?

Edit: To be clear, my concern is that we already use W/L as a metric, and we use Efficiency to guesstimate how much of that win ratio is padded.  Trying to make your rating correlate perfectly with win ratio seems to make it meaningless.

Edited by AMartin223, Nov 29 2012 - 00:53.


HOTA_CHATON #6 Posted Nov 29 2012 - 00:55

    Major

  • Players
  • 11696 battles
  • 11,756
  • [T0TS] T0TS
  • Member since:
    09-28-2011
What a total waste of time and effort.  Why do people get on here and try to make this a life and death situation.  I can't do this as I don't have the stats to or I can do this as my stats are
good enough to.

Can't you just let this be a game, as it was really intended,fro the start?

Nice effort but extremely useless.

PanzerXO #7 Posted Nov 29 2012 - 01:01

    Captain

  • Players
  • 13166 battles
  • 1,902
  • Member since:
    05-23-2012
Please keep us up to date with this.  I like what I've read, and look forward to getting back to my comp so I can check out your excel file.

How are you acquiring the data?  Selective viewing of Dossiers?  Would it help to convert the formula into a database and collect a larger sample?

if you continue, would be interested in seeing a way to parse recent battles / performance.  7 / 30 / 60 / 90 day stats or last x number of battles.

SHISHKABOB #8 Posted Nov 29 2012 - 01:04

    Major

  • Beta Testers
  • 8526 battles
  • 13,139
  • Member since:
    12-06-2010

View PostHOTA_CHATON, on Nov 29 2012 - 00:55, said:

What a total waste of time and effort.  Why do people get on here and try to make this a life and death situation.  I can't do this as I don't have the stats to or I can do this as my stats are
good enough to.

Can't you just let this be a game, as it was really intended,fro the start?

Nice effort but extremely useless.

no one's making it a life or death effort, there are simply people out there who *enjoy* crunching numbers. I am an example of one of those people.

We just like doing this kind of stuff because it is a puzzle, we like solving puzzles and we like to solve them with math and numbers and computer programs and stuff like that.

it's just an extension of the game, honestly, because crunching the numbers and doing this stuff is just as much fun as shooting tanks

Praetor77 #9 Posted Nov 29 2012 - 01:04

    Captain

  • Players
  • 29723 battles
  • 1,675
  • Member since:
    05-01-2011
One more thing, this wnEfficiency should be a MUCH better tool than normal efficiency to compare player ability in a given tank, as is analyzed here:

http://vbaddict.net/wot

If you check that website you will see the top 5-10 efficiency players for each tank always has very low damage, but around 12-20 capture/battle. Using wnEfficiency would surely improve actually comparing players ina a fair way...since capping tops at 3.2, defense at 2.2, and damage for a vehicle of a given tier is properly normalized, unlike normal Efficiency.


Lastly, let me give you a screenshot of the top 40 players of Havok and G according to efficiency. On the left hand, you have ranking in Havok+G according to efficiency, ranking according to WNEfficiency, player name, Games Played, Victories %, Battles Survived %, Destroyed, Spotted, Damage, Capture Points, Defense Points, Average tier, WN2-Efficiency score, and finally Efficiency score. Colored by red (below Havok+G average), yellow (between average and +1 standard deviation), green (between 1 and 2 standard deviations), and violet (above 2 standard deviations).
Posted Image
The better normalization of damage according to average tier played of WN when compared to efficiency, together with cap limit, is what drops TheYankeeClipper from 1st place of efficiency ranking to 24th place. Wiking rises in the WN ranking and stands out from the rest, since despite his "low" 1449 average damage, he has managed that, playing an average tier of 6.12.
Similarly _Endo_ drops from 4th to 37th place. This is due to his excessive capping (4.26) and the fact that his whopping 1851 average damage was achieved while playing an average tier of 8.2.

I believe WN is fairer and more statistically sound than PR and efficiency.

I am open to all kinds of criticisms, comments, questions, etc.

PS: G+Havok analysis:
http://www.mediafire...zgx39hvnxlzwfeo

Edited by Praetor77, Nov 29 2012 - 01:20.


Garbad #10 Posted Nov 29 2012 - 01:05

    Major

  • Beta Testers
  • 29232 battles
  • 14,206
  • [-G-] -G-
  • Member since:
    10-02-2010
Where does one get winEff? I haven't seen this yet, but it looks to be impressively predictive.

The wnEfficiency formula is:
frags*(250.0-level*9.0) + damage*540/(31.7*tier^1.83) + 115.0*spotted+130.0*defense(capped at 2.2/battle)+55.0*capture(capped at 3.2/battle)+ (Winrate(capped at 60%)-48.5)*42

My only concern about this is the lack of some way to measure the value of exp/game, ie, spotting and such.

NeatoMan #11 Posted Nov 29 2012 - 01:05

    Major

  • Players
  • 23737 battles
  • 14,382
  • Member since:
    06-28-2011
Win rate and any efficiency measurement will never correlate to a high degree. Efficiency is an individual measure, while win rate also includes the influence of platoon mates, and to a lesser extent company and CW battles.   In the top clans, or with top players that platoon together, the relationship will be skewed toward higher win rates for any given efficiency.

Just look at what CarbonWard is doing with a platoon of three unicums.  Their efficiency is probably not skyrocketting to the extent that their win rates are.

PanzerXO #12 Posted Nov 29 2012 - 01:05

    Captain

  • Players
  • 13166 battles
  • 1,902
  • Member since:
    05-23-2012

View PostHOTA_CHATON, on Nov 29 2012 - 00:55, said:

What a total waste of time and effort.  Why do people get on here and try to make this a life and death situation.  I can't do this as I don't have the stats to or I can do this as my stats are
good enough to.

Can't you just let this be a game, as it was really intended,fro the start?

Nice effort but extremely useless.
. Isn't this your third or fourth tirade on this subject in the last week?

Your horse is dead.

Praetor77 #13 Posted Nov 29 2012 - 01:07

    Captain

  • Players
  • 29723 battles
  • 1,675
  • Member since:
    05-01-2011
Data was collected from here:
http://wot-news.com/...s/us/1000000105
and here:
http://wot-news.com/...s/us/1000001695

I would be awesome to be able to upscale this, but it is out of my reach, just putting the formula out to you guys to see what you think... :D

Garbad, sadly that is the one weak link here, if we had average damage spotted/game, that would make my day! :D I calculated wnEff using Excel, and getting stats for each player from each clan from wot-news.

Edited by Praetor77, Nov 29 2012 - 01:13.


Praetor77 #14 Posted Nov 29 2012 - 01:18

    Captain

  • Players
  • 29723 battles
  • 1,675
  • Member since:
    05-01-2011
Oh, aslo forgot to mention, in average, how much each stat contributes to total WNEfficiency total:

15%: (frag part of equation)
45%: (damage part of equation)
10%: (Spot part of equation)
09%: (Cap part of equation)
11%: (Defense part of equation)
10%: (Winrate part of equation)

Edited by Praetor77, Nov 29 2012 - 01:21.


Praetor77 #15 Posted Nov 29 2012 - 01:27

    Captain

  • Players
  • 29723 battles
  • 1,675
  • Member since:
    05-01-2011

View PostAMartin223, on Nov 29 2012 - 00:51, said:

One major concern.  You are using W/L as your benchmark.  While I agree W/L is a good indicator, isn't finding a metric that best mimics it kind of pointless?  Wouldn't it be better to say, get players to keep track of their non-platooned public matches only and see how the various metrics relate to winrate in those games, then base your metric off of those ratios?

Edit: To be clear, my concern is that we already use W/L as a metric, and we use Efficiency to guesstimate how much of that win ratio is padded.  Trying to make your rating correlate perfectly with win ratio seems to make it meaningless.

You bring up a good point, I invite you to download the xls, look at the stats for players, or even better, paste your clan stats into page 2 of the xls, and look at those stats and try to compare the numbers efficiency, WNEfficiency for each player in your clan and see which is more fair and exact when compared to your in-game knowledge of those player´s true tanking abilities.

I had no way to batch-retrieve PRs, I did them by hand, each one of the 200... :(

WR is actually included in the calculation of WNEff, as it also is in PR. When checking correlation with WR, you can see weird things for PR, like having it drop off in an almost 90 degree angle for people with PRs higher than 1600... doesn´t happen with WNEff, and also WNEf is the only one stable throughout it´s entire range.

weesh #16 Posted Nov 29 2012 - 01:29

    Major

  • Players
  • 13374 battles
  • 2,274
  • Member since:
    09-11-2012
Edit: I was ninja'd.  Nothing to see here.

It is nice to see the stats weighted to their ability to create a win.  This is could be the best formula that we have at this time.

But I have reservations.

You have plots of kills, damage, cap, defense, and spots versus WR.  But there is another stat in your forumla that you didn't make a plot for: WR.  Why didn't you correlate WR to WR?  Because that makes no sense!  If the end all be all in WR, why would we EVER look at efficiency?  We could just look at WR and be done with it.  Efficiency is attempting to tell us something more.  Something different.  WR should not be in the calc.  The very fact that the BASIS of the calc is to correlate everything to WR seems wrong to me.

Edited by weesh, Nov 29 2012 - 01:32.


PanzerXO #17 Posted Nov 29 2012 - 01:34

    Captain

  • Players
  • 13166 battles
  • 1,902
  • Member since:
    05-23-2012
Have you given any thought to the platooning problem?

I do like a calculator / predictor that more closely correlates to win rates - but have a concern that anything that too closely ties itself to win rate as shown by top clan members at the upper end may not reflect an individual's actual performance; but rather his win rate as a party of good players forming a team within a team.

My concern is that corrections you make using top tier Clan aligned players will skew the results for players who do not platoon.

Praetor77 #18 Posted Nov 29 2012 - 01:42

    Captain

  • Players
  • 29723 battles
  • 1,675
  • Member since:
    05-01-2011

View PostPanzerXO, on Nov 29 2012 - 01:34, said:

Have you given any thought to the platooning problem?
I do like a calculator / predictor that more closely correlates to win rates - but have a concern that anything that too closely ties itself to win rate as shown by top clan members at the upper end may not reflect an individual's actual performance; but rather his win rate as a party of good players forming a team within a team.
My concern is that corrections you make using top tier Clan aligned players will skew the results for players who do not platoon.
I made no corrections using G and Havok data, I just analyzed it to see what would happen. The formula is supposed to be best at calculating an AVERAGE player´s stats and his contributions to his team. I think the ACA + ACA-T comparison is good at showing that. I wanted to see how "fair" wnEff was at measuring top player performance, and I honestly think it does a pretty good job.

Nevertheless, there is a mechanism in place to "correct" for excessive platoon and TC play, and although crude, it works. Winrate is capped at 60% which is a personal estimate of the maximum you could reach playing solo (well, not you or me, maybe Garbad or Carbonward :D).


View Postweesh, on Nov 29 2012 - 01:29, said:

Edit: I was ninja'd.  Nothing to see here.

It is nice to see the stats weighted to their ability to create a win.  This is could be the best formula that we have at this time.

But I have reservations.

You have plots of kills, damage, cap, defense, and spots versus WR.  But there is another stat in your forumla that you didn't make a plot for: WR.  Why didn't you correlate WR to WR?  Because that makes no sense!  If the end all be all in WR, why would we EVER look at efficiency?  We could just look at WR and be done with it.  Efficiency is attempting to tell us something more.  Something different.  WR should not be in the calc.  The very fact that the BASIS of the calc is to correlate everything to WR seems wrong to me.

I added WR due to my clanmates input, had not incorporated it into the formula at first, but they convinced me, and here is why:

There are small things every good player does which do not appear in any of the other stats. Choose targets, make sure your arty stays safe, track an enemy on the top of the railroad crossing so your teammates can take him out before he comes over, ram and/or track the scout which is racing for your team´s arty, keep your teammate alive by shooting the tank about to kill him instead of another target, etc. etc. etc.

This is why I decided to include it, though at a relatively low percentage of the final score (around 10% in avergae as I said above). The actual calculation is 42 wnEff points for each percentage your winrate is above 48.5%.

You could download the xls, and just chance the wnEF formula to not add the WR cells, and presto! :D
Then you could compare those stats to whatever you wish. PR, wnEF+WR, efficiency, WR, etc.
You are most welcome to play around with the xls.

Edited by Praetor77, Nov 29 2012 - 01:46.


diego999 #19 Posted Nov 29 2012 - 02:09

    Major

  • Beta Testers
  • 27914 battles
  • 4,108
  • [ACA] ACA
  • Member since:
    11-22-2010

View PostPanzerXO, on Nov 29 2012 - 01:34, said:

Have you given any thought to the platooning problem?

I do like a calculator / predictor that more closely correlates to win rates - but have a concern that anything that too closely ties itself to win rate as shown by top clan members at the upper end may not reflect an individual's actual performance; but rather his win rate as a party of good players forming a team within a team.

My concern is that corrections you make using top tier Clan aligned players will skew the results for players who do not platoon.

Some of the data gathered for the calculations belong to several players who like to play mostly alone. I'm one of those.

Garbad #20 Posted Nov 29 2012 - 02:09

    Major

  • Beta Testers
  • 29232 battles
  • 14,206
  • [-G-] -G-
  • Member since:
    10-02-2010

View PostPraetor77, on Nov 29 2012 - 01:42, said:

Nevertheless, there is a mechanism in place to "correct" for excessive platoon and TC play, and although crude, it works. Winrate is capped at 60% which is a personal estimate of the maximum you could reach playing solo (well, not you or me, maybe Garbad or Carbonward :D).
I would agree with this, but given I have soloed my KV-5 to 65%/68% depending on how you count it, the cap is clearly above 60%. Perhaps 65% would be appropriate.

It also still "feels" a little off. Perhaps a tad more on KPG and exp/game somehow.





Also tagged with statistics, stats, noobmeter, performance, efficiency, metric, compare, quality, rating, Garbad

16 user(s) are reading this topic

0 members, 16 guests, 0 anonymous users