diff --git a/docs/technical/kitscore.html b/docs/technical/kitscore.html new file mode 100644 index 000000000..cec896d00 --- /dev/null +++ b/docs/technical/kitscore.html @@ -0,0 +1,3177 @@ + + + + + KitScore + + + + + + + + + + + + + +
+

KitScore

+

Memo

+

A performance metric that makes sense. This metric can be applied to any game, virtual or real. Specific pieces of information relating directly to World of Tanks Blitz will be provided within parentheses and with a "WoTB:" prefix.

+

Consumption of Observations

+

KitScore is driven solely by the behavior of other players and finding meaning of a new observation within a pool of condensed data based on previously observed data. Generally, we will be provided with a number of statistics (the xix_i of an observation) per player (WoTB: xfragsx_\text{frags}, xdamage_dealtx_\text{damage\_dealt}, xspotsx_\text{spots}, etc.) including a target statistic (the xjx_j of an observation) that all other statistics try to maximize (WoTB: "winrate" which we will defined later as xjx_j).

+

If the statistics are cumulative, they will have to be normalized (WoTB: statistics fetched from Wargaming are indeed cumulative even for a single player; xfrags=ΣxfragsΣxbattlesx_\text{frags}=\frac{\Sigma x_\text{frags}}{\Sigma x_\text{battles}}, xspots=ΣxspotsΣxbattlesx_\text{spots}=\frac{\Sigma x_\text{spots}}{\Sigma x_\text{battles}}, xj=ΣxwinsΣxbattlesx_j=\frac{\Sigma x_\text{wins}}{\Sigma x_\text{battles}}, etc.). Each player will have such average statistics, giving us many points of data to work with. If the statistics are not cumulative, then you will be dealing with multiple observations per player which will only help improve accuracy.

+

It’s Impossible to "Find What Makes the Metric Tick"

+

WN8, a widely adopted metric in the World of Tanks and World of Tanks Blitz communities is plagued with problems. One of them being that people often "what makes the metric tick." In other words, people easily exploit the rudimentary nature of WN8 to score undeservedly high.

+

In WN8, the gains of some rewarding categories overshadow the losses of others. KitScore makes this nearly impossible as it punishes a player for scoring well solely high in a certain category. You will never exploit KitScore. Period.

+

Generating Reasonable Values

+

TODO: there's like no runaway cases

+

Symbol Interpretation

+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
SymbolInterpretation
AhumanA_\text{human}The human readable version of the raw score AA.
AAThe raw KitScore.
CCAn arbitrary coefficient.
IIThe number of statistics including the target statistic. This means that there are I−1I-1 entries for xix_i and 11 entry for yy for each observation.
iiThe statistic index of where there are II entries.
jjA discriminator for the target statistic. There may never be more than 11 entries for jj.
mmThe degree of the polynomial regression.
nnThe number of observations.
aaAn "atomic" score which is a function of the $i$th statistic.
wwThe weight of the $i$th atomic statistic.
+

Computing the Score

+

The final metric AhumanA_\text{human} is an arbitrarily scaled value of the raw score AA where CC is any coefficient of your liking. BlitzKit uses C=103C=10^3.

+

Ahuman=CAA_\text{human}=CA +

+

The final metric AA is the weighted average of individual scores aa which are functions of the $i$th statistic.

+

A=∑wiai∑wiA=\frac{\sum w_{i}a_{i}}{\sum w_{i}} +

+

The "atomic" score aa is the difference between the predicted target statistic Pi↔j(xi)P_{i \leftrightarrow j}(x_i) acquired using a polynomial regression of degree mm and the actual newly observed xjx_j.

+

ai=xj−Pi↔j(xi)a_i=x_j-P_{i \leftrightarrow j}(x_i) +

+

The polynomial regression of $m$th degree can be written as follows:

+

Pi(x)=β0x0+β1x1+β2x2+⋯+βmxmP_i(x)=\beta_0x^0+\beta_1x^1+\beta_2x^2+\dots+\beta_mx^m +

+

Polynomial regression coefficients β\beta can be acquired through basic linear algebra where an m×mm \times m of constants is multiplied with m×1m \times 1 of unknowns which equals another m×1m \times 1 of constants.

+

[∑k=1nxi0∑k=1nxi1∑k=1nxi2…∑k=1nxim∑k=1nxi1∑k=1nxi2∑k=1nxi3…∑k=1nxim+1∑k=1nxi2∑k=1nxi3∑k=1nxi4…∑k=1nxim+2⋮⋮⋮⋱⋮∑k=1nxim∑k=1nxim+1∑k=1nxim+1…∑k=1nxi2m][β0β1β2⋮βm]=[∑k=1nxjxi0∑k=1nxjxi1∑k=1nxjxi2⋮∑k=1nxjxim]\begin{bmatrix} +\sum_{k=1}^n x_i^0 & \sum_{k=1}^n x_i^1 & \sum_{k=1}^n x_i^2 & \dots & \sum_{k=1}^n x_i^m \\ +\sum_{k=1}^n x_i^1 & \sum_{k=1}^n x_i^2 & \sum_{k=1}^n x_i^3 & \dots & \sum_{k=1}^n x_i^{m+1} \\ +\sum_{k=1}^n x_i^2 & \sum_{k=1}^n x_i^3 & \sum_{k=1}^n x_i^4 & \dots & \sum_{k=1}^n x_i^{m+2} \\ +\vdots & \vdots & \vdots & \ddots & \vdots \\ +\sum_{k=1}^n x_i^m & \sum_{k=1}^n x_i^{m+1} & \sum_{k=1}^n x_i^{m+1} & \dots & \sum_{k=1}^n x_i^{2m} \\ +\end{bmatrix} + +\begin{bmatrix} +\beta_0 \\ +\beta_1 \\ +\beta_2 \\ +\vdots \\ +\beta_m +\end{bmatrix} + += + +\begin{bmatrix} +\sum_{k=1}^n x_jx_i^0 \\ +\sum_{k=1}^n x_jx_i^1 \\ +\sum_{k=1}^n x_jx_i^2 \\ +\vdots \\ +\sum_{k=1}^n x_jx_i^m +\end{bmatrix} +

+

The weight ww is a slice of the Pearson corelation matrix. The numerator is the corelation between the $i$th statistic and the target statistic jj while the denominator the sum of the corelations between the $i$th statistic and all statistics but the target statistic jj (including itself which will always result in r=1r=1).

+

wi=ri↔j2∑ik≠jri↔ik2w_{i}=\frac{r_{i\leftrightarrow j}^2}{\sum_{i_{k}\neq j}r_{i\leftrightarrow i_{k}}^2} +

+

Understanding the Weights

+

Some may find the weight equation arbitrary; it evidently isn't. The goal of ww is to eliminate over-rewarding the same behavior represented in multiple statistics. It makes all atomic statistics concurrent by leveraging the Pearson corelation matrix.

+

A simplified example will clarify the idea. First, we compute a Pearson corelation matrix for I=3I=3 statistics. We have x1x_1 and x2x_2, the statistics that are trying to maximize xjx_j. Reminder: a Pearson corelation matrix finds the rr value not only between xix_i and xjx_j but also between all combinations of xix_i and themselves. Here, r2r^2 is calculated using polynomial regression corelation analysis to the $m$th degree.

+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
x1x_1x2x_2x3x_3xjx_j
x1x_1110\boxed{0}0011
x2x_2110011
x3x_31111
xjx_j11
+

In the case above, r1↔22=0r^2_{1 \leftrightarrow 2}=0 meaning that both statistics have absolutely nothing to do with each other. Hence, the player will be awarded for optimizing both individually. And since r1↔j2=r2↔j2r^2_{1 \leftrightarrow j}=r^2_{2 \leftrightarrow j}, they should both be rewarded equally; in other words, they should have equal weights.

+

w1=11+0+0=1w_{1}=\frac{1}{1 + 0 + 0}=1 +

+

w2=10+1+0=1w_{2}=\frac{1}{0 + 1 + 0}=1 +

+

w3=10+0+1=1w_{3}=\frac{1}{0 + 0 + 1}=1 +

+

Now consider a case where x1x_1 and x2x_2 are fully correlated. It would be a bad idea to reward them equally; hence, their weights suffer.

+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
x1x_1x2x_2x3x_3xjx_j
x1x_1111\boxed{1}0011
x2x_2110011
x3x_31111
xjx_j11
+

w1=11+1+0=0.5w_{1}=\frac{1}{1 + 1 + 0}=0.5 +

+

w2=11+1+0=0.5w_{2}=\frac{1}{1 + 1 + 0}=0.5 +

+

w3=10+0+1=1w_{3}=\frac{1}{0 + 0 + 1}=1 +

+
+
+ + + + \ No newline at end of file