mirror of
https://github.com/blitzkit/blitzkit.git
synced 2026-10-21 10:47:04 +02:00
123 lines
8.0 KiB
Markdown
123 lines
8.0 KiB
Markdown
# KitScore
|
|
|
|
## Memo
|
|
|
|
A performance metric that makes sense. This metric can be applied to any game, virtual or real. Specific pieces of information relating directly to World of Tanks Blitz will be provided within parentheses and with a "WoTB:" prefix.
|
|
|
|
## Consumption of Observations
|
|
|
|
KitScore is driven solely by the behavior of other players and finding meaning of a new observation within a pool of condensed data based on previously observed data. Generally, we will be provided with a number of statistics (the $x_i$ of an observation) per player (WoTB: $x_\text{frags}$, $x_\text{xp}$, $x_\text{spots}$, etc.) including a target statistic (the $x_j$ of an observation) that all other statistics try to maximize (WoTB: "winrate" which we will defined later as $x_j$).
|
|
|
|
If the statistics are cumulative, they will have to be normalized (WoTB: statistics fetched from Wargaming are indeed cumulative even for a single player; $x_\text{frags}=\frac{\Sigma x_\text{frags}}{\Sigma x_\text{battles}}$, $x_\text{spots}=\frac{\Sigma x_\text{spots}}{\Sigma x_\text{battles}}$, $x_j=\frac{\Sigma x_\text{wins}}{\Sigma x_\text{battles}}$, etc.). Each player will have such average statistics, giving us many points of data to work with. If the statistics are not cumulative, then you will be dealing with multiple observations per player which will only help improve accuracy.
|
|
|
|
## It's Impossible to "Find What Makes the Metric Tick"
|
|
|
|
WN8, a widely adopted metric in the World of Tanks and World of Tanks Blitz communities is plagued with problems. One of them being that people often "what makes the metric tick." In other words, people easily exploit the rudimentary nature of WN8 to score undeservedly high.
|
|
|
|
In WN8, the gains of some rewarding categories overshadow the losses of others. KitScore makes this nearly impossible as it punishes a player for scoring well solely high in a certain category. You will never exploit KitScore. Period.
|
|
|
|
## Generating Reasonable Values
|
|
|
|
TODO: there's like no runaway cases
|
|
|
|
## Symbol Interpretation
|
|
|
|
| Symbol | Interpretation |
|
|
| ---------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------ |
|
|
| $A_\text{human}$ | The human readable version of the raw score $A$. |
|
|
| $A$ | The raw KitScore. |
|
|
| $C$ | An arbitrary coefficient. |
|
|
| $I$ | The number of statistics including the target statistic. This means that there are $I-1$ entries for $x_i$ and $1$ entry for $y$ for each observation. |
|
|
| $i$ | The statistic index of where there are $I$ entries. |
|
|
| $j$ | A discriminator for the target statistic. There may never be more than $1$ entries for $j$. |
|
|
| $m$ | The degree of the polynomial regression. |
|
|
| $n$ | The number of observations. |
|
|
| $a$ | An "atomic" score which is a function of the $i$th statistic. |
|
|
| $w$ | The weight of the $i$th atomic statistic. |
|
|
|
|
## Computing the Score
|
|
|
|
The final metric $A_\text{human}$ is an arbitrarily scaled value of the raw score $A$ where $C$ is any coefficient of your liking. BlitzKit uses $C=10^3$.
|
|
|
|
$$
|
|
A_\text{human}=CA
|
|
$$
|
|
|
|
The final metric $A$ is the weighted average of individual scores $a$ which are functions of the $i$th statistic.
|
|
|
|
$$
|
|
A=\frac{\sum w_{i}a_{i}}{\sum w_{i}}
|
|
$$
|
|
|
|
The "atomic" score $a$ is the difference between the predicted target statistic $P_{i \leftrightarrow j}(x_i)$ acquired using a polynomial regression of degree $m$ and the actual newly observed $x_j$.
|
|
|
|
$$
|
|
a_i=x_j-P_{i \leftrightarrow j}(x_i)
|
|
$$
|
|
|
|
The polynomial regression of $m$th degree can be written as follows:
|
|
|
|
$$
|
|
P_i(x)=\beta_0x^0+\beta_1x^1+\beta_2x^2+\dots+\beta_mx^m
|
|
$$
|
|
|
|
Polynomial regression coefficients $\beta$ can be acquired through basic linear algebra where an $m \times m$ of constants is multiplied with $m \times 1$ of unknowns which equals another $m \times 1$ of constants.
|
|
|
|
$$
|
|
\begin{bmatrix} \sum_{k=1}^n x_i^0 & \sum_{k=1}^n x_i^1 & \sum_{k=1}^n x_i^2 & \dots & \sum_{k=1}^n x_i^m \\ \sum_{k=1}^n x_i^1 & \sum_{k=1}^n x_i^2 & \sum_{k=1}^n x_i^3 & \dots & \sum_{k=1}^n x_i^{m+1} \\ \sum_{k=1}^n x_i^2 & \sum_{k=1}^n x_i^3 & \sum_{k=1}^n x_i^4 & \dots & \sum_{k=1}^n x_i^{m+2} \\ \vdots & \vdots & \vdots & \ddots & \vdots \\ \sum_{k=1}^n x_i^m & \sum_{k=1}^n x_i^{m+1} & \sum_{k=1}^n x_i^{m+1} & \dots & \sum_{k=1}^n x_i^{2m} \\ \end{bmatrix} \begin{bmatrix} \beta_0 \\ \beta_1 \\ \beta_2 \\ \vdots \\ \beta_m \end{bmatrix} = \begin{bmatrix} \sum_{k=1}^n x_jx_i^0 \\ \sum_{k=1}^n x_jx_i^1 \\ \sum_{k=1}^n x_jx_i^2 \\ \vdots \\ \sum_{k=1}^n x_jx_i^m \end{bmatrix}
|
|
$$
|
|
|
|
The weight $w$ is a slice of the Pearson corelation matrix. The numerator is the corelation between the $i$th statistic and the target statistic $j$ while the denominator the sum of the corelations between the $i$th statistic and all statistics but the target statistic $j$ (including itself which will always result in $r=1$).
|
|
|
|
$$
|
|
w_{i}=\frac{r_{i\leftrightarrow j}^2}{\sum_{i_{k}\neq j}r_{i\leftrightarrow i_{k}}^2}
|
|
$$
|
|
|
|
## Understanding the Weights
|
|
|
|
Some may find the weight equation arbitrary; it evidently isn't. The goal of $w$ is to eliminate over-rewarding the same behavior represented in multiple statistics. It makes all atomic statistics concurrent by leveraging the Pearson corelation matrix.
|
|
|
|
A simplified example will clarify the idea. First, we compute a Pearson corelation matrix for $I=3$ statistics. We have $x_1$ and $x_2$, the statistics that are trying to maximize $x_j$. Reminder: a Pearson corelation matrix finds the $r$ value not only between $x_i$ and $x_j$ but also between all combinations of $x_i$ and themselves. Here, $r^2$ is calculated using polynomial regression corelation analysis to the $m$th degree.
|
|
|
|
| | $x_1$ | $x_2$ | $x_3$ | $x_j$ |
|
|
| ----- | ----- | ----------- | ----- | ----- |
|
|
| $x_1$ | $1$ | $\boxed{0}$ | $0$ | $1$ |
|
|
| $x_2$ | | $1$ | $0$ | $1$ |
|
|
| $x_3$ | | | $1$ | $1$ |
|
|
| $x_j$ | | | | $1$ |
|
|
|
|
In the case above, $r^2_{1 \leftrightarrow 2}=0$ meaning that both statistics have absolutely nothing to do with each other. Hence, the player will be awarded for optimizing both individually. And since $r^2_{1 \leftrightarrow j}=r^2_{2 \leftrightarrow j}$, they should both be rewarded equally; in other words, they should have equal weights.
|
|
|
|
$$
|
|
w_{1}=\frac{1}{1 + 0 + 0}=1
|
|
$$
|
|
|
|
$$
|
|
w_{2}=\frac{1}{0 + 1 + 0}=1
|
|
$$
|
|
|
|
$$
|
|
w_{3}=\frac{1}{0 + 0 + 1}=1
|
|
$$
|
|
|
|
Now consider a case where $x_1$ and $x_2$ are fully correlated. It would be a bad idea to reward them equally; hence, their weights suffer.
|
|
|
|
| | $x_1$ | $x_2$ | $x_3$ | $x_j$ |
|
|
| ----- | ----- | ----------- | ----- | ----- |
|
|
| $x_1$ | $1$ | $\boxed{1}$ | $0$ | $1$ |
|
|
| $x_2$ | | $1$ | $0$ | $1$ |
|
|
| $x_3$ | | | $1$ | $1$ |
|
|
| $x_j$ | | | | $1$ |
|
|
|
|
$$
|
|
w_{1}=\frac{1}{1 + 1 + 0}=0.5
|
|
$$
|
|
|
|
$$
|
|
w_{2}=\frac{1}{1 + 1 + 0}=0.5
|
|
$$
|
|
|
|
$$
|
|
w_{3}=\frac{1}{0 + 0 + 1}=1
|
|
$$
|