Skip to content

A probabilistic player rating system for Counter Strike: Global Offensive, powered by machine learning

License

Notifications You must be signed in to change notification settings

phil-holland/csgo-impact-rating

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation


A probabilistic player rating system for Counter Strike: Global Offensive, powered by machine learning


Latest release Build Status Go Report Card License: MIT


How it WorksPrediction ModelDownloadUsageBuilt With

How it Works

Impact Rating uses a machine learning model trained on a large amount of historical data to predict the probable winner of a given CS:GO round, based on the current state of the round. A player's Impact Rating is then calculated as the amount by which their actions shift the predicted likelihood of their team winning the round. Therefore, players are rewarded purely for making plays that improve their team's chance of winning the current round. Conversely, negative Impact Rating is given when a player's actions reduce their team's chance of winning the round.

A player's performance over the course of a complete game can be summarised with their Average Impact Rating (AIR) - the mean of all single-round ratings.

Example

Two simplified examples of in-game scenarios are shown in the diagram below. Both describe a single CT player getting a triple kill, but they only receive impact rating in one scenario. In the first, the CTs are left in a 5v2 situation in which they are highly favoured to subsequently win the round. In the second, the remaining CTs are forfeiting the round, not attempting a late retake. A triple kill at this point does not alter the CT team's chance of winning the round.

Case Study

An in-depth case study analysis of an individual match between Mousesports and Faze can be found here: case study. This analysis breaks down the events of a round, showing how model prediction values change accordingly, and the resulting effect on player Impact Ratings.

Calculating Impact Rating

Internally, the state of a round at any given time is captured by the following features:

  • The number of CT players alive
  • The number of T players alive
  • The mean health of CT players
  • The mean health of T players
  • The mean value of CT equipment
  • The mean value of T equipment
  • The round time elapsed (in seconds)
  • The bomb time elapsed (in seconds) - this is zero before the bomb is planted
  • Whether the bomb is currently being defused
  • Whether the bomb has been defused - this is required to reward players for winning a round by defusing

Inputs for each of these features are passed to the machine learning model, which returns a single non-integer value between 0.0 and 1.0 as the round winner prediction. This value can be directly interpreted as the probability that the round will be won by the T-side.

For example, a returned value of 0.34 represents a predicted 34% chance of a T-side round win, and a 66% chance of a CT-side round win.

This concept is applied to every in-game event that causes a round's state to change from the end of freezetime to the moment the round is won. Players that contributed to changing the round outcome prediction are rewarded with a appropriate share of the percentage change in their team's favour - the sum of these values over a particular round is their Impact Rating for that round.

The following action categories are considered when rewarding players with their share:

  • Dealing damage
    • This rewards players who damage an opponent's health
    • This also punishes players for taking damage themselves, and for any team-damage
  • Trade damage (if an opponent takes damage very soon after they themselves have damaged the player in question)
    • This rewards players for taking damage so that a teammate can damage their attacker (a.k.a "baiting" for someone else)
  • Flash assist damage (if someone takes damage whilst blinded by a flashbang thrown by the player in question)
    • This rewards players for flashing an enemy who then sustains damage
    • This also punishes players for team-flashing their teammate into taking damage
  • Successfully retaking
    • This rewards players who win rounds by retaking and defusing the bomb - all living CTs are rewarded when the bomb is defused
    • This also punishes T-side players who cannot prevent a defuse whilst alive

Prediction Model

Whilst the machine learning aspect of Impact Rating can in theory be implemented using any binary classification model, the code here has been written to target the LightGBM framework. This is a framework used for gradient boosting decision trees (GBDT), and has been shown to perform very well in binary classification problems. It has also been chosen for its lightweight nature, and ease of installation.

Model analysis and instructions for how to train a new model can be found here: model analysis.

Download

The latest Impact Rating distribution for your system can be downloaded from this Github project's release page (for 99% of Windows users, this means downloading the csgo-impact-rating_win64.zip file).

Download Latest Version

Extract the executable and the LightGBM_model.txt file to a directory of your choosing - add this directory to the system path to access the executable from any location.

Usage

CS:GO Impact Rating is distributed as a command line tool - it can be invoked only through a command line such as the Windows command prompt (cmd.exe) or a Linux shell.

Usage: csgo-impact-rating [OPTION]... [DEMO_FILE (.dem)]

Tags DEMO_FILE, creating a '.tagged.json' file in the same directory, which is
subsequently evaluated, producing an Impact Rating report which is written to
the console and a '.rating.json' file.

  -f, --force                Force the input demo file to be tagged, even if a
                             .tagged.json file already exists.
  -p, --pretty               Pretty-print the output .tagged.json file.
  -s, --eval-skip            Skip the evaluation process, only tag the input
                             demo file.
  -m, --eval-model string    The path to the LightGBM_model.txt file to use for
                             evaluation. If omitted, the application looks for
                             a file named "LightGBM_model.txt" in the same
                             directory as the executable.
  -v, --eval-verbosity int   Evaluation console verbosity level:
                              0 = do not print a report
                              1 = print only overall rating
                              2 = print overall & per-round ratings (default 2)

For general usage, the above command line flags can be ignored. For example, the following command will process and produce player ratings for a demo file named example.dem in the working directory:

csgo-impact-rating example.dem

A full per-player Impact Rating report will be shown in the console output.

Processing Details

Processing consists of two internal stages: tagging and evaluation.

1. Tagging:

First, the raw demo file is parsed from start to finish, creating a "tagged file" in the same directory as the input demo with the extension .tagged.json. This is a JSON file that describes the key events of the demo, each "tagged" with any players who have contributed to that event. A complete example of a single "tagged" event is shown below:

{
  "tick": 251249,
  "type": "playerDamage",
  "scoreCT": 6,
  "scoreT": 7,
  "teamCT": {
    "id": 2,
    "name": "mousesports"
  },
  "teamT": {
    "id": 3,
    "name": "FaZe Clan"
  },
  "players": [
    { "steamID": 76561198083485506, "name": "woxic", "teamID": 2 },
    { "steamID": 76561198068422762, "name": "frozen","teamID": 2 },
    { "steamID": 76561197997351207, "name": "rain", "teamID": 3 },
    { "steamID": 76561198201620490, "name": "broky", "teamID": 3 },
    { "steamID": 76561197988627193, "name": "olofmeister", "teamID": 3 },
    { "steamID": 76561197991272318, "name": "ropz", "teamID": 2 },
    { "steamID": 76561197989430253, "name": "karrigan", "teamID": 2 },
    { "steamID": 76561197988539104, "name": "chrisJ", "teamID": 2 },
    { "steamID": 76561198039986599, "name": "coldzera", "teamID": 3 },
    { "steamID": 76561198041683378, "name": "NiKo", "teamID": 3 }
  ],
  "gameState": {
    "aliveCT": 3,
    "aliveT": 2,
    "meanHealthCT": 80.33333333333333,
    "meanHealthT": 100,
    "meanValueCT": 1233.3333333333333,
    "meanValueT": 6325,
    "roundTime": 58.8359375,
    "bombTime": 0,
    "bombDefusing": false,
    "bombDefused": false
  },
  "tags": [
    {
      "action": "damage",
      "player": 76561198201620490
    },
    {
      "action": "hurt",
      "player": 76561198068422762
    }
  ],
  "roundWinner": 1
}

Note: if an tagged file already exists for the input demo, this stage is skipped by default.

2. Evaluating:

Secondly, each event saved in the tagged file is evaluated with the machine learning model, producing a predicted round outcome probability. These probabilities are then used to calculate player ratings for each round, and their overall average over all rounds. This is printed to the console window - an Average Impact Rating table for an example demo is shown below:

> Overall:

Team           Player         Average Impact (%)   |   Damage (%)    Flash Assists (%)    Trade Damage (%)    Retakes (%)    Damage Recv. (%)    Alive (%)
----           ------         ------------------   |   ----------    -----------------    ----------------    -----------    ----------------    ---------
FaZe Clan      olofmeister    -4.605               |   7.132         0.319                1.259               0.000          -13.293             -0.022
FaZe Clan      rain           -1.889               |   10.494        0.057                0.183               0.000          -12.608             -0.014
FaZe Clan      coldzera       4.640                |   19.461        -0.797               0.340               0.005          -14.307             -0.063
FaZe Clan      NiKo           1.437                |   8.293         0.081                2.151               0.005          -9.046              -0.047
FaZe Clan      broky          0.428                |   9.891         1.187                1.912               0.005          -12.569             0.001
mousesports    woxic          0.054                |   10.532        0.626                1.482               0.076          -12.715             0.053
mousesports    frozen         14.941               |   19.581        1.006                1.807               2.466          -9.887              -0.033
mousesports    ropz           -6.264               |   5.485         0.030                0.856               0.069          -12.761             0.057
mousesports    karrigan       -2.735               |   8.918         0.962                0.655               0.076          -13.370             0.024
mousesports    chrisJ         -3.252               |   9.492         0.000                1.592               0.145          -14.526             0.045

All calculated statistics are saved to a "rating file" with the extension .rating.json in the same directory as the input demo. Along with player rating summaries, this file contains the inferred probabilities at each event, and the changes in player ratings through each round.

Built With

  • demoinfocs-golang - used to parse CS:GO demo files
  • leaves - used to process LightGBM models internally
  • pflag - used to build the command line interface
  • pb (v3) - used for progress visualisation
  • LightGBM - used for model training/round outcome prediction
  • Optuna - used for hyperparameter optimisation during training