Mobile Games A/B Testing Analysis
The objective of this study is to analyze the impact of gate placement on player retention in Cookie Cats, a mobile puzzle game developed by Tactile Entertainment. The analysis focuses on an A/B test conducted to evaluate whether moving the gate from level 30 to level 40 affects player engagement and retention.
Data was collected from 90,189 players who installed Cookie Cats during the A/B test period. The dataset includes player information such as user ID, gate version (control or test), number of game rounds played, and retention rates at 1-day and 7-day intervals.
Exploratory data analysis was conducted to understand player engagement levels, distribution of game rounds played, and overall retention rates. Visualization techniques were employed to gain insights into player behavior and identify patterns in the data.
A hypothesis was formulated to test whether there is a significant difference in player retention between the control group (gate at level 30) and the test group (gate at level 40). Statistical methods, including bootstrapping, were used to assess the certainty of observed retention differences.
The analysis revealed insights into 1-day and 7-day retention rates for both AB-groups. Findings were interpreted to determine the impact of gate placement on player engagement and long-term retention.
Based on the results, it was concluded that retaining the gate at level 30 is associated with higher player retention compared to moving it to level 40. The study provides actionable insights for game developers and stakeholders to optimize gate placement strategies and enhance player engagement in mobile games.