Creating a KPI Guidebook: game overhaul
Situation: It is unclear how traditional slots players gravitate and resonate with hybrid-genre features over the crucial D16 core loop.
Do we overhaul from the ground-up with a new studio or optimize existing app?
Summary: Mixed-Methods. Diary Study. Hybrid-genre, 16 days of gameplay.
I led the design, recruitment, moderating, analysis, and reporting of the following:
-
KPI Development: Control Mastery, Efficacy, Enjoyment, etc.
-
Align behavioral analytics across difficulty curves and pinch-points.
-
6 diary survey entries + Qualitative sessions.
-
Created a guidebook for new 3rd party development agency to follow.
IMPACT
-
Set the standard for studio-centric custom attitudinal KPIs.
-
Highlighted revenue risks in downplaying genre-loop features.
-
Accelerated revenue recovery via increased D7 conversions.
-
Guided launch timing and hybrid-genre marketing plan.
-
Informed resource allocation of > $6,000,000.
TASK
Deep-dive research plan that developed a scorecard of crucial gameplay factors.
Qualitative playtest sentiment of archetype motivators converted into within-subjects, repeated measures design.
Meaningful Choice
Variety of options presented with palpable impacts?
-
Challenge growth; are challenge varieties scaled well?
-
Decision autonomy; do decisions feel ‘player controlled’?
-
Customization features & Town expansion; flexible enough?
Growth
Is there appropriate evolution of Gameplay & character development?
-
Narrative lead-In engaging & sustained?
-
Tow expansion growth; pacing & scope.
-
Are player capabilities & talents strengthened?
Efficacy
Does RTS achieve the fundamental engagement & purpose that the player values?
-
Progression & Self-Empowerment
-
Talent & Tactics growth
-
Connection & Achievement pathways provided?
Goal Interest
How do players goals & the game’s facilitation of them change over time?
-
Primary Goal benchmarks
-
Objectives presentation & facilitation
-
Competition against ‘House’ / Slot-Gold growth
Enjoyment
Does the game’s mechanisms engage, excite, and motivate elevated play?
-
Thrill & captivation ratings
-
Emotional gratification & Feature ranking
-
Retention & Recommendation likelihoods
ACTION
I lead study design, recruitment, analysis, and reporting.
Collaborated with Tooling for cross-game recruitment.
Studio GM, 2 Product leads, and 1 Game Designer were brought in to review metrics & study design.
-
Game’s guidance; Confusing or Clear & Effective
-
Effective tutorialization – learn & implement cycle provided?
-
Strategy & Tactics empowerment
Control Mastery
Mechanics and UI capabilities taught effectively to put player on a path to expertise?
SURVEY 1
SURVEY 3 (Repeated 4x)
SURVEY 2
SURVEY 4
GRANULAR RESULTS
This research enabled confidence in a new vision, overhaul of resources, and narrowing of marketing strategy.
Determined the game needed significant overhaul while establishing which gameplay loops mattered most.
Narrowed down extent of customization features while also highlighting where monetization was persuasive.
