Card Targeting in Clash Royale

Background

• data scientist in Clash Royale
• with Supercell for 4½ years, first in Boom and joined Royale before beta (2+ yrs)
• leading machine learning at Supercell

Company culture

• High quality games
  • Thinking big, with small teams
• Four global games
  • One game in beta
• CR dev team is less than 20 people
  • for game dev, this means killing many games before launch
  • for live ops, this means being smarter about scaling => today’s talk

Clash Royale

• real time PvP battling game
• card collection game
• free to play
• launched globally Jan 2016
• unique gameplay on mobile

Live ops in Clash Royale

• Rich and dynamic player experience
  • purpose of live ops: improve player experience
• Global player base
  • add variety: global player base, diversity of progression
• Everyday eventfulness
  • add eventfulness: new exciting things going on
• Content schedule ≠ game update schedule
• Live opsy since 2016 fall
• Examples:
  • different game modes available as special events
  • special offers and sales
  • offer to celebrate release of new cards
  • limited time boosters
  • sales to celebrate big events like release of touchdown mode

Goal: tailored live ops

• Towards personalized player experience
• Relevant content
• Relevant timing

…with a small team

• Automate day-to-day events

• Human focus on high profile events

• most live ops been global

  • limited by small team size, PvP fairness

• growing card collections ⇔ personalize card choices

  • personal playing style, hog player/2v2 player/skilled challenge player

• bread and butter liveops

• use machine learning to help

Interlude: machine learning?

• machine learning is an alternative way to create the “programs” i.e. input–output logic
• data is the elixir of machine learning
  • input–output mapping is hidden in data
• training: processing input–output data to find correlations
  • can take anything from minutes to weeks, depending on task and data
• trained model: imitate input–output relationships as seen in training data
• machine learning logic is called “model”, but it’s still a piece of software
• model is software that can be trained on data
  • data changes => retrain model
  • often an uninterpretable black box
• machine learning: software program that is learned from data, instead of writing manually
• two stages: training and application (inference)
  • training: offline
  • application: live

Clash Royale card targeting

• potential signals to consider
  • Card?
  • Pricing?
  • Stack size?
  • Near upgrade?
  • Card in deck?
  • Request from clan?

• card and deck choices vary naturally from player to player
• focus machine learning on picking a good card
  • manual design: stack size and pricing
  • output: card
  • input signals: use in deck, near upgrade, request

Problem statement

Provide dynamic content

• that players find interesting → purchase rate
• that reacts to changes in playing style → time-dependent signals
• without being spammy → filter weak offers
• without breaking the game economy → cooldowns

Solution sketch

1. Create offer inventory
   • graphics assets
   • price points, stack sizes
2. Get training data
   • data for machine learning
3. Train binary classifier
   • train model
4. Go live

Problem:

• Input data ✔
  • signals: card collection, decks, card requests, … => available
• Output data ✘
  • labels: is this card relevant for that player ⇔ would they like to purchase this

Clash of Clans offer targeting

• Potential signals to consider
  • Pack?
  • Troops used in war?
  • Troops used to attack?
  • Troop upgrades?
  • Pricing?
  • Resource amount?
  • Building upgrades?

• similar approach as in CR: use machine learning to pick a good pack
• manual design: resource contents and pricing
• output: pack
• input signals: troop usage, troop and building upgrades, TH level

Targeting model

• task: would player want to buy giant as special offer?
• 21 binary outputs: binary classifier (logistic regression)
• collection: card levels, amount missing from next level
• battle decks: cards used vs. not used; last 1, 7, 30 days
• card requests: requested vs. not requestsd, last 1, 7, 30 days
• account: XP and arena levels

• Clash: 1500 input signals, 7 output classes
• 100 hidden nodes
• one hidden layer
• utilize combinations of signals: XP level 8 AND requesting card BUT not playing
• signals are mapped to targeting using ML model
• program logic doesn’t have to be written by hand
  • tradeoffs: how important is card collection vs. battle deck

Technology stack

• data collection: need live offers
• model training: manual training and validation
• rarely retrained, e.g. when adding new cards, or if data distribution changes
• model application: daily batch job, re-run targeting based on newest input
• application: monitor performance

Real world details

• Highly imbalanced data
• Interactions between events and offers
• Which metric to optimize?
• Testing and validation
• Staged model development
• Reproducible models and targetings

Players are getting value in Clash Royale

• baseline is flat
• we’ve seen that ML model improvement is sustainable,
• lift w.r.t. normal chest shop
• novelty value
• accuracy coverage tradeoff
• significant lift in business metrics

Players are getting value in Clash of Clans

• statistically significant lift between rule-based and ML
• practically not good enough, given additional complexity of ML vs. rule-based
• lesson learned: ML is not a silver bullet
• task is quite narrow, only choosing among 7 packs

Learnings

• Pay attention to player experience
• Machine learning can be a useful alternative to manual heuristics
• Simple models like logistic regression provide a lot of value
• Align the machine learning loss function with players’ interests
• Fast iteration is very important
  • fast iteration but not by all means
• need a good model already at launch
• ML is a tool that can replace heuristics
• make sure cost vs. benefit is there
• ML core model is 10% of the work, 90% is integration
• go live faster by bootstrapping with available data