Talk |

Building a Game AI for Clash Royale

How and why to teach a ResNet to play mobile games – talk given at Helsinki Data Science meetup
YouTube link
Data in Gaming – Helsinki Data Science meetup (17:26)

Clash Royale

  • Battling and card collection game
  • Player vs. player
  • Real time battling
  • Three minute battles
  • Mobile & free to play
  • Launched 2016
Clash Royale battle screenshot
Two bots enter, one bot leaves © Supercell

Game AI

  • What? Computer controlled characters
    • Some game AI in all games
  • Why? Game content, player tutoring, game testing, …
    • Bosses, practise opponents, tutors, assistants,
    • Our focus: bots
  • How? Rule-based systems Imitation learning from game replays
    • Traditionally rule-based: designed, not learned

Clash Royale: casual PvP?

Casual PvP?
Red king or blue king? © Supercell
  • Game AI serves game design
    • Our game design purpose?
  • Losing 50% of the time feels brutal
    • There is always a loser
  • What if everyone had 70% win rate?
    • 70% not scientific

Rich observation space

Observation space
Observations are gameplay frames © Supercell
  • Two players: blue, red
  • 32 x 18 tiles
  • Cannot deploy everywhere
  • Waiting is always possible
  • 100+ types of units
  • 0–100 units on board
  • 1–14 power levels
  • Health 0–4000
  • Context: deck, time, …

Dynamic action space

Action space
Actions are card deploys © Supercell
  • Discrete actions
  • Theoretical action space >50k
  • In practise 1–2000 actions
  • Wait is always possible

Network architecture

Network architecture
It's like a ResNet but for mobile games © Supercell
  • What’s the simplest thing that can work
    • image/digit classification!
    • This might work!
  • Supervised learning
  • Memoryless
  • Use the real game: full board, all cards, exact mechanics
  • Residual net playing Clash Royale
    • 21 conv. layers
  • Input is in “pre-digested” symbolic form, no edge detection
  • Small input resolution
    • => don’t need 150 layers
  • Structured data like deck, elixir, time
  • Mix context in with spatial data
  • Context is broadcast to all spatial locations

Results

  • Quantitative evaluation ¯\_(ツ)_/¯
    • Proxy metric: “correct” action prediction accuracy (4.4%)
    • Frame-by-frame imitation performance does not guarantee anything about actual gameplay skills
    • Prediction errors accumulate over time, as network is run recursively
  • Empirical playtesting 👍
  • A/B testing 🎉
  • Net win rate: 72%

Live system

  • Trained on 70M frames of game replays
  • In production since 2018
  • Live for low/mid trophies
    • Extension of tutorial, to relax skill requirements
    • => not deployed for endgame players
  • Memoryless = stateless ❤️
  • Served more than 1012 frames
    • GPU serving, C++, LibTorch

Slides

Title slide