3rd Party Hero Statistical Testing -- UPDATED 25 November 2023 -- You can find me at GtG Forums, BGG, and Discord under the yakguardian/Yak Guardian Screenname.
Deck Tracker -- SotM Stuff
Back when I first started building custom Sentinels decks (Yak Guardian, Young Justice, Matriarch Redeemed) for the then-planned-to-be-much-larger Crossover Crisis set, I took the first working deck (Young Justice) to my gaming group where my friend Scott was convinced that the deck was overpowered. Tired of arguing with him about it, I started working out a means of statistical testing of Hero decks. Given all the variables in Sentinels, I decided the first step was to control as many of those variables as I could. I also needed a scenario that was relatively fast, generally winnable , and not too swingy. Ultimately, I chose Ra, KNYFE, and the Hero being tested, versus Baron Blade at the Mobile Defense Platform. The MDP was selected because it mostly stays out of your way. As I was using Ra and KNYFE in the scenario, that gave me 24 (pre-OblivAeon) Heroes to test to for a baseline. I dug into my old Statistics textbooks and concluded that with that population size, I needed at least 5 data points for each to have a fair distribution. Thus I went with 5 wins for the stats (except for Win Percentage).
In addition to Win Percentage, the stats I decided to collect were Survivability, Per Round Damage Output, Support, and Destruction. In general,
Survivability is calculated as the percentage of their maximum HP that the Hero has left at the end of the game.
When I was building and testing my Young Justice deck, my friend Scott felt that it was overpowered. I, and others, did not feel so, but I respected Scott's opinion enough to undertake a massive statistical analysis of the Heroes. Using the 26 (pre-OblivAeon) Heroes as a baseline, I have now tested not only Young Justice but also over 100 additional custom (Licensed, my own, and other Third-Party) Hero decks. By and large, I have found nearly all of them to be balanced. (Some Cauldron and OblivAeon Heroes have skewed higher, and I suspect the Earth Prime Heroes will as well, but the analysis is still ongoing.)
When OblivAeon was in playtesting, many felt that the Heroes were skewing tougher than the Multiverse era Heroes. I was among them. As a result, I have treated them as a seperate group and left my stats based on the 26 original Heroes.
My testing methodology is: Ra, K.N.Y.F.E., and the Hero being tested, versus Baron Blade, at the Mobile Defense Platform. I run the scenario until I have achieved five wins. Except for Win Percentage, the stats are based solely on the victorious games. I include stats for Survivability (percentage of original HP left), Per Round Damage Output (abbreviated PRDO; not damage dealt but total damage put out into the universe, including hero deck triggred self-damage, damage boosts, overkill damage, but excluding damage reductions), and number of Suport Cards (cards that boost other Heroes: allowing Plays, Powers, Draw, HP regains, Hero damage boosts, Non-Hero damage reduction, etc.). I also record number of cards that allow Ongoing Destruction, but mostly as a curiosity as a Hero without it in this scenario leaves the only hope against Baron Blade's Ongoings as KNYFE's 2 cards.
Having the baseline values (Survivability 32% Standard Deviation 9.04%, PRDO 4.22 SD 2.0535, Support 6 SD 5.547), I have used it to playtest OblivAeon and Earth Prime Heroes, and custom Heroes (both my own and other people's). It is not, by any means, a perfect tool, but I feel it is a useful tool, especially during development.
The summary of the stats to date can be found here.
| Expansion Name | Win Rate | Survivability | PRDO | Support | NOTES |
|---|---|---|---|---|---|
| Multiverse Era Heroes [BASELNE] | 86% | 32% | 4.22 | 6 | 26 Heroes (2 are part of the Testing Scenario.) |
| OblivAeon Era Heroes | 94% (+0.57 SD) | 40% (+0.88 SD) | 5.76 (+0.75 SD) | 8.18 (+0.39 SD) | 11 Heroes |
| Cauldron Heroes | 90% (+0.29 SD) | 35% (+0.33 SD) | 3.38 (-0.41 SD) | 10.50 (+0.81) | 22 Heroes |
| Earth Prime Heroes | 2 of 12 Heroes complete | ||||
| Crisisverse Heroes | 7 tested so far (final Hero count TBD) | ||||
| Riftverse Heroes | 6 tested so far (final Hero count TBD) | ||||
| Freedom Force Heroes | 92% (+0.43 SD) | 36% (+0.44 SD) | 3.64 (-0.28 SD) | 7.38 (+0.25 SD) | 8 Heroes |
NOTE: Cut-off for inclusion in this table is 8 Heroes from a single source. (I'm willing to accept (civil) arguements for why I should lower that threshold.)
Web page Copyright 2023 Michael J. Ahlers.
Sentinels of the Multiverse is the property of Greater Than Games. The Cauldron was created by Matthew Bishop. Earth Prime is the property of Green Ronin. All other items are the intellectual property of their respective creators.