{"id":17585,"date":"2012-10-24T10:16:49","date_gmt":"2012-10-24T15:16:49","guid":{"rendered":"http:\/\/www.shamusyoung.com\/twentysidedtale\/?p=17585"},"modified":"2015-06-03T13:20:26","modified_gmt":"2015-06-03T18:20:26","slug":"trusting-the-system","status":"publish","type":"post","link":"https:\/\/www.shamusyoung.com\/twentysidedtale\/?p=17585","title":{"rendered":"Trusting the System"},"content":{"rendered":"

In the coming week of Spoiler Warning<\/a>, we have a conversation about how gradual some of the upgrades are in Mass Effect 3. I’m talking about the things that give 10% more damage, or increase the radius of area damage by half a meter, or other baby-step upgrades. This conversation reminded me of another problem with complex damage systems. As a programmer, this is something that’s been bothering me for ages.<\/p>\n


\n
The rules are beyond your comprehension.<\/td><\/tr><\/table><\/p>\n

For our worst-case scenario, let’s consider the system for incoming damage in your typical online game of the massive variety. The game is set up so that the enemy (usually called a mob<\/a>) is trying to kill the player character. To do this, it has to reduce the player’s hitpoints to zero. There are a lot of layers of mechanics between the mob and the player death state:<\/p>\n

1. Control<\/strong><\/p>\n

You knock the monster down, knock it back, stick it in place, confuse it, transform it into a sheep, or otherwise stop the mob from initiating an attack. This is actually a pretty straightforward behavior and since the outcome is binary (it either attacks or it doesn’t) it’s not terribly difficult to see the impact it has on the fight. However, things get increasingly murky from here on…<\/p>\n

Star Wars The Old Republic<\/td><\/tr><\/table><\/p>\n

2. Prevention<\/strong><\/p>\n

Players can inflict conditions onto mobs to make them attack more slowly, hit less hard, recharge their magical energy less quickly, limit their attack range, or otherwise limit their combat effectiveness. The mob will still attack, but those attacks will be some percent less potent. <\/p>\n

However, the attack does take place unless the player has…<\/p>\n

Champions Online<\/td><\/tr><\/table><\/p>\n

3. Evasion<\/strong><\/p>\n

You might be partly invisible, or dodge due to being very nimble. Maybe the game has a formally recognized dodge mechanic. Whatever. The mob takes a shot that may fail to connect. Maybe your ability simply makes the “next” attack miss, or perhaps this is a probability roll of your ability to dodge vs. their ability to hit. <\/p>\n

If the player doesn’t dodge, then perhaps they will use…<\/p>\n

Guild Wars 2<\/td><\/tr><\/table><\/p>\n

4. Blocking<\/strong><\/p>\n

This typically only applies to characters with shields, but it’s also sometimes used in a magical sense where the player has a shield. Again: Maybe there’s a shield that can absorb a single attack, regardless of strength. Maybe the player is under an effect that will automatically block X attacks. Maybe this is a probability roll of your [fortitude] vs. their ability to batter aside your shield. Whatever. If the attack is blocked, then a majority (perhaps even all) of the incoming damage is discarded.<\/p>\n

Any damage that makes it through blocking will then proceed to…<\/p>\n

TERA Online<\/td><\/tr><\/table><\/p>\n

5. Mitigation<\/strong><\/p>\n

Here is where your “toughness” stat comes into play, or whatever stat they’re using for extremely durable characters in this game. If you’ve got great armor or high stats, then the incoming damage numbers will be reduced by some factor. <\/p>\n

Once the damage has been mitigated, it finally gets to the health bar and its time for…<\/p>\n

Guild Wars 2<\/td><\/tr><\/table><\/p>\n

6. Absorption<\/strong><\/p>\n

Finally we’re subtracting some health away from the player’s hit point bar. If the mob can inflict sustained damage faster than the player is being healed, then eventually the player will drop dead. <\/p>\n

This is a very bare-bones look at the system. We’re ignoring magical resistance, elemental damage, damage over time, damage reflection, and conditions on the player. We’re also ignoring the similar path followed by outgoing damage and aggro<\/a> mechanics. Basically, we’re taking about the simplest form of damage: A mob hitting you with its pointy-stick of choice. Even this simplistic interaction has many layers that might have a lot of funny math going on. <\/p>\n

In just one example: You’ve got innate stats and the armor bonuses of perhaps a dozen or so pieces of armor. How do those stats stack up? If your stats reduce incoming damage by 20% and your armor reduces it by another 30%, are those stats additive or compounding? If a mob has just hit the player for 100 damage then perhaps the 20% is added to the 30% reduction for 50% damage reduction. Result: The player suffers 50 damage.<\/em> On the other hand, perhaps the 20% is applied first, reducing the damage to 80. Then the 30% is applied, reducing it to 56. Result: The player suffers 56 damage.<\/em> <\/p>\n

Most players just ignore the number-crunch. They look at two bits of armor, and pick whichever one has the higher numbers. No need to worry about what’s going on under the hood. This is usually my approach to playing a game. <\/p>\n

World of Warcraft<\/td><\/tr><\/table><\/p>\n

But as a programmer, my problem is this: <\/p>\n

How does anyone know if the system is actually working properly?<\/strong> <\/p>\n

This system is simply too chaotic for normal QA testing to detect bugs. Sure, big<\/em> problems will show up easily enough, but this system is made almost entirely of small systems chained together. Even if it were feasible to have a team of QA testers examine every power against every mob with every piece of gear at every level, and even if they understood the mechanics completely, I seriously doubt it would be possible for them to spot problems without some serious statistical analysis tools. <\/p>\n

If that sword that offers 3% more<\/em> critical damage was actually doing 3% less<\/em> damage, how could you possibly tell? If the armor bonus of your boots was never applied to melee attacks, ever – would anyone know? What if that condition you repeatedly stack on an enemy to make them weaker is actually only effective the first time, and otherwise ignored? What if, instead of adding the critical chance of both of your equipped weapons, the system only uses the main-hand bonus unless you’re using pistols in which case it uses the lower of the two? What if the condition the enemy puts on you to make you slightly more likely to miss is never actually cleared until you log out or change zones?<\/p>\n

Now, you might say that you’d notice because the numbers in your character screen wouldn’t look right. But you’re assuming the character screen is being honest with you. Josh found a bug in Guild Wars 2 where an Elementalist can select a trait that will make certain powers cool down faster. After selecting the trait, the listed cooldown of Mist Form dropped from 90 seconds to 75 seconds. However, when you actually use<\/em> the power, the cooldown is still 90 seconds. The tooltip is lying. The trait is probably completely useless. Note that the only reason Josh is aware of this bug is because you can set the game to display the cooldown timer. Without this, he’d need to use a stopwatch to see the problem. If this trait impacted random damage rolls instead of a fixed timer, then it would be extremely difficult and time-consuming to conclusively observe this bug. <\/p>\n

If you’re fighting alone, then fights are generally too short to observe anomalies. If you’re part of a group, then any anomalies are easily attributed to the activity of other players. All you can see are damage numbers, which are derived from random rolls and run through the numeric meatgrinder above. Even if by some miracle you do notice a problem with the numbers, are you really going to be able to detect this as a bug? Maybe you’re lagging. Maybe another player has thrown some other conditions or buffs into the mix. Maybe you’re just noticing a few bad rolls in all the noise. Maybe a recent patch changed things. Maybe you’re just failing to understand the undocumented mechanics. <\/p>\n

In your typical MMO, think about all the observable<\/em> bugs you see. The one corner where people fall out of the level by hopping. The way magical swords don’t generate particle effects after you get out of the water. The way the “salute” animation is incorrectly applied to characters in motion. The way on-screen chat bubbles get clipped if you have the camera zoomed all the way in and your game resolution is different from your desktop resolution. Bugs can often be rare, situational, and obscure. <\/p>\n

If bugs that can be easily observed and reproduced end up lingering for months, then how much worse is it for bugs that people might not be able to notice? Any system that can’t be properly tested is bound to have bugs in it. And unlike visible bugs, there might be incentives to avoid<\/em> looking for or fixing mechanical bugs.<\/p>\n

Consider a programmer who is scrolling through the code one day and finds this:<\/p>\n

\r\n\/\/Reduce incoming fire damage according to the fire resistance.\r\nif (DamageType == DAMAGE_FIRE) { \r\n  Damage -= FireProtection;\r\n}\r\n<\/pre>\n
After some head-scratching, they realize it should actually be this:<\/p>\n
\r\n\/\/Reduce incoming fire damage according to the fire resistance.\r\nif (DamageType == DAMAGE_FIRE) { \r\n  Damage -= (Damage * FireProtection);\r\n}\r\n<\/pre>\n
World of Warcraft<\/td><\/tr><\/table><\/p>\n
So your ring that granted 10% fire protection was actually just reducing all fire damage by 0.1 hitpoints. This bug only applies to fire resistance bonuses from non-exotic jewelry that isn’t player crafted. In this game, perhaps fire damage isn’t a big deal until you get near the level cap, so here we have a bug that only applies to certain characters with certain gear of a certain quality, and even then the bug is only important if they happen to be facing a lot of fire damage. <\/p>\n
Nobody has noticed this bug in the year or so since it was introduced. Now for the really tricky questions for the programmer: <\/p>\n
Do you fix this bug? Is that going to introduce balance problems with PvP? Will making the required change introduce a bug elsewhere? Do you really want to announce to players that the precious gear they bought and quested for actually didn’t work right until now? Or maybe you should just fix the bug quietly?<\/p>\n
Quest of the Dungeon Swords Online. Patch notes for build 1.133:<\/p>\n
    \n
  • Fixed bug where players could fall out of level in The Citadel of Dragon Fighting.\n<\/li>\n
  • Lorik Questsayer will only give the quest to kill the Dredrat King if the player has opened the way to the Dredrat Warrens.\n<\/li>\n
  • Fixed the issue where bloom lighting would cause slowdowns with certain NVIDIA cards.\n<\/li>\n
  • Made slight change to how fire damage is calculated in some situations.\n<\/li>\n
  • Players can now re-bind the key to display scoreboards in Battlewatch PvP arenas.\n<\/li>\n<\/ul>\n<\/blockquote>\n
    This doubt grows on me the more I play these games. As the mechanics become ever more impenetrable, I can’t shake the notion that maybe I just spent five gold on a sword for no reason because the numbers don’t work the way the tooltips claim. <\/p>\n
    Do you trust the system?<\/p>\n","protected":false},"excerpt":{"rendered":"
    In the coming week of Spoiler Warning, we have a conversation about how gradual some of the upgrades are in Mass Effect 3. I’m talking about the things that give 10% more damage, or increase the radius of area damage by half a meter, or other baby-step upgrades. This conversation reminded me of another problem […]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[55],"tags":[],"class_list":["post-17585","post","type-post","status-publish","format-standard","hentry","category-game-design"],"_links":{"self":[{"href":"https:\/\/www.shamusyoung.com\/twentysidedtale\/index.php?rest_route=\/wp\/v2\/posts\/17585","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.shamusyoung.com\/twentysidedtale\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.shamusyoung.com\/twentysidedtale\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.shamusyoung.com\/twentysidedtale\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.shamusyoung.com\/twentysidedtale\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=17585"}],"version-history":[{"count":0,"href":"https:\/\/www.shamusyoung.com\/twentysidedtale\/index.php?rest_route=\/wp\/v2\/posts\/17585\/revisions"}],"wp:attachment":[{"href":"https:\/\/www.shamusyoung.com\/twentysidedtale\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=17585"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.shamusyoung.com\/twentysidedtale\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=17585"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.shamusyoung.com\/twentysidedtale\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=17585"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}