So let’s say you’re a stone-age Homo sapiens and you’re having a really bad day. You’re separated from your tribe, you’re hungry, you’re thirsty, and you’re exhausted. Nobody’s invented McDonald’s yet, so you have to forage for your food. You don’t know what vitamin C is, but you do know that for days you’ve had a powerful craving for something tart.
Then you come across a plant you’ve never seen before. Fruit is growing on it. You don’t know if it’s safe to eat, but you don’t have much of a choice. You take a bite and Woooooooooooooah. It’s got water, it’s got calories, and it’s got vitamin C. That wave of euphoria you experienced when you ate the fruit was caused by the release of dopamine in your brain
So Borderlands 3 is finally out. It’s gotten a mixed reception so far. Like I said in my last video, it doesn’t have the same charm that Borderlands 2 did, and a lot of that is because of the way the reward system works. It might be a great game, but it’s a lousy skinner box. I think most gamers know what a skinner box is at this point, but I rarely hear people talk about why Skinner Boxes work the way the do.
The concept of a Skinner Box comes from behaviorist B. F. Skinner. The skinner box is a real device – properly called an operant conditioning chamber – and it actually covered a lot of different stimuli, both positive and negative. Doc skinner made animals go hungry and he applied electric shocks to others. He was a giant in his field, but he was definitely no Dr. Dolittle. The actual skinner boxes were probably not a fun place to be, but in gaming we’ve borrowed the term to refer to a gameplay mechanic that delivers irregular rewards. The original skinner box was a scientific apparatus, but now we use the term to describe gameplay mechanics. What’s really going on is that we’re using an exploit in the learning systems of our brain to create pleasure. Here’s how it works:
Learning Has Diminishing Returns
Getting back to that stone-age Homo sapiens, that blast of dopamine isn’t just so our primitive human can feel better. Yes, dopamine feels good, but it’s also linked to the formation of memories. Let’s say you find another bush a few days later. As you savor your prize, you start to think, “Hey, the last time I found one of these, I was in a valley, in a sunny patch, near water, with some sand around. Just like this time. Maybe in the future I should seek out these sorts of places to see if I can find more of this fruit.”
The dopamine doesn’t just feel good, it also tickles the parts of your brain that look for patterns and build muscle memory. Basically, this is the brain’s way of saying, “Whatever you did to make that happen, try to do it again because that outcome was really good.”
Under normal circumstances, this works as intended. It doesn’t matter if you’re performing music, shooting hoops, telling jokes, solving a crossword, or learning to punch lawless thugs unconscious. Every success gives you a little kick of dopamine, allowing you to gradually fine-tune and hopefully perfect your performance.
And that’s the trick here. Whatever you’re learning, it needs to be something with a non-obvious solution. If the solution to a problem is straightforward, then your mind isn’t going to be attracted to it. Everyone is wired a little different, but most of us don’t get a thrill from pressing a win button.
We only get rewarded when we succeed at something we haven’t mastered yet. Or at least, that’s where you get the really big rewards. You might enjoy a nice hit of dopamine the first time you clear a challenge room in Batman without taking a single hit. You might cheer. Do a fist-pump. If there’s a friend around you might try to get a high-five out of them. Maybe you’ll instagram it, or call your mom. Whatever. I’m not here to judge how you celebrate. I’m just saying that it won’t be nearly as special when you do that for the tenth time. And when you do it for the one hundredth time, you won’t even change the expression on your face. The triumph has become routine, and you need to tackle a new challenge if you want another big dose of dopamine. Your brain is telling you that if you want more rewards, you need to push yourself to overcome something new.
But what if there was something where you could regularly enjoy success without ever mastering it? What if there was a task that you could never improve at matter how long you practiced, even though you could still succeed once in a while? A task like that could theoretically reward you forever.
The good news is, such an activity does indeed exist. The bad news is… it’s a slot machine.
The Slot Machine
Those spinning wheels and flashing lights in gambling devices aren’t just there to make the game more “fun”. They’re required for the game to work at all. If you had a slot machine where you dropped in your money and it instantly gave the result, then it would lose the ability to trigger the release of dopamine in most people. Your brain does that because it’s trying to get you to find a pattern, and the tactile, visual, and audio cues all exist to create a bunch of information for your brain to analyze. Sometimes you win, usually you lose, but when you do win your brain will still give you that sweet, sweet dopamine, hoping that you’ll crack the code. If there is no code to crack, then the reward will never stop.
The interesting thing is that all of this seems to happen on a subconscious level. You can tell a gambler that a slot machine is pure randomness and that the lights and sounds don’t mean anything. They can know and understand this idea, but that doesn’t stop them from getting more dopamine when they win.
What Doc Skinner discovered is that we – both animals and humans – will fixate on a system that gives irregular or unpredictable rewards. If you stick an animal in a box and give them a button to dispense food, then once they push the button a couple of times they’ll have it figured out and then they’ll only push it when they’re hungry. But if the reward is irregular – if it’s random whether or not they get food – then the animal will press the button a lot, even when they’re not hungry. The irregular behavior makes your brain fixate on something until you can solve the pattern. A simple food button wouldn’t work on a human, but if you add enough spinning wheels and flashing lights, the complexity might be enough to make them curious.
Some people say things like, “I don’t gamble because I understand math.” It’s true that the math says that betting against the house is a bad idea, but this isn’t really why people gamble. Plenty of people with gambling problems can explain the odds and the toll that gambling takes on their life. It’s less that they’re confused about the mechanics and more that they’re looking to justify doing something they already know is a bad idea. Our brains are really really good at rationalizing and coming up with reasons to reward ourselves right now.
“I’ll just have one drink and then I swear I’m going straight home,” says the alcoholic.
“I’ll double my workout tomorrow,” says the person who wants to cheat on their diet.
“I think my luck is about to change,” says the gambler who’s been losing all night.
“But it’s on sale!” says the compulsive shopper who’s already maxed out two credit cards.
Deep down, these people know they’re wrong. The person is rationalizing a short-term reward at the expense of long-term setbacks because they’re really craving the reward. That’s a pretty common problem for human beings to have.
Maybe all of this sounds insidious, but a skinner box is a system that can be used for good or evil. It can destroy people, but it’s also the mechanism that entertains us, helps us learn, and motivates us to pursue our long-term goals.
Diablo and Borderlands are both games that harness this built-in reward system to make a game more fun. Note that this works best when it’s added to an already-satisfying game. Ideally you’re playing a game where the gameplay is fun, the story is interesting, and the visuals are good. If you take an already-good game and add a skinner box on top of it, you wind up with the kind of game that can keep you engaged for months or weeks, and that you can come back to years later. If you don’t have that stuff – if the gameplay is shallow, the art is ugly, and the story is lazy nonsense – then you wind up with something that’s just a skinner box. A lot of mobile titles wind up here. It can still hook a few people that have a strong proclivity for skinner box traps, but the majority of people won’t give the game a second look.
Welcome to the Casino
Gamers are frustrated with current-gen loot boxes and other microtransaction-based reward systems because instead of working with the mechanics of the game, it works against it. Defenders of loot boxes claim that nobody is forcing you to buy them. While that’s true, it doesn’t repair the damage that loot boxes inflict on the design. I want a game where I can steadily earn things through gameplay, and sometimes I’ll get a little bonus reward of a rare drop or whatever. A loot box driven game stonewalls that steady progress in order to push me to buy spins on their slot machine. In order to sell more loot boxes, the steady rewards are placed so far apart that the game feels empty if you’re not spending money.
I don’t know what happens next for loot boxes. I know a lot of us hate them, but I also know that a very small number of people pour a ton of money into them. Consider the person who spent $62,000 on Runescape. If an 18 year old MMO can make that much from one person, imagine how much current-gen games are making from loot boxes. If a thousand people boycott a game because of loot boxes, it only takes one person spending $62,000 to make up for all of those lost sales. The AAA area of this hobby is pivoting away from making games and towards making video game branded casinos, and the result is not pretty.
Good Robot Dev Blog
An ongoing series where I work on making a 2D action game from scratch.
A programming project where I set out to make a gigantic and complex world from simple data.
Internet News is All Wrong
Why is internet news so bad, why do people prefer celebrity fluff, and how could it be made better?
The Plot-Driven Door
You know how videogames sometimes do that thing where it's preposterously hard to go through a simple door? This one is really bad.
The Game That Ruined Me
Be careful what you learn with your muscle-memory, because it will be very hard to un-learn it.