Let’s talk about the Skinner Box and its impact on video games in the last 20 years

Behaviourism's legacy and the implications for young people

I miss the video games of old, before the businessmen got involved. In its infancy gaming was pure, it was a landscape of variable quality, hobbyist enthusiasm, and genuine passion and creativity. Nowadays… well, let’s just say it is a landscape designed to capture and exploit human attention with the express purpose of making as much money as possible.

It makes me quite sad really. I remember a time when games had “cheat codes” to unlock different models for your character, different outfits or secret levels. A time when there was no internet, companies could release a half-broken game and then release patches for you to download to fix it months later. The product you got was complete, for good or bad, and so developers had higher standards to hold to if they wanted to be successful. Now many games, even high budget “triple A” games, are released broken. The developers then “apologise” and promise to fix them over time, with no guarantee and no consequences except their reputation. Not only this but content is cut, or unreleased. Those fun alternate character designs you used to unlock with a cheat code or secret achievement? Yeah, those are paid for separately now. They are “Downloadable Content (DLC)” or “Pre-Order Bonuses” designed to force you to spend more money, and spend it earlier before reviews are released.

Debatably these things are necessary evils, benefiting the stability of the company. Ultimately what they choose to include or not is up to them, and this is the norm now. I’m just sayin’ I miss the old days!

Today’s topic is really the one that tips me over the edge regarding calling some of the major gaming corporations “MegaCorps” rather than publishers or businesses, etc. I use that specific term because I love sci-fi, and particularly cyberpunk novels. They are almost universally dystopian, and usually involve a giant, all-powerful corporation (MegaCorp), so wealthy it can operate outside the law or even is the law, hell bent on an “anything for profit and control” mentality at the cost of human life and suffering.

Not all games developers are evil, but there is a significant proportion that are demonstrably immoral. I genuinely fear for the impact this has on young people; they are exposed daily in both social media (we’ll deal with this beast in another article) and video games to extremely psychologically manipulative corporations and there are almost zero safeguards in place. Lawmakers are too slow to regulate the industry and so they are left to the wolves in many cases. Most adults aren’t even educated on this topic.

My one redeeming hope is to rely upon the savvy and intelligence of the young people I speak with. Education is my only weapon in this battle. I speak openly in my classrooms with my students about the fact I play video games. I tell them what tactics to watch for. I never judge them (who can judge a child for falling into a psychological trap they don’t even know exists?) The toughest part for me, is that I know even educating many will not be enough to save them from these tactics, but for now it is all I can do.

Ok, brace yourself, it’s unpleasant, and we’re going in.

Skinner Box Games Design

A skinner box simply put is

a box or cage in which a subject (usually an animal) may be isolated from outside influences and studied; used in operant conditioning experiments, where the subject can operate a leaver to obtain a reward or avoid a painful shock.

Originally developed by B. F. Skinner, the famous psychologist who developed his theory of Behaviourism. You will remember I talked about him in my previous post “So what pedagogy do you adhere to?”. I concluded during that post that I believe strongly in many aspects of behaviourism as a form of learning, and this article will hopefully show you why.

A skinner box, is used to condition animals (usually rats) to operate a button or lever (perform a task). Through repetition and reinforcement an animal is trained to act in a specific way. It is always built on a [Stimulus → Correct behaviour → Reinforcement] process. For example, one of the simplest forms of the Skinner box is:

1. Stimulus: Rat is shown tasty treat in clear plastic bucket she cannot reach.

2. Correct behaviour: Rat pushes button / pulls lever.

3. Reinforcement (reward): Rat receives food pellet.

Skinner was investigating the question of “Can an animal be conditioned to behave in a specific way in response to specific stimulus?”

His findings can be summarised as “Yes, by applying specific stimuli, you can cause the animal to respond in a particular way.” Truly, a luminary. I jest of course. Do not dismiss this idea as “obvious” or simple, it is possibly the most influential concept of the 20th and 21st century. It has been used to shape human behaviour before his experiment, I’m sure. Since Skinner published his research it has become ever more prominent, never more so that right now in 2023. To understand the problem, we must go deeper.

Variable-Ratio Reward Schedules and Pigeon Pilots

In addition to his rats in boxes, Skinner trained pigeons to peck a disk to receive food. He demonstrated that the rate of pecking could be controlled by changing the schedule of reinforcement. This is where it gets dangerous, pay close attention.

If food was delivered every tenth peck (a fixed-ratio schedule), the pigeon would peck at a high rate until receiving the food pellet, and then stop until it next wanted food. A rational behaviour in line with basic food gathering.

However, if food was delivered unpredictably (a variable-ratio schedule), the pigeon would peck at a steady rate without taking breaks, even after receiving a pellet. The uncertainty of when the next pellet would come would compel the pigeon to keep pecking.

This experiment ended up forming the basis of the pigeon-guided missile system, Project Orcon (Organic control), towards the end of the second world war. Don’t believe me? Here’s the video footage of the pigeon pilots in training!

Two things are worth noting here. Firstly, pigeons are intelligent. Particularly with regard to visual matters. Pigeons have achieved all of the following tasks successfully (thanks to Psychology today for the article!) :

1. Pigeon’s can discriminate a previously unseen Picasso, from a previously unseen Monet painting. (Watanabe et al., 1995)

2. Pigeons can determine malignant vs benign tumours in radiology images. (Levenson et al., 2015)

3. Pigeons are on par with primates when it comes to counting, unlike most animals. (Scarf et al., 2016)

4. Pigeons can recognize words. (Scarf et al., 2016)

5. Pigeons have exceptional memory for visual patterns. (von Fersen and Güntürkün, 1990)

I don’t know what I expected to learn when I began writing this article, but I promise there is no hidden pigeon agenda in my writing.

Back to the topic at hand, enough of this flapping about!

“Game” Design elements

Note for this section that while I do directly compare some design elements, in many cases these are fine, and genuinely fun. If the gameplay itself is entertaining, and the skinner box addition is a part of that design and not what I would describe as “weaponized” for addiction, then generally it isn’t terrible. We’ll get to those types of games later.

Let’s examine an exceptionally popular game right now, Diablo 4. Recently released by Activision-Blizzard, their fastest selling game of all time totalling $666 million in just 5 days. The game is themed around hell and demons; hence they reported this particular financial milestone ($666M) rather than a more typical number such as gross sales in first week.

In the game players slay demons and other enemies, hoping for them to drop rare and powerful equipment for the player to equip, in turn strengthening their character and allowing them to go on and slay further enemies, including the most challenging bosses in the game. The loot does not always drop you understand, only sometimes, it is a variable-ratio reward schedule meaning, it doesn’t drop after X number of kills of monster, but instead the item can drop after any number kills but at a low chance.

This is objectively fun game design. Hunting the super rare items by slaying the most powerful bosses in challenging combat. Optimizing your character to do this quickly and safely allowing you to find the items quicker and reach the highest challenges. Oftentimes the gear can change your character in meaningful ways. By this I mean that rather than a powerful item just having higher “strength” so that you hit harder, instead the gear would modify how your skills work, introduce new skills, or fundamentally change how some of the game’s mechanics work. An example of such an item can be seen here:

Health globes restore health to the player when picked up. Normally they have a chance to spawn when you slay an enemy. This item is a good example of an item that fundamentally changes the games mechanics making for interesting gameplay.

I love these types of games. They’re my favourite genre, I have played many over the years and I fondly remember 2010, where with my wife as my co-op team member we beat Titan Quest: Immortal Throne on the toughest difficulty (her as an archer, I as a shaman if you’re curious).

Even further back as a 14-year-old I had a laptop capable of playing Diablo II: Lord of Destruction. I used to be up half the night in boarding school playing! I even loaned my laptop to my good friend Rob once I’d got tired of the game, and he did exactly the same. Good times. To this day I can quote the lines from it, and remember the music, the incredible music.

However, I will remind you of Skinner’s conclusion, and show you the other side of the coin:

If food was delivered unpredictably (a variable-ratio schedule), the pigeon would peck at a steady rate without taking breaks.

Flash back to 2012, Taiwan.

An 18-year-old named Chuang rents a private room in an internet cafe and plays Diablo 3 for 40 hours straight without taking any breaks for food or rest. He passes out, and when an employee of the cafe checks on him, he wakes up, stands up and then collapses. Chuang is rushed to hospital but pronounced dead on arrival with cause of death unknown. It is thought to have been a blood clot from remaining in place for nearly two days straight.

I’m sure I don’t need to explain the link between this specific piece of game design, and the outcome observed being identical to the rat’s behaviour.

If this seems bad to you, then I shall warn you, this is not even the tip of the iceberg. This is a drop in the ocean and genuinely has not happened often. It is the most extreme example of video game addiction causing death, and usually there are significantly worse factors at play than the escapism leading to video games. I’m concerned that someone, somewhere analysed this news with the statement “how will that affect our quarterlies?” No, in fact, I’m sure someone did. Let’s get into the much, much worse practices that are adopted.

While it is worth noting this direct comparison, you should keep in mind the idea of operant conditioning (stimulus → operate lever → reward) for the entire suite of practices I’m about to list out.

Freemium - $98.4B of revenue in 2020, 78% of all digital games’ revenue for that year. More than twice the revenue as traditional games.

Note we’re moving now from “making game mechanics” addictive, to “making the entire game addictive”.

Freemium refers to games you can download and play without any cost. The word is made from combining Free and Premium together. You have likely seen many of these games on the Apple and Play stores. Prominent examples include Raid: Shadow Legends, Candy Crush Saga, pretty much anything by Zynga or King. It’s scary to think the penetration of this insidious market is so profound that major brands around the world have accepted it and are garnishing their own profits from its design. Including the likes of, Harry Potter, Pokémon, Disney and just about every other brand. The “design” has been accepted without any meaningful critical challenge.

Allow me to convince you they should be challenged, and challenged fiercely. Most games of this ilk feature at least one but usually all of the following systems:

1. Free-to-play

   The game is free to download and play. By making a game free at the point of entry the market is opened to significantly more potential customers. This particularly impacts children, who often do not have access to the disposable income needed to buy every game they want. It is also fundamental to get players into your game initially so that you can begin with the other strategies.

2. Pay-to-win

   1. Player. vs. Player

      You will note that most of the games of this type have some form of competitive player vs. player gameplay. Ostensibly this is pitched as a fun way to challenge your friends, compete on the leaderboards and so forth. But the reality is it is an uneven playing field designed to incentivise you to purchase a competitive advantage. In most competitions in the world all players start equal, not so in freemium games. Your starting point depends on many factors designed to create inequality, either by equipment, unlocked/bought characters, your “level”, available resources, or a host of other ways you will find that you are inevitably not evenly matched. Most of these companies will pitch the “matchmaking” as competitive, not balanced.

   2. Resource-Gating

      The next mechanic is resource gating. A resource refers to any unique spendable or earnable currency, equipment, character or advantage. This can be as simple as limiting the “energy” of players to X number of actions per day. Inevitably you will find that as soon as your energy is spent, the game will provide you a situation to respond to that requires energy to act! This will often be applied in conjunction with the FOMO strategy outlined in point 5. Don’t worry though, you can always buy more energy. This is a direct example of negative reinforcement operant conditioning. You are the rat, and you are avoiding a negative reinforcement by engaging in the behaviour the company desires.

3. Premium Currencies

   1. Premium currencies, you’ve all seen them: Gold bars (Candy Crush), V-bucks (Fortnite), Riot Points (League of Legends). They are ubiquitous. They have infiltrated near every genre at every scale, they are in everything from purpose built Freemium apps to AAA titles from major publishers like Sony and Microsoft. Why? Profit, of course. Why do they work?

   2. They are difficult to refund as they join a “pool” of points. They allow the corporations running the game to bypass typical consumer protections like credit card chargebacks, again you will be told this is to help protect companies from fraud, but make no mistake this is profitable for them, if you buy an outfit for your character with V-bucks, you’re committed, there is no option for buyer’s remorse.

   3. They obfuscate cost to deliberately impair value judgements. Here is an example of the costs of premium currency in the game Ingress made by Niantic the same people who make Pokémon Go. Tell me, how much does it cost to buy 3 key lockers as efficiently as possible?

      

      Example premium currency costs and possible purchases.

   4. We’re not done. Notice the CMU costs vs the amount of CMU you can buy. If you wanted to buy any one item on the list, you cannot do that and not “waste” money. You will always be left with some amount of leftover currency, a trick to manipulate the sunk-cost fallacy to get you to spend more. Another negative reinforcement conditioning process.

4. FOMO strategies - A good friend of mine who is significantly better as Psychology than myself tells me this technique is more closely related to behavioural economics than pure behaviourism. Still, it is well worth your time to understand it.

   1. FOMO strategies are exactly what they sound like. An attempt to manipulate the “feeling of missing out.” They are all variations of negative reinforcement operant conditioning. Here are just some of the multitude of variations.

      1. Loyalty/Login Bonuses - Daily/Weekly/Monthly the player is reward for opening up the game and clicking the login reward button. If you miss this reward your streak resets and you won’t get to the higher tier rewards.

      2. Limited-time deals - Offers of discounted currency or items that expire “in 24 hours” etc. This have huge numbers of variations not limited to discounts.

      3. Limited-time False-scarcity deals - Bearing in mind digital items have no physical cost to produce once they are made, it is somewhat laughable that some are sold under conditions such as “only during this month, never again!” or to “only 100 people can buy this sword!” Of course this scarcity is completely false.

      4. Battle Passes - Purchasable “additional content”

         Battle passes “reward” (positively reinforce!) you for doing various activities within the game, such as beating certain levels, or earning certain amounts of experience. They are insidious. On the surface these look like fun stimuli for gaming challenges, in reality they push the player to desperately log in trying to complete the Battle Pass so they don’t miss out on the rewards because guess what, the battle pass has a limited duration and if you don’t complete it in that time, you don’t get those rewards. This time-constraint in freemium games naturally forces the player towards the freemium currency to boost their energy/power/whatever to try to complete the pass in time. Battle passes are in most games nowadays. Freemium and regular gaming have both been assaulted by this menace.

I know that because you’ve read a lot, and the list ended, you probably think we’re done. I’m sorry, we’re not.

But surely “Real” games don’t do this? Let me talk to you about Loot Boxes, FIFA packs, Gacha and worst of all… Whale-Hunting.

The Loot Box is thought to have been first introduced by a game called MapleStory as far back as 2004. MapleStory is a giant of the MMORPG genre and arguably one of the founding games of it, in fact it is still live today and one can still load it up and play… for free, of course. The loot box was called “Gachapon ticket”, ironic considering I will be addressing specifically Gacha in a section later.

In principle they work like this:

1. Company releases a bundled group of items. Some very rare, some not so rare. In most cases the “big ticket” item will be something new, and very powerful.

2. You cannot buy these items directly, you can only buy a “ticket” / “box” / “chest”

3. You purchase chests at a fix cost and when opening them you receive a random item from the selection box. Due to the randomized nature of this if you want a specific item from the chest (you always do, the best one) you will typically have to spend much more than you would be willing to spend in a one off purchase.

Until recently the chance to receive a specific item has been a hidden value. This remained the case until companies were forced to reveal the percentages by lawmakers in 2020. Apple mandated this for App store games in 2017, Google did so for Play store games in 2018.

Suprisingly to me, as soon as this was implemented some big publishers, including Psyonix, publisher of the massively successful Rocket League, immediately began dropping loot boxes. It would be tempting to think they recognised they were about to be exposed for using exploitative tactics on their consumers. Cynically I figured they simply wouldn’t care, and would continue anyway.

Would it shock you to learn this is not considered gambling? It shocked me when I discovered it years back, but the distinction is written in law. By the very fact that it is impossible to get a “nothing” result from a loot box, it is not considered gambling. Even if a possible result is an item that is near-worthless that you already possess 5 of from the previous 7 boxes. Even if you cannot resell the item in any way to another player. Still not gambling. Just psychologically, functionally, and emotionally the exact same. Legally though, nope.

FIFA packs

FIFA packs for me embody the destruction of an excellent gaming concept in preference of pure monetary greed. Initially back in 2009 EA Sports introduced to FIFA the game mode “FIFA Ultimate Team”. In short: Fantasy football, in a football video game, where you play as the fantasy team you have created. I’m no football fan but I recognise excellent game premise and design, and this is exactly that. It must have been a great time to be a FIFA fan. Very quickly it became the most played game mode in FIFA and over the years has had genuine development of mechanics such as player chemistry, online seasons and lots more. Today it is still incredibly successful:

I don’t think it’s hard to see this is an excellent game idea. But, EA, well known for their shoddy business practices, decided from the get go to design it with the express purpose of selling loot boxes FIFA packs. How does one acquire players for their ultimate team? Yep, that’s right, buy FIFA packs. Purchase a pack, hope for a rare player, repeat ad nauseum.

This financial engine created $1.62B of revenue in 2021. Because behaviourist psychology works on children. Hell, it works on adults too, but children and early 20’s are the main targets of this system, according to a poll by users of the official FIFA forums in 2019, FIFA’s core demographic is 12-15 year olds. EA does not release this information publicly, what a surprise. Do you think it might be because:

• 93% of Gen Alpha and 91% of Gen Z have spent money on in-game purchases during the past 6 months.

• Gen Alpha players spend the most on gear, playable characters, and in-game currencies. Gen Z players spend the most on in-game currencies, gear, and playable characters.

Perhaps teenagers are famous for the poor management of money, inability to make good monetary value judgements, and highly impulsive? Making them easy targets for psychologically predatory tactics? Nah, its probably just that kids like football, right? That’s got to be it.

Gacha

Gacha is a catch-all term that describes games with loot box mechanics. It was pioneered in anime games with the collecting of “5-star” characters via loot-box style mechanics, though it inevitably is now present in freemium games across the board.

Whale-Hunting

Whale-hunting refers to the deliberate targeting of specific players, so called “Whales”, within freemium game models. The name comes from the fact that they most spending in freemium game models is done by very few players. Whales can spend upwards of $1000 a month on freemium games, and much much more than that too. Yes, you read that correctly. Its difficult to find data on the numbers for specific companies as many guard this information, perhaps because publicising their moral bankruptcy isn’t seen as a good idea, but from what statistics are out there we know that the smallest percentage of users, often 0.1% or less, are responsible for 50% of the revenue. In other businesses they might be described as “lucrative clients” but in not in the mobile games market. No, no. They favour the much more dehumanizing “Whales.” Whales come in different types and their spending patterns differ accordingly but the goal for the game companies is simple: Find them and extract all their money. I know I’ve written far too much today, but please do watch this talk from Pocket Gamer Connects in 2016 given by the CEO of Tribeflame. Torulf Jernström. If I haven’t convinced you yet, I’ll leave it to the man who opens his talk with:

“some of you will probably be slightly shocked by all of the tricks here but I’ll leave the morality of it out of the talk, we can discuss it if we have time, later”

followed by clearly insincere laughter. Can you think of a more villainous way to open a talk on monetization strategy?

After watching that for the first time, back in 2016 I remember feeling utterly disgusted. To this day I’ve never purchased an in-app freemium style purchase. I am ruthlessly critical when I evaluate games as a result. Oh also let’s note, at an entire developer conference on this topic, no one asked any questions at the end regarding the morality, there was time in the end but they simply do not care. Torulf’s use of language and callous laughter at the target

So what can we do?

I know this has been somewhat doom-and-gloom, it is the unfortunate reality of gaming in the modern era. I didn’t even really touch on “games as a service” but with what I’ve taught you in this article, you should be able to see through most manipulations now so long as you think critically and follow my driving instructor’s early advice, “be paranoid, you can’t trust anyone in this world.” He was a nice man.

I implore you. Spread the word. Inform people, educate them on the perils, but most of all, educate children on them. Knowledge isn’t a perfect defence, and I’ve no idea how to get regulation implemented, but at the very least if people are made aware of all the micro-strategies at play, if they understand the concept of operant conditioning, they have a better hope than most at spotting when they are being targeted for manipulation.

As always, thanks for reading.