I’m not sure who will find this interesting. This is an AI analysis of a ten year old videogame. This entire endeavor will sound absurd to people familiar with the game in question, and hopelessly esoteric to those that aren’t. Still, I’m putting this up in case there is someone else out there who is just as peculiar as I am, in that I find this sort of thing intensely compelling.
About a month ago I wrote a Starcraft scenario which allowed you to observe a game between AI players. I’ve been curious about the quirks in the Starcraft AI and I’ve wanted a chance to see them do their thing in a deterministic environment. I learned some surprising things about this ten-year-old gem. While the races themselves are very nearly balanced in the hands of humans, it turns out the AI is a lot better at using some races compared to others.
|This is a very pixelated map of The Hunters. The map has eight starting locations, marked by the colored squares. Players are randomly placed on the map, with one random spot left empty.|
I’d usually let the game run overnight and check on the results in the morning. (Yes! This was my solution for being too busy to play computer games, I programmed a game to play itself for me!) A game normally takes a couple of hours, although rarely one will end in an hour, and several became endless stalemates.
At first I just set the difficulty to “normal”, but I found that the computer players were far too likely to consume all the resources on the map, go broke, and then just sit there. I’d start a game before going to bed, and when I came back in the morning I’d find the battle was down to three sides who couldn’t make any fighting units. I changed the difficulty to “Insane”, which auto-cheats by giving itself 2,000 minerals and gas anytime it goes broke, meaning the thing is always rolling in resources. This made sure that most battles came to a proper conclusion. Although this made battles larger, more spectacular, and a little more chaotic, it didn’t seem to affect who won. I ran many overnight battles with both AI setups, and while higher difficulty made for inflated scores, over many games the results painted a clear picture.
I dislike this auto-cheat for a number of reasons, mostly because it negates a lot of the strategy in the game. None of the players can go broke, but the AI still plays as though resources were important. The only true way to knock a player out of the game is to annihilate their core base with all the critical buildings in it. Expansion bases are (mostly) worthless in a game like this. Yet the computer still builds and defends expansions (because that’s what it’s programmed to do) and still wastes time attacking enemy expansion bases. This introduces a bit of luck into the game: Who wins depends a lot on positioning. The AI tends to attack the nearest base, not the most important one. Sometimes the nearest base is the core base. Sometimes the nearest is an expansion which is pointlessly destroyed and rebuilt over and over again.
Like I said, in the long run it didn’t really change the fact that one race continually came out on top.
Without the unpredictable actions of a human involved in the match, the AI fights like clockwork. Games have a perceptible rhythm to them. They all build their first couple of buildings within a few seconds of each other. Even these variations are probably the result of minor changes in the layout of each base. If the bases were somehow shaped the same, the players would probably all build in perfect unison.
They build an initial attack force. On insane difficulty this attack force is huge – probably sixteen or so zealots or the given race equivalent. On normal difficulty the force is smaller, but the behavior is the same. They all leave the base at about the same time, and (as far as I can tell) attack a foe totally at random. Sometimes this foe will be a neighbor, and sometimes it will be someone all the way across the map.
This is the most chaotic and unpredictable part of the game, and it’s basically a crapshoot. Usually one or two players get taken out. Nobody has any defenses at this point, so when an enemy comes knocking, their survival is a matter of luck: The defender has an attack force elsewhere on the map. Is that force still intact, and can it be recalled before the place is destroyed? Are there enough of them left to save the place? A player can also do really well if they are attacked by two people at exactly the same time – the attackers end up wiping each other out and leaving the buildings alone.
The AI seems to wait until this initial attack force is nearly all dead before entering the next phase: It builds defenses and another attack force. Again, this force is sent out. Once gone, the AI begins trying to build its first expansion base.
The rest of the game is a series of escalating attack waves. As they add more buildings onto their main base they work their way up to air units, and this is where you start seeing some differences in how well the computer handles the different races.
If you’ve never played Starcraft, you’re probably not even bothering to read this, but just in case:
The Zerg are a bug-like race. Think H. R. Giger Aliens or Starship Trooper aliens. Their units are usually cheap, plentiful, and weak. They are made to be mass-produced and sent to their deaths. The Protoss are highly advanced aliens with lots of fancy technology. Their units are very expensive but very durable. Their units are usually less numerous. Their shield technology encourages you to try and make sure they survive a battle so you can recharge their shields and send them off to fight again. They are not intended to be disposable units. The Terrans are the humans and fall somewhere between these two extremes in terms of cost vs. unit lifespan. It’s actually far more complex than this, but that’s the general, high-level, broad, big-picture overview. Volumes have been written on these three races and we can’t begin to cover it all here. Just remember: Zerg = cheap and weak, Protoss = expensive and tough, Terrans = Somewhere in-between.
I expected the AI to excel at using the Zerg, since their intended usage fits nicely with how the computer behaves: Churn out masses of units and fling them at the enemy without regard to casualties. I expected Protoss to be the worst, since the send-them-to-die policy of the AI would end up squandering a lot of very expensive units.
Without a doubt, the AI is far better as using the Protoss. It’s not even a contest. The only time a Protoss would get wiped out would be if they had a stroke of bad luck in the initial first-wave attack, which is always a game of Russian Roulette. Assuming they survive that, a Protoss AI is probably going to go the distance. The only foe they have to really worry about is the other Protoss. In the dozens of games I ran, only once did I see a game where the top spot wasn’t occupied by a Protoss. Most of the time two protoss players occupy the top two slots in terms of scoring.
|This is the outcome of a very typical battle. Protoss kicking butt. Zerg coming in second. Terrans going extinct in some kinda hurry.|
The Terrans were unquestionably the worst in the hands of the AI. Despite the fact that there were (usually) three Terran players, thus slanting the odds in favor of a Terran-dominated game, they never got above third place, and often the Terran players would occupy three of the bottom four slots. They were more likely than the other races to get wiped out in the first rush, and if they survived they had a terrible time doing real damage to their foes. Despite the fearsome power of the Terran Battlecruiser, the AI was very shy about building them. Even with limitless resources, it never built more than a few and it never used them very well. Even late in the game it was building Wraiths, and it would not quit making them no matter how absurdly poorly they performed. The Terrans seemed to maintain a smaller force. It used Siege Tanks poorly, sent Marines to fight without Medics, and generally fought like a complete tool.
The computer did alright with the Zerg, although it was moronic with its mix of air units. Guardians were often sent in with too little support, making them easy fodder for enemy defenders. If it used Defilers, it was likely as not going to hit its own units when using plague.
|A familiar battle: Siege Tank vs. Mutalisk. Also popular is the Valkyrie vs. Dragoon fight. The Terran AI always seems to have the wrong unit, at the wrong time, in the wrong place, doing the wrong thing, and not nearly enough of them.|
Against other AI, the Templar are bumbling comic goofs. They will drop Psi Storm on single enemy units and hit a bunch of their own guys in the process. They will blunder through fortified territory attempting to reach a unit deep inside, and get cut down before they even get close.
I’ve come to suspect that the AI cheats a bit and detects clusters of units which have been grouped by hotkey by human players. This is very naughty if it’s true. What’s worse is that peeking at how my hotkeys are set up seems to be central to its decision making. Deprived of that bit of cheating info, the Templar is helplessly stupid. Boo.
I’m note sure why the thing is so bad at utilizing Terrans. Aside from the issues I mention above, it just seems less aggressive overall. It also has a penchant for building base defenses (bunkers, towers) in places where a base should go, effectively rendering a viable expansion useless. It will attempt to lunch nukes without bothering to cloak the Ghost first. It will risk the painfully expensive Science Vessel in order to irradiate something of very low strategic value. It makes small numbers of all units instead of focusing on a few and using them well.
Still, this was an amusing experiment. If you have the Broodwar expansion for Starcraft you can try it out yourself. You can get the map here:
You DO remember where your Starcraft disks are, don’t you?
The Gradient of Plot Holes
Most stories have plot holes. The failure isn't that they exist, it's when you notice them while immersed in the story.
Dear Hollywood: Do a Mash Reboot
Since we're rebooting everything, MASH will probably come up eventually. Here are some casting suggestions.
Here is a 13 part series where I talk about programming games, programming languages, and programming problems.
The Best of 2014
My picks for what was important, awesome, or worth talking about in 2014.
Zenimax vs. Facebook
This series explores the troubled history of VR and the strange lawsuit between Zenimax publishing and Facebook.