AI Follies: Targeting

By Shamus Posted Wednesday Aug 19, 2009

Filed under: Programming 49 comments

So, your AI routine is running. Taking into account for the enemy field of view, light levels, camouflage, distance, cover, and general threat awareness (like, are there any suspicious dead bodies lying around?) the program has determined that the bad guy has, in fact, spotted the player. Time to start shooting.

Not so fast.

Just as bad guys will always know where the player is unless you tell them not to know that, they will always hit the player flawlessly unless you make them miss. Left to their own devices, bad guys will be able to ding the player right between the eyes with a Luger at a half mile. Auf wiedersehen, dummkopf!

Like figuring out if you can see the player, figuring out if you can hit them is likewise going to keep your programmer busy for a long time.

Area Targeting

You want to shoot the player. But where do you aim? The head seems a likely spot. But remember the I-can’t-see-you-you-can’t-see-me problem we had with detection. If the player is crouching behind cover with their butt sticking out, the AI needs to be smart enough to forgo the headshot and aim for the sticking-out stuff. If not, players will eventually realize they can take cover behind ridiculously small things – anything that can cover their head.

Weapon Behavior

stalker_town.jpg
The obvious thing to do is to make sure the AI will [seem to] live under the same rules as the player. Effective range, crosswind, weapon accuracy, and movement all need to impact the accuracy of the shot. A lot of games seem to have two sets of rules: One for players and one for AI’s. STALKER was really bad about this, in that weapons were far more accurate in the hands of enemies than they were in the hands of the player. Each bad guy most likely had a single “accuracy” rating that was applied to all situations. You would be crouching, with a good-quality weapon, with the bad guy perfectly lined up, but the bullet spread would cause you to miss. The bad guy would be not just standing but walking, using the same weapon but in worse condition, and would hit you more often than you hit him. On easy difficulty. This was flat-out unfair and frustrating.

But having enemies operate under the same rules as the player means the AI will need to be aware of how each weapon should be used and how to correct for it. This isn’t as much of a problem for linear indoor tunnel games – just put the shotgun guys in small rooms and the snipers up on catwalks and you’re all good. But once you’re trying to make a freeform game where the player can approach an encounter from many different angles you can no longer dictate how the battle will play out, and thus you can’t just hand your AI a random gun and trust it to Do The Interesting thing.

Some weapons are fine for running around and spraying bullets, but others require that you stabilize yourself. This variety makes the weapons more interesting, and lets players have a lot of freedom in selecting arms to suit their play style. But this depth and variety comes back to bite the developer when it comes time to write the AI. It will look stupid if the AI is using the sniper rifle by shooting from the hip, or trying (and failing) to snipe you from the guard tower with his sidearm. Now the AI needs to understand effective range and be able to weigh all sorts of very hairy tradeoffs. To wit:

Hmm. The shotgun isn’t particularly useful at this range. I should get about twenty meters closer. But that means leaving cover and rushing the player. Is it worth it?

That last question is an incredibly complex value judgment. The AI needs to weight just how much it’s getting out of the weapon versus their chances of survival if they move. I doubt any game has really solved it, which explains why sometimes the bad guys fire away uselessly at long range, or dash out into the open and certain death. When they make the wrong decision they come off as either morons or lemmings.

But this type of decision-making falls more under “behavior” than “targeting”. I’ll come back to it later in the series.

Aim

The bot in the distance is using a gun which shoots green orbs which travel in a perfect line without ever dropping.  Note how scattered they are.  A human would have to shake their mouse very fast to get this effect.
The bot in the distance is using a gun which shoots green orbs which travel in a perfect line without ever dropping. Note how scattered they are. A human would have to shake their mouse very fast to get this effect.
Foes should not have superhuman aim. The old-school approach was to just have the enemy determine the perfect shot (an effortless calculation) and then have their aim randomly spread around that point in some sort of Gaussian distribution. This is serviceable in many cases, but falls apart when the enemy is using visible projectiles. In Unreal Tournament, low-challenge bots would spray projectiles like they were shooting from a sputtering garden hose. This distribution never tightened. Even if both the bot and their target were stationary, the bot would keep the same bullet spread. Human players don’t shoot like this unless they’re being electrocuted while playing.

ai_targeting2.jpg
A struggling human player will generally begin aiming with large, sweeping motions, making smaller and smaller adjustments until they settle on the target, even if it’s moving. The target can quickly change relative velocity to foul this process. The result is human players weaving and hopping to keep their relative velocity as unpredictable as possible. For AI, the accuracy should start out very low on a moving target and go up to near 100% as the AI draws a bead on the player. It doesn’t make sense for the AI to miss 30% of the time when they’re shooting at an immobile player at point blank and also missing 30% of the time when they’re shooting at a sprinting player at thirty meters. Just having them miss 30% of the time (with the number going up as distance increases) will not produce believable behavior. More importantly, it will homogenize enemy combat effectiveness, thus making player choices less meaningful (why bother to engage in a way that’s unfavorable to the weapons the enemy has, if they will perform just fine anyway? Might as well just charge headlong into every fight) and thus less interesting. Battles will start to all feel same-y, and people will complain that the combat feels “bland”.

Beyond this, how the AI aims depends a lot on what sort of game you’re making. Are you going for a “tactical shooter” type simulation like Operation Flashpoint, STALKER, or WWII game? Or is this a run-and-gun action game like DOOM or Max Payne? (FEAR is an interesting example, since the AI behaves like they’re in a tactical shooter but the player can act like they’re Sgt. Bullet Sponge. You’re Duke Nukem up against the foes of Operation Flashpoint. This might be a big part of the appeal of the AI in that game. They act like like a plausibly realistic squad, but you can charge in using slo-mo and see them at work instead of fighting them from a distance where their teamwork isn’t as visible.) In a more action-styled game, you actually want a bit of unrealistic behavior because you’re trying to keep up the action-movie feeling of the hero rushing through a hail of bullets. The bad guys are supposed to miss, up to a point, but they need to just “nearly” miss. Ideally they should be using lots of tracer rounds, and when they do decide to miss they should be aiming in such a way that their bullets still pass somewhere in front of the camera or strike nearby objects that will do interesting things. Glass bottles shattering, wood splintering, pipes leaking.

Friendly Fire

Kane & Lynch
Ignoring the fact that the officers seem to be firing into the crowd, or the fact that they shot up the wall for no apparent purpose, and the fact that one of them seems to have taken cover in FRONT of the squad car… Actually, that’s an awful lot to ignore. Anyway, Officer Dumbass (right) is being pretty relaxed about shooting over the head of Officer Fodder. (center)
Assuming the AI is playing the part of rational human soldiers (as opposed to heartless aliens, mind-controlled forces, or robots) then the AI should hold fire if it would be at risk for hitting friendlies. Either you’ll have enemy soldiers mindlessly shooting their comrades in the back, or you’ll have them shooting through their comrades to hit the player. Either one is an AI failure. Most games have nailed this, but I think foes are usually inhumanly aggressive when shooting around their allies. They basically do a hit test and see if pulling the trigger would result in hitting any ally. If not, they fire away. This results in bad guy #1 standing shooting at you, and bad guy #2 standing ten feet behind him, shooting just 10cm over #1’s left ear.

Ammo Conservation

This is one that games have yet to tackle (among games I’ve played, anyway) but it’s a major annoyance of mine. A bad guy pins you down behind some cover and then just plinks away at your hiding spot. Endlessly. Even if you hold your position for five minutes, the AI will just drill away at you without ever worrying about running out of ammo. And when you finally DO get out and kill him, he has nine bullets. Maybe he shot at you for five minutes, or maybe you took him down in a surprise attack. Either way, you always kill him nine bullets short of empty.

It would be nice if suppressing fire started out intense, then fell of to bursts, then eventually tapered off to single rounds. (Depending on the weapon.) You do want the bad guys to try to pin down the player, but you don’t want them to abuse their gift of infinite ammunition. They should at least pretend they don’t have infinite ammo. (And actually running out of ammo is out of the question. It would make the game very exploitable, and programming proper behavior for a weaponless AI would be very difficult. You don’t want to spend ages developing and polishing a situation you don’t want to have happen in the first place.)

And while we’re on the subject of cover fire, AI is usually very bad about cheating when you go into hiding. If you duck behind some crates and slip out of the room, the AI generally won’t continue to douse the crates in suppressing fire. Instead, they move to give chase because they’re clairvoyant and know you have left the room, even if they can’t see you. There was one game (I think it was FEAR, but I’m not positive) where the AI didn’t cheat like this and I was able to flank them. I came around and found a group of guys blasting away at the spot where they had last spotted me. This was a very impressive moment, and blindsiding them was a right and proper reward for my quick thinking.

 


From The Archives:
 

49 thoughts on “AI Follies: Targeting

  1. UTAlan says:

    This series is both making me want to go learn how to program my own game and to stay away from it at all costs. Some very complex problems you are presenting, but definitely ones that need to be addressed by minds greater than mine.

  2. somebodys_kid says:

    I agree! This is a superb series and is rekindling my interest in programming AI as well.
    I play far too many FPS games, and in my estimation, the best combat AI was in F.E.A.R. Those fights never got old, and I eagerly looked forward to engaging my next squad of replicas. The best NON-Combat AI I found was in the No One Lives Forever series (the conversations you could overhear whilst skulking about were in and of themselves wonderful). Those two games had the best discovery and reactive AI to my sneaking (haven’t played Thief so maybe I’m wrong).
    The Infinite ammo quandary has always bugged me, though. Take STALKER, for example. In the early stages of the game, I’m conserving my 25 AK rounds like they were gold coated and some punk bandit saunters up and spits 150 AK rounds at me (with uncanny accuracy as Shamus implied above) before I take him down with my second to last round. He should not have more than 60 rounds when he finds me that early in the game and should ALSO be on single shot mode.

  3. Rutskarn says:

    I don’t think I’ve played a game with truly brutal targeting issues. I mean, I’ve played games with most of the common mistakes above, but never the STALKER-esque games where enemies had far better accuracy than the player.

    When I watch Spoony’s LPs of SWAT 4, however, I see a lot of that. He’ll shift his weight to lean through a door, and his aiming reticule leaps a few inches, to the point where it would literally be difficult to hit something a few feet away. Meanwhile, at the opposite end of the room, an untrained enemy runs out, fires a single round at him from the hip, and kills him instantly. That’s not even an exaggeration–I’m pretty sure that exact thing happens to him a few times.

  4. Henebry says:

    That AI in the last photo isn’t so dumb as you think: Officer Fodder has taken cover in front of the car because he’s afraid that Officer Dumbass will clip him accidentally.

  5. Kyrionus says:

    Enter the Matrix was terrible for the unlimitted ammo milarky! You’d be hiding behind a pillar while the SWAT team empty crazy amounts of ammo at you, and then they die with 7 bullets left… EVERY TIME (then again, EtM had enough OTHER problems that it wasn’t too bad).

    I’m reminded of playing the early levels of Red Faction 2 on Hard, where a single hit tends to leave you down a health kit, and just crouching behind the indestructible terrain (in a Red faction game… There’s somethnig wrong there somewhere) whilst my computer ally stood up next to me soaking bullets with their plot-based invulnerability and emptying their infinte ammo into the enemy until they were all dead.

    One game (well, series) that didn’t do infinite ammo for enemies (although it’s not an FPS in any way) was Baldur’s Gate. The enemy archers would use their arrows and then have to resort to melee weapons, the spell casters would (eventually) have to resort to punching or knifing (daggering?) you etc. How an FPS would cope with all the enemies having to punch you is an interesting question though…

  6. Benjamin Orchard says:

    There are things about the UT AI which both bother me and which amaze me. Of course you have to address WHICH version of UT you are referring to. I’ve been playing UT2k4 most recently, so that’s the version that I’m most familiar with. The bots there do both some amazingly stupid things (in terms of breaking reality for a given difficulty) and some amazingly smart things (again in terms of considering the AI).

    What I think is amazing about the AI is how well it holds up to user modification of the weapons. Having played the ballistic & some other mods quite a bit, it’s pretty impressive to watch the AI handle that nearly as well as it handles the base game. It’s not perfect, but it IS good.

    But there are definitely times when the AI is able to target people when you wouldn’t believe it possible. Which is frustrating on low difficulties. You expect it on high difficulties, but not on low difficulties…

  7. Decius says:

    Regarding the aiming, just have the AI weapons have the same inherent inaccuracy (cone) as the player weapons, add a “inaccuracy” value based on the actor’s inaccuracy, and adjust for target movement speed.

    I’m not sure if this is a ‘targeting’ issue or not, but one of the decisions that need to be made is “do I wait in ambush?”. I can’t even figure out a decision tree for that one.

    1. WJS says:

      Well, a good place to start would be “Am I alerted of the player?”. If a bot is alert, he needs to decide between “ambush” and “hunting” states, favouring “ambush” if the player is heading his way and “hunting” otherwise. While in “ambush”, he shouldn’t open fire on the player, who may not have spotted him yet. If the player approaches close enough to give him a good shot, or starts shooting at him, he should switch to “combat” state.

      I’m sure this could be refined a lot further, this is just top-of-the-head thinking.

  8. Adam says:

    These articles really make me want to play around with some AI scripting.

    Does anyone know of an AI sandbox environment that lets you tweak the AI actor’s script without making your own game first? Specifically I’m looking for something like morrowind that already has the open, sandbox feel to it. Even something 2d that looks like a JRPG but retains the open world with NPC AI would be good.

  9. Factoid says:

    I am appreciative of the situation AI programmers are in with the ammo conservation issue.

    A lot of games are less interesting without ammo conservation for the player, so it would be a game-breaker if all you had to do to stock up on common ammo was catch a well-supplied guard unawares.

    And if the AI had limited ammo, they might be more realistic in how much they spray, and they’d have to have new tactics for when they got low, but what if they run out. In real life a determined foe would then improvise a weapon, or switch to hand-to-hand or melee combat. That would be awesome if a game could do that well, but that’s a whole new realm of AI problems to solve now, as well as new animations and gameplay mechanisms.

    The happy medium would seem to be a bot with limited ammo who never lets himself go completely dry without retreating or switching to kamikaze mode as he’s down to his last bullet.

    Once you loot the corpse if no ammo was left there should be none to find. If he was still loaded it should give you some appropriate reward, but never anything like 15 fully loaded clips, unless the game you’re playing is Serious Sam.

  10. Stringycustard says:

    The new Splinter Cell game is making the fact that the AI only work in regards to your last known position a major feature. It’s implemented in a slightly gimicky way, and I’m not sure if all AI on a level know about your last position or if they have to “tell” each other where you were. Also, it’d be interesting if they aren’t merely restricted to that but can extrapolate possible paths you’ve taken from that point and search those (e.g. last known position was next to a doorway, and before that was 5 foot from the door, perhaps the player went through the door and into the corridor). I realise the extrapolation is a fairly advanced thing to calculate (and might have to take into consideration plausible goals that the player could have in order for one path to take precedence over another).

  11. Somebody Else says:

    I’ve definitely been dinged between the eyes at 100 meters, while hiding behind bushes, in a storm, at night, with a shortbow. It tends to be somewhat less than entertaining when done repeatedly.

    I remember the Heroes of Might and Magic series – turn-based strategy – giving each ranged unit a limited quantity of ammo, and when it was out, that unit would close to point-blank and start thwacking their foes, often with some kind of auxiliary close combat weapon, like daggers or whatever. It was interesting.

  12. Jeremy says:

    Anyone here played Deus Ex? A lot of the NPCs, both friendly and foe (the ones that weren’t invincible anyway) could run out of ammunition. On some difficulties their aim was so appalling that in most scripted fights between UNATCO and the NSF they’d soon finish off the few pistol magazines they each have, their solution being to start hacking away at each other with knives.

    The one in Battery Park is especially noticeable for this.

  13. Eldiran says:

    This is an excellent series of posts, Shamus. I found the anecdote about your old Pacman game especially interesting. Keep it up!

  14. Jazmeister says:

    Here’s an art game I’d like to see: something modelled on FEAR where you could not lose, as the central mission. A game where every effort is made to protect you while hurtling you through the most hazardous situations you’d expect from any other straight FPS. Like a roller coaster.

  15. SireCh says:

    I wonder what the big studios do regarding reusing ai code across games. Do they start the whole thing from scratch? Can you do a library/framework/whatever general enough, so that it can be easily adapted for new needs? I guess that between time constraints, efficiency vs performance and the whole “too abstract to be useful” problem this is almost intractable. Plus, I don’t see studios collaborating to make one ai library that everyone can use.

    On the other hand, say epic is developing gears of war. Some of the targeting code has to be at least similar to the code in the unreal series, isn’t it? It really looks like an interesting area to tackle.

  16. Neko says:

    @Adam: Not sure if it’s exactly what you were looking for, but I remember WolfenQt, a demo of some cool new things added to the Qt libraries, which includes a Wolfenstein-style Nazi soldier powered by a little bit of QScript. If you were really keen, you might be able to adapt that to your needs.

    The neat thing is that you can edit the script right there in the ‘game’ world.

  17. Tacoma says:

    I think it may make an interesting game if enemies dropped what they actually held. I’ve always felt it was stupid that you can never loot your enemies. Dwarf Fortress lets you loot everything down to the individual socks, but your enemies wear stuff of a different size than you, so all you can do is melt down their metal goods and make good Dwarven gear. But at least it’s something. Of course the player will get rich if he can loot his enemies’ guns and sell them … but there are ways to limit that too.

    In the case of looting guns and ammo, the game would have to realistically portray the weight of the equipment. Stuff is heavy! Encumbrance should have perhaps 20 states from Nude to Immobile. At some point running and jumping becomes impossible. It’s just not worth carrying all that phat loot around.

    Or, for simplicity, just have the Diablo-style equipment space and have objects take up more spaces if they’re heavier. Ammo would take up one space per two magazines (one space per three magazines of handgun ammo), grenades one space each, shotgun shells one space per twenty shells. A knife or handgun would take up one space, an SMG four, a shotgun or rifle six. Worn body armor would reduce movement but not take up space, carried body armor would just take up space.

    Second he needs to spend the time to haul it around. STALKER was good about this part at least – if you wanted to move an arsenal around from one camp to the next it would take many trips at full encumbrance.

    Third you need to find someone who wants to buy your stuff. The merchant will not offer anything like full price for the goods. Half is an OK compromise. And if he already has a lot of that type of equipment left unsold he simply won’t buy it. Or he will offer one-twentieth value just to take it off your hands. And he will have a limited stash of money to give you – and unlike Fallout 1 and 2 you can’t just wait 24 hours to let him recharge his full money stash.

    In terms of the AI enemies, ammo conservation seems worthwhile to explore. A group of enemies won’t ALL plink down suppression fire. They’ll coordinate so you’re pinned down just enough. If it looks like they’re all down to just one spare clip, they send a guy to run to the ammo storeroom and pick up some extra.

    Speaking of, ammo should be ridiculously available if you get to a well-stocked armory or shop, yet impossible to find just lying around the wilderness. People at a practice range can easily burn off hundreds of rounds at one session with a handgun. If a compound has 50 dudes with pistols and assault rifles stationed there, they should each carry 3-4 rifle magazines, a spare pistol magazine, and double that amount (4-5 rifle and 2 pistol per man) should be in the armory at absolute minimum.

    Yes this means you can raid the armory and keep them from restocking. But you can’t carry it all. You can hide it from them in the air ducts though! But while you’re doing all that you could have been shooting them …
    You could set the armory on fire after you loot it. But then you can’t come back to stock up again.

    I just think in this case the standard conventions of gameplay do work, but you don’t have to use them. The standard conventions of real life work just as well, but differently.

    1. WJS says:

      It would be nice to see more realistic (as in mechanics, not graphics) games. It’s kind of ridiculous how strong genre conventions are in gaming, held over from when they were originally dictated by technology. And people will actually defend this gamification, claiming that any attempt, any attempt at all to make things make more sense wouldn’t be fun, when the truth is that nobody has ever really tried (or one game did badly, and is them crucified as an example of what happens when you break dogma).

  18. Broggly says:

    Jaz: The problem is the players, perverse as they are, will find some obviously fatal situation to walk into and laugh at the absurdity. Like stand right in front of an enemy who just can’t manage to kill them. Of course given a well designed game this could be fun, with the player trying to find how the character will get out of this horrible situation they’ve set up.

    There’s a reason why predestination makes such great fodder for comedy.

    Tacoma: If you could wear enemy body armour, there’d be all sorts of things to do with the AI. For one, they wouldn’t shoot at you straight away just for acting a bit wierd (unless they’d figured out there was someone in disguise at the base). More likely they’d come up and tell you to stop doing that. Maybe they’d see your face, and realise you’re in disguise (“Hey! That’s not Svenson!”). What would be more interesting is if you could wear face-covering helmets. You might see the enemy get jumpy and have some friendly fire incidents. They might all take their helmets off leaving you “odd man out”. Not to mention the ramifications of taking an officer’s uniform and ordering them around…

  19. Knut says:

    I remember playing one of the first “Rainbow six” games(first one maybe? Can’t quite remember) where the opponents would always miss the two first shots, and then hit with every one after that, regardless of weapon, range etc. Was cool the first two times it happened, but got sort of predicable and dull after that :P

  20. Jennifer says:

    I find it amusing that the game is most impressive when you’re able to out-smart the AI soldiers and blow them away–but in a reasonable fashion.

    It occurs to me that another thing that might be fun to add (and relatively easy) would be to make the AI “afraid” of being shot–if you’re aiming at a given bot that has detected you, their aim should get a lot worse as they try to Get The Heck Out Of Dodge. One of the most “interesting” behaviors in tabletop games is dealing with fleeing or retreating enemies–so much so that I’ve seen party wipes happen when the bad guys decided to withdraw instead of slug it out.

  21. Armagrodden says:

    The second Rainbow Six game tackled the infinite ammo issue. After a certain number of shots, the AI characters have to duck behind something and reload; so the game actually kept track of what each AI was armed with and how many bullets it had in a clip.

    This isn’t to say that they’ve completely overcome targeting issues of course; I’ve had too many experiences in which the bots would round a corner and instantly kill me with one shot from the hip to say that.

    (Also the behavior Knut describes must come from the first game, because it isn’t true of the second.)

    1. WJS says:

      That isn’t solving the infinite ammo problem. Unless they are only able to reload a limited number of times, and the game has a way to successfully handle the case when they run dry, that isn’t even coming close.

  22. Daemian Lucifer says:

    I think that in half life AI tends to shoot at where you were and you can flank them.But I might be wrong,since this situation rarely happens,and it might actually be because of your squad fodder.

    Anyhow,discussing the AI in a shooter is all fine and dandy,but really its not the biggest problem.Are you going to brush up with strategies as well(especially real time ones)?Those are really THE bane of AI.

  23. scragar says:

    What I don’t get is why these enemies are so stupid, if someone is hiding behind cover for 2 minutes I’d certainly never assume that continuing shooting at it is a good way to spend my time, especially if I haven’t seen any signs of life behind the cover in some time.

    I know it’s hard to write something for that, but I assume implementing an ammo limit and having the guards retreat when they get low(and call for help early) would be a good idea, this could easily be countered by giving the player a limited ammo capacity themselves, rather than the strange world of most FPSs where you can carry 10 guns, each having between 20 and 500 ammo, requiring said character to drag half an armoury around when going into battle.
    Having the guards fire for so long, before pulling back to get some more ammo, or find better cover themselves sounds like much more realistic behaviour than firing endlessly at a pillar, or tower of cardboard boxes(which stop bullets how?).

  24. Arquinsiel says:

    In the first two Call of Duty games the AI controlled enemies would rather often skip firing altogether and charge in to beat me to death with their rifle butts. I think this may have been implemented in some form of “if path is clear” and weapon damage priority listing thing. They also had a tendancy to do it to me while I was reloading, which is when I would normally shoot at someone that far away. Oddly enough, that would generally kill me far more often than being shot at would, proportionally speaking.

    1. WJS says:

      That doesn’t sound too bad; “Jump him while he reloads” is a very viable tactic if combat is happening at close range. And shooter combat always happens at close range. That long range bit you’re thinking about now that I said that? Probably only about fifty meters, if that.

  25. Decius says:

    Something that just came to me: Instead of each actor making decisions, create a ‘sqaud’ pseudo-actor. Give the squad access to the information held my the actors in it, and make decisions as a whole. Then pass simple directives to the members (like ‘Move here’ ‘Search here’ ‘suppressive fire here’)

    The information the squad needs includes known enemies, known secure areas, and objectives (Defend, Search and destroy, patrol, ambush, survive).

    The only way I can figure out how to prioritize strategy is to start by making decisions randomly – each time a decison has a ‘positive’ outcome, make it more likely, and the opposite for ‘negative’ outcomes. Run that AI on AI for enough iterations, and you will reach a Nash equilibrium. Then continuing versus a human playtester should eventually produce a fairly believable mix of reactions.

    Leaving the ‘learning’ function in place in the release should result in the enemies learning from the player, as well. I hesitate at including a ‘debrief’ algorithm, where the AI determines what it should have done and does that next time. (Having someone watch the other entrance after the distraction?)

  26. rofltehcat says:

    Targeting…
    reminds me of when we played BF2 vs. bots… 50 bots vs. 5 players, just for fun.
    They often hit very bad, so you normally don’t lose much health and can kill 5 or so without any problem.
    But then once in a while a bot from the other side of the map rolls 2 natural 20s at once and scores a headshot (with a MP from 300 metres), killing you instantly.

    Pretty stupid :/

  27. Jennifer says:

    It sounds like the greatest problem here is using a very simple mathematical operation to model highly complex behavior. While your math doesn’t have to be anywhere near as complex as real behavior, having two variables is much better than one (30%? Come on) and likewise three would be better than two, etc. In a tabletop game, this would be a nightmare, but this is precisely the sort of thing a computer can handle quickly and easily–and once you have enough interacting variables, you achieve what you want, which is complex behavior that is “intuitively” predictable. What I mean by that is that it forces you to use approximations and estimates instead of just exploiting the formula.

    The other half of that equation is using variables that twig off of things the player can *observe*, like whether a bot is moving or not, whether the player is moving, what type of gun they’re using, the range, the visibility conditions, etc. It probably doesn’t matter so much *what* the conditions are as long as they’re easily observable.

    Bleh, I’m thinking out loud and reinventing the wheel here, sorry. Anyway, I will comment that the sine-wave graph thingy is inaccurate, because the “correction” should not be as large as the initial error. People don’t oscillate like that, they don’t overcorrect *exactly* as much as they initially overshot, they tend to split the difference.

    Which reminds me that I read an engineer’s response to an interview question that tackles this very problem. You could get a LOT of variety in a game just by having different enemies use different types of solutions to this problem, with the stupid inaccurate ones using, say, the naive solution while the more accurate ones use more optimal solutions. Add in a check for a “solution reset” (they lost their targeting solution and have to either start over or start several steps back at least) and you’ve got a good start to an accurately inaccurate AI.

  28. Carra says:

    Weird, just started playing FEAR for the first time today. And so far, the AI seems to be quite good.

    And maybe I should push that slowmo button from time to time…

  29. Ell Jay says:

    I want to say thanks, Shamus, for the time you’ve put into addressing this issue on the blog here. I honestly hadn’t given nearly this much thought into how difficult it is to make “enemy” AI believable, and it makes it that much more incredible, in hindsight, to think of games where enemy AI never or rarely made itself noticeable (i.e., Half-Life 2).

    I also understand better now why they seem to make so many games where the “enemy” are zombies or monsters or other non-gunplay entity (come to think of it, Half-Life 2 for stretches).

  30. MelTorefas says:

    I am absolutely loving your AI posts, Shamus. AI in games is something I’ve been into since I started writing scripts for WCIII, and your articles are hitting all the right notes. :)

    “Human players don't shoot like this, unless they're being electrocuted while playing.”
    Glad I wasn’t eating or drinking anything when I read that.

  31. equinox216 says:

    This response is going to wax long-winded. So you’ve been unnecessarily warned, but I feel better saying it.

    As an opening caveat: I’m not a programmer. I am an avid gamer/FPSer (or was, while I had the necessary disposable income and hardware. I have consoles now, and am a little out of the loop because I feel that playing FPS with a thumb stick is like kissing your sister. Not MY sister, mind you: she’s as pure as the driven snow and a paragon among women. I mean YOUR syphilitic dog of a sibling.). And I’m a sociologist to boot, so that amalgam’s where my perspective is coming from.

    You can’t ‘program’ or ‘schedule’ or ‘plot’ or ‘predict’ (and therefore hope to generate) complex response, at least not unless you intend to make The World’s Most Exhaustive Conditional List in place of what should be AI. A lot of the failures in concept being described here, both in general AI thought and “How do I, Botty McBotterson, use this weapon best? But also like a fallible hoo-mon, which I am playing against?”, seem like they arise from direly befouled conditional lists. Some are governed by simple prioritization, some by slightly cross-interactive formulas (when do I shoot or charge? What are the numbers?), but they seem to all eventually run into immersion-breaking or enjoyment-scourging priority clashes, or if you’re unlucky, both.

    I’m going to dip “briefly” into academia, here. My work, theoretical and experimental, is in exchange networks. I build and test networks with the MOST simplified elements (a stripped-down ‘exchange’ consists of “How will you and I share 24 points? Or, will that other guy offer me a better deal? I only have one trade to make, after all.”), because otherwise you can’t pick out what facet of structural power you’re getting data about if you have too many variables in the air.

    Despite this maximal attempt to simplify, you still get very complex results. The more actors you have, the more complicated the whole system (and its output) becomes, regardless of how simple you make each actor’s priorities, because of interactions of the elements: structure and actors. But some things are mostly clear-cut: an actor in a strong position has to truly screw the pooch to not profit, and that’s usually a result of experimenter error in not instructing them properly, or the subject totally not grokking how truly few rules you’re asking them to follow. Like, “Just earn points. ‘Fairness’ to other actors is meaningless in this context.” You’d be surprised how hard ‘fairness’ is to instruct out, and how much it’ll bork your test. (Figuring out ‘fairness’ effects and their cousins is basically an entire subfield.)

    Anyway, with that in your short-term memory, consider how you take those ideas/concepts/problems and then try to make complex artificial behavior. You don’t make the conditional checklist; what you do is try to give the simplest guidelines possible. For ‘modern warfare’ (or if you want to wax sci-fi, ‘alien horde’) AI, that means squad combat. For simplest squad combat, that means roles.

    Say you have an 8-man platoon: leader, riflemen, and specialists. Your riflemen have certain individual engagement priorities, and instructions for how to interact with the environment, support their squadmates, and get an uncertain situation back to one they know how to deal with. Your specialists (demolition, radioman, medic) have similar considerations but skewed different ways: the demolitionist throws grenades/plants bombs, the radioman hides in back supporting the leader and calling for air support, the medic keys on squad injury. They all have [enemy-friend-environment-task focus] programming, but they all pursue those axes of behavior in different ways. So you can come up with “relatively” simple general instructions for a ‘grunt’, with variations for each role. You can even give each instance of a role a degree of variation, like ‘courage’ on high for a medic increases how hard he’ll try to break cover and hand out band-aids, and on low is how close to the ditch he stays.

    And then the leader’s continued existence determines how on-task each person who can ‘hear’ him (hey, radioguy’s down, he’s gotta shout) will stay. And sometimes he’ll ‘yell’: “Fall back,you have NO IDEA what you’re doing, assholes!”.

    Each role is variably ‘good’ at tasks/aim/whatever in their specialty, so medics can’t do much with their pistol but run like the dickens, and riflemen aren’t the speediest but can probably hurt you, and radioguy has the adult diaper and the ‘Pls send napalm thx’ button.

    You get simpler roles to program for, by determining axes of priorities and skills, and by keeping any given role’s focus on their main ones high, but you get fewer concept failures because you give them short lists, and you give them a ‘bug out, support, and regroup’ option.

    Which gives you super-complex squad vs. player interaction (Do I shoot the leader? Can I FIND the leader? I see the radioguy, that coward!) from simpler instructions to each AI actor, and gives you an easier time hammering out environmental interactions because then you just apply the abilities in degrees as role-based ‘skill’, and it doesn’t become STALKER or a Quakebot railgun fight, but it’s still interesting and challenging…

    because you generated ‘complex’ from ‘multiple simple’ instead of ‘multiple complex’.

    Though, for all I know, some programming paradigm has already integrated this approach. I’ll shut up now.

  32. Jennifer says:

    equinox216, having complex squad vs. player interaction doesn’t do anything for you when you’re not going up against a squad. I mean, yeah, that’d be a good approach when you are talking about a group that is supposed to work together, but you have to write complexity-creating AI for individuals, too.

    One way you could have your cake and eat it, too, in that situation would be to have a long list of behaviors that only operate when the unit is solo–when they hook up with other units, behaviors on the list get turned off so that they work effectively with however many teammates they’ve currently got.

  33. The list of prioritised behaviours is often referred to as emergent AI – and therein lies the rub. If you change something about that list, it has wide-ranging effects.

    In other words, fixing the problem that AI isn’t taking cover enough could break the tendancy for AI to advance to forward positions.

    The benefits of that solution are that it takes very little processing power to work out what the AI should do next – just go down the list until you get one action whose conditions match the situation

  34. BvG says:

    This guy writes about the problems and considerations needed to make a human-like quake 3 bot. Very much recommended: Brainworks

    However, no recent posts are on that topic, so you gotta search the archives…

  35. Mayhem says:

    I recall the AI in the Jagged Alliance games being pretty good. You’d get varying levels of alertness, ranging from (ignorant patrol) to (run to the gunfight) but they had morale as well (run and hide if outnumbered) and varied behaviour from (charging bull) when wielding a shotgun to (camping bastard) with a sniper rifle. The AI also had the same rules as the player in terms of action points – they had a limited amount they could do each turn, and weapons accuracy was controlled by the same system as the player – I believe the ai part was in judging what percent of AP to spend on a shot.
    Granted as a turn based sim it operated under very different design ideas to a FPS, but some things are common – if one AI knew where a player char was then so do the rest. If they knew only a rough area they would converge on the site then spread out and search. You could manipulate this by throwing rocks to make noise to expose them too. I liked the balanced feel from having the pc sims and the enemy sims operating under the same fog of war though.

  36. Mayhem says:

    A neat trick could be to make the harder difficulty bots start exploiting the known game flaws for advantage – instead of fixing the flaw, make it work for them. As players find more, you can work them into your code.

    Kind of like Dungeon Keeper – there were lots of ways to exploit the game to make it easier. A fan released the ‘Ancient Keeper’ campaign mod that replaced all the levels with ones that used those very exploits to make it fiendishly difficult. Essentially if you didn’t work out the right way to cheat you had no hope.

  37. toasty says:

    I’m loving this series! Its really informative. I feel for video game programmers now.

  38. equinox216 says:

    [email protected]: I thought that one through, too; I think the ‘role’ selection gives you a template for a given unit type in solo situations, as well. Any given place you encounter enemies, they’re going to have motivations, right? Not every badguy’s a badASS, is he? The rifleman engages, but retreats if it’s taking a beating; the demo falls back but lays traps; the radioguy runs like hell but flails and tries to get help’s attention however he can. Once you figure out your axes, you can create character variation templates. I mean, a medic isn’t going to stick around if he’s the only one turning into a sieve. He’s lousy with his pistol, and he KNOWS it.

    I played TF1, but have only listened to TF2 chatter. Each class’s strengths and weaknesses in teamplay seem to be engineered around semi-unique weapon and skill sets, but that doesn’t mean those strengths/weaknesses only determine their group behavior, does it? Between map objectives and class matchups, choices get made somewhere between the attractors of ‘what you need to get done’ and ‘what you can do’. Your fallback list is achieved by dropping some behavior pressures in the absence of X, which is why a solo medic would run, a solo rifleman wouldn’t stay between you and the leader/radio, etc.

    [email protected]: Emergence isn’t exactly a gaming-unique condition ;) . I do battle with it every time I design an experiment that involves humans. I think where game AI goes not-so-I belly-up is when the hard lists take the place of something more elegant, like a priority system that tallies interactions. After a while of hearing noises, then finding nothing, that guard should reach a tipping point and phone that crap in regardless. After getting shot at with a chain gun, some poor mook with a pistol isn’t going to emulate Rambo; he’s going to run the hell away. You can even make ‘unintelligence’ a PART of AI character development, rather than a feature of terrible prioritization (as manifested through the hardwired checklist that gets enemies stuck halfway between assault and cover, flipping back and forth and doing neither).

  39. Lex Icon says:

    “Ideally they should be using lots of tracer rounds, and when they do decide to miss they should be aiming in such a way that their bullets still pass somewhere in front of the camera or strike nearby objects that will do interesting things.”

    Prototype has a bad problem with this. If you’re running through a firefight between the infected and military then it looks really neat and realistic.

    But as soon as you stop, you realize they only shoot in front of you, or slightly to the sides. And if you’re disguised, they don’t change behavior. Nothing more annoying then being “shot next too” with tank cannons while I’m disguised as a military commander.

    Also, the military will fight to stay in front of me, meaning if I stand in an intersection full of infected and look down the alleyway, the tanks will up and start piling down that way. Change my angle, and they back up and start down the other way. It’s like some absurd puppetry magic.

    There are some other problems, but those are the most glaring relevant ones.

  40. About STALKER: Weapon condition doesn’t affect accuracy, just frequency of jamming. Also when I played, I was much more accurate than the enemies. I engaged them at distances where they had a very hard time hitting them, and I was able to (still missing more than half my shots) pick them off and emerge mostly unscathed.

    “A human would have to shake their mouse very fast to get this effect.”

    “programming proper behavior for a weaponless AI would be very difficult”: No, it’s trivial. Deus Ex does it. They switch weapons (to a melee weapon, often), and if they have no more weapons, they start fleeing.

    1. Shamus says:

      Alex: It’s only trivial if you want a game where soldier mount a suicide banzai charge when they run dry.

      “Fleeing” is not always clear either. Leave cover and flee over open ground?

      1. WJS says:

        I don’t think the decision between “make a break for it” and “curl up behind cover and cry” is that hard, it’s not that dissimilar from deciding when to break cover to attack, and that’s a case that you can’t band-aid by preventing them from running out of ammo and giving them the survival instincts of berserkers. The issue is “if they break, where do they run to, and are players so bound do genre conventions that they will take offence at the enemy running away rather than fighting to the death”. I can easily see players objecting to that, either because they feel cheated out of their kill, or they – *gasp* – might actually feel bad about shooting a fleeing enemy in the back.

  41. John Magnum says:

    Recently, I tried the Crysis demo, just to see what all the graphical hubbub was. (Spoilers: Three years after it came out, its graphics aren’t actually that impressive–there’s a handful of games that’ve come out this year or last year that look just as good. Crysis looks better if you add a bunch of AA outside the game itself and if you add some particle effects mods, but the same is true of Call of Duty or Battlefield or Just Cause.)

    I didn’t even manage to get through its first few missions, because the shooting was so obnoxious. The player is given a tiny handful of ammunition, and most enemies require a couple headshots or half a dozen bodyshots to take down, on Normal difficulty. Therefore, your superhuman engine of destruction must scavenge the corpses of your Korean adversaries and carefully count his shots, using single-shot firing mode quite often. This is reasonable enough if they want that kind of play, although it’s bizarre to enforce ammo conservation on a game where you’re playing that aforementioned superhuman engine of destruction.

    Unfortunately, your enemies have no such compunctions. They will all blaze away on fully automatic fire as soon as they spot you, and they will continue to blaze away until SOMEBODY dies. Then, when you loot their corpse, you find two dozen bullets.

    It was frustrating nonsense. So I spent about thirty seconds thinking “Wow, that foliage looks quite nice,” and ten minutes thinking “Wow, the actual gunplay is really pretty lame.”

    I’m sure if I played more, or had tried the stealth aspects, I would have experienced still more ridiculous AI follies. Blah.

  42. WJS says:

    That decaying sine curve seems a pretty good approximation (when adjusted for 3d, of course), although it would probably be better to implement a simple system that will produce such a shape emergently, rather than try to calculate it directly. You don’t, for example, want the enemy’s aim to consistently wobble around your position regardless of if you’re standing still or sprinting from cover to cover.

    Pick a random vector and apply a force to it each frame, and they should quickly zero in on you if you’re standing, and lag behind you if you’re running. That seems about right for what we want, I think.

  43. Lars says:

    Same rules for opponents, YES. That is why I hate the rubber band AI of the racing games of the past 10 years. If you are driving really good, the AI has always the chance to catch you in the last 5 curves. Grunt.
    That effect was best visible in Juiced. It should have had a great AI, but I played it on an underpowered machine, and AI was calculated after graphics. In a 7 lap race an opponents car rammed the plank and got stuck. I rounded him at least three times. Than I occasionally hit his car and got him free. After that he paced the 3 laps in 30 seconds. Without any physics regulations. From that point on, I just hated that stupid game and this AI mechanics.

Thanks for joining the discussion. Be nice, don't post angry, and enjoy yourself. This is supposed to be fun. Your email address will not be published. Required fields are marked*

You can enclose spoilers in <strike> tags like so:
<strike>Darth Vader is Luke's father!</strike>

You can make things italics like this:
Can you imagine having Darth Vader as your <i>father</i>?

You can make things bold like this:
I'm <b>very</b> glad Darth Vader isn't my father.

You can make links like this:
I'm reading about <a href="http://en.wikipedia.org/wiki/Darth_Vader">Darth Vader</a> on Wikipedia!

You can quote someone like this:
Darth Vader said <blockquote>Luke, I am your father.</blockquote>

Leave a Reply

Your email address will not be published. Required fields are marked *