Human Ingenuity Beats AI’s Machine Precision

By Shamus Posted Wednesday Feb 13, 2019

Filed under: Column 138 comments

My column this week talks about the final game between pro Starcraft II player MaNa and the AlphaStar AI. As you’ve probably figured out from the subtle hints in the title, the human won this time around. My article talks about how MaNa won and gives some historical contextIn this case, me talking about games I was playing 20 years ago counts as “historical context”. to the matchup.

How AlphaStar was developed:

They seeded the system with some of the basics of the game: Building a base, creating units, attacking the opponent. From there:

  1. Make many copies of the AI. Each of these copies is an agent.
  2. Randomly alter the behavior of each agent by slightly scrambling its parameters. Build units slightly sooner. Build different units. Send units to random spots on the map. Attack at different times.
  3. Most of these changes will be nonsense garbage, but a few will be slight improvements. To figure out which ones are best, have the agents fight each other. Part of the reason the team chose StarCraft II for this project is that Blizzard opened up its API to make it easy to connect an AI to the engine that runs the in-game rules. It’s possible to have two agents play an entire 20 minute match in just a few seconds, because you can run the game as fast as you like. The system is deterministic, so you get the exact same result at 100x speed as you do at normal speed. This means the agents can play an entire tournament worth of games in seconds. (Assuming you’ve got a beefy supercomputer to run everything on.)
  4. Out of the myriad of goofball malformed agents, a few will rise to the top. Keep these, and discard the rest.
  5. GOTO 1

Because of this, every AlphaStar agent has experienced about 200 years worth of StarCraft games. Assuming the average game would run for 10 minutes if experienced in real time, that comes to just over 10 million games of StarCraft.

Ten million games of practice, just to be able to play one race against the same race on one particular map using one particular strategy. This is obviously a very brute-force way of developing AI and it doesn’t really reflect how meat-based thinking machines operate. It’s obviously much more like evolution than cognition.

When Humans want to master a task, they don’t try random things. Instead they visualize a goal like “I want to win the game” and then try to imagine what steps they need to take to make that happen. Their guesses might be terrible or wrong, but they obviously don’t need to daydream 10 million games in their head before they come up with an idea like, “Maybe I should build more siege tanks and fewer turrets”.

But that’s the trick, isn’t it? We don’t really know how humans do their thing, even though we’re all humans and we’re doing it right now. AI is an amazing field and we’ve made fantastic strides. Brilliant people have worked very hard and gotten amazing results. But we’ve barely put a dent in the problem.

Ah well. At least we got some fun StarCraft games out of it.

 

Footnotes:

[1] In this case, me talking about games I was playing 20 years ago counts as “historical context”.



From The Archives:
 

138 thoughts on “Human Ingenuity Beats AI’s Machine Precision

  1. Karma The Alligator says:

    20 years ago probably counts as actual history. Maybe.

    When Humans want to master a task, they don’t try random things.

    I would disagree based mostly on if they have access to tutorials or other form of help, and if they’re experienced enough with the task. If there are no instructions, they will try everything and see what works, although less so if they have enough experience to make educated guesses.

    1. Jabberwok says:

      We can apply prior knowledge to new tasks using logic and understanding of systems, even a problem or task that’s never been encountered before.

      1. Karma The Alligator says:

        To a point, sure, but that falls under the experience category (granted, I made it too specific). If you have no idea on how to pilot a plane, you’re not going to get anywhere.

        So in fact it would be a third category that I forgot: how complex the task is.

        1. Jabberwok says:

          Hmm, trying to think of how to apply this comparison. So maybe a human that’s never flown a plane before could think to themselves: “Well, I probably shouldn’t steer the plane toward the ground, as things that touch the ground at high speed tend to die.” Whereas an AI of this sort might think: “Let’s crash into that mountain and see whether it makes things better.”

          1. DerJungerLudendorff says:

            But you could also consider that experience.

            Since the human knows that the ground is hard, and things that go fast and go into hard things tend to break a lot.
            They may not know much about aircraft, but they do know about physics, even if it’s only on an instinctual level.

            1. Seax says:

              I think you missed the point: we humans are able to apply prior knowledge, gained with seemingly unrelated experience, while machines can’t. In the example of flying a plane, if you have more than one chance, you could learn to fly that plane much faster than an AI (counting iterations, not real time), because you’ll probably be more methodical than a random AI.

              This ability of asociation and metaphor, and our ability to make sense (i.e., to amplify structures masked by random noise) are the great human advantage, for now. They are also what machine learning can help us tone down. AI can help us remove “overfitting” models, that we humans are so fond of.

              1. Daimbert says:

                I recall when doing a Cognitive Science AI course that there were models that tried to use metaphor and analogy to learn instead of either rules and inferences or neural nets. I haven’t looked into how well they did, though. Neural nets are really, really “big” right now.

              2. Paul Spooner says:

                I would suspect that NN AI is more guilty of overfitting than humans are. We like simplicity and leisure too much to get stuck counting blades of grass for our definition of a lawn.

          2. decius says:

            A human with zero prior information about flying a plane would sound like a sperm whale in the same position.
            “It’s so big and round… I think I’ll call it GROUND. I wonder if it will be friends with me”

            1. Paul Spooner says:

              Deep cut! From Douglas Adams, “The Hitchhiker’s Guide to the Galaxy” no doubt.
              Though, really, a human with no context for an airplane is hardly a human at all. The geometry of the whole thing, including the controls, all have affordances for human physiology.

              1. decius says:

                Indeed! But we still say that the pro players have “10 years” experience, rather than addressing the fact that they have tens of millions of years of evolutionary training in things like basic pattern recognition.

                1. Ciennas says:

                  Because barring time travel accidents, we all start with that.

                  1. decius says:

                    Yes, and the AI *doesn’t*. It’s got maybe 30 years of human computer science behind it, or 0% of the evolutionary pressure humans have.

            2. Jabberwok says:

              We still need SOME knowledge to make logical connections of course, but the logic is still there. I would expect any caveman to understand that “shiny bird going too fast to land like regular bird like I learn in Caveman Physics.” On the other hand, my understanding is that this AI can’t even apply its knowledge to a different map in the same game. That’s like needing a thousand hours of flight records from the correct model aircraft in order to ‘grasp’ a concept that a human could get from falling down once.

              But as Shamus said, we don’t totally understand what happens inside the human brain. We could just call these AI simulations “thinking” if we want, but that does seem like a stretch.

              1. Blake says:

                “my understanding is that this AI can’t even apply its knowledge to a different map in the same game”

                It wasn’t trained on anything but the one map, but I think the approach they’re taking would be able to work, they’d just need some nodes trained to identify the critical regions of the map or just train it for a week with the whole map pool each season.
                Pro players don’t like unfamiliar maps either.

    2. Sleeping Dragon says:

      The problem being that what a human will try will be based on them making some assumptions, maybe wrong assumptions but still. What the AI offshots will try will actually be sort of random (within given parameters). For example one such offshot might just randomly decide that it will run its army into the enemy base but just keep it walking back and forth inside it never allowing it to attack. Another one will do something like destroy one building then run all the units back to the base, run them across the map again, destroy one building, run back to base. These will be aborted in favour of a more reasonably acting branches and once those are trimmed it becomes a matter of stuff like whether the variant that attacks production buildings first or workers first comes out on top, but at no point does the AI decide that “if I attack workers I will harm their economy” or “if I destroy the production buildings they won’t be able to make more units as I destroy the rest of the base” they just pick the options at random and whichever comes out superior is more likely to be picked as a new seed.

  2. Dreadjaws says:

    AlphaStar had a massive army and was moving towards MaNa’s base to crush him yet again. MaNa flew a couple of units directly into the heart of AlphaStar’s base. These units were not a serious threat to AlphaStar. They certainly would have done some damage, but AlphaStar would have been fine if it just ignored them and proceeded with its plan to crush MaNa’s base. Instead, AlphaStar turned the entire army around and marched it all the way back home to deal with these two attackers. As soon as the ground army arrived, MaNa flew his units away. With nothing left to do at home, the AlphaStar army once again began the march across the map.

    Once the AlphaStar army had left home, MaNa returned with his two units and again began doing bits of annoying damage. In a game between humans, this type of attack is called harassment. It’s not designed to kill the opponent, but instead is intended to distract them and force them to build units to deal with it. AlphaStar was unable to cope with this type of harassment and fell for it again and again.

    Here’s the thing, though: there’s a word for using this practice against the AI, and it’s called “juggling”. It’s a very old strategy that players have used for this kind of game for decades, so I’m actually a bit puzzled that the AI developers didn’t think of finding a way to counter it.

    You think they’d make the AI notice when the opponent is trying to get them away from their base by making them go back and forth. Once the AI figured that was happening they’d simply have their attack force ignore the small attackers and continue to the base, while making a couple new units to deal with the former.

    Granted, players could still find a way to adapt to this (by, say, moving their entire attack force to the enemy base on the side), but still, seeing how old this practice is it should be a given that it would be one of the very first things they’d try to counter.

    1. Mephane says:

      AlphaStar was trained by playing against different copies of itself. So it would first need an AlphaStar copy that actually uses a juggling tactic in order to learn ways to deal with it.

      1. Dreadjaws says:

        Sure, but the starting AI should already be aware of certain strategies. They have to start knowing something after all.

        1. Shamus says:

          The thing is, it’s hard to input something that complex. Harassment isn’t something concrete like a build order, it’s something you can do, in certain situations. The whole point of the project was to feed in some basic games and see what came out the other side of those 10,000 games.

          And maybe they DID have juggling in there somehow, but after 10k games of rapid evolution those features were lost. If juggling is ineffective in AI vs. AI, then the AI using it will die out. Then after a thousand or so generations, that knowledge and the related countermeasures will be lost.

          Really, that’s what these matches were all about. They’d had the game play itself, and now they wanted to see what it could and couldn’t do against people.

          1. Scampi says:

            An alternative interpretation: The Winner AI was the one DOING the juggling but was never itself victim to it, thus doesn’t know how to beat it.
            I think it’s very possible to know how to execute a strategy without knowing effective measures against it.
            Also: Juggling seems to be a very “chaotic” strategy to me, something that would be really hard for a machine to learn/develop left to itself, as it would, I believe, include insane amounts of random chance.

            Another implication would be that there might be another among the survivor agents capable of handling juggling.

            1. DerJungerLudendorff says:

              Perhaps it needs a better idea of damage tradeoffs involving bases?

              Like if I don’t use my units to defend myself but do something else, can I do more damage than I will take, and is that beneficial to me right now?

              Alternatively, how long will it take to create a second army, and is it better to spend resources creating new units or spending the time for existing units?

              1. Daimbert says:

                Neural net AIs are limited to “input-response”. They don’t reason. So it could defeat it if it recognized it as a strategy and came up with an appropriate response — build some defensive units/static defenses — but it can’t really reason out a trade off here. And this one is particularly problematic because the initial harassment could be just harassment, or it could be the vanguard of an actual army invasion. If the AI waits to turn around its forces until it knows that it is, the game could be over if it’s a full scale assault.

                1. decius says:

                  “Neural network” includes humans.

                  The inability to make sophisticated reasoning about what reserves the opponent has and whether the harassers can be ignored (or repelled with less than the entire army) isn’t inherent in the neural network- but it is hard, and 200 years of evolution is not enough to teach it; chordates had about a billion years, and most of them aren’t very good at things like that.

                  1. Daimbert says:

                    Well, that’s the theory, but the lack of semantic information in neural net AIs that humans have access to is somewhat problematic for that idea.

                    The fact that inference engines seem to work the way we normally reason also call the neural net implementation into question.

                  2. Cubic says:

                    “Neural network” includes humans.

                    The NN algorithms are inspired by the human brain but that doesn’t mean they ultimately work in the same way. It’s an analogy, like a ‘tree’ in computer science, which doesn’t have much to do with a tree outside of my basement.

                    1. Abnaxis says:

                      Indeed. Whenever I see someone say a thing like this a cringe a little, if only because I used to think the same way before I learned just how much complexity there actually is in the human neurological system.

                    2. decius says:

                      I wasn’t talking about the algorithms. I was talking about the hardware that they emulate.

                      It’s possible to build, in hardware, an engine that implements a neural network. With a few hundred added complications, that’s what a nervous system does.

                      The nervous system of one worm has been implemented in software, and when connected to a mechanical worm, it exhibits essentially the same behavior.

                    3. Cubic says:

                      Note that the brain is analog while the NN is digital. The brain works by synaptic interconnections and chemistry and electric spikes, the NN works by matrix multiplication. And so on.

                      The last time I looked at simulating C. elegans (which I assume is the worm you mention), they hadn’t gotten very far. But that was five or ten years ago.

                      Brain simulation, like IBMs Blue Brain, seems to require a lot of hardware, possibly including special-purpose circuitry, but has AFAIK not yielded strong results yet.

            2. Matthew Downie says:

              The winning AI probably wasn’t doing juggling, because its tactic appeared to be “make one massive army and send it to attack the enemy”.

              1. decius says:

                I’m pretty sure the AI was seeded with a bunch of real games between top humans. Top humans never juggle, because they never play against people who can be juggled.

                1. Scampi says:

                  I believe to have read something along those lines as well.
                  While a top player never juggles, isn’t it still something that can basically be treated as a kind of harrassment, which I’d assume the AI would be “aware” of? Couldn’t the AI still just build a few new units to try and defend the base or some defensive structures instead of being caught with its pants down drooling over itself by trying to bring the entire army back to the base?
                  I’m not sure about the degree of mobility of SC2 armies, though, which would obviously factor to take into consideration imho. Do Protoss (still) have a quick way to return to base and possibly back to the field if needed?

                2. Zak McKracken says:

                  Top players can’t be juggled (though I certainly can…), but top players still harrass, or drop their opponent’s base when they notice that their army is out of position. So this is indeed something I would have expected the AI to have an appropriate reaction for.

                  …but maybe that behaviour died out during training for some reason?
                  The AI was able to split its army to attack MaNa from two sides, and reacting to a drop by moving your entire army around is not something it would have learned from humans (unless it was watching me play), so I guess it must have been some machine-machine thing. For some reason the AI training sessions must have stopped doing drops at some point.

                  1. decius says:

                    One appropriate reaction to seeing a couple units near your base killing your doods is to bring your army back and make sure there isn’t a bigger force hiding just out of your sight.

                    Harassment that doesn’t use the perfect escape characteristic of flying units will be defeated by that tactic. Meanwhile, it still has the ability to prevent all damage to its stalkers, so it still wins.

            3. Mistwraithe says:

              Pretty sure this isn’t the case – if the winning AI was doing juggling then as they continued to evolve it would have been put up against many more versions of itself which would have also been doing juggling. The winning AI most definitely played against many, many variations of itself. That is really the weakness – as it got better there would have been a definite risk of overspecialisation in that direction.

              1. Scampi says:

                I guess you’re totally right there.
                My point was more in the direction of: We have seen a limited amount of possible current agents, and if any juggling AIs developed at all, they would likely not have been eliminated from the process due to being likely to be regular victors. A thing I keep wondering: is the knowledge from the games afterwards integrated into the AIs? Or are they left to only the knowledge they gained first hand? If the second is true, there just might be no good way to find a valid counter to such measures, if juggling tactics appeared rarely and are practically designed to be fought through by decisive means instead of being paralyzed in confusion.

                1. Sleeping Dragon says:

                  It gets more complicated, if juggling was present within the initial set but the AI developped “always win” strategies against it and so it was sifted out, then later the strategies the AI came up with made it vulnerable to juggling again (or “made it forget” how to deal with juggling in more human terms) would the randomization algorithm be advanced enough to sprout a new “juggling” branch on its own or would it never come up again with different variants just clashing big armies against each other and becoming increasingly optimized in that (superior micro) while leaving themselves open to other strategies?

                  And if a new juggling branch came out, it started winning, then the AI, through the proces of natural selection, reverted to “anti-juggling” strategies wouldn’t it effectively be stuck in a loop of becoming repeatedly resistant and vulnerable to juggling. Or would the variants get stuck in a rock-paper-scissors situation where the “massive army AI” beats the “anti-juggling AI” but falls to “juggling AI” with ultimately a human making an arbitrary call which one to use as future seed? Or provide them all as separate agents hoping they will win against whatever human throws at them but knowing full well they are vulnerable to particular strategies if the human has the time to try them out or lucks into them (which is what I think might have actually happened here).

    2. Daimbert says:

      The thing is that the whole point of a neural net is to NOT in any way code in any explicit rules into the system, and simply let it learn from experience. The agents could have learned how to counter it if one of the agents had stumbled upon it as a strategy and tried it against the others, but if they didn’t or those ones weren’t very successful — potentially because of other flaws in their strategies — then the agents wouldn’t have faced it and so wouldn’t have developed a strategy to counter it.

      All they could do now is have the agents play against that strategy to develop a counter-measure.

  3. John says:

    The problem with the kind of optimization that AlphaStar is doing, indeed the problem with most kinds of mathematical optimization, is that you tend to wind up at a local maximum rather than a global maximum. In other words, you may find a point (in this case a strategy) that produces better results than all other nearby points (minor variations on that strategy) but there’s no guarantee that it produces the best possible results. AlphaStar also has a problem in that they are doing constrained optimization. They aren’t looking for the best strategy; they’re looking for the best strategy consistent with whatever strategy framework they’re using. An AI can’t harass or respond to harassment appropriately if it lacks the capacity to understand what harassment is.

    1. Thomas says:

      It’s worth noting the AI does have harassment strategies – and it did try a warp-in with Mana. If you watch the reply, the AI doesn’t understand the ‘null’ space of the map. It sends flying units to track the flying drop ship unit, and eventually settles on sending its army outside the base to intercept it – except theres part of the map there where flying units can stay and ground units can’t reach.

      The AI gets stuck trying to tell its ground units to attack the flying units in this ‘null’ space. It doesnt seem to understand that that’s impossible.

      The AI team did some really best things to try and avoid local maximums. They don’t select ‘a’ winner, they select a deliberately diverse range of winners. They also alter the objective functions for each agent – some are tasked with winning the game, but some are required to win the game with more of a particular unit for instance.

      It obviously wasnt good enough. It’s worth noting the AI that lost to Mana had 7 days training time, whilst the one that beat him had 14 though. Perhaps with more time the AI will understand dead space and drops better.

      There’s reason to think that – the 7 day AI that beat TLO didn’t understand chokepoints, but the 14 day AI that beat Mana generally did.

      1. Cubic says:

        The AI team did some really best things to try and avoid local maximums. They don’t select ‘a’ winner, they select a deliberately diverse range of winners. They also alter the objective functions for each agent – some are tasked with winning the game, but some are required to win the game with more of a particular unit for instance.

        It would be more convincing with less puppeteering by the team though. But perhaps the ‘self-playing bootstrapping’ idea simply is not as effective for Starcraft as it is for Go and the rest.

    2. Zak McKracken says:

      I think that makes at least some amount of sense, especially since the core of the algorithm is a steepest-descent optimisation:

      The AI evaluates moment-to-moment what to do, and after every game, the decision algorithm is tuned so that in a given situation that occurred in the game, it will do what the winner did. It probably does not have enough parameters to fit them so that it will exactly reproduce all the games it was trained on, so it will just be an approximation. It is well known that there are tons of local optima when fitting a neural network this way, but there are countermeasures, such as “island hopping” (you use steepest-gradient descent, but with multiple starting points, find several local optima and pick the best) etc. …

      I think another factor would be that “whatever the winner did” may not be the best strategy. Especially in Starcraft, there’s a lot of back and forth. Player A can do everything right, while player B makes a few tactical blunders but then wins with a lucky baneling landmine or something similar.

      And third but not least, the “moment-to-moment” thing may be a big factor, too. The AI does not “plan” ahead, does not have a model for how the game will play out. It only regards the current situation and then decides, based on how other games in the database concluded that looked similar at one point. I guess it would be theoretically possible for the AI to include another neural network representing the opponent, to predict the opponent’s next few actions and play the game for a bit in fast-forward while deciding what to do in some situation, but that would require a large multiple of the compute power it currently uses, and it’s already not a lightweight.

  4. Echo Tango says:

    Phew. A couple more years before we’re all enslaved by robot overlords. :)

    1. Matthew Downie says:

      The war against the machines will be won by tricking them into marching their army of invincible terminators back and forth between two points indefinitely.

      1. Scampi says:

        Reminds me of the German fairy tale of “Der Hase und der Igel”, included in the brothers Grimm’s tales, where the rabbit ridicules the hedgehog for being a bad runner and is challenged to a race in turn. The hedgehog’s (identical looking) wife waits at the finish line, claiming to be the hedgehog. The rabbit demands repeated revenge until he dies of exhaustion after 74 lost races.

        1. Lino says:

          The rabbit demands repeated revenge until he dies of exhaustion after 74 lost races.

          Because how else would you know it’s a German fairy tail? Do they cut him up and eat him afterward, or is this the kid-friendly version of the story?

          1. Scampi says:

            Please, enlighten me: What gave it away? The arbitrary number of races? The demand for revenge? Or the moral, wherein the offended cheat is the heroic victor to the reader? Or is it the “insanity”?;)

            1. Lino says:

              Yeah, that and the fact that, as is, it probably wouldn’t make a good fit for a PG-13 Disney movie :D

              1. Nah, that’s WAY too straightforward. What SHOULD happen is that the enraged rabbit should kill the hedgehog wife and nail her up over the door to his house. Then the hedgehog sneaks in at night and takes her down, buries her in the forest, and pretends that he brought himself back to life. He then offers to teach the rabbit this trick, culminating in the hedgehog convincing the rabbit to murder his entire family and then kill himself.

                Then the hedgehog skins the rabbits, takes the skins to town, and convinces a merchant that they’re magic skins that will turn rocks into rabbits by having his friend the turtle pretend to be a rock, and then start moving around when the hedgehog places the rabbit skin over the turtle.

                The merchant then takes the rabbit skins and tries to sell them to the King, but he’s unable to get the rock to start moving around, so the king executes him.

                The hedgehog and turtle then get married and move into the wealthy merchant’s house.

                1. Droid says:

                  Simply beautiful! Ah, fairy tales used to be so much better in the olden days…

                2. Shamus says:

                  Disney needs to adapt this as soon as possible.

                  1. Hector says:

                    Throw in a song where a hedgehog princess dreams of escaping her dull life of luxury, education, and loving family for adventure.

                3. Scampi says:

                  At the very least, please do a full version of this tale for me to read to my own children:D

        2. RFS-81 says:

          Damn, those fairy tales are always way more *grim* than the versions I remember being told as a kid.

          1. Syal says:

            If the moral isn’t written in blood it’s not a proper moral.

        3. shoeboxjeddy says:

          Which was really mean of the hedgehog, since we all know they can “really move”. But I guess this was inevitable, since they also “have an attitude” and this is the sort of sassy trickiness we should come to expect.

          1. tmtvl says:

            But were they blue? That’s the real question.

            1. Nimrandir says:

              If only the rabbit had looked for the telltale chili dogs.

            2. Sleeping Dragon says:

              But… but it’s the red ones that go faster…

              Also, who looked at a hedgehog and thought “this needs to be more sonic”?

  5. Daimbert says:

    Ten million games of practice, just to be able to play one race against the same race on one particular map using one particular strategy. This is obviously a very brute-force way of developing AI and it doesn’t really reflect how meat-based thinking machines operate. It’s obviously much more like evolution than cognition.

    Deliberately, in fact. In the olden days when I was going AI the approach was even called “Genetic Algorithms”.

    The key factor here is that these systems deliberately don’t include explicit reasoning or semantics in them. So there’s no where inside the neural net where it stores or reasons from specific beliefs. Without that, it’s really hard to get any kind of theory of mind. All it can do is react to an input or stimulus, which lets it often seem to SIMULATE reasoning, but there’s no way for it to do it or show its reasoning. There are many cases where a neural net comes up with a surprising but ultimately right answer and the people using it spend lots of time trying to figure out what the reasons or things it “noticed” to come to that conclusion.

    There ARE AI systems that explicitly reason and learn from reasoning (inference engines, expert systems, and probably some new ones by now). These tend to be great at reasoning tasks and terrible at anything that isn’t. By contrast, neural nets are great at pattern matching and terrible at reasoning problems. So perhaps AI has gotten too enamoured of neural nets and needs to look at the benefits of reasoning systems again.

    My personal theory is that the best AI is going to be something with an inference engine front-end and a neural net back-end, where the inference engine learns what problems are best to offload to the neural net.

    1. Hector says:

      Would you say these are more like iptimization algorithms instead of an AI? Because it sounds a lot like a very complicated program fundamentally solving Travelling Salesman problems rather than making decisions.

      1. Daimbert says:

        There’s entire fields and at least one entire branch of philosophy dedicated to figuring out if these things are actually intelligent or not, starting from Searle’s Chinese Room and going on (and on and on and on [grin]) from there. The argument against neural nets is that it doesn’t actually know anything and only reacts to stimuli, which is why you can actually hook it up to a completely different set of inputs and outputs and it might actually work, or fail miserably. The argument FOR them is that they loosely model the structure of our brains, and so if they aren’t capable of being intelligent based on that structure it seems like we can’t be either. But I agree with you that neural nets don’t actually seem to make decisions, which is why I tend towards the inference engine model.

        But I just ordered some books on AI and so might find out what this “Deep Learning” thing everyone keeps talking about is.

        1. Geebs says:

          Arguably, the major point of commonality between neural nets and actual neurons is that they both begin with the same letters.

          Neural nets are basically just very good at interpolation, as a function of their ability to weight outputs in response to a complex input, right? It’s not actually “learning” any more than, say, a regression model does.

          We can judge intelligence when one starts extrapolating wildly on insufficient data, and then gets cross when somebody corrects it.

          1. Cubic says:

            We will not consider an AI intelligent until it starts trolling the comment section. The second-level Turing test.

            1. Zak McKracken says:

              I think trolling should be easier for an AI than for a human … feed a Markov chain generator with a large enough number of real comments, then let it loose. It’ll generate things that look like comments but don’t make too much sense, people will try to point out the flaws in the reasoning but it will happily go on spouting nonsense … I think that should work nicely.
              For added flavour you could selectively pick troll comments to train on, but I don’t think that would even be required.

              1. Cubic says:

                You have a point there, OpenAI had a press release in the last few days about building a convincing fake news generator. For some reason they’re not open sourcing it … I guess that means the comment section is a solved problem.

        2. Sleepy the Bear says:

          For what it’s worth, my understanding is that deep learning is just neural networks stacked on top of each other.
          Now the field might do clever things with activations, network architecture, and how to speed up training.
          But it’s fundamentally just neural networks with a cool marketing label.

          The Deepmind blogpost gives some details on the actual architecture, where it sounds like they might have implemented
          something similar to your idea about the frontend policy on top of some neural networks trained via reinforcement learning.

          The classic reinforcement learning approach is optimizing a Markov decision process using discounted rewards to determine the optimal play. Its basically solving the Bellman optimization problem, but with neural networks being used to transform screen inputs, to generate sequences of actions, and choose an action.

          All of these algorithms are glorified pattern matching devices. Make the networks big enough and deep enough, and it can in principle match or act on anything. But it’s currently not really an artificial general intelligence. The question is, at big enough scale, does that distinction matter?

          Out of curiosity, did you have any favorite references on those more classic AI approaches?

          1. Cubic says:

            I would recommend Russell and Norvig’s book for a solid introduction to the ‘classic approach’.

            https://www.amazon.com/Artificial-Intelligence-Modern-Approach-3rd/dp/0136042597/

            However! Note that the high-level approaches with inference and expert systems and all had their shot and basically failed back in the day. That’s why there was an ‘AI winter’.

            Modern NNs have after all considerably advanced the state of the art, that’s why everyone’s excited about them now. (Having many orders of magnitude more computing power and data to apply also helps.)

            1. Daimbert says:

              Neural nets have been around and potentially exciting people for decades now. They seem to have become hot again, but it’s still to be seen if their detriments will crop up again …

          2. Daimbert says:

            Thanks for that. It’ll be interesting to see if it’s something new or a rebranding of something I’d already heard about.

            I unfortunately don’t have good references for the classic models because any sources that I might have had are WAY out of date …

          3. Zak McKracken says:

            All of these algorithms are glorified pattern matching devices.

            That is an important thing which people often overlook. All that neural nets can do at the moment is “for any X find me the correct Y”, given a ton of training data — and they can still go quite wrong, either because they didn’t have enough data for certain situations, or because the data itself has some bias.

            Make the networks big enough and deep enough, and it can in principle match or act on anything. But it’s currently not really an artificial general intelligence. The question is, at big enough scale, does that distinction matter?

            I don’t think we’re nearly at large enough scale, but I’ve been listening to a ton of YANSS, and it would seem as if a lot of our own brain’s function is also just reflexive pattern matching, paired with a reasoning engine which spends most of its time coming up with justifications for what we just did, then convincing us that we had a reason and a plan.
            The thing that appears to set humans apart is that some of the time (really not often!), the reasoning engine will override the reflexes, or construct a story which brings our actions in line with some thematic arc, and that then creates some amount of coherence between our emotional state (which influences reflexive actions), our conscious and thoughts and our actions.

            Ignoring the fact that artificial NNs are still miles behind our own brains, they could do similar things when paired with reasoning engines and a few other things yet to be determined — but they would still be machines programmed to achieve some goal. If you want to generate an artificial person, you’d need to have some objective function in there which relates to the AI itself, and which can be changed by the AI itself. Aaand then you’d need to set it up in a way that this objective function actually makes the AI do sensible things.

            … I’d say the bigger risk is that banks, insurers and states (also Internet companies) are trusting NN way too much already and too many of their decisions are already made by “AI”, be it NNs or rigid decision processes (“computer says No”). We are already ruled by machines, way more than I’m comfortable with.

      2. Majromax says:

        > Would you say these are more like [o]ptimization algorithms instead of an AI?

        The training process for a machine learning algorithm is an optimization problem, against a loss or error function (like “don’t lose”). For continuous-parameter models like neural networks (or “deep learning”), the entire training regime is a large-scale gradient descent process, with regularization and metaparameters chosen to attain the fastest-possible learning rate while avoiding local minimums.

  6. Owlbear says:

    This is actually the exact same strategy I would use on the AI in Warcraft one, and I’m quite mediocre at these games.
    I wonder if AI will always have these kinds of blind spots, even when it reaches something like real cognition.

  7. Gargamel Le Noir says:

    That’s nice but if Alphastar won 10 matches before, isn’t it already allowed to invade Earthrealm?

    1. DerJungerLudendorff says:

      Only if all Earth armies can only communicate through written messages, and are only allowed to use 18th century flintlocks.

    2. Scampi says:

      Right, but you fail to notice: This is the first match in the series of tournaments, where Mana tries to take control of the combined realms. And once Mana is at its peak, you don’t want to live in the sixth world anymore.

      1. Nimrandir says:

        Does this make Serral Johnny Cage? I’ve never seen him wearing sunglasses.

      2. Hector says:

        The Sixth World?

        Just remember: Shoot straight, Conserve ammo, and Never deal with a Dragon.

        1. Nimrandir says:

          Unless you’re playing the Genesis game, in which case you have to deal with a dragon.

          1. Hector says:

            (LOL, but then, that’s Shadowrun for you. The thing you absolutely don’t want to to do is the only option you have.)

    3. Nimrandir says:

      That’s if it wins ten tournaments. Since it hasn’t even been entered into a tournament yet, we’re safe.

      1. Gargamel Le Noir says:

        Right, forgot the exact rules of Starkraft Kombat! Thanks!

  8. Joshua says:

    So, the basic flaw of these AI discussions is that the AI is incapable of “learning”. Who else thought of WarGames after reading this article?

    1. Daimbert says:

      Which is a bit of a misleading way of putting it. The AIs can learn, but what they can’t do is adapt on-the-fly and in-game. For that, you’d have to turn on their learning algorithms — that they used in the thousands of games in the initial training set — which wouldn’t help them in-game and in fact even between games wouldn’t help much because neural nets need lots of repetitions to guarantee that they learn a reasonable response.

      An inference engine could do that by learning a new rule/response, and learning it faster than a neural net typically will, with less risk of overwriting previous strategies. But it then needs explicit rules for what it does.

      1. Scampi says:

        I don’t like the implication this would have for using the same method to teach the AI things other than “kill at top efficiency”.
        Can anyone imagine an AI learning such a thing as, for example, surgery by this method? Or, as it was brought up, piloting planes? Nursing children?

        1. Nimrandir says:

          Hopefully the design teams know better than to use Surgeon Simulator and NES Top Gun for the training exercises.

        2. Matthew Downie says:

          Piloting planes via neural networks actually sounds possible, because flight simulators exist. After crashing ten million times (in simulation), it would probably know what to do well enough not to crash a real plane.

          Surgery is harder; you’d need to simulate a human body for it to work with.

          Childminding is harder still. You’d need to be able to simulate children. Then it would have to spend millions of accelerated hours of time randomly experimenting on these children to see what gives the best results. But simulating children is a much harder AI problem…

          1. decius says:

            Flight simulators exist- and they might even be accurate enough simulations that the AI somewhat learns how to fly a plane, rather than only how to fly the simulator.

            I don’t know how well flight simulators model things like low-level wind shear, gust fronts, wake turbulence, or any of the other things that make flying hard.

            1. Matthew Downie says:

              Landing gently is the only difficult bit of piloting, as far as I can tell.

              I reckon I could make an AI for a completely autonomous passenger airplane. It would have a success rate of, say, 90%. That’s got to be enough, right?

              1. Scampi says:

                As long as it means that’s how much of the plane will still be functional after landing, I guess it’s about par for the course by my experience.
                If we’re talking passenger survival, we might be in a bit of trouble. If we can concentrate the failures among a few select airway companies, we might be lucky.

              2. decius says:

                If you can make an AI that can understand verbal ATC commands in a controlled environment with 90% accuracy, market it as an improvement to the FAA ATC training sim.

                You don’t even need to be able to control real planes or understand nonstandard phraseology or adjust for errors in instructions, just make the simulated planes do exactly what they were told to, instead of being instructed to “turn right at delta and contact ground” and deciding to take off, proceed directly into the downwind, and stop listening.

              1. decius says:

                “Autoland” isn’t an AI feature, it’s a hardcoded algorithm that uses information from the ILS glide slope and localizer (radio signals that tell the avionics how far they are displaced from two geometric planes)
                the glide slope is a plane at a small angle from horizontal; if you at the correct altitude on the correct final approach course, you are near that plane.

                The localizer signal plane is a vertical plane down the runway centerline; again, if you are on a correct final approach course, you are very near that geometric plane.

                Autoland just adjusts the control surfaces of the aircraft in a predetermined manner that should keep the aircraft on those radio signals. Being a computer, it responds faster than a human to changes. Being a computer, it also can’t realize if the radio signal it is following is being distorted by something (for example, in good weather an airport vehicle might drive through the localizer antenna chasing a piece of lightweight debris blowing across the airfield; that can cause interference with the analog signal, and any interference would cause aircraft instruments that were already on course to indicate that they were off course. A human pilot in good weather conditions would either just not notice their instruments saying they were off course, or notice and ignore them, because the pilot is looking out the window at the runway to determine if they are on course. (In bad visibility, vehicles are prohibited from entering areas where they could interfere with those signals while aircraft are using them to navigate).

                Yes, there’s actually a procedure for ATC to remind a pilot who has told them that they intend on using Autoland in good weather that somebody might be interfering with that signal. Because if Autoland receives a radio signal indicating that it’s too high, it’s going to descend regardless of the distance between it and the ground- if the signal is working as intended and suffers no interference, autoland will descend and hit the ground on the runway.

                1. Cubic says:

                  Depends on what you put in the term ‘artificial intelligence’. Many a researcher has complained that once you write down how it’s done, it’s no longer considered ‘AI’. Autoland can apparently do better than a human pilot in bad weather, so it’s already kind of neat. I’m sure it can be improved.

                  (Bonus remark: it also points out one of the problems with current AI approaches — your pilot can’t train by landing millions of planes, but has to learn much quicker than that. I think this is known as ‘one-shot learning’ in the field.)

            2. beleester says:

              I doubt anyone’s tried it for flight simulators yet, but a common tool for training an AI in simulation is to randomly shake up its environment so it learns what to do when things don’t go as expected. Suddenly raise the force of gravity a little, push it off balance at random times, randomly drop controller inputs, etc. And then when you put it in control of a real robot arm and it ends up slightly out of position because the motors are a little dodgy, it’s like “Oh, I’ve seen this before” and it can correct what went wrong.

            3. Zak McKracken says:

              I just imagined the AI finding a glitch in the flight simulator that lets it skip to the landing immediately, learning to exploit that and then, on its first day in a real aircraft, pressing some random combination of buttons while frantically twitching the thrust lever around, forever, because that totally worked in the simulator!

              1. decius says:

                “Set throttle to -2147483648
                Reduce throttle by 1
                You have arrived at your destination.”

          2. Zak McKracken says:

            As others have said: All autopilots are using static algorithms, no neural nets. The first reason is that neural nets are too new, and the second is that you cannot prove that a neural net isn’t going to do something stupid in some situation, but for a simple software algorithm, you can.

            I know someone who was developed a NN to use in the control of an (aerodynamically unstable) small drone, as a student project. Don’t know if they actually used that; it was more a case of “could this work”. It can definitely work but it would not be a good idea to use it on a passenger plane. Those have very stable behaviour, with only the exception of a few fringe cases, which very rarely come to attention because even flight simulators don’t know about them until they happen in reality — that means you couldn’t train a NN on them as you wouldn’t have data that contains them. And a stable algorithmic control law is usually much better to have than a NN for which it is impossible to predict how it will behave if something unexpected happens.

        3. Thomas says:

          Decent AI at the moment are dependent on having masses of high quality data. They’re training AI which can make good medical diagnoses because we have a ton of data in that. It uses trial and error on the data, not the patients.

          Similarly Amazon algorithms make better book recommendation systems than humans.

          But it soon becomes hard to have enough good data. The great thing about games, is you can have the AI generate as much data as it needs.

        4. decius says:

          If you can simulate a task and measure success, this type of AI can learn it from the simulation.

          If your surgery simulator differs from actual surgery, or the success condition of the simulation differs even slightly from your actual success condition, this type of AI should not be expected to learn about reality from the simulation, even if a human would have.

    2. Scampi says:

      My respect for having the best name to make this comment, btw.:)

      1. Joshua says:

        Ha! I guess it was one reason for that movie sticking around with me in the early 80s.

    3. Matthew Downie says:

      The AI is creating an algorithm for playing Starcraft. The algorithm isn’t capable of learning; it’s the process that creates the algorithm that learns (or pseudo-learns, or whatever), because it’s ‘imagining’ millions of games of Starcraft, and then using that ‘thought process’ to create a set of rules by which you can win at Starcraft against most opponents.

  9. kikito says:

    > they obviously don’t need to daydream 10 million games in their head

    We don’t daydream 10 million games, but we do still traverse a decision tree. We are much better at pruning our tree though: we have *context*, which the computer lacks.

    Computers can get context from two places. One is the programmers; they can hand-code some of it in by putting rules like “if you want minerals, create more workers”. The other is brute force: trying millions of random things and “seeing what sticks”. Even when doing that, “seeing what sticks” is still defined by the programmers, so that’s still partially hard-coded.

    > Ten million games of practice, just to be able to play one race against the same race on one particular map using one particular strategy.

    The human, on the other hand, needed a *lifetime* of sensorial stimuli (plus nourishment) to be able to extrapolate the context of the videogame :). The machine actually has the upper hand in “time since creation”. I’m not sure about “resource cost”.

    1. Xeorm says:

      On the other hand, if we assume each game on average takes 10 minutes (which is probably too quick, but let’s roll with it) then 10 million games would take ~190 years to go through. The computer did it very quickly due to computer shenanigans but the player’s still doing it much, much faster.

      1. Thomas says:

        On the other other hand, the level of quality and amount of sensory data we receive to train our brains though (never mind evolution) dwarfs the amount of data contained in 10 million games of starcraft.

        I wouldn’t be surprised if staring at an autumn forest for 10 minutes uses more information.

        1. decius says:

          Human eyes max out at around 10-20Mbit/sec. https://www.quora.com/What-is-bandwidth-of-optic-nerve

          Not all of that is going to be useful data.

  10. Leonardo Herrera says:

    Our brains *do* have the benefit of millions and millions of test iterations. It’s called evolution, and guides most of the basic functions of our brain. Our brain has wired-in a lot of the basic abstractions of reality (otherwise, Hellen Keller would have never “learned”). Our brains have way more advantages than any current AI in this matter. Of course, AI can do this training way faster than us (an AI “generation” takes fractions of a second instead of many decades as is in our case.)

    1. Adam says:

      Its also, possibly, what “dreams” are. Though again, maybe that’s because we’ve evolved to be able to dream.

      1. Paul Spooner says:

        Not sure what you’re referring to, but my best understanding is that dreams are the subconscious coming into view of the conscious. That is, it’s the part of us (certainly across our personal history, and likely across our heritage, and possibly even all humanity across space and time) that is doing automatic processing coming into view of the part of us (individually, this time, probably) that is paying attention. The subconscious itself probably does a lot of testing, as does the consciousness in imagination and, to come full circle, playing games.

  11. Syal says:

    Most hand-coded AI today — the kind you find in commercial video games — have the same problem. The AI can’t learn from its mistakes because that would require it to somehow rewrite itself.

    Not sure if that counts as a problem with built-in videogame AI; you want the machine to be beatable at every level, so exploits for the super high difficulties are more of a feature.

    1. Ander says:

      Depends on your goal. AI could be seen as a simulation of various levels of human play, in which case the discoverable exploits aren’t features per se. They might be acceptable anyway, depending on how the AI performs otherwise

  12. Daimbert says:

    Damn it, Shamus, now you’ve made me want to get back into doing AI again, as if I don’t ALREADY have enough projects that I never do!

  13. decius says:

    Wait, were the AlphaStar agents genetic algorithms or reinforcement learners? I thought that, like AlphaGo, they were reinforcement learners, but your initial summary is of genetic algorithms.

  14. Joe says:

    “Mother nature worked on this problem for billions of years, and as far as we can tell humans are the best she could come up with.”

    God, that’s a depressing thought.

    1. Paul Spooner says:

      Is it? I think humans are pretty great. Not alone in that either:
      https://youtu.be/VNR6cbq7iyk

      1. Syal says:

        I think humans are pretty great.

        I know I am!

        1. madoni says:

          I’m pretty great, but i’m trying to lose some weight already, ok?

      2. Scampi says:

        I’d ask you to convince me, but I sure won’t hold my breath.
        And while I think Peterson is right in a lot of ways, even in a lot of ways that are really uncomfortable for me, I could never agree with this specific assessment.
        Not even with Picard.

        1. Shamus says:

          Whoops. Someone dunked on Jordan Peterson in the comments. I know just enough about the guy to know he’s really controversial in some circles. I deleted the comment for bringing him up. It wasn’t until now that I clicked on Paul’s link and saw Paul was the one who brought him up. So… sorry to the author of the deleted comment.

          I acknowledge this is an interesting topic and it’s related to the discussion we’re having. But some people are just too hot for this site, discussion-wise.

          And for those worried this shows some moderation unfairness on my part:

          1) I had the same policy for Anita Sarkesian.
          2) Yeah, I’m probably biased. Look, I do my best to make sure everyone feels welcome and like they can come here to talk about videogames without having their anger buttons pressed. It’s a hard job, it doesn’t always work, but I do what I can.

          Let’s drop this topic. Thanks!

          1. Scampi says:

            Well, Shamus, thanks for letting us know.

            I, for my part, didn’t know the man was persona non grata on your behalf by himself. Sorry for reacting to Paul’s link from my side. I won’t talk about it anymore, as I agree he makes people angry and uncomfortable (as I said: he makes *me* uncomfortable, and I agree with him on some if not many things. I don’t care enough about him to spend any more time listening to him than necessary, though, where necessary means: “I’m reading news and for some reason, he comes up as a topic.”

        2. Syal says:

          (Going to assume that “this topic” is referring to Jordan Peterson and the moderation policy, and not the underlying “how half is the glass” topic.)

          So let me tell you about the time I talked myself out of a good mood.

          Underway in the Navy, I lost my ID; I had it in my shirt pocket like an idiot, leaned forward to clean a wall, and it tumbled out and vanished. For those of you who don’t know, if you don’t have your ID, you can’t leave the ship. And when you’re underway, you can’t get another ID for about a month, just because of how the mail works. So all of a sudden I’m boat-locked; no more seeing foreign countries, buying tourist trap trinkets, drinking and subsequently throwing up in new and exotic locales. Just sitting on the ship, watching ship movies (which are going to be Shark Boy and Lava Girl on repeat because the guy who picks the movies got pissed about people constantly requesting ones he didn’t have).

          About an hour later, someone hands me my ID. It didn’t fall in the ocean, it just landed on a lower deck somewhere. Suddenly I’m once again free to see the sights and drink the smells and buy that tourist towel they sell in every country in Europe, with the dragon standing on the pile of treasure. And I am ecstatic.

          So after I calm down a bit I start thinking about how weird that is; I started the day with my ID, and I ended the day with my ID, and yet I’m really, really happy about having my ID. When you think about it, it doesn’t make a lot of sense.

          Well… I kept thinking about it. How does it make sense to feel good about losing something and then finding it? If anything, the temporary deprivation should make you feel worse. That would make sense. I should feel worse, not better. Wouldn’t that make more sense? And so, having convinced myself of the logic of feeling worse not better, my good cheer dissipated.

          Obviously the problem was not in my feeling good, but rather trying to rationalize it with a faulty metric. Instead of abandoning the framework when it didn’t fit the facts, I shoved the facts into the framework until they warped into what I would expect, and the day was worse for it.

          All that to say; if everything looks dark, it may be your framework casting shadows.

          1. Scampi says:

            Well, if I’m to judge, I’d say you never knew how much you’d miss your ID until you needed it and it was missing.
            Now compare this to my “experience”: Imagine you’d lost the ID, realized how much you needed it and never gotten it back or received a new one because bureaucracy failed to deliver it to the correct address.
            That’s how it would likely happen if I had lost mine. Or maybe I’d get an ID at a point when it was no use to me anymore.
            Maybe people actually ARE great…they just seem to forget it when dealing with me.

            1. Syal says:

              My current theory is happiness is based on rate of change from a comparison point. No ID was compared to having an ID, and caused headaches; then Yes ID was compared to No ID and caused elation. The default comparison is your ordinary state of things (or someone else’s ordinary state of things), but that doesn’t have to be what you use.

              The old half-full/half-empty example is directly related to the result; it’s based on whether you consider a full glass to be perfect, or merely adequate, so a half-filled glass is either halfway to perfect, or significantly under competency. It also implies the direction forward; “I only need to fill half this glass” versus “this glass will be empty in half the time a full one would take”. You can improve your outlook by imagining an ideal outcome, and then comparing the situation’s similarities to it, instead of the differences from it; round up instead of down.

              and never gotten it back or received a new one because bureaucracy failed to deliver it to the correct address.

              So the framework shadow is “if it’s sent to the wrong address, no one will correct it.” That brings up another, much less illuminating ID story; my new driver’s license was expected to arrive months earlier, so I had to call the State’s main DMV office and endure 20 minutes of the worst hold music I’ve ever heard (some squealing saxophone thing that I assume was intended to make people hang up) in order to get them to send me another one, but then they did and it arrived like it was supposed to and everything was fine.

              If you know losing an ID will create significant problems, you can mitigate them ahead of time. Apart from just keeping it somewhere it won’t get easily lost, you can stockpile supplies so you don’t need to get them ID-less later, spend enough time around people in ID-related jobs that they’ll recognize you by sight. You can even order another ID ahead of time as a spare; you’re probably not supposed to have multiple copies, but if the ID is that important and getting a new one is that much of a crapshoot it’s more a choice of which rules to break so break the ones that make life easier. Your outlook can improve just by assuming problems are within your power to fix. (Even if they aren’t, assume they are and you just haven’t hit the answer yet. Worst case, you’ll end up where you’d be starting otherwise.)

              So, like… assume everyone is great, and that you just have to discover the ritual to unlock their latent greatness.

              1. Scampi says:

                I appreciate your effort in explaining this to me.
                Sadly, part of it is already known to me-the part about the classic full/empty glass. I view it as directional and for people who claim “half full” was the optimistic version, I add “still” as a qualifier, and suddenly the direction is reversed. I don’t really care about the metaphor.
                The other half is really nice, but it seems to assume I’m a blank slate with no previous experience of living among humans.
                My experience is a mix of impotence (my possibility to influence results ahead is basically 0-even preparing for the worst case means only being ready for how things will play out) and the knowledge that people will go out of their way to make things as uncomfortable and annoying to me as possible.
                From lying to my face in a way I can only find out to be a lie once I’m out of reach (wasting my time to research things I specifically asked the person tasked with presenting me with information for) to making decisions that everybody except me profits from to making me take tiny stutter steps to a target inches away from me so the solution of obvious and in theory easily solvable problems takes the maximum time possible (but only for me) or stays entirely out of my range.
                Negativity on the scale I exhibit is not something you’re born with. It’s thrust upon you by people around you who decide you are just someone they decide to abuse because he’s there.
                So…I guess I was wrong about my previous idea of what would happen to my ID in the example. It would not just get lost. It would be sent to me, get misdelivered, and afterwards deposited at a place I wouldn’t be able to reach without a valid ID. That sounds more like it.
                It’s the kind of hurdle that “could” in theory be defeated, but the specific problem at hand would make the item useless if I had the solution at my disposal.

                From my perspective: everyone I ever suspected of having the potential for greatness prove to me especially how talented they were in screwing me over. Optimism consistently bore rotten fruit, while negativity at the very least always allowed me to realize some people were not entirely as useless and scummy as I had suspected them to be in the first place.
                Had I to chose, I’d pick negativity anytime, since in my experience, it’s also the necessary result of optimism.

                1. Syal says:

                  Particular advice would need more particulars, but the immediate answer is “work somewhere else”. Secondary answers include including your boss in all information requests so you can maybe get the liar fired, and finding out why your stuff isn’t getting the support the other folks are. And if it’s why you think it is, work somewhere else.

                  For prior bad experiences; a punch to the eye will swell it closed, not open. First thing to do is assume there’s something to see besides punches.

                  Last story I’ve got I think; a blind lady with dementia. Her family would put her dinner in front of her, and she would forget it was there. Then she’d tell them she was hungry, they’d tell her her food was in front of her, and she would adamantly declare there was nothing there. Made no attempt to see if there was food, just assumed it was a trick. Nothing to do about it but wait until she changed her mind. She was the one going hungry.

  15. Majromax says:

    > Ten million games of practice, just to be able to play one race against the same race on one particular map using one particular strategy. This is obviously a very brute-force way of developing AI and it doesn’t really reflect how meat-based thinking machines operate. It’s obviously much more like evolution than cognition.

    This is a fairly generic problem for AI research. Humans can lean a lot from one or two examples, and we learn about as much from failure as we do from success. Machine learning algorithms, however, tend to require massive training databases to approach human-like capabilities.

    Overcoming this problem is a major challenge in machine learning research. One approach here is transfer learning, where an AI is first trained on one task (like image classification) and then re-trained to do something else only peripherally related. The hope is that some of the more fundamental “skills,” like texture recognition, can be shared – in the same way we don’t first have to learn to see in order to figure out Starcraft. However, it’s still a long way from that to understanding (say) a Day9 instructional video to better-optimize a build order.

  16. manitoba says:

    I wonder what a human brain would do under the same conditions that AS has.

    Let’s say, for instance, that i invented a game called Paxglaw and you has to play it. But i cannot tell you what the rules are because we cannot talk, i can only tell you if you won or lost. This game is different from anything that you ever heard, played or even imagined before, even the goals cannot be compared (you cannot say “ah, it’s like chess but with a bit of wrestling” nor anything).

    You don’t have a database of 125.000 generations of humans that came before you to use, it’s really a thing entirely new (even called it a “game” gives the wrong idea because that’s allows an analogy. Paxglaw is nothingl ike this AT ALL).

    Well, in this hypothetical scenario, are we that different from what AS is doing?

    1. Daimbert says:

      This, though, is what makes me skeptical about the excitement over neural nets as, at least, being the solution to general intelligence, because what you’ve said is, basically “Well, if you stripped out all the semantics and meaning from the input we’re getting, how good would WE be at it?”. But this, then, highlights the issue: the neural net, or at least the algorithm AI runs with neural nets, is totally lacking the semantic information and meaning, and can never actually ever get it. But it is precisely that information and meaning that lets us do intelligent things and learn. So if it can never have that information, then how can it ever be really intelligent? It’s CERTAINLY not going to be doing it the way we do, even if that’s one of the claims for using it.

      And inference engines CAN use that. You can explain the rules of the game to it. You can give it advice on how to play. It could literally learn how to play the game by reading books on the theory behind it. And it could learn in-game by building a rule that tries to predict what other players will do and correct it if their opponent does something else. So an advanced inference engine could and should learn the way we do.

      And it highlights the difference in learning styles. An inference engine learns slow, while a neural net learns fast. An inference engine does want a lot of data, but it wants to parse it carefully and doesn’t want or need repetition, while a neural net wants MASSIVE data and LOTS of repetition. Which set seems more like what we do?

    2. Syal says:

      So the first thing a human would do is try to find out what losing Paxglaw does, and the second thing they’d do is make up an arbitrary goal of their own and work toward that one. Then they’ll use their personal goal as a benchmark against Paxglaw. If they can routinely succeed at their goal, they’ll make a new goal that builds upon the first goal.

      Depending on how losing at Paxglaw affects their own personal goals, they might just ignore Paxglaw entirely at that point, unless they find out winning is amazing and much better than what they’ve been doing.

  17. Zak McKracken says:

    Is the replay of that last match available somewhere? All I seem to find is the same video Shamus linked to, which is a recorded video which only shows what MaNa was seeing.
    I’d like to watch something like the other games, where a proper replay was recorded, and the spectator can see everything on the map. That’d allow for much better analysis of what the AI did.
    One thing that I believe happened is that the AI did not actually send its army out to MaNa after the first drop but rather sent it to kill the warp prism — the army then lingered on the closest spot on the ground, rather than actually doing anything.

    1. Shamus says:

      The AlphaStar had the replay available on their site, which is where I got it. I’ve lost track of the URL, though.

    2. Syal says:

      Winter’s video had it from Alphastar’s perspective, though again it’s not normal replay vision. Starts at 1:26:35 if you’re interested.

      It looked like it started sending the army across the map, brought them back for the first drop, lost vision on the prism, started sending them across again, brought them back for the second drop, illuminated the prism so it couldn’t disappear, and then parked the army uselessly next to the void until illumination wore off.

      1. Zak McKracken says:

        Thanks a bunch, that’s also a really useful perspective, and it shows that Alphastar really seems to have a problem not just with moving the entire army to counter a drop but also with where to look …

    3. Zak McKracken says:

      Just did another search and found this, which is in German, but it shows the standard spectator view, so may be of interest even for those who don’t understand what the guy says.

  18. Zak McKracken says:

    Just occurred to me that if the AI was trained on AI-vs-AI games, with the “enhanced” vision of the earlier games, there would not have been very much strategy possible which takes into account what your opponent has seen and what conclusions they might draw from it etc..
    One of the most difficult things for humans is to keep track of everything on the map, and not to overlook the dropship that scoots by the peripheral vision of one of your units. This also means knowing where to look, at all times. If the AI can see everything at the same time during training, then it wouldn’t bother to learn that because it’s perfect already. With the restrictions in the final match, however, having to move the screen about would consume some APS, and at the same time the AI would probably not be very efficient about it, either, because it hasn’t had as much training data for this mechanic.

Thanks for joining the discussion. Be nice, don't post angry, and enjoy yourself. This is supposed to be fun. Your email address will not be published. Required fields are marked*

You can enclose spoilers in <strike> tags like so:
<strike>Darth Vader is Luke's father!</strike>

You can make things italics like this:
Can you imagine having Darth Vader as your <i>father</i>?

You can make things bold like this:
I'm <b>very</b> glad Darth Vader isn't my father.

You can make links like this:
I'm reading about <a href="http://en.wikipedia.org/wiki/Darth_Vader">Darth Vader</a> on Wikipedia!

You can quote someone like this:
Darth Vader said <blockquote>Luke, I am your father.</blockquote>

Leave a Reply

Your email address will not be published. Required fields are marked *