Experienced Points: Graphics Are Sometimes Hard

By Shamus Posted Tuesday Feb 10, 2015

Filed under: Column 47 comments

As luck would have it, a bunch of people all sent in related questions at once, so this week I’ll answer three different questions about graphics programming.

I don’t have much to add to the column, so instead I want to whine that STRAFE isn’t going to meet its Kickstarter goals, and that makes me really sad.

A couple of weeks ago on the Diecast I was talking about combining modern lighting with retro-90’s low-fi graphics. I’ve actually dabbled in that idea myself. I’m sort of obsessed with it. But work will begin again on Good Robot very soon, and I expect that will keep me busy enough. I just don’t have time to work on mixing lighting models with anachronistic graphics styles. Having STRAFE was going to be the next best thing.

Ah well. Maybe in six months.


From The Archives:

47 thoughts on “Experienced Points: Graphics Are Sometimes Hard

  1. Daemian Lucifer says:

    But work will begin again on Good Robot very soon

    Valve soon?

    1. Trainzack says:

      By the time we land a man on mars, he will give us a date.

    2. silver Harloe says:

      one would hope, since he rated it above *spoiler warning* in the header, that he plans a little sooner than that :)

    3. Zak McKracken says:

      Edit: whooops, answered to the wrong post — this was supposed to go under DL’s next one!

      “All the way to the horizon” is usually a finite distance, and more often than not, it’s determined by where everything either fades into the haze (in slightly older games more like a dense fog…), or what you think is the horizon is really just the skybox.

      LOD (level of detail) means replacing things by simplified representations of themselves (low-res geometry, textures), but many things will just disappear (grass, trees, houses… ), and after a certain distance, there’ll be just a matte painting of the background (the afore-mentioned skybox). If it’s done nicely, the player won’t notice the transition, but there definitely must be a maximum draw distance for everything.

  2. Daemian Lucifer says:

    Wait,isnt view distance more about “at what range does stuff go from high poly to low poly” than how far you can see?Because there are plenty of games where you can see all the way to the horizon no matter the view distance,but stuff popping in and becoming more detailed will change depending on where you put the slider.

    1. 4th Dimension says:

      That is Level of Detail. View distance might control the scaling of at which distances they use what LOD. In many games viewdistance is how far away is terrain obscuring fog.

      1. James says:

        which reminds me of some awful pop-in “issues” some games have, where grass magically rises from the ground as you approach (SWTOR in recant memory is a massive abuser of this) or things just appear from the ether.

        Visual Culling looks interesting, only drawing what is visible to the PC (culling things behind houses/trees/4k polygon rocks (Fuck you bethesda) and i hope it catches on, could allow greater detail with steady 60fps.

        1. guy says:

          Visual culling is already standard. It’s just sometimes cheaper to draw something that gets hidden than to check for line of sight.

          1. Alan says:

            To expand on guy’s thought, this is a good place to use spare fill rate (and some throughput). The GPU is incredibly specialized, but for those specialized tasks it’s mind bogglingly faster than a CPU. Visual culling (generally*) needs to be done on the CPU. So if you’ve got fill rate and some throughput to burn, it’s faster to just ask the GPU to draw stuff that won’t actually end up on screen. (This is an interesting difference from 90s games where unnecessary drawing was really expensive. Much effort was spent on clever culling techniques and in many cases designing levels so that the culling techniques worked well.)

            * – GPUs are doing some culling. Polygons facing away from the camera are just dropped. Anything that’s outside of the view frustrum (what you see) is dropped. If something else has been drawn in front, the GPU can determine (on a pixel-by-pixel basis) that it’s a waste of time to figure out details like which color to draw. Potentially you could teach the GPU to do additional culling for special cases, but the GPU’s simplicity mean it’s not practical to create a general solution.

          2. Xeorm says:

            My impression of visual culling is that it’s not usually done during the rendering phase. It’s fairly expensive to do on anything except simple polygons, and isn’t guaranteed to work to help during any worst cases and is instead going to make them worse. It’ll help the average case, but the average isn’t as important as any worst cases.

            Instead, it’s mostly done pre-rendering in other ways. Like setting up hallways with sharp corners so you know which objects are visible and which aren’t by which portion of the hallway you’re in, rather than doing it during rendering by polygons. Or only worrying about npc’s that aren’t obscured by effectively-rectangular buildings.

            1. guy says:

              Actually, one of the big steps in rendering is culling everything that falls outside of the camera’s field of view. There’s also a step that removes all the polygons on the backs of objects, and some techniques that will relatively cheaply remove most occluded polygons prior to running shaders. They’ll tend to err on the side of drawing extra stuff over doing thousands of calculations per polygon and then rendering nearly all of them anyway.

              1. Xeorm says:

                The steps mentioned earlier were about removing polygons occluded by other polygons, so removing based on camera angle would be different in the given context.

    2. Phill says:

      The two questions overlap to some extent. Draw distance is the maximum distance you bother to render a given model. Level of detail is switching between different models with different numbers of polygons (and also potentially different fragment shaders – not much point doing complex reflection calculations in the shader for the polygons in a shiny metal helmet when the whole thing is three pixels across, whereas when it takes up 20% of the screen it makes a difference).

      Different categories of model have different draw distance / LOD bevaiour. Terrain, being generally composed of large polygons, non-animating with relatively simple shaders, is cheap to draw, and you can use various techniques to dynamically adjust the polygon size at various distances, so you can actually draw quite distant terrain stuff quite cheaply. Plus a lot of the work can be pre-calculated.

      More complex static models like buildings are similar, but there is a limit to how low poly you can make them. At the extreme you either draw a textured box or nothing. While with terrain there is still benefit in drawing very distant terrain even with polygons hundreds or thousands of meters across – it lets you pick out landmarks like mountains still. With buildings you can’t go that low, and increasing draw distance means that at some point you can’t reduce the cost per building any more, while the number of buildings is still potentially going up as distance^2. (while with terrain you can make the cost more or less linear with distance).

      Animating, detailed models like characters and animals are more limited still, since they tend to be more detailed, have more complex shaders, can’t be simplified as far, and are animated (requiring extra CPU load to track and manipulate all the relevant animations).

      Hence you get what you see in WoW, where you can see distant mountains pretty well. Buildings (the kind you can go inside, rather than ‘landscape’ ones) and similar features (barrrels and other static scenery objects) tend to pop in closer, although hopefully not too noticably. And characters are only drawn pretty close by. And terrain widgets, like billboard grass doodads, and drawn even closer still, since they are pretty numerous and small and beyond a short distance would merge with the floor anyway.

      (How well doodads are handled is another question: James mentioned SWTORs grass which is an example of it not working well – you can see the doodads behaving artificially quite easily, whereas I don’t recall seeing similar behaviour in GW2).

    3. Spammy says:

      An interesting example of bad Level of Detail was Resident Evil: The Mercenaries 3D for the 3DS. Mercenaries 3D was (I think) running on an unrefined version of the engine they’d use for Revelations. One of the things that’s really noticeable is that their Level of Detail ranges just jump from one to another. If you’re using a sniper rifle you could watch a fuzzy model of a baddie halfway across the map suddenly gain details as they shamble towards you.

      Although now that I think about it, I wonder just how much they improved in that regard with Revelations. I think Mercenaries 3D might have been really obvious about it since Mercenaries maps were all taken from the wide open spaces of RE4 and RE5. Revelations takes place in usually more closer indoors environments. So I’m not sure now whether they improved their Level of Detail techniques (although I bet they did, Revelations looks a lot better than Mercenaries) or just keep things so close they don’t need a lot of Level of Detail shifts.

      1. Felblood says:

        I wouldn’t bank on that.

        When it comes to making games look good, having a powerful/versitile engine is less important than knowing what your engine does well.

  3. Blake says:

    “The jump from 1080p to 4k requires four times as much fill.”.

    Huh, I’d just assumed 4k would be something like 4320p, but nope, after wikipedia research I found (what I presume was) marketing decided to switch to horizontal not vertical to make for bigger numbers (4k is 2160p).

    I guess I’m saying your article was educational even if I already knew the graphics side of things :p

    1. The Rocketeer says:

      This confused me, too. For a while, I thought resolutions had skipped a generation and become sixteen times larger by area.

      1. Ravens Cry says:

        I bet that’s what They want you to think.

    2. James says:

      Think of 1080p as 2k, and 720 as 1k, im sure 3k is a thing but its kind of esoteric and a odd ratio.

      1. Zak McKracken says:

        1080p is close to 2k horizontal resolution (1920) but the 1k analogue to that would be 960 x 560 … 560p is (close to) regular old TV resolution.

        1280×720 has (a little less than) half as many pixels as full HD (and twice as many as SD), so it’d be more like 1.5k … well, actually 1.3k.

        The whole accuracy thing doesn’t seem to be taken serious in this business, since 4k:=3840 and 2k :=1920 … I would have liked 2k to be 2048×1152* and so on. But I guess if you can have 12% fewer pixels and still call it 2k, why wouldn’t you (as long as you’re not the consumer)?

      2. Wide And Nerdy says:

        Some tablets and phablets, such as the Galaxy Note 4* have a 1440 by 2560 resolution. So there’s something close to 3K. I guess its really 2.5K. Still, adds up to a pixel density of 550ppi on a phablet. Its insane. I can’t wait for the monitor or television that can match that pixel density.

  4. MadTinkerer says:

    “I want to whine that STRAFE isn't going to meet its Kickstarter goals, and that makes me really sad.”

    Half of me wants STRIFE to succeed, because it looks like they have a good base for a solid product. Half of me wants them to fail if it means Shamus gets motivated to do his own take on it.

    Meanwhile, Looking Glass Studios are trying to resurrect themselves with a new Underworld game on Kickstarter and, more importantly, the original staff of LGS, but they haven’t met their goal yet either. I’m pretty sure that would be a worse tragedy than STRIFE not getting made.

    1. Zak McKracken says:

      I’d love for STRAFE to succeed, too. I am, however, really not their target audience, so sorry for not taking part. The pitch is hilarious and wonderful but it’s just not my type of game (and wasn’t in 1996, either).

    2. Andy_Panthro says:

      I jumped right in with the new Underworld game, the original is an absolute classic and I hope the new one can live up to the name.

      It’s also being set in the same dungeon (The Stygian Abyss), so it will be interesting to see how much they keep similar and how much they alter (I think it will be very different, but no doubt there will be a couple of places which give a nod to the original).

      [edit] should have put a link to the kickstarter I suppose: https://www.kickstarter.com/projects/othersidegames/underworld-ascendant

  5. DaMage says:

    One addition:
    One of the other reason when you have a huge draw distance is that it means your depth buffer will be less accurate. The depth buffer is not linear, closer objects are have much more depth to work with then objects that are far away and the closer the objects gets to the end of the viewing frustum the more likely they will get the same depth and overlap on the screen.

    I hate fill rate, if you are building a software renderer and don’t have access to the hundreds of GPU cores then it just tanks your FPS. In fact I can easily do all the matrix/vector maths for 30,000 vertices, but then struggle to render past 800×600 resolution.

  6. Eschatos says:

    STRAFE is failing, and a kickstarter about exploding cats made five million dollars. There is no god.

  7. harborpirate says:

    I can’t help but think that the video for Strafe didn’t capture its full potential. The concept is great, but I think it takes it one step too far on the gross-out factor; resulting in turning something that could have had mass appeal into something more niche.

    For me the bridge too far is the mother weeping while trying to reassemble her childs exploded head. Despite being ridiculous, it somehow hits just a little too close to home emotionally. The fact that this scene is near the end of the video hurts it twice as badly for those that dislike it; see the study referenced in this TED talk for how a memory is colored disproportionately by the way in which it ends: http://www.ted.com/talks/daniel_kahneman_the_riddle_of_experience_vs_memory?language=en

    Oddly, you know what would have been hilarious? The mother going completely insane and throwing herself out the window in a ridiculously exaggerated fashion, rather than going over to her headless child.

    The other option that would would have had better market appeal? Have the kid get his face melted but, rather than it exploding, he pops up and turns out to be just fine in the end (he doesn’t even notice that he has no face). The fact that the kid is in no pain and seemingly doesn’t care at all about his missing face means that his mother’s shock and horror could be played up for laughs to the nth degree.

    Anyway, I’m not sure how much the pitch video actually matters on Kickstarter campaigns. A bad video will absolutely kill a campaign, but how much more does a slightly more appealing video bring in than an “average” video?

    Oh, and since I’m here: Shamus, well done on the column, I enjoyed it.

    1. MichaelG says:

      Yeah, I didn’t get much of a feel about the game itself. The video seems all about people getting addicted and killed by the game.

    2. Zak McKracken says:

      Actually, I’d have wanted the mother to go over to that computer and beat the crap out of that game.

      Also, maybe the pitch should have highlighted the game itself more than just the gore aspect of it. We don’t know if the gameplay is any good, only that there’s lots of blood in it, and that it’s probably rather hard.

    3. Nixitur says:

      Yeah, I got interested in STRAFE when Shamus talked about it on the Diecast, then got completely repulsed by it when I watched the video.
      I would have preferred a boring dev-talking-to-the-camera video over this nonsense. It honestly made me feel slightly sick.

      I’m aware that not backing a game because of the trailer is just as irrational as buying a game because of the trailer, but that video irritated and angered me, so I’m really not in the mood to support these people.

  8. kunedog says:

    Wow the site has a “Boobs” section now . . . oh, the font just changed.

  9. McNutcase says:

    View distance is particularly interesting to me right now because I recently started yet another play of Morrowind, this time with the Morrowind Graphics Extender XE installed. Pretty much all that does is up the draw distance significantly, but it’s a massive change in the feel of the game, and two unexpected side effects have occurred: first, remember Cliff Racers? Of course you do. EVERYONE remembers Cliff Racers, to the point that Jiub was declared a saint for exterminating the things. Funnily enough, when you can see them a long way off, they’re much less annoying; you know they’re coming, you can just be ready and deal with them. Second, remember how much the directions people gave you in-game sucked? The way you’d have to guess at when to turn, and you’d need to backtrack to find the blasted path again if you ever managed to find a landmark, since finding the landmark meant you’d gone too far? It turns out, if you can see more than a hundred yards or so, landmarks actually function as landmarks. I know for a fact I spent several hours getting lost doing one Mage’s Guild quest back in 2003; with the increased draw distance I now have, it was an easy hike with clear directions.

    Admittedly, there are some undesirable side effects, such as having a giant dwarven ruin just sort of looming over the Legion fort just outside Balmora (and from the gate of said fort, you can see the Balmora silt strider stop!) but overall, the lack of annoyance from the Cliff Racers and the vastly better in-game directions more than make up for that.

  10. Nick Powell says:

    On the subject of low-res graphics with modern lighting, here’s a skyrim screenshot with the Ultra Low Graphics mod (Which vastly reduces the texture resolution) and ENB (Which hooks into DirectX and replaces the lighting with its own more up-to-date version)


    Incidentally, the guy who did that seems to have turned skyrim modding into a full-time job.

    1. Riley says:

      Man that’s gorgeous…

      1. 4th Dimension says:

        From a distance. I’m not so sure it would look as good up close where you can see that textures are gradients and are fuzzy.

        1. Stuart Hacking says:

          For me, it kind of brings to mind a first-person Runescape: Youtube Video

          1. Asimech says:

            Looks nice otherwise, but all the people-made stuff looks a bit off. Lower “texture softening” would probably fix that, but I don’t know if that’s actually possible to implement.

          2. Nick Powell says:

            I think that wouldn’t look nearly as shit if it still had shadows enabled.

    2. Merlin says:

      Skyrim, by art director Bob Ross.

  11. tmtvl says:

    “Grpahics”? Spellcheck, you failed me, why!

    Anyway, thanks for answering the question. I know you’ve discussed in in one of your coding series, but it’s nice to get a reminder every now and again.

  12. gggggaaaaarrrrrrreeeeeetttttt says:

    I’m taking a graphics algorithms class and heck yeah graphics are hard. Drawing lines is tricky.

  13. Wide And Nerdy says:

    Regarding my bounding box/physical mesh question, I was more thinking about other types of interactions between objects, even melee attacks. I’d like to see my melee weapon interact a little more realistically with opponent. Granted I’m probably only going to see that in some kind of dueling game where you only have to render an arena and you can focus most of your resources on the interaction between the two duelists. Or maybe you can get away with some scripted tricks and canned animations, but I’d like to see more complex physical simulation in melee focused hack and slash type games in the future. Even if it involved playing a game where the opponents a encases in weird bubbly forcefields so that the physical interactions can remain basic while having high fidelity to the visuals.

    But fair enough, shooters are the dominant type of gameplay so I can see where it rules design consideration and groupthink (and i use that term neutrally here). Particularly with hitscan weapons (now that I know what that is) I can see how you can cheat and keep your physics basic.

    I’ve also noticed that anytime you do turn on physics, a simulation slows way down. In Second Life when they introduced meshes, you’re allowed to produce high fidelity physical meshes but thats also one of the most effective ways to crash a server. Was hoping for insight into that. Even something as simple as dropping 50 physics enabled objects into a room can suddenly get you single digit framerates.

    1. guy says:

      Actually, Shamus’s answer applies to melee combat too. Now, of course you could spend more processor cycles on collision detection and fewer on other things. You could make your collision detection twice as expensive and have half as many things in the scene. You could make the AI worse.

      However, cutting graphics would not help nearly as much as you might think. See, gaming computers have CPUs and GPUs. The CPU these days probably has four cores, which can each do several billion operations per second and have things like pipelines and sophisticated branch prediction. The GPU has a lot more cores that can perform highly specific operations much faster. So with graphics, the CPU sends a bunch of information about vertices of every polygon in the scene to the GPU. Then the GPU does a bunch of matrix and vector math, sets the color value of every screen pixel, and sends that to the monitor. Lighting, bump mapping, and actually applying textures all happen on the GPU, with only mild effort on the CPU’s part. Collision detection is going to be on the CPU; even if the GPU can handle it the CPU is going to need to know the results before it can act on them, and it takes time for information to travel between the two.

      As for physics, that’s mostly because it’s a lot of math. The actual equations are complex and they need to be done a lot of times.

      1. guy says:

        I suppose I should expand on pipelines and branch prediction. So, the simple way to do a CPU is to have each instruction complete in one cycle and at the beginning of a cycle it starts an instruction and completes at the end of that cycle. However, with pipelining, each instruction is broken into multiple steps that each take one cycle. Each cycle, ideally a new instruction goes into the first step, all the others go to the next step, and one completes. The cycles are shorter, because the single-instruction implementation has to do all of the steps in one cycle. It’s not quite the single-instruction time divided by the number of steps because there’s some overhead and the steps can take variable amounts of time and the cycle length needs to be the longest step or things go terribly wrong, but the pipeline is much faster assuming that it’s kept full. However, there are two main problems for keeping it full.

        First, if a prior instruction is going to write to a register that a later instruction needs to read from, the later one needs to stall until the prior one writes. That can generally be resolved in part by rearranging instructions to produce the same result without having to pause, though that isn’t guaranteed to be possible.

        The second problem is branching, where a condition is evaluated and then either goes to the next instruction or jumps to an instruction somewhere else in the program. Obviously, having the pipeline stay empty until the branch completes is very bad. So branch prediction mechanisms guess which way it’s going to resolve and put the instructions down that path into the pipeline. Then if they guess wrong those instructions get thrown out. More sophisticated processors might be able to start up both paths. Graphics cards have extremely poor branch prediction if any.

    2. Piflik says:

      Physics simulation is hard. The calculations itself aren’t that sophisticated, but you have to do them many times for many objects (sometimes at a higher rate than the visible FPS).
      Since the equations determining physics in our real world are highly complex and non-linear, there often aren’t analytical solutions to a problem. The only real way to do physics in realtime simulation is numerics, and then you have to work with numerical inaccuracies (in addition to floating point accuracy which is bad enough in itself*). The longer your physics timestep is, the bigger the error becomes, and while you will never get a 100% accurate simulation using numerics, you have to at least get a plausible result.
      Collision detection alone is a problem that uses lots of tricks to speed up the process, like spacial partitioning datastructures (grids, octtrees, hashing ect) to early discard object-pairs (or even parts of objects) that are definitely not colliding before going into detail and test vertices against triangles to determine the exact point and direction of a collision. And even something sounding so simple, like reliably detecting a collision between two oriented bounding boxes, is really not a trivial task.
      (You have to transform both boxes into the local coordinate system of Object A, so that it lies exactly inside the -1 to 1 cube, and if any corner of Object B has a coordinate inside that cube, you have a collision, and then you have to transform that point and the penetration depth back to world coordinates to calculate an impulse and correct the boxes position to resolve the penetration, or you have another collsion in the next frame…but then you still only have tested the faces of Object A against the vertices of Object B, so you have to do it the other way around, too…and then you are still missing edge-edge collisions…testing arbitrary meshes for collision is magnitudes more involved)

      *Floating point accuracy is one of the reason why levels tend to be smaller in modern games. The further you are from the origin of your coordinate system, the bigger the errors from floating point arithmetics become, and physics simulations are the first to suffer from it.

  14. guy says:

    Gah, mistyped my address.

  15. WWWebb says:

    The bottleneck question reminded me of Super Bunnyhop’s Gaming beyond 60Hz video. If anyone hasn’t seen it, I highly recommend it.

Thanks for joining the discussion. Be nice, don't post angry, and enjoy yourself. This is supposed to be fun. Your email address will not be published. Required fields are marked*

You can enclose spoilers in <strike> tags like so:
<strike>Darth Vader is Luke's father!</strike>

You can make things italics like this:
Can you imagine having Darth Vader as your <i>father</i>?

You can make things bold like this:
I'm <b>very</b> glad Darth Vader isn't my father.

You can make links like this:
I'm reading about <a href="http://en.wikipedia.org/wiki/Darth_Vader">Darth Vader</a> on Wikipedia!

You can quote someone like this:
Darth Vader said <blockquote>Luke, I am your father.</blockquote>

Leave a Reply

Your email address will not be published. Required fields are marked *