Project Good Robot 21: Resource Usage

By Shamus
on Oct 6, 2013
Filed under:
Good Robot

You know what the hardest feature of this game was so far? Fullscreen mode. Well, not so much fullscreen as the ability to allow the user to change the window size, which crops up most often when toggling fullscreen mode. I don’t think it was the most time consuming feature, but it was certainly the most effort for the most mundane and uninteresting payoff.

It’s not something you can skip, really. I’m long past the point where I’ll put up with a game that wants to restart because you changed the resolution. We’re all accustomed to being able to smack alt-enter to toggle fullscreen or to resize a window by dragging. It’s just part of making civilized software and not something a developer can leave out. On the other hand, it’s infuriatingly difficult and troublesome and adds all kinds of unwelcome complexity to systems that would otherwise be graceful and elementally simple.

Ever wonder why some old games (and a few modern ones) won’t change the resolution until you restart the program? I’ll tell you.

The problem has to do with how video memory is allocated. Let’s imagine this blank canvas is your video memory:

gr21_mem1.jpg

When the game starts up the silly user has the resolution set to a hilarious 640×480. So we need to allocate enough memory to hold that image. Actually, we need two of them. These blocks of memory are called the front and back buffers. The idea is that the game shows you one image (the front) while it’s busy drawing the other (the back) and then swaps the two. It’s always showing you one and drawing on the other. This swap is the end of a frame, and doing this switch is what gives us the “framerate” thing that the kids are always talking about.

So when we start up OpenGL or DirectX we grab some memory for our front and back buffers:

gr21_mem2.jpg

Now, we might also need a few more buffers: Depth buffer, stencil buffer, accumulation buffer, or whatever those fancy folks working on the Cry Engine use to make their pixels so shiny. But you get the idea that we need to take a chunk of video memory to hold the stuff we’re drawing.

We’ll also need to grab some more memory to hold our texture data. We might have a small number of huge textures, a modest number of medium textures, or a multitude of small textures. It kind of depends on the game. Either way, in most cases textures will take up more video memory than anything else.

gr21_mem3.jpg

To the other coders: Yes, I know memory is linear and not arranged on a grid like this. But that’s not important for the conversation we’re having and this is much easier to visualize.

We might also have some vertex buffers. That’s where we take a bunch of polygons and send them to live in video memory so we can refer to them quickly. Instead of sending all 10,000 polygons of Master Chief’s head every frame we can just say to the graphics card, “Hey man. Remember that head I gave you a while back? Draw that.”

So let’s put some vertex buffers into video memory:

gr21_mem4.jpg

And finally if we’re doing bump mapping or somesuch then we need vertex and fragment shaders. These things are just bits of code, so they’re small. But they do exist in video memory.

gr21_mem5.jpg

So fine. We’re drawing this 3D scene. We’re rolling along, rendering stuff and giving the user aliens, Nazis, Zombies, or robots to shoot. Everyone’s happy. But then the user decides they don’t like 640×480 mode and they switch to something like 2048×1152. Suddenly…

gr21_mem6.jpg

Suddenly many times more memory is needed for those front and back buffers. (And stencil, depth, etc.) This space is gained by purging everything from video memory and starting over. This always happens, even if there’s tons of memory left.

I’m sure there’s a good reason for this. That’s not sarcasm. I don’t know enough about the low-level videosystems to know why this purge happens, but a lot of people have been obsessing over these software layers for over a decade, and there’s no way this is the result of “laziness”. I’m sure the answer would be very technical and would come down to performance.

It doesn’t matter. The point is, at any given frame we could suddenly discover that every single thing we’ve ever sent to video memory has been wiped out. Keep in mind that establishing these resources is a big percentage of the time you spend at the loading screen. If you’ve ever wondered why videogames tend to enter this massive funk just after you change the resolution, this is why. The program has to stop and re-load every dang thing. (The textures take far longer than everything else combined. My images above are not at all to scale. If they were, then the canvas would be about sixty times larger than the front/back buffers, the textures would be at least twice the size of the LARGE buffers in the last image, and the shader programs would be too small to see.)

gr21_screen1.jpg

It’s really annoying to have to design a system to handle mid-game re-initialization like this, but it’s what you have to do. Making the user restart is obnoxious and barbaric, but if we try to do ANYTHING without restoring the lost data we will end up drawing a heap of garbled mess. And then crash. (Trying to render from a missing vertex buffer is suicide.)

There’s other stuff that’s not kept in video memory that also ends up getting killed by this reset, like OpenGL display lists. In my case, this means it kills all of the fonts and the HUD.

The user re-sizes the screen and suddenly you have to re-init textures, vertex buffers, lists, shader programs, and fonts. You have to do these in the right order, since some systems contain others. What a mess.

On the upside, this acts as a great stress test. If your program can survive several successive re-sizes without crashing, leaking a bunch of memory, or displaying a bunch of ugly artifacts, then it (probably) means you’ve got a solid foundation on your program.

gr21_screen2.jpg

Performance-wise, I’m pretty happy with the program.

Some Trivia for the Curious

The project has ~40k lines in 84 source files. 2,500 of those lines are comments. The largest source file is 15k lines of code, but that’s a silly OpenGL thing called OpenGL extentions wrangler. It’s mostly a bunch of conditional statements to figure out what OpenGL features are available on the user’s system and how to get to them. The largest part that I’ve written personally is one tenth the size of that; the Robot AI is just 1,550 lines of code.

I’ve never heard of Cyclomatic complexity before, but according to the source-examining tool I just downloaded my McCabe VG Complexity score is 4,175. I’m sure somewhere out there is a computer scientist who knows what that means and will find it interesting.

We’re currently on week ten of the project.

It’s a total of two seconds from the moment I start the program to the moment when I can start playing. I hate loading screens and I think I’m going to be able to meet my goal of making a game that doesn’t have one. So far I can fly from the starting area to the end of the game in a single unbroken journey. There’s a tiny little stutter when you move down to the next level, but I’m sure I can get rid of that.

Memory-wise, I’m doing okay:

gr21_performance.jpg

My in-game memory usage is a good bit less than the memory used by Pac-Man at its main menu. I could get that even lower, but this is already ridiculously low. You can see it’s already using less memory than a couple of browser tabs in Chrome. (I think one of those is Facebook.) I’m trying to make efficient software, not mash my program down to some teeny tiny size for bragging rights.

How much more am I going to use in the way of resources? I think I’m basically done consuming texture memory. I won’t need more unless I fill my atlas texture, and I’m using less than a third of it:

gr21_atlas.jpg

I suspect I’ll eat some more memory with new sound effects. Other than that? I guess this is about what I can expect from the final product, minus optimizations.

I’m still not to the point where I’m ready to start optimizing and testing to see what the final system requirements need to be. But at this point I’m feeling confident that I’m not making any egregious errors or doing anything obviously stupid.

Enjoyed this post? Please share!


2020202019Ninety nine comments. Just... need... one... more!

From the Archives:

  1. Dev Null says:

    I hate loading screens and I think I’m going to be able to meet my goal of making a game that doesn’t have one.

    Ok sure, but you’re still going to make me go through 7 menu options – each with its own separate confirmation dialog in a slightly different portion of the screen – before I can exit the game, right? Because I’m pretty sure that’s required by law.

    • Akri says:

      No lie, I have honestly had nightmares about being unable to exit a game. No matter what I clicked on it wouldn’t end, and somehow simply turning off the computer or walking away wasn’t an option. I don’t even know why it was a big deal, but dream-me was absolutely panicked about the situation.

      • anaphysik says:

        I had a dream where the exit option in a game was bugged – it led you to the charselect screen, and only after confirming a choice did it (without warning) shut down.

        It wasn’t frightening or anything, but I do recall being ticked off at the evidently crappy QA testing.

      • Eruanno says:

        The first Assassin’s Creed game… aaaaaahhhh. There were like four loading screens before I could finally get out of the damned thing.

      • D-Frame says:

        You should try “B-17 Flying Fortress – The Mighty Eighth” then. This one will turn your nightmares into a psychosis.

      • D-Frame says:

        You should try “B-17 Flying Fortress – The Mighty Eighth” then. This one will turn your nightmares into a psychosis.

    • DrMcCoy says:

      Also, straight up not being able to quit the game at all during long cutscenes.

      Just had that happen in Richard & Alice, which I found out uses AGS and therefore runs on GNU/Linux with that community-built port. Just wanted to quickly test if it does indeed works, and then it wouldn’t let me quit during the intro. I dropped to the console and did a killall -9 ags.

      • Nathon says:

        I bind a key combination to xkill.

        • Peter says:

          Same here.
          When i use my (gaming) win8 box however, i use superf4. It sends off an “end task” on ctrl alt f4, which enables me to do mostly the same.
          I loathe the menu dance to close whatever (mostly games), but it’s also useful to end frozen programs without having to resort to a task manager.

          More on topic, this thing combined with the resize part of Shamus’ post makes me think of the rock paper shotgun posts about what games should/shouldn’t do. They really should be a mandatory read for anyone developing games.

          • ET says:

            Here’s the link to the article, if anybody’s interested.
            I believe the author updates this post with later posts, so the post date from two years ago is only half-true.

          • Endominus says:

            Do you mean this article? Most of that stuff seems basic to anyone who actually considers users when doing interface design. To be fair, I’ve seen some big games completely fail them.

            You know, I’ve never understood the logic of hiding the savegames in a completely different directory from the rest of the data. Presumably it’s to preserve in case of uninstall, but I can’t see the issue in moving the save file there during uninstall, should the user choose to keep the files; that way, people who use add/remove programs or just delete folders directly (*shudder*) will stillget rid of that data.

            • James Schend says:

              Windows has a Named Folder for saved games, if you put them anywhere else your game is buggy. Naturally most games get this wrong.

              • Mephane says:

                Yes, safegames are supposed to be put into Users//Documents/My Games//. Or was it without the company specific sub-directory. If there are several of those that appear to be from the same company with slightly different wording, which one is the right one? Should it even be within “My Games”, or directly in “Documents? Or somewhere in AppData? Local or Roaming? With or without company name there? Maybe under “All Users” and not your own user directory? What happens when you run the installer with an separate admin account (you do have one and use the computer as a non-admin-user-account, do you?), does the game think the stuff belongs into the directory for that admin account? In this case, does it matter when you directly use the “launch game now” option at the end of the setup – will this launch the game as admin or as a normal user? What about this MMO launcher/updater/crappy intern C# weekend job (nothing against C#, but these launchers appear to use it particularly often) that suddenly demands admin rights at the end of the download, or when it is started for the first time, or every single time? Should I close it after all downloading and patching is done, just to be sure that the game won’t run as admin and might save the game and settings into the admin account directory, not mine?

                Then there’s also this “Saved Games” thing which looks suspiciously like a mere “library”, and not a proper directory, and appears not to be used ever anyway. So you want to back up your safegames? Better start searching the entire hard disk for the files. Did I mention all of the above also has alternative names in your local language, and whatever a dialog presents you might not be the actual directory name on the hard drive?

                If there were one definite, consistent, unchanging place for safegames, I’d concur that this is the right place for the stuff. But given the choice between the current mess and just having an obvious “safegames” place right inside the game folder, I’d always choose the latter.

                • Mephane says:

                  Weird, I lost the ability to edit that post. The first directory structure listed should actually look like this

                  Users/[user name]/Documents/My Games/[company name]/[game name]

                  except I used <> instead of [] and the blog software either edited them out for being wrong HTML tags, or left them there as unclosed wrong HTML tags. In any case, this is what should have been there.

                  (Had to edit this addendum after seeing that <> got removed here, and used the HTML escape sequence instead.)

                • John Lopez says:

                  I *hate* games that mix user data with the game, simply because it makes the job of backing up my machine that much harder.

                  No, I don’t want to use my cloud backup space or even space on my removable drive for all the music, sound, graphics, textures and code. I have some way to re-install the game already.

                  But when the save data gets lumped in, now it becomes a project to *find* it and then add a new rule and probably mess around with permissions because the game is in the protected Program Files area.

                  SAVE THE GAME IN MY DATA DIRECTORIES. I don’t care where. Make your own folder, use the Microsoft Program Guidelines (god forbid someone read the docs), but don’t turn your game’s save file into a research project to back up.

                  • ET says:

                    Would it be inappropriate for games to put save data into {game folder}\{name of user logged in or running the game}\{saves}?
                    I know Linux gives each user a proper home folder, so they could dump it into ~\.gamename\saves\ or something, but since Windows has such a bad history with this…
                    Would make it trivially easy to find your save games, and makes it user/account-proof.

                  • Zukhramm says:

                    And I hate the reverse for th same reason. Instead of just taking my game drive as it is, I now have to go hunt for rogue data.

                • Richard says:

                  The Windows “Known folder” rules are relatively simple, except that they are basically impossible to determine and nobody at all follows them.
                  Most of these existed in WinXP, but weren’t enforced until Vista – which of course broke everything.

                  The “short” version that’s applicable most of the time:

                  – Application files go in %ProgramFiles%. Nothing else can ever be in there.

                  – Cache files go in %LOCALAPPDATA%, either in a GUID folder or /Temp/GUID depending on whether you’re hoping it’ll still be there tomorrow.

                  – “Hidden” user-specific but not machine-specific data (usually preferences) go in HKEY_CURRENT_USER/Software/… in the Registry, or %APPDATA%. (%APPDATA% is the roaming one)

                  – “Hidden” machine-specific but not user-specific data goes in %ProgramData%. Don’t use it unless you have a really, really good reason.

                  – “Public” user-specific data goes in %USERPROFILE%
                  There is no such thing as public machine-specific data.

                  In all cases except “Public” userdata, you should put it in a GUID-named folder, not your company name. Nobody ever does of course.

                  The most serious problem is when applications hard-code one or more of these locations. Do this and I KILL YOU.

                  The next-worst is writing to %ProgramFiles% or %ProgramData% at run-time. This requires Admin privileges, so don’t. Installation-time only.

                  See MSDN KNOWNFOLDERID for more, yet unclear information.

                  • Alan says:

                    It’s so frustrating. Because everyone breaks the rules, Microsoft can’t make them hard rules without getting blamed for breaking software that currently works. Because Microsoft doesn’t make the rules hard, no one has incentive to change from what they know. Add in that frequently doing it right is a nuisance, and you end up with painfully slow progress. A real shame, because if everyone played by the rules, modern Windows is pretty well organized.

              • RandomCommenter says:

                Windows also wants me to put everything in C:\ and every game in C:\Program Files right in the middle of other unrelated programs. Thankfully, I’m allowed to have multiple partitions and install my games somewhere else should I choose to do so.

                By all means stick your saves in some obscure official subfolder by default, but given the lack of coherence in that default hierarchy, giving the user the option to choose save location should be mandatory.

              • Zukhramm says:

                Until Windows let’s me choose what drive the user folder goes I will consider games storing data outside their own folder outright evil. I should NOT need some 20 extra GB on my OS drive.

                • Kalil says:

                  At least on Win7, I’m able to have most of the contents of my users folder – ‘my documents’ and ‘desktop’ – stored on another hard drive. If you right-click on ‘my documents’, go to ‘properties’, and ‘location’, you can relocate it wherever you want.

        • DrMcCoy says:

          Well, I got a second monitor running a second X screen to which I normally switch to using a key combination to kill offending programs, but R&A even didn’t let me do that.

          The ironic thing is, some games tend to die themselves when I switch screens. For example, Valve games, for some reason, minimize themselves when they lose mouse focus (which I hate), often dying in the process (which I hate even more).
          Some games are fickle like that in Wine as well.

    • Brayduck says:

      I’ve found exiting from the first Assassin’s Creed to be a complete nightmare. I have never even once exited the game ‘normal’ way, only with Alt-F4 or Task Manager.

    • Daemian Lucifer says:

      And the game needs to disable alt-F4,alt-tab and ctrl-alt-delete,so that you must use its ingame menus to exit.No cheating out of the feature!

    • Mephane says:

      Antichamber comes to mind, though in a different way. Alt+F4 is disabled in the game, to exit, you have to hit Esc once to get to the “hub” room, then hold Esc pressed in order to exit the game itself. I don’t see why in reaction to Alt+F4 the game couldn’t just behave the same, internally, as what it does when you exit using the “intended” sequence.

    • rofltehcat says:

      Obviously, Shamus needs to use all that saved time on the loading company logos. He probably doesn’t have one but he can probably cobble one together from the Diecast drawing. Make it slowly inch into the picture, fly around, zoom in or whatever it takes to make it as long and excruciating as possible.

      Then also make intros for every single person that helped you.
      And of course cancelling those doesn’t work. Or if it does work, it boots you directly back to desktop if you press escape one too many times.

      • Paul Spooner says:

        I would suggest that the program parametrically generates a new “company splash screen” during loading every time you start the game, but then the loading time would be longer. Maybe there can be an optional “turn on loading splash screen” checkbox in the options menu.

        • Daemian Lucifer says:

          “I would suggest that the program parametrically generates a new “company splash screen” during loading every time you start the game”

          Hey!Only fake “good” suggestions!

          • rofltehcat says:

            With Shamus’ experience in procedural generation of stuff, this would surely be an easy task for him! The splash screens would probably look like looking through a kaleidoscope but at least they’d be unique!

            The amount of splash screens when starting a AAA game as well as the mad amount of $$$ involved in them must mean that he could surely earn a lot of money by making even more splash screens!

        • Corran says:

          Terraria has a fun feature where it puts random quotes in the game’s title bar / taskbar button.

          I’ve been playing way too much Terraria since the 1.2 update arrived…

    • Neil Roy says:

      This was something I warn people about in my game. I tell them in the docs which they probably never read, about all the keys the game uses and it explains that if you press ESC, it quits the game, no confirmation dialog, no extra spoiler screens etc… it just quits. I think I worded it something like “No are you sure prompts so you know… be sure!”. I hate it when a game does that, although in the case of pressing ESC rather than the QUIT option on the menu, on my next version I may add in an ARE YOU SURE, but only if they are in the middle of a game and it hasn’t been saved recently. But not for the menu option. I do this on my level editor if the level has changed and not been saved yet, it will remind you that your changes haven’t been saved yet and ask if you are sure, which is reasonable, but that is it. And I have keyboard defaults so you can quickly get by these prompts.

      A game that stands out for me for being bad when it comes to exiting the game is Command and Conquer: Generals. I think from the game screen, you have to click something like 5 times before you finally exit the game totally.

  2. Mephane says:

    I wouldn’t call 124mb “okay”, but negligible. Like another browser tab – who cares? If you CPU and GPU values are as low, I doubt you even need something like minimum specs other than “has to be a PC”. Don’t forget RAM is getting even larger. We’re on a trend where the size of memory – RAM or persistent, becomes irrelevant, but speed and latency – CPU, GPU, internet – are the bottlenecks, and from all you’ve written it appears that Good Robot is nowhere near hitting any of those, either.

    • Nathon says:

      CPU and GPU speeds are not likely to be bottlenecks for much of anything. Most modern systems’ bottlenecks are in I/O: Memory bandwidth and latency are the big ones. If your program is big enough that some of it has to be stored on disk, that’s another huge latency hit.

      Obviously, network latency is off the charts. If you’re waiting for packets to tell you what to draw, you’re going to be waiting a long time.

      A little over a megabyte of resident space in memory though…that’ll fit in most L2 caches these days. I’m impressed.

    • Cuthalion says:

      It becomes more relevant with things like netbooks. Mine, for example, which tries to run Windows 7 “Basic” on only 1GB of RAM. So, I basically have maybe 300MB to work with for programs. I know, I know. I’m crying, too.

      • Volfram says:

        I have tried testing my game on my netbook, but the Intel graphics card does not support the only version of OpenGL that I can work with. I’d worry about it, but the Intel graphics card on my tablet handles it just fine, so I don’t really expect it to be an issue by the time I get to release.

      • Peter says:

        Depending on the netbook, you might be able to break it open and add/replace the RAM. It’s definitely worth a shot. Just be sure to check what kind of RAM you need.

    • rofltehcat says:

      Also don’t forget about smartphones! Memory may not be the huge issue with them but their CPUs seem to have a huge variety in power as well as power consumption.

      Running several things (normal background stuff + Kindle + music player + other apps that didn’t terminate properly) at once on my aged low-mid-range smartphone can lead to it either getting “stuck” (have to wait for everything to load properly and calm down) or draining the battery like mad.
      But at least I can call people with it, that’s the most important thing about it, anyways.

  3. Erik says:

    I’m not quite sure how to ‘score’ cyclomatic complexity, but i do know what it means. Basically if you have a loop, in a loop you increase your cyclomatic complexity, and every programmer worth its salt knows that if you do that too many times, its bad :)

    foreach ($a as $b)
    {
    foreach ($b as $c)
    {
    while ($c !== $a)
    {
    if (true)
    {
    continue 4;
    }
    }
    }
    }

    • DanMan says:

      Cyclomatic complexity for an entire project is basically a worthless number unless you’re trying to estimate things like cost of re-write, cost of ongoing maintenance and the like. It’s a pretty number that you can plug into equations and give other pretty numbers to management.

      However, cyclomatic complexity at the method and class level are useful. The typically agreed-upon number is 10-20 complexity per method. If your method exceeds 20 cyclomatic complexity, you should consider refactoring the code. At the place I work, a single method with greater than 10 cyclomatic complexity actually will not compile.

      • MrGuy says:

        Rant. This is exactly why I’m so loathe to report technical metrics like cyclomatic complexity, unit test coverage, etc. on projects to anyone who’s not actually on the team.

        They are stupid simple to abuse by someone (invariably someone in management) who doesn’t know what they mean, but have some vague notion that “the number should move this way!” and think setting targets for things they don’t understand is “helping.”

        “4,175 seems like a lot. Let’s get that number down under 4,000!” “Our unit test coverage is 81%? Let’s get that up to 85%!” Never mind that the person in question doesn’t understand the number, what it’s measuring, what the value means, or why it’s at that level.

        That’s not to say technical metrics like code style violations, cyclomatic complexity, test coverage, etc., don’t have their place. But they only add value when they’re understood by the people who write the actual code we’re measuring.

        • John Lopez says:

          The key to cyclometric complexity is understanding that it is only going to grow with program size. Where valid comparisons can be made is between functions or programs that claim to be doing the same thing: reducing the cyclometric count and getting the same results means you have simplified your code.

          Which is a good thing for testing and probably bug counts. But as you say, pointy hair bosses are unlikely to compare the complexity of two codebases doing the same thing, but just use it as a talking point with no substance.

    • Retsam says:

      Well, it’s not loops that add to cyclomatic complexity, but conditionals. Granted, 99% of loops involve conditionals, (well, assuming whatever is calculating the complexity is smart enough to realize that a “conditional” like “if true” isn’t really a conditional at all) so saying that loops add to complexity is true enough.

      But, yeah, cyclomatic complexity is just a rule of thumb for trying to avoid writing unreadable code. (Along with other rules of thumb like “Are you coding in perl?”) Sometimes it’s useful, sometimes less so. Like, personally, I find a set of mutually exclusive switch statements quite clear and readable, but it’ll give a pretty high complexiy.

      • ET says:

        Perl at least lets you write decent-looking C-style code.
        Although perhaps you chose Perl as an example, to avoid the language which must not be named.

        • silver Harloe says:

          Nah, Perl has a bad rap because

          a) regexps are top level language components and so much more flexible than a whole slew of startsWith() and strstr() type functions. So Perl code tends to contain a lot of regexps, which many people find illegible

          b) while you CAN write C-looking code, you do not HAVE to, and many people do NOT, so while I will produce perfectly legible Perl, if you download the average Perl app, there’s no guarantee that it will be legible

          • ET says:

            Well, that’s really a problem with the culture and/or individual programmers in front of the keyboards isn’t it?
            While languages can try to force you to write good-looking code (and honestly, Python 3.X is the king here), some people will always produce bad code.
            Their bosses should either sit them down and force them to learn better habits, or fire them.
            Most of the crap-looking code is produced by people who are also doing other horrible things, like prematurely optimizing their code to shave microseconds (bytes) off of the run time (space), instead doing proper algorithm analysis and/or using a better algorithm, which will shave off hours (gigs) of run-time (space).
            Big-Oh notation (and theta and…I can’t remember the rest) is your friend, people! :)

  4. Wow. I’m so glad I chose to use someone else’s rendering engine for my project – that kind of back-endy dealing with the weirdness of hardware is something that I am loathe to do.
    It’s nice to have some stats about your code and performance – although I’m not sure what to think about it, just knowing about it is interesting.

    • Volfram says:

      Yeah. I started reading this and immediately panicked and ran over to do resize tests on my program, because it doesn’t go *BONK!* every time you resize the screen, and I did almost no work to make that happen.

      Granted, I haven’t been working with fullscreen mode…

      • Richard says:

        Some toolkits offer “You have requested a logical image size of X by Y. I shall do that and silently rescale it to whatever the user actually chose.”

        Which is lovely, because you just pick some fixed logical size and no longer care about the user changing the ‘real’ image size.

        Which is hideous, because if the user makes the image smaller hoping for a higher framerate, it actually gets slower due to the additional scaling step.

        Others simply try to do all the memory management for you, which is great as long as you always, always do everything through the toolkit.

  5. Ilseroth says:

    I have asked a question like this before but not quite so directly.

    Are there any websites, books or wikis in particular for developing a basic graphics engine in C++ (using OpenGL)

    My personal experience is two years of classes of C++ and two years of Java (high school taught C++ in their advanced programming classes, college taught Java *sigh*)

    I’d prefer to go back to C++ though it has been a solid 7 years since I touched it (Jesus really, 7 years? time flies) learned mostly the general stuff, used in all programs (so that we could more easily go to any programming specialty)

    Originally I went to school for game development, I am going back now for something else, but I still wouldn’t mind pursuing making a game i my free time. I use a unfortunate amount of brain power on a day to day basis designing game systems that will most likely never get any use and I would like to fix that.

    So yeah, links to any good graphics C++ programming tutorials, books, wikis that are particularly good (there are thousands of them so I am kinda going in blind.

    • James Schend says:

      If you want to test game systems, you’d be better-served learning something like C# (in the guise of MonoGame, Unity or XNA-and-derivatives) or whatever Unreal’s using for scripting these days (is it still Unreal-C?)

      This stuff Shamus is touching is super low-level– he’d probably be done by now if he’d started with Unity. (For example, everything in this post he spent time on is stuff you get “for free” using any of those game libraries.)

      Basically, what I’m saying is:
      1) If you want to develop graphics engines, that’s fine– but don’t have a realistic expectation of testing game systems on them for a long, long time
      2) If you would rather test game systems, use something high-level first. Heck you could even prototype in GameMaker or Flash. Once it works out, then find a pre-made game development environment and port your systems and artwork to it.

      You can’t rapid-prototype in C++. C++ is only rapid when running code; it’s 10 times slower for writing it.

    • Volfram says:

      I started learning SDL over at LazyFoo and then moved on to NeHe when I wanted to learn some basic OpenGL, and I still check back with them for some of the newer basic techniques.

      These days I usually just Google-search the function I’m curious about and get info straight from the OpenGL wiki, but I couldn’t do that without the foundation laid by LazyFoo and NeHe.

      If you’ve been programming in Java for the last 7 years instead of C++, I would HIGHLY recommend checking out C# or D. C# is interpreted and supported by Microsoft, and if you want to get into Xbox programming you’ll pretty much have to learn it. D is more open, compiled, cross-platform, and IMO easier to get started on. D and C# solve largely the same set of problems, and both use a source code style you’ll have become used to working with Java(this is what happened to me)

      SDL and OpenGL were also both written in C, so you don’t have to resort to C++ just for that.

      I do not advise trying to build a graphics engine in DirectX. It’s Microsoft-only, somewhat more painful to work with, and tends to run a bit slower than OpenGL.

      If you’ve got about 4 years of programming experience, you have all the tools you need to do anything you may want. At very least you’ve got the tools to learn new tools on your own.

      • Ilseroth says:

        Hah, no I simple haven’t been programming for the last 6 years or so, way out of the loop, granted when I have had occasion to read code written by others I have been able to at least figure out what they are doing and how to do it, assuming they aren’t literally the worst commenter ever.

        As per the response to start off with another engine first, I suppose I could, I loaded up the free version of unity and just found myself confused…

        I suppose if I want to learn some stuff, the best thing to do is find some open source stuff that has been coded together already so that I can deconstruct it (as that’s the fastest way for me to learn something in programming)

        Those websites you linked seem to be helpful, what is a good free compiler nowadays (that isn’t spyware >.<)

        • Volfram says:

          For C or C++, I have always used GNU GPP. MingW is a no-stress precompiled version for Windows.

          For D, I use the official Digital Mars compiler. Doesn’t support 64-bit Windows binaries. You can also get your hands on a GPP-based version, but it requires some work to get it working, and the middle of a project is the wrong place for that.(You may find it’s worth the effort.)

    • MadTinkerer says:

      In my experience: don’t. You can make a game engine from scratch in C++, many have. But it’s not a hobby thing. You shouldn’t even try unless you’re doing it for enough money to justify the time it’s going to take to learn. Instead:

      1) Use Game Maker or Unity or other game engine with comprehensive scripting ability (and more recently: CROSS PLATFORM AWESOMENESS for both of those). People have made lifestyle-supporting money off of Game Maker and in a few rare cases even RPG Maker and Ren’Py if you’re willing to put in the effort to make it stand out from the RPGM and Ren’Py crowd.

      2) Grab the Doom or Quake or Voxlap or Source source code and use an existing code base to make your engine variant.

      3) Use a really, really, really good library set, such as App Game Kit (v2 coming soon!). This will smooth the learning curve a lot, and although you won’t have control over some of the “close to the metal” stuff, it will enable you to actually maybe get something actually done.

      4)And if you really must make a new game engine in C++ with OGL, from scratch, as a hobby/learning experience, there’s no one book. There are SO MANY books. The books NEVER END and you really should take a few weeks just to research exactly which ones to buy using your budget (library book sales are invaluable).

  6. otakun says:

    given that your memory footprint is so small anyway, can you just pre-allocate the front/back memory footprint for the “unused” screen resolutions? It’s wasteful but would eliminate the need to reload all the textures etc. Just keep the areas pre-allocated, don’t actually fill them, then if the user switches resolution you just start using the appropriate front/back memory instead of the old one.

    • Tektotherriggen says:

      That may be feasible if he only supports a few resolutions, but if Shamus is trying to write a truly “polite” program he’ll probably want to allow the player to type in any desired resolution, which would be impossible to predict.

      I know that I get a perverse sense of satisfaction from taking an old game (like Homeworld) and running it at a resolution and aspect ratio that its designers never dreamed of.

      • James Schend says:

        You could still reserve enough for, say, 1920×1200 resolution off-the-bat, which will cover 99.9% of the common cases for the next 5 years. Obviously you’ll still need to resize on the fly for resolutions higher than 1920×1200. So it doesn’t save a ton of work, but it might make your game a lot more responsive.

        To Ben’s point: while it’s true supporting higher resolutions can raise your video RAM requirement, the problem here is that there’s no link between video RAM and screen resolution in the first place. So you have no way of knowing that that guy with the 128 MB card has dualie 2k monitors, and your system requirements are shot to shit anyway.

        Better solution would be to set the minimum requirement to support the highest monitor size in use today (which should still be pretty generous), then since you’ve decreed that’s the minimum go ahead and use your space.

      • RCN says:

        Homeworld? Have you ever seem Total Annihilation? That game allowed such outlandish resolutions for 1998 (when 640×480 was actually the high end) that the initial missions of the game don’t even FIT inside the larger resolution possibilities. Sure, it leads to instant crashes if you try to scroll the screen, but it was impressive.

    • Ben says:

      I don’t think that would work very well. For a fullscreen-only game, you only have to support about 10-20 different settings for common resolutions, but each of those is between 3 and 10 megabytes just for the buffer data. An extra 150 megabytes on video RAM can disqualify a whole class of low-end cards with otherwise enough power to run the game.

      But if the game has a windowed mode, suddenly you can resize one pixel at a time. With an upper limit of 2560×1440, there are something like three and a half million possible combinations, and if you support higher resolutions like 4k the numbers get even more insane.

      • Daemian Lucifer says:

        Cant he do something in between then?Pre-allocate memory for resolution bellow a certain threshold,for example.

      • James Schend says:

        Oh I assumed the grandparent meant: pre-allocate memory for the highest common screen resolution. Not: pre-allocate multiple screen buffers for different screen resolutions.

        Note on the former case, even if you’ve allocated a 1080p buffer, nothing’s forcing you to actually draw 1920×1080 pixels in it– you can easily draw a small image in a big buffer, the only problem arises if you’re trying to draw a big image in a small buffer.

      • rofltehcat says:

        Don’t most games just use the predefined common resolutions in windowed mode? I play mostly in windowed mode and it is pretty annoying if if forces a 16:10 resolution on you, meaning you can’t have other stuff active on the same screen as easily.
        It is also pretty annoying when games don’t have a borderless window mode: What good is a 1920×1080 window mode when the bottom get clipped because I can’t push the window up far enough for it to fit the screen?

      • guy says:

        You can just reserve enough memory for the largest supported resolution and then allocate however much of the reserved portion you need. The downside is that at any lower resolution you’re tying up some amount of memory for no reason, but it would theoretically prevent needing to reallocate the rest of the memory.

        That might not be possible depending on the graphics engine, though, since it might demand the buffers be adjacent in memory. There may be some hack to handle that.

        • Paul Spooner says:

          Yep, this sounds like the way to do it (if that’s what you want to do). We know he can get the maximum supported resolution because a fullscreen mode exists. So, you just grab the “full screen” resolution, pre-allocate the memory for that, and then just render to a sub-section of the memory if the program is windowed.

          But what if you scale the window to be larger than the “full screen” resolution? What if “full screen” is smaller than “stretched to double monitor” size? What if the computer has an absurdly large full screen size, and the user doesn’t want to allocate that much memory? What if you’re running a server, and you don’t actually need to render anything at all? So many problems.

  7. otakun says:

    another note – you mentioned that you trust the wisdom of the Elders as far as not knowing why the Purge happens. But as a vehicle for exploring 2D game programming, perhaps GoodRobot is the perfect vehicle to test that wisdom. Why not break the conventional wisdom and see what happens?

    I do a lot of scientific analysis programming (in MATLAB mostly) and I always find myself chasing dead ends or tangents. When I’m not being lazy, i preface those dead ends with enough logic so that I can just flip a switch and remove it if it breaks stuff. I’m sure you have more elegant ways of doing it than I do (I usually use bitmasks… it’s a masochistic thing)

    • Steve C says:

      Shamus is already going trailblazing being his own one man development studio. It’s enough that he’s floated the idea on his blog and maybe inspired someone else to trailblaze that problem. Keep doing what you’re doing Shamus. Though Shamus could work on solving P = NP in his spare time.

    • Nathon says:

      There is a much more elegant way to do it. It’s called source control. The best part is that it’s easy. You can even use it on matlab programs.

    • James Schend says:

      In what way would Shamus reverse-engineering video drivers be a good use of time?

      The goal is to write a video game.

      And ditto with Nathon says: source control branching solves your Matlab problem. Why would you put that stuff in the code itself? Just invites bugs.

  8. Peter H. Coffin says:

    Given that not being able to run at native resolution is probably a “fail” state to begin with, is there any solid reason not to peek (at startup) at what the native resolution is, decide that’s the max that someone’s allowed, and allocate frame buffers to accommodate that, then just not use a lot of it of the player wants to old-school it in 640×480? (I get that it’s not *efficient* that way, and that’s got an element of inelegance about it, but if the user experience is such that you’re removing the whole “reload to change resolutions” bandage in trade, that’s a long-term gain..)

    • James Schend says:

      Note: to be the technical pedant (but being a pedant is good for stuff like this) in Windows, at least, monitors are hot-swappable. You can determine the native resolution at program start, but it might be different 10 seconds later when the player begins a game.

      That said, your idea covers the vast majority of cases.

      And hey, terrible game programmers (if any are reading this!)? While we’re at it: audio devices are hot-swappable too! Stop failing to recognize my microphone because I use a USB headset and don’t keep it plugged-in all the time! Stop giving “no audio device” errors when I unplug it! Grump! Huge pet peeve of mine.

  9. Retsam says:

    Alt-Enter? What black magic is this? I’ve always just tried F11 or F12, and if neither works, dig through settings. That’s good to know for future reference…

    • 4th Dimension says:

      Video players also use Alt+Enter. Also I think Unity by default changes from fullscreen to windowed mode on Alt+Enter.

    • ET says:

      Alt-Enter has been the defacto standard shortcut for switching between full-screen and windowed mode since…the 90s?
      I don’t even know how or why it happened.
      My guess is a combination of Windows 98 and black magic.

      • Decius says:

        {Alt, enter} was the windows 3 keystroke combination to ‘maximize window’. I’m not sure if there was a precedent for that one.

        • Atarlost says:

          No. The key combo for switching a DOS application from windowed to fullscreen was alt+space. I can see why they stopped using that, though. It tended to cause problems with games since both alt and space tend to look attractive to bind to things like jumping and shooting.

  10. swenson says:

    No comments on the monitor stuff, just wanted to say I like the industrial level an awful lot. Quite the interesting contrast from the jungle level.

  11. guy says:

    I would guess the purge happens because the alternative is moving the entire contents of the video memory, which would presumably involve copying sections to other memory so you don’t overwrite memory waiting to be transferred. You could alternately put the buffers at the end instead of the beginning if you feel like having to move the buffer every time you load a new texture instead of moving everything else when you resize the screen. Since GoodRobot loads its entire texture data into memory at once, this might actually be an acceptable tradeoff, but Crysis, for instance, needs to swap textures in and out of memory a lot.

    Of course, the most important question is “Do you want to rewrite the OpenGL libraries to support this one game?”

  12. Matt Downie says:

    I like that you’re comparing it to a giant bloated version of Pac-Man. About twelve years ago I knew someone who was working on putting a version of Pac-Man into one of those joysticks with five early arcade games on, the sort you plug directly into a TV. His task was to take the original Pac-Man assembly code (which must have been tiny, like 48K) and make it even smaller (by removing two-player mode, for example), due to them using ultra-cheap hardware for the joystick.

  13. Bruno M. Torres says:

    I’m not a professional, but I have my thesis about this memory problem: It has all to do with pointers.

    Every element has a position in video memory. This is why we can’t just move everything to a new position: The references will be lost. They have to be remade, one by one.

    There are two – bad – workarounds for this:

    1 – You set a upper limit to framebuffer and alloc resources above it. Then we will either limit the max resolution of the game OR waste useful memory if this max resolution is never used.

    2 – You make a pointer to the start of the resource heap and all the resource pointers will be relative to it. Then you can just copy the heap to a new place and change this pointer. You’ll also make your video memory slower, because instead of checking $(address), you’ll have to check $(base+address). Also, you’ll have to make an entire (slower) memory management system to handle something you’ll change once or twice in your life.

    To finish, you could adapt #2 to speedup the resolution change, maybe: Making a list of pointers, a memcopy and changing the list? Or is it too impractical?

    • ET says:

      I’d say it’s both dead pointers and memory limits/waste.
      Good work on coming up with this!
      I so seldom work with anything low-level, that this didn’t even occur to me until reading your post! :)

    • Atarlost says:

      But when you have the sort of low level access a video driver bloody well should there’s a third solution.

      You know how big your memory is, or should. You know how big your buffers are because you have to be told that to allocate them. Instead of putting them at 0 and [size] and starting the textures and stuff at 2*[size] put them at [max_address]-[size] and [max_address]-2*[size] while allocating all the fiddly little stuff normally starting at 0. You may still need the ability to do a purge when you go over your video ram, but you won’t have to actually do a purge unless a resize puts you over because your resources don’t move when you resize your buffers.

  14. Paul Spooner says:

    Very interesting! I was never clear on how the graphics card memory was allocated.
    So, if you have a ton of polygons and stuff, that all goes in the vertex buffers? Do vertex buffers work for nested vertex transformations, like animated armatures? Or, do those vertexes need to be transfered in their new location every frame?
    Is it possible to pre-draw more than one frame? Like, draw several buffers four or five (or way more?) frames in advance to guard against spikes of render load? If the buffers are a small part of the memory load, it seems like this would be an effective method of smoothing out rendering. Maybe that’s nonsense though. I really have no idea.

    • Volfram says:

      Anything you send to the video card to draw will be stored, however temporarily, in video memory as a buffer. This means vertex buffers for vertexes, pixel buffers for the screen and textures, and shader fragments for shaders.

      Newer versions of OpenGL do everything in shaders, meaning even if you just hand OpenGL a list of vertexes to draw, it’ll still load them up and say “Here Video Card, have a vertex buffer!”

      I assume(I have NOT done any work with shaders, so I don’t know for sure) that you can do most forms of vertex animation using vertex shaders these days. This wasn’t always the case, shader programming languages have gotten pretty impressive in the past ten or so years. I remember reading almost a decade ago that the new (at the time) nVidia cards “support if/else statements.” Really it just depends on whether you want your video card or CPU to be calculating the animations.

      Some graphical engines support what is called “Triple Buffering,” which is, yes, drawing a second backbuffer if the first one hasn’t been used yet. As you draw more backbuffers, though, you end up increasing the delay between what the player is doing and what you’re showing them, so for interactive purposes you don’t really want more than 2 backbuffers. For a noninteractive purpose, you might want to draw backbuffers until you run out of video memory.

  15. Amstrad says:

    One of the best recent examples of a game that doesn’t allow you to resize at all is FTL. When it came out I was interested in trying it out, but the discovery that it didn’t support resolutions below 1280×720 immediately rendered it a non-option for me as I’m stuck on a 4:3 LCD that with a max resolution of 1024×768.

    • Decius says:

      Running entirely on sprites is the cause of that; adding another resolution would double the art required.

      • Ivy says:

        I mean, not really? I don’t know the technical know-how, but there are lots of games that scale sprites with window size. It’s not always PRETTY, but it’s certainly doable with no extra art.

      • ET says:

        There’s tons of image-scaling algorithms, many of which could resize the entire screen buffer quickly and with a decent enough quality to allow other resolutions without any extra art cost.
        Most graphics and/or game libraries give them to you for free, so there’s really no good reason that FTL doesn’t allow other sizes.

  16. Neko says:

    Funny that a post about resource usage was the one that just kept timing out when I attempted to load it last night. Were you having hosting issues, Shamus, or fending off spammers? Perhaps there was just a bubble in the internet’s pipes that needed to be pushed out.

  17. Shamus you should use Process Explorer (part of the Sysinternals suite of tools).
    You can add/enable a GPU column to show alongside with the CPU one, in addition you also have have CPU and GPU graphs.
    And if you right click/open properties of your listed program you can see the threads in your program and their resource/CPU use.

    Like seen here http://www.album.de/img1/big/245/24538297494870.jpg
    or here http://betanews.com/2011/07/20/process-explorer-15-adds-gpu-monitoring/
    and here http://cloud.addictivetips.com/wp-content/uploads/2012/02/thread-2.png

    It’s not the same as using a actual source integrated profiling tool, but it’s a great help to notice any general issues.
    And Process Explorer is miles ahead of Task Manager (though I think Mark did add a few features there).

    Anyway, http://technet.microsoft.com/en-us/sysinternals/
    The Sysinternals Suite (menu on the left) is highly recommended).

    Process Explorer, Process Monitor, and Autoruns are probably my favorite tools in that suite, Process Monitor to track issues with files or registry or loading etc. And Autoruns to clean up a lot of crap or fix issues manually on a system, and Process Explorer I’ve already gushed about above so…

    I’ve found Process Explorer and Process Monitor invaluable when coding or tracking down issues.

  18. TehShrike says:

    I just have one request: please PLEASE have the game auto-detect the desktop resolution of my primary monitor and launch in full-screen at that resolution, instead of launching at some other resolution and resizing all my open windows and throwing them across all my monitors.

    I hate it when a game does that. If I can’t stop it from doing that the next time I run the game, I just uninstall it.

    • Sounds like a bug to me (might depend on the OS also), once you quit the game it the desktop should return to normal again.

      Also, there are two forms of fullscreen, a maximized borderless window will cause the taskbar on Windows to automatically hide. Giving the illusion it is fullscreen.

      So you basically have:
      Windowed
      Windowed Fullscreen
      Fullscreen

      On a modern system (hardware and OS) the performance difference should be minimal between them.
      One drawback with Windowed Fullscreen is that some games do not handle the mousepointer properly so if you have multiple monitors you might have fun trying to shoot in your notepad or browser window.

      Windowed and Windowed Fullscreen are the most multitasking friendly, real fullscreen is usually the only way that older games support (you may be lucky and they have a commandline flag or ini option for windowed)

      Windowed Fullscreen is the rarest though which is a shame as in my eyes it’s a great middleground.

  19. Anachronist says:

    “I hate loading screens and I think I’m going to be able to meet my goal of making a game that doesn’t have one.”

    I can see wanting to avoid a loading screen when switching resolutions. But in general (when starting the game), what would you put in place of a loading screen? Isn’t that something you just have to accept when you have a procedurally-generated game?

  20. Neil Roy says:

    This is one thing I have liked about the Allegro 5 library, you can easily create screens, borderless fullscreen windows being my favorite, windows, and you can resize the windows on the fly and Allegro will handle that nicely for you.

    I don’t have a clue what is involved in creating and managing window resizing on my own, but I can easily modify my game to handle that no problem. Right now in my current game I have it create a full screen window based on the users desktop size and then add in borders where nessecary depending on the aspect ratio and then it uses some nice hardware translation etc… to resize everything nicely. Or you can simply run it in a window. Allegro 5 also handles most image formats (JPG, PNG etc) as well as sound (OGG etc). My current game I have OGG music and sound effects and PNG for my graphics, all my game data is stored in a ZIP file which can be accessed from within my game so it all stays in the zip (which I renamed with a .dat extension). You basically point the game inside this zip file and it treats it like a normal file system, where you load from it normally. Allegro uses the PHYSFS library for this. Best of all, there are people that have precompiled the Allegro library for a variety of compilers like MinGW (which I prefer) as well as various Microsoft Visual Studio versions. I highly recommend it. Oh, and it has all the 3D, hardware blending etc as well as truetype font and bitmap font support… you name it.
    You can check it out at Allegro.cc which has forums, files and freely hosts Allegro projects. Allegro also uses SDL etc. It is cross platform as well, so compiling your project for Linux, MAC, Windows etc… is not a problem.

  21. Someone essentially
    assist to make significantly posts I’d state. That is the first time
    I frequented your website page and so far? I amazed with the
    analysis you made to make this actual publish incredible.
    Wonderful task!

  22. Hello just wanted to give you a quick heads up and let you know a few of the images
    aren’t loading correctly. I’m not sure why but I think its a linking issue.

    I’ve tried it in two different browsers and both show the same outcome.

Leave a Reply

Comments are moderated and may not be posted immediately. Required fields are marked *

*
*

Thanks for joining the discussion. Be nice, don't post angry, and enjoy yourself. This is supposed to be fun.

You can enclose spoilers in <strike> tags like so:
<strike>Darth Vader is Luke's father!</strike>

You can make things italics like this:
Can you imagine having Darth Vader as your <i>father</i>?

You can make things bold like this:
I'm <b>very</b> glad Darth Vader isn't my father.

You can make links like this:
I'm reading about <a href="http://en.wikipedia.org/wiki/Darth_Vader">Darth Vader</a> on Wikipedia!

You can quote someone like this:
Darth Vader said <blockquote>Luke, I am your father.</blockquote>