Project Octant Part 1: Introduction

 By Shamus Apr 30, 2012 129 comments

At PAX this year, during the Q&A session at the Escapist Movie Night someone asked about Project Frontier. This is still a thing people ask about. Remember Project Frontier? That was this thing:


Link (YouTube)

People ask me why I don’t work on it anymore. There are a lot of reasons, but the most important of which is: Because it was done.

It’s true that it wasn’t a game, and I occasionally talked and fantasized about making it into a game, but deep down my real driving goal was just to make my procedural generation ideas and prove they could work. I thought it might be nice if someone took an interest in the project later, and I always hoped it might lead to opportunity the way some of my other projects did, but at the heart of it was a desire to validate my ideas with a working proof-of-concept.

Once I’d gotten to the video above, I had 90% of what I was after. Sure, I could have done ten times more work to finish a game and get the other 10%, but I’m sure you can see why that route wasn’t particularly alluring. I had bills to pay and a book to finish, and so Frontier was shelved.

So almost exactly a year later, the programming bug has bitten again. I tried to ignore it so I could continue work on my next book. This resulted in me sitting with a blank look on my face, unable to write anything because I’d rather be programming. Which led to me closing the word processor and playing videogames, because I wasn’t getting anything done.

I’ve come to the conclusion that the only way to get moving on any of these other projects is to get the programming bug cured as quickly as possible. I can spend a couple of weeks staring into space, or I can spend them writing code. Might as well write code.

I seriously considered working on this project in the dark. I’ve already invested several days into the project, and I’m only writing about it now because I seem to work better when I can write about what I’m doing. And since I’m writing, I might as well make them blog posts, right?

I know what people are going to say. “Ugh. A MINECRAFT CLONE.” In fact, I’ve already gotten this treatment from a friend, so I can only imagine how THE INTERNET will respond.

However, there are a couple of technology ideas I want to play with, and doing a Minecraft -style world is the simplest and most direct means of playing around with those ideas.

There are three main concepts that I want to explore:

1. The Qt Development Environment

I want to mess around with Qt. This is a cross-platform SDK AND development environment. This is odd. The development environment is where a programmer works. They generally look like this-ish:

ide_vs2010_small.jpg

It’s basically to coding what a word processor is to writing. You type in your code. You compile your program. When it crashes, the IDE can help you find where. It provides a bunch of tools for keeping your code organized and managing the thorny dependency issues you might encounter.

There are a lot of these environments, usually many for whatever language you might want to use. Java, C#, Python, whatever. For nearly all of my career I’ve hammered away at Microsoft development environments. Once in a while I try something else, just to see if I’m missing out on anything. I keep coming back to Visual Studio. (I just accidentally typed that as “Visual Stupido”. Hm.) I’ve never been terribly impressed with Microsoft’s operating systems, but their developer tools are top-notch.

But who cares, right? I mean, this is Qt:

ide_qt_small.jpg

It’s got different windows. So what? Well, we’ll see. The pitch with Qt is that by using their tools you will be making 100% portable code. Write your program and it should run on PC, Macs, or X11. Since it’s generally free to use, it might be worth leaving behind Visual Studio. That’s the theory, anyway. The only way to really know how the Qt platform holds up is to write something meaningful with it. Any idiot can load up example programs and get them to compile. It’s another thing to try to bend the tool to your will.

2. Octree

Octrees are a major concept in 3D programming, yet I’ve never written one. Hard to believe, but there it is.

An octree is a way of dividing up 3D space. There are a lot of ways to use them. In my own work, I’ve used a lot of quad trees in my day, which which is basically the same thing but in 2D form. (Like this.) In fact, fair warning: Odds are good that at some point I’m going to call something a “quad tree” instead of an “oct tree”. I’ve made that mistake several times in my own code over the past few days and then wondered why it wouldn’t compile. I guess I’ve typed the word “quad” a LOT of times in my career.

An octree just takes a 3d space and divides it into eight sections by dividing it into half along every axis:

2x2cube.jpg

I’ll get into why it’s important later.

Yes, this problem is “solved”. I could type “octree” into Google and get any number of open-source solution to the problem. In fact, Michael Goodfellow made one in his very first entry of Let’s Code. I could download his code and call this done. But my goal isn’t to have an octree, but to write one. It’s one thing to see a diagram showing you “this is how you use a crescent wrench”. It’s another to pick up a wrench and take something apart with it. Writing a bit of code can put the concept into your mind in a way that reading about it in the abstract really can’t. The concept then becomes a general-purpose tool that you might be able to apply to different problems or use in new ways.

3. Perlin Noise

Here is where the Minecraft comparisons will come in, because to most people octree+Perlin noise=Minecraft. Fine. But I still want to do it.

So those are the goals. I have many other small goals in mind, but this is basically it. As with Project Frontier, this project could die at any time.


A Hundred!209There are 129 comments here. I really hope you like reading.


  1. kikito says:

    Hey Shamus,

    Would you consider opensourcing the projects that you are “shelved”? I’d like to give a look to the source of Project Frontier (and procedural city) if you don’t mind. At the very least, they deserve an entry somewhere on the pcg wiki.

    • silver Harloe says:

      He did release Procedural City: http://www.shamusyoung.com/twentysidedtale/?p=3237 . I’m still hoping to peek under the hood at Project Frontier to finally make his posts on it make final sense.

      I’m also hoping he’ll explain what Perlin Noise is ere too long. He was going so very, very explanatory for most of this post then just stopped defining things right when I needed it most :) I’m wondering if it has anything to do with how you store all that dang data. Forget 3D rendering for a moment, just how do you hold on to a world the size of which is implied by a need for procedural generation while also keeping track of everything that players have changed in the world. I mean, conceptually, anything can just have more harddrive space thrown at it, but you have have it all indexed right to load it as players move (or teleport) or change things. Scary.

      I’m also really, really curious to see what the next few weeks get him to say about the performance of Qt v. C++. Seems like the game industry clings to the latter because they have to to keep the FPS up.

      • Jon says:

        I enjoyed playing around with procedural city which was only possible for me because he open sourced it, and someone else ported it to Linux. I think it would be great if this was open-sourced, because IMHO it has the potential to be the bedrock of some pretty interesting future games, or at least game concepts. Too many of the actual minecraft clones out there spend 100% of their dev energy re-implementing the minecraft engine verbatim and never get past it to the game. This could bootstrap some people!

      • Elec0 says:

        Perlin Noise is a type of procedural black and white noise. He’s used it before, unless I’m mistaken, to get better looking landscapes. I know I have. Wikipedia has an article on it.
        And actually storing the data is a big problem, but binary data is really pretty small, although it can get huge when you’re dealing with something like Minecraft. You’ll also notice that Minecraft’s savegames aren’t all that large in comparison to how big you’d think they are.

      • nmichaels says:

        Qt is a C++ framework. It can be used from other languages, but it’s written in C++ and Shamus will be using C++ to play with it.

      • Atarlost says:

        The beauty of storing procedural worlds is that you don’t have to. You can re-generate them from your seed or seeds any time you need to. Discreet elements like vegetation do this easily, though terrain generated this way could wind up looking tiled.

        Just generate the part of the world you need. When the player wants to see another part generate it from the appropriate seed instead of loading it from disk. Generating content on the fly is a bottleneck, but I’m not sure it’s a worse one than disk access and the tricks sometimes used to eliminate disk access caused loading screens will probably work for this too.

        Then you have your deltas for your “designed stuff” and “mutable terrain that has mutated.” This goes back to Shamus’s article on video compression. It should work pretty much like that. You have one keyframe and one set of changes for the initial game state, then another for mutable terrain and destructible or moveable objects. You can make a good case for moveable objects not staying where you put them though so you only have to track some of those. Generally just the ones that were part of the designed stuff in the first place and those stored in secure locations.

        Designed stuff can be “slap a town here.” with the town, but not the NPCs inhabiting it yet, also being procedural.

        • silver Harloe says:

          “Then you have your deltas for your “designed stuff” and “mutable terrain that has mutated.” This goes back to Shamus’s article on video compression. It should work pretty much like that.”

          Actually, in a game like minecraft, it seems the “mutable terrain that has mutated” takes over in developed areas. And that’s why I’m curious how it stores so much and accesses it so quickly.

          • Nick says:

            Depends on your method of storage vs display – if all you need to know to draw a sword in that square is that it is a sword, you need one identifier stored that tells the graphics that. Your graphical processor can then generate points and draw lines according to it’s rules, but on disc/in logic storage you only need space for the one value to say that.

            This is why we use models for things in games, rather than moving and storing the individual points on all character models at all times

          • Uristqwerty says:

            Minecraft itself just generates terrain in 16×16 squares, saves what it generated, and just loads it later, no fancy keeping track of changes or anything. It also happens to load rather slowly and take a lot of space to store it all if you do any reasonable amount of exploration.

            Try setting the view distance to maximum, travel a bit (into ungenerated or at least unloaded areas, preferably somewhere reasonably flat), build a nice tall pillar, then stand on it and look into the distance. It tends to take a while to fully fill the area in front of you, and if you turn quickly, you can see nearby stuff appear as you turn.

            What it does do, is to load things nearby in front of you first, because there usually is enough content there that you can’t see the more distant parts that are taking a while. Being able to look down at the world, and get a good view of everything for a fairly large distance, reveals the rate it actually manages to create content.

  2. 6b64 says:

    Personally, I like if Shamus talks about programming too. I think prsesenting this an enjoyable way is quite difficult. I study mathematics, so we have plenty of dry and abstract courses, but programming theory tops it all. And Shamus, don’t listen to the Minecraft complaints. It doesn’t matter as much (to me at least) what you do as what you’ve got to say about it.

    • Deoxy says:

      And Shamus, don’t listen to the Minecraft complaints.

      Well, since Minecraft was just a Dwarf Fortress clone with better graphics, I don’t put any weight on those complaints, anyway.

  3. Thor says:

    Another coding series? Yes! Yes! Yessssssssss! That’s exactly what I was hoping for. That, and a Skyrim LP.

    Besides your Bethesda games LPs, the programming posts are what I enjoy most on your site.

    • Woodthorn says:

      Bethesda Games LPs? Have I missed something potentially really awesome or are you confusing Shamus with Rutskarn?

      • Tharwen says:

        They did Fallout 3 a while ago.

        Anyway, more programming! This pleases me greatly.

        • Eruanno says:

          And Fallout: New Vegas. (Which is an Obsidian game! I know! But it’s built with a Bethesda engine and published by Bethesda, so it’s kinda Bethesda-ish-but-not-quite.)

          • Thor says:

            Yes.

            Fallout 3

            Fallout: New Vegas (Yes, technically not a Bethesda game. Blah, blah, blah.)

            I am so ready for Reginald Cuftbert, the Nord! Although, I don’t know where you are going to find a bonnet (or a tophat). Oh, and Josh must be the player. It couldn’t be any other way.

            Some inspiration? (NSFW!)

            • IFS says:

              I don’t know, Kajiit are better at sneaking and punching, which are clearly Josh’s strengths. Either that or an orc, since they have the most broken racial ability in the game.

            • some random dood says:

              WARNING: That NSFW link above not only has swearing, but also juvenile trash-talk. If you find phrases such as “rape her for her armour” (referring to killing – which is better?) to be offensive, you may want to skip. At the very least, skip the advert so you do not reward the person who made that vid.
              (Note: the phrase used above may not be exact – it was approximately that, and I have no intention of going back to get the exact words.)

              • Thor says:

                Mate, I wouldn’t take it too seriously! After all, he is portraying a badass Viking. Vikings are not popularly known for their kindness. He’s not actually advocating that anyone should go on a rampage in real life. According to the number of views, upvotes, and positive comments, I think most people got the joke. Relax and lighten up!

    • Andrew says:

      I might actually watch a Skyrim LP :)

  4. Nersh says:

    That’s cool, but why do they call them ‘trees’?

    • PurePareidolia says:

      Simply put they’re called trees because they “branch out”. So each division can subdivide again, and each of those subdivisions can further divide, with each new division referred to as a series of “branches” of the octree. Octree literally referring to splitting into eight branches each time.

    • MrPyro says:

      Shamus will probably get into this later, but when you split things into quad- or oc- trees, you split them up into sections (I’m going to refer to these sections as octs, because it’s an octree. I have no idea if that’s a real term). Each oct then may get split up into another bunch of octs, so you’d end up with an oct inside an oct, which will then contain other octs. The easiest way to map the relationships between your octs is a standard mathematical tree structure; hence octree.

      If that didn’t make any sense, Shamus will probably explain it much better soon.

    • Rax says:

      Because visualized they look something like this (courtesy of a quick google image search for “tree programming”). Nevermind that this is another kind of “tree”, I just used this one because (if you turn it around) it looks more like a “real” tree than an octree would.

    • Sumanai says:

      Because programmers like trees but are sort of like vampires. Instead of blood they drink the metaphorical tears of the users as they fail to compile, install, run or beneficially use the program. You might think that all those super annoying bugs are accidental, but no, they feed the coders.

      Shamus is a bit special in that he doesn’t need the tears, which is why he doesn’t release his programs for wide use. I don’t know what the source of his power is, however. Maybe the tears of other programmers who want to get their hands on his source code?

      • Dave B says:

        That sounds like the premise for the next bestselling romantic fiction! “Naive young user meets ageless programmer with a soul. Was it meant to be?”

        Shamus, when you publish that book and become wealthy beyond the dreams of avarice, I don’t need credit for the idea. Just send me the source for Project Frontier. ;)

        • Sumanai says:

          I don’t think Shamus could write bad enough for it to turn into a bestseller. If nothing else I don’t think he would feel good about making the programmer into big enough of an asshole for it to work in the “romantic” genre.

          Maybe satire?

  5. PurePareidolia says:

    Awesome, I can’t wait to see what you do with it.

  6. MrPyro says:

    People ask me why I don’t wok on it anymore

    Minor typo; guessing that should be ‘work’.

    There’s a ‘cooking up code’ joke in there somewhere but work is sucking my sense of joy away.

  7. Gravebound says:

    “I’ve never been terribly impressed with Microsoft’s operating systems, but their developer tools are top-notch.”

    Speaking of Microsoft’s operating systems, were you ever going to make a post on your impressions of Windows 7? Or did you already post that and I just missed it?

    • Woodthorn says:

      He replied up there. ^

    • Shamus says:

      Yeah, I forgot about that.

      Short version: Win 7 is the new XP. I can live with this. I have a few small fiddly complaints, but it’s pretty trivial given the age difference.

      Farewell XP. That was the first Microsoft operating system that I didn’t think was sabotaged by some sort of anti-usability agents.

      EDIT: Moved my this, which was replied to the wrong comment. Pardon me, I seem to be new to the internet.

      • Peter H. Coffin says:

        Concur utterly. 7 just gets out of the way in the way that a good OS should. I don’t want to pay attention to anything involving “make computer go” more than once a week, and 7 more or less manages this.

        • Tse says:

          Unless you don’t turn off UAC… and you use the start menu to get to the important parts of the OS, I still have trouble getting to the network settings any other way.

      • Deoxy says:

        Short version: Win 7 is the new XP. I can live with this. I have a few small fiddly complaints, but it’s pretty trivial given the age difference.

        So, several years go by, and they manage to create a new OS that is finally as good as… that other one they put out 10ish years ago. Standard Microsoft “progress”.

        But a little more seriously (maybe), it was predictable that 7 would be at least decent; since Vista failed to penetrate the market, and XP was enjoying an incredibly long life, MS was risking the XP “standard” going wild, becoming a stand-along standard that was emulated by better OS’s (so people could run the MS-compatible software they want, which is the primary thing that ties them to MS).

        In short, their entire business model was at stake. A real shame they realized that – we missed a good chance for escaping from them.

        Though yes, the Visual Studio environment is actually pretty good… other than being an absolutely incredible resource hog. But then, I’d be surprised if it weren’t, coming from MS.

  8. Simon Buchan says:

    Hey Shamus is trying out stuff! Neato. You’re not alone in like Visual Studio: they get so many tiny things right in their text input alone, using other programs – even really excellent, dedicated “purity of text” programs like Sublime Text – drives me a little crazy at all the niggly little things they get “wrong”. If only Visual Studio 11 didn’t look so butt -ugly….

    Octtree’s are cool and all, but I’m more of a kD tree guy. Remember to make your parent/child relationships implicit in their embedding array index for the last 3 or so levels for performance!

    • Anorak says:

      I’m all for KDTrees too. Just finished implementing one in my raytracer, which can now render the Stanford Dragon in ~10 seconds. Of course, with optimisiation it might get a lot faster, but probably never real time :(

      • Simon Buchan says:

        Try out CUDA – or if you’re lazy, try throwing an inner loop at Microsoft’s AMP. It’s pretty baddass what that can do.

        • Anorak says:

          I did try out OpenCL, for a different project, and found it confusing. I’ve never used AMP, but I’m trying to remain as platform independent as I can. Also, my specific implementation is in Java. It used to be C++, but I got to a point where to progress any further I would have had to do massive code restructuring, so I decided to start from scratch in a different language. Are there OpenCL bindings for Java?

          • Simon Buchan says:

            Wait, you’re raytracing in Java, and you want performance? You’re a braver man than me! I suppose with some effort you could mush all the important data into an array or two to get around pointer-chasing and object headers trashing your cache, but it seems like a lot of work. I assume you got rid of any object allocation inside your loops? (Nothing against Java here – C# has the same issues, though support for value types helps a lot)

            I’m assuming you’ve googled it already, but http://www.jocl.org/

            Yeah, the problem with MS’s programming stuff, especially recently, is that they’re getting a little too good at making Windows-specific code look attractive. I even get all wistful looking at the C++11 support in VC11 that I can’t use for work because they dropped XP support :( Fortunately, in the case of AMP they’re trying to convince other platforms/compilers to adopt it, (including irrevokable patent grants) which would be great if it takes off.

            • Anorak says:

              Yeah, I know I’m pretty foolish to expect any kind of speed from Java. I’m also at the mercy of the Garbage Collector the entire time. I must say though, I wish I’d stuck with c++ when I decided on a rebuild, I very much miss pass by reference.

              On object allocation: I thought that in Java (don’t know how c# manages it), scope doesn’t actually effect performance because it all gets compiled to the same byte code anyway? I might be dead wrong there.

  9. psivamp says:

    I’m very interested in this project and the concurrent branching out into other tools. My computer just ate itself, but prior to the armageddon of this weekend, I had four IDEs in Windows that I wasn’t happy with and a copy of Visual Studio which I could make work, but that also chained me down to the Windows OS family. Somehow it never occurred to me to check out Qt’s dev tools even though I’ve installed them in linux before.

    Also, I should get off my butt and try to do some oct tree implementation. A month ago, at the programming assistance center (I work at a university helping beginning computer science students write programs in Java and C) I was helping students with a quad-tree program and the professor hadn’t told them anything about why you might want a quad-tree or what it was good for. I pointed a couple of people to Miguel Ceparo’s page to show them oct trees in action and basically said quad trees are the same deal but in 2D space. At the time, I really wanted to whip something up built on oct trees to show them something cool, but my OpenGL experience is limited. And, by limited, I mean that I figured out how to convert height maps to polygonal surfaces so that I can visualize functions in three dimensions, but I haven’t learned how to write a shader or even played with anything more advanced than the two most basic of the built-in shaders.

    This is getting really long-winded. Also, oct trees might be a good place to start on my latest pipe-dream of an idea for a programming project.

    tl;dr folks: I want to read this because I am looking for an open-source cross-platform development environment and I think oct trees are neat. Also, I suck with OpenGL.

  10. noahpocalypse says:

    Wait… Octrees? 3D quad trees?

    *gasp*

    You can make caves with those, can’t you?!

  11. DrMcCoy says:

    Eh, my IDE is vim, which does not look like MSVC. :P

    • Simon Buchan says:

      Does vim really have enough language knowledge to be called an IDE? At this point, I practically define an IDE by having code-hints/Intellisense. Unfortunately, that means I can’t find any Python or Scheme IDEs :) (actually, I lie, Microsoft’s Visual Python extension is pretty kick ass… when it works)

      • Anachronist says:

        When I use vim, it pays attention to the file type and highlights the language keywords appropriately, as well as identify matching parentheses, brackets, and braces. It apparently has the capability to recognize HTML as well as PHP, C++, C, and so forth. And of course it lets you split your view of a file or edit multiple files at once.

        I have developed a full-fledged complex PHP/MySQL web sites using only vim. If you’re used to the ‘vi’ way of editing (and most folks these days aren’t, but for my part I can’t fathom EMACS), then vim will be quite comfortable.

      • Ragnar says:

        Both vim and emacs support code-hints yes.

        Anyway, you might like Eric as a python-IDE. http://eric-ide.python-projects.org/

      • wererogue says:

        In addition to the above, I’d add that vim can generate ctags files which can be used to navigate by variable, function etc., rather like Visual Assist’s GoToImplementation and Find References.

      • Erik says:

        While I’m a long time vi/vim/gvim user, for Python and Tcl I’ve pretty much moved to ActiveState’s Komodo. The free version is just an editor, which I use for Python projects at home; the paid version is a full IDE with integrated debugger/appBuilder, which I use for Tcl at work. It has full tooltip language context help, at least. I’ve not used Visual Studio, so I can’t do a full comparison. But for the editor at least, the price can’t be beat.

        (Yes, Tcl is old-school and I’d rather be using Python, but I was using Tcl for a decade before I tried Python and this place has a large existing base of Tcl testing scripts that they needed help with.)

      • Zak McKracken says:

        Python IDEs:

        As Ragnar pointed out, Eric is not a bad option (on Windows)
        Apart from that:
        Eclipse has the Pydev plugin, which makes it work with Python pretty well (and very cool thing if you already use eclipse). It runs anywhere
        I’m using Spyder on Linux (I hink it runs anywhere) which has the same functionality as Eclipse, but I like it better for the smaller scripts I’m doing with Python since you don’t need to have a project Directory on your hard drive and can just open a python file with it, and there you go (but you _can_ still manage a project as in Eclipse or any big IDE)
        And then there’s Komodo, which has a free and a commercial version. Worked fine for me, but at some point it started crashing my computer, so I had to abandon it. That was a few years ago and not with the current version. It’s also at least on Linux and Windows.

      • Simon Buchan says:

        Thanks for the replies guys! Komodo looks pretty kick-ass for a full Python(/scripty) IDE: I’ll have to try it out some time to see how accurate it is (I have a tendency to use weird libraries, features, even builds of Python, that can trip up Visual Python). I remember trying out Eric, but for some reason I didn’t end up using it – I should try it out again. For now I’m fine without a proper IDE, but I’m starting to get to the point where a graphical debugger especially would be saving me time. For the more scripty stuff I’m digging Sublime Text though: every editor should have search as good!

  12. arron says:

    So Shamus going to work on code for 20 minutes each day whilst Josh, Rutskarn, Mumbles and Chris comment on the stuff on the screen? I like it. It’s a winning idea.

    And then we can have a in-depth argument about the number of spaces in tab indenting and other important “In development Spoiler Warning” issues we talk about here.. ;)

  13. swenson says:

    Oh goodie, another programming series! The one on Project Frontier was fascinating to me. I dabble in programming, but I’ve never done anything with graphics (3D or otherwise), so it was a very interesting look into that aspect. I look forward to seeing yet another aspect of it!

  14. Arvind says:

    Programming posts are back! Yay!

    I want to try out Qt, but I heard it doesn’t play well with other APIs – it needs to take over your main loop apparently.

  15. Mark Erik says:

    For what it’s worth, programs using the Qt SDK can be written using any IDE/editor. Obviously Qt Creator is specialized for Qt, but you can do it with VS too.

    I haven’t touched Qt in about a decade, but I have a very good impression of it compared to other C++ toolkits. MFC code is ugly (MACROS!) and Win32-specific, wxWidgets is cross-platform but is modeled after MFC, GTKmm uses an ugly toolkit, etc. Qt has a good object hierarchy, has a nice API, and pretty solid documentation.

    I’ll definitely be keeping an eye on this series.

  16. Moewicus says:

    The thing about Project Frontier is that it looks so great to
    just walk around in. I hope Shamus does something more with it eventually; it could be quite the meditative experience. I’ve found that with expansive and beautiful games like Wind Waker and Minecraft (one of these is prettier than the other) I inevitably find myself roaming the countryside/ocean watching the days pass, and doing this almost as much as I play the central part of the game. (I am weird.)

    • Anachronist says:

      Not weird. Because I personally dislike combat, when I encounter a lovely game my own inclination is to wander around and explore. Tera looks attractive to me from that standpoint (although I hear it simplified the combat, so maybe I’d enjoy that part too).

    • Neery says:

      I would absolutely pay some money just to be able to walk around Project Frontier for a while. It looks beautiful and interesting, and I’d really like to be able to explore it, even if that’s all I can do in there. Shamus, if you don’t want to open-source it, at least consider selling it as a zen-game-like thing to those of us who like to wander around computer games?

      • Adam F says:

        I’d second this. Actually what I really want is to have it as a screensaver like procedural city. If it’s finished, why not just release it?

      • Zagzag says:

        I would also pay just to have a verion of Frontier that worked, and that I could use to run around precedurally generated landscapes.

        Please please make it available. If you charge for it, then fair enough, and if not it looks like I’ll be using the donate button again.

    • Dave B says:

      When I started playing Lord of the Rings Online, I loved the art and world design so much that I wished I could wander around in it without having to constantly fight an endless stream of bears and spiders.

  17. Anachronist says:

    The programming stuff is what keeps me coming back to this blog. Fractal terrains, Procedural City, Frontier, all great stuff. Thanks Shamus. I echo the sentiment expressed above: if you decide a project is done and you’re moving on, never to return, perhaps consider making it open source.

  18. GiantRaven says:

    Nothing wrong with a ‘Minecraft clone’, so long as it strives to offer a different kind of experience to Minecraft itself. Terraria and the upcoming Castle Story are clearly influenced by ideas in Minecraft yet managed to differentiate themselves in distinctly different ways.

    People who complain about new games being developed containing Minecraft-esque ideas irritate me. Do they also complain about the waves and waves of Doom clones that come out every year?

  19. Sweet! I’m glad you’re getting some coding done again. It would be great to be able to be creative in exactly the way we want all the time, but the different creative portions of our soul need time to rest and recover. Like you said, you may as well be coding in the meantime!

  20. If it makes you feel better about the coding bug, these posts are what keep me coming back to your site. :)

  21. Matt says:

    Yes! These posts are why I started reading this site. I’ve missed them.

  22. Chris Headley says:

    Thanks for bringing this back, I am happy to hear the coding bug has bitten again. Look forward to more posts. Appreciate your developer insight, thoughts on design, and the typical pains of being a developer.

  23. wererogue says:

    Everybody seems to love Qt but it has only ever given me grief :/

  24. CaptainMaybe says:

    You know, I was just thinking about switching to integrative biology or English or public health, and then I saw this and couldn’t help but think: “Oh my god, this is so freaking cool!

    Now I will never be able to choose a major…

    Guess I better just stick with CS.

    Btw, awesome post. I had no idea about octrees–very interesting.

  25. JPH says:

    Speaking as someone with no programming experience whatsoever, I have to say I enjoyed your Project Frontier series if only because I thought it looked beautiful and was excited to someday play in it.

    I’d love it if you did something with it. Doesn’t have to be something terribly complex; I’m just fascinated by that cartoony world-building engine you created.

  26. Alex says:

    I’m really glad that you’re starting up on this again. I was looking at the old ones from last year while I’ve been messing around in Unity. These posts actually inspired me to make a game and, while I was looking at trade schools like the Art Institutes, my friends recruited me to make a game. So, thank you for that.

  27. adam says:

    octree+Perlin noise=Minecraft

    Those people are silly. Perlin noise is a foundational pillar of procedural content generation and you can’t really avoid it if you want to do anything procedural.

    BTW, Shamus, consider using Simplex noise instead.

  28. harborpirate says:

    Shamus, I would buy into a kickstarter, even if it were just to compensate you for open sourcing the code and assets that you’ve created for this when you’re done. I get the feeling that there are quite a number of people that wouldn’t mind taking a crack at that last 90%, if they just had something to start with; because getting something off the ground that actually works is the hardest part. (What you’ve done may not be as time consuming, but it requires a greater degree of technical expertise.)
    Lots of programmers can’t write 3D code from scratch, but give them a working prototype and they can modify it to do really interesting things. I’d pay for that head start, because to my knowledge, no “procedural 3D world toolkit” exists, and I think the Indie dev world could really use one.

    • This is a neat idea. I’d be glad to see some of the Project Frontier code in the package too. I think the main reluctance is that someone might want to buy the whole deal or pay to get exclusive access to the code. A scenario which would be ruled out by selling the code to the open source pool.

      That, and all coders (myself included) suffer from “bad code” phobia. It’s really scary releasing code you made, especially code you made yourself, and super especially something you made for fun. On top of that there’s the fear that someone will look at your code and say “Why did you do it that way?” and then demonstrate that your whole program (that you worked so long and so hard on) can be replaced by a Java Bean or something else equally insulting.

      I’m not saying any of these would necessarily happen. I’m just saying there are a lot of reasons someone might not want to release code, even working code, to the public. So, while I would be happy to see some of this code, I am equally happy letting Shamus sleep at night by NOT seeing it.

      • Shamus says:

        While I didn’t expect to sell the code, I was sort of hoping that the project might attract now opportunities the way my webcomic opened up the games journalism thing. But I think if that was going to happen, it would have happened by now. In fact, keeping the code is probably decreasing the chances of this now.

        And yes, I do suffer from a bit of bad code phobia. This is especially true with a project like Frontier where I was prototyping at a phenomenal rate. This is not going to be tidy, resume-ready code.

        Still, there’s probably a lot to be learned by putting it out. I’ll probably be doing so at some point this week.

      • Tse says:

        I may not know code, but I do know what you mean by “bad code” phobia. Showing your work always includes a little bit/a lot of uneasiness. I usually hesitate before showing other people my work, because I’m not sure what they would think of my mistakes and the general way I do things.
        I know every mistake I made and every way I could’ve done better which only exacerbates my worries.

        • Lord Nyax says:

          Ditto. I make videos for fun, and showing someone a video you threw together for the fun of it for the first time is scary. Even if you know it’s not awful, it’s stil scary. Even though I’ve never had somebody make fun of my videos (to my face at least) it’s still scary. Showing your work is just plain scary! Especially since I can see every little thing that’s wrong with it and can’t stop myself from wondering if they see it too! Still, it’s better to go through with it than keep it to yourself.

  29. Tse says:

    Any idea on where I would have to start in order to make cross platform games for smartphones and PC? I am a total newb when it comes to programming, but I like the logic of it, especially the way Shamus explains it. I’d like to make 3D games, because I AM an architect (jobless as I may be), after all.

    • Dave B says:

      Well, I’m probably not the person to ask, but I’ll throw this out there anyway. My impression of things is that you might want to look into:

      1. Java, as it is supported by Mac, Windows, and Linux. Java is also the language used by the Android SDK.

      or

      2. Microsoft XNA Game studio. It is a way to make and publish games for Windows, the Xbox 360, and Windows phones.

      • Tse says:

        I can’t publish to Android from my country (mobile developers don’t want to lose programmers to independent development), but I’d rather program for more than just windows. Thanks for the info, I’ll look into Java.

        • psivamp says:

          Java is also a “nice” language. It watches your back for you and tells you when you’re screwing up — unlike C and it’s more closely related languages which more often just blow up at run-time. Also, computing speeds have come far enough since I first learned Java for Java to be a realistic language to write things in.

          The university I attend/work for teaches their beginning CS majors Java to show them the ropes. The engineers who only take one class learn C. I tell people fairly frequently that Java is “nice” and C is “powerful and fast, but like a gun without a safety.”

          • Simon Buchan says:

            Java’s a bad newbie language – enforcing encapsulation rigor on people who are still trying to figure out what ‘while’ does (let alone a function!) seems like a bad idea. Python is a good match for several reasons: you don’t have to introduce anything you don’t need to, the syntax is minimal to the point of being trivial (no need to explain when, and when not to use ‘;’), it practically enforces good style through it’s layout syntax, etc…. The fact that it’s easy to set up, and a genuinely useful language that you can use in the real world gives it that bump over more academic languages like Logo, Scheme and Forth for me.

            On the other hand, it’s still a pain to set up graphics – even installing easy packages like PyGame with PIP requires that you have the Visual C++ 2008 (to match the compiler used for Python itself) installed.

            If we’re in formal education, I’d then recommend C (not C++) for a second year so they learn how machines actually work (I would, of course, never recommend C over well used C++ for real work), then start them on Scheme so you can start teaching them real computer science, instead of this stupid “programming” nonsense.

            But to answer the original comment, I’d seriously consider C++. You will get things wrong. Things will be hard. There is a lot of bad tips and “cargo cult”ing. It doesn’t change the fact that it’s the native language of systems everywhere, lets you use every API directly, and it’s blazing fast. Getting into it now is good timing, since C++11 cleaned up a lot of rough edges and made it much more usable (“vector<unique_ptr> load_meshes(string path)” is now not only the correct way, but the fastest)

            • psivamp says:

              I agree with much of your post. Java is a terrible language to start off on. It is overcomplicated for teaching basic computation and flow control, then it coddles you with all of its built-in exception handling and checking array lengths for you at run-time. C was the third language I learned, but I think it makes a wonderful starting language. C is, after all, the father of a good chunk of our modern work-horse languages (C++, C#, Objective-C, Java). It also relates more closely with what the computer actually ends up doing. (Okay, loops are a high-level construct. There’s no such thing as a block of code and all of your conditions and loops are really just jump statements when you run it through disassembly. The point being that it’s much closer than Java.)

              I personally wouldn’t have changed the program over from teaching C++ to teaching Java. Like I said, it’s a “nice” language, but nice isn’t always a good thing. Java lets you just write code willy-nilly without understanding the underlying structure. Which is decidedly not a good thing when you’re supposed to be learning.

              Edit: I also try to plug AltDevBlogADay.com at work, but it doesn’t seem like many of the CS students who come in have the coding bug — which I find to be a little disheartening.

          • some random dood says:

            I remember someone once comparing C to an F16 fighter jet – “When it flies, it’s beautiful, fast and graceful; but when it crashes, boy does it make a mess!”

    • Exetera says:

      This is probably not a suggestion you’ll hear from anyone else, but I’ve gotta suggest Javascript+HTML5 Canvas or Javascript+WebGL. True, the language is a bit quirky at times, and WebGL is a pain in the rear to troubleshoot, but what you write in it (*well, not with WebGL) will run on absolutely everything, guaranteed. Only problem is, everyone can read your source…

      Fun bonus: WebGL is a wrapper for OpenGL ES, so a port to mobile devices is relatively simple.

    • Piflik says:

      I would always recommend Unity. It is very easy to learn…so easy in fact, that even I can create a game…and I didn’t really have any programming expertise before staring on it.

    • adam says:

      I don’t really agree with the people saying Java is a bad newbie language. I started off with C and C++ but those languages can become overwhelming quickly. Trust me. Java is great in that it’s a very encouraging experience. The standard and Apache Commons APIs are powerful enough to let you do a lot of cool things with a minimum of fuss–unless you consider fuss having to abide by (mostly) good object-oriented practices.

      I’ve heard Python is also a good place to start. But again I don’t think that having to learn about semicolons and stuff like that is a big deal.

      Strongly typed languages are easier to read and understand imo and starting someone in a weakly typed language or one with the facility to do either feels like asking for trouble when they’re confronted with C++ or Java in a real-world situation (I’ve seen it at my own workplace). If I were a CS professor I would start with C, C++ and Java, which would get you a peek at the underlying structure while not punishing you for not understanding it all.

      • silver Harloe says:

        If you want to knock out junk with marginal performance fairly rapidly, by all means learn all the Java you want.

        If you want to be a good programmer who can really understand what’s going on, learn C. Learn Lisp. Learn at least one assembly language. Learn Prolog for a bit of a mind bender. Learn a stack based language like Forth. Learn Smalltalk. Learn SQL (not as much of a language, but still). Learn a concurrent language (I learned Ada because I did this in 1989, but there’s probably better choices by now, perhaps Erlang). Read the dragon book on compilers and write a compiler. Then go ahead and learn Java and/or C#, and you’ll be much much better at it than your coworkers with no facility to grok what the virtual machine is doing. Then memorize the Gang of Four book (Design Patterns) and learn how to do objects in Javascript for a different approach to the whole class/object thing (prototype inheritance instead of class inheritance), and you’ll be much much better at OO design than your cohorts. Read the devil book (Design and Implementation of the BSD OS), and you’ll be in a much, much better place to understand why the libraries are the way they are in C and C++ and, by extension, why a lot of libraries in other languages have some of the quirks they do. If you’ve done all that, you’ll understand why learning a new language is both trivial and hard (one the one hand, it’s just syntax and you can pick that up in a few hours. on the other hand, the real difficulty of learning a language is becoming familiar with the libraries). Do not expect the process of becoming a great programmer to take less than 10,000 hours of real, hard work.

        Sadly, I don’t know how to extend that list past that point to also learn graphics and windows programming and otherwise be useful for making games.

        • Dave B. says:

          While I can’t disagree with anything you said, I want to point out that not everyone who asks, “What programming language should I learn” is asking “What programming language(s) should I learn to be a master programmer?” Some people really mean, “What language will get me from zero to making a game the quickest.” The simplest tool for the job might be ugly, or inflexible, or just a bad tool in general, but should I ignore what they want and give them what I *think* they need? This is a genuine question. What does your experience tell you?

          • silver Harloe says:

            Oh, fair enough.

            I wish I knew the answer to “zero to making a game the quickest,” myself, especially since I’d be starting from somewhere slightly above zero.

            I suspect it depends on the kind of game you want to make: if it involves 3D graphics and many users, you may need to be a master programmer (AND have a bunch of experience and knowledge beyond what I listed) to have a shot at anything reasonable. But you could probably start making a flash game in a week :)

            My experience with teaching others is pretty limited – just to coworkers, really, and often as a matter of us teaching each other rather than having a master/grasshopper relationship. I suspect, though, you should tell people the truth when they ask: “yes, you can make junk, though perhaps junk with whizz bang, with things you can learn in a month, but this is what you need if you have real plans…”

            See, the problem I think people will bump into (though I could be way wrong about this) when they start learning to program by using some handy game library or whatever is that when they get to the limits of the what the library’s author thought of and what to add or change a feature, they will likely get lost – without understanding the system under the library, they they will lack a basis to make intelligent guesses about whether their feature is just “extend that class and we’re happy” or if it’s “well, there’s a reason they didn’t have that feature and you’d need to fundamentally re-architect part of the library to make what you want happen”. It’s kinda like using Sharepoint – you’re doing _great_ as long as you’re designing a website that looks and acts just like one of the sites the authors thought of, but you’re against a wall when you want to do something they didn’t think of.

            • Tse says:

              It’s true that I’d like to dabble in programming. Becoming a master programmer may become a far-off goal, but I would like to make something in 3D first. I don’t want to spend years learning programming only to find it’s not something I like. That’s why I’ve chosen Unity3D with Java script for my first step. I’ve even found a wonderful series of tutorials in my native language (Bulgarian):
              http://vbox7.com/play:fd6160fa&al=2&vid=8523980

              P.S. The ability to make game levels out of my architectural visualizations is also an important goal for me. I want to be able to do something few other architects can.

              • Dave B says:

                I’m glad that you have found a way to get started. In my opinion, the best way to learn programming is to do programming, and tutorials are a great start.

                I am also happy to see an architect get into game level design. Good luck with your projects!

            • Dave B says:

              …you should tell people the truth when they ask: “yes, you can make junk, though perhaps junk with whizz bang, with things you can learn in a month, but this is what you need if you have real plans…”

              A very good point. This is probably something that a lot of people either aren’t told, or don’t listen to.

        • adam says:

          “If you want to knock out junk with marginal performance fairly rapidly, by all means learn all the Java you want.”

          What the hell are you talking about? I don’t see any evidence anywhere that Java produces junk with “marginal” performance. That’s pure nonsense. Java is used everywhere and modern JVMs can produce native code that is every bit as fast as C++ code (depending on the task/hardware optimizations/etc, obviously). Don’t believe me? http://www.ibm.com/developerworks/java/library/j-jtp09275/index.html

          (That’s just one of many resources which refute your statements)

          As for the rest of what you said, you’re talking about pointing someone new to programming at the deep end and then asking them to swim to the shallow end. Why? What’s the point of that? Most people don’t or can’t start at the deep end. You start with something that you can easily wrap your brain around and build up from there. Why would you start someone off with zero programming experience learning assembly? That’s such a bad idea I can’t even produce a response to it other than NO. I started out learning C and C++ but I did not understand half of what I was doing. It was not until I moved to Visual Basic (this was years and years ago) that I started to get a grasp of what programming was all about. At that point I was able to move forward and relearn C and C++ with a much better grasp of what was going on, focusing on the nuances instead of the entire concept.

          There’s a reason introductory programming courses at university Computer Science programs start students out in Java. Claiming it’s a bad language, for starting out or otherwise, is either ignorance or elitism.

          If someone wants to learn how to program games, I would first make sure they had SOME experience with programming. Then, yes, I would likely point them toward C++. But that’s a completely different question.

  30. Blake says:

    I’ve missed your coding blogs, and having not written an octree either I’m interested to see whether you run into any issues with it.

  31. Bstrdsmkr says:

    I’d love to see the “generated” concept taken a step further and a sort of “Let’s Code” series. Have the commenters suggest and discuss new features, then pick your favorite and show us how you’d go about implementing it.

    For example, I love to see you go through the process of generating medieval towns to be placed in Fontier =D

  32. Yay more coding posts!

    I too get the itch to code, and there is no other way to scratch it. I need more programming in my day job.

  33. Retsam says:

    If Shamus is annoyed at being distracted from other projects like writing his book by a “coding bug”, he’s certainly finding no sympathy here. More like “Whoo! More coding!”

    • some random dood says:

      Surprised no-one has yet berated Shamus for not working on becoming ambidextrous. That way he could write novels with his left hand, and program with his right and put more of his mighty brain to work at the same time!

  34. Adam P says:

    Shamus, I would love it if you took this new project to game development instead of proof of concept. I’m still sold on the idea of Steamtown from however long ago. :D

  35. John Beltman says:

    Hi Shamus,

    I would still dearly love for you to write a book on how to program.

    The things that you have become known for are your programming and your sense of humour. You have written before about the unsuitability of learn to program books. I understand that maybe you have always wanted to be a fantasy author but it seems crazy to me to ignore the established audience you already have on this blog, and your talents that got you the audience in the first place.

    It is interesting how you say you have the programming itch at the moment. Perhaps you could take turns – fiction, non-fiction, fiction, non-fiction. This way you could scratch your programming itch as well. i.e. The Witch Watch, How to Program the Shamus Way, The Witch Watch 2, The History of Computer Games, …

    Warmest regards,

    John.

    • Darkness says:

      Hmph. That is a really good idea. The way Shamus explains things but keeps it at a reasonable level is really nice. I can show his stuff to others and they don’t get lost. When I explain I just continually get frustrated since I know most of the list of what THEY don’t know and cannot get past the but you don’t know … anything … realization. I thought being a teacher was a great idea until I had a couple of these sessions under my belt. Now, not even a little bit.

    • Cuthalion says:

      This. If you are at all inclined to write a book, or even a tutorial series, on how to program, I think it would be great.

    • Bubble181 says:

      Yup. Programming for Dummies is a crappy book that could be much improved upon by a braindead weasel; Shamus might actually make it an entertaining read even for grandmothers who don’t ever want to code. That takes quality writing, which is sadly missing in “technical” areas far too often.

  36. Bryan says:

    Nuts. Qt is the one UI library that I haven’t had to install in the last ten years or so. It looked like absolute *crap* at the time (2002-ish), so I stopped using it for personal stuff, and wrote everything new in gtk. (Mostly pygtk. While python is entirely different from C and C++, it sure worked well for what I was doing with it. Not to say it’d work for this, of course.)

    Anyway, eventually I was able to stop rebuilding qt when I rebuilt this system. (Moving off KDE helped as well, of course.)

    Now you’re going to make me go reinstall it (and presumably qtgl, or whatever the “get an opengl context for your qt window” addon control was), aren’t you. :-/

    ;-)

    • Darkness says:

      What are you using now? !KDE tools? Which distro?

      I am mostly command line stuff. I hate wasting cycles on GUI. I need ALL those cycles or my code is fast enough. Never fast enough.

      Edit: messed up KDE query.

      • Bryan says:

        No KDE installed. I run twm, basically, and only because it was part of the X11R6 bundle back when that was a thing that existed (before xorg, though it was also in the single xorg bundle back when that was a thing that existed).

        http://www.linuxfromscratch.org/lfs/view/development/
        http://www.linuxfromscratch.org/blfs/view/svn/

        Been running LFS ever since v3.3 IIRC. (Back before BLFS existed. Some time around 2001.) Qt was only built on the first of the LFS systems I set up, I believe. It was installed by default on the many installations of Mandrake that I ran before it, but that’s kinda hard to avoid on a distro.

        Though I haven’t been running /development/ or /svn/ for quite a long time now. (No time to rebuild the whole system.)

        For GUI programs I write, unless they’re ports of Shamus’s code, it’s gtk and (critically, since I get a lot more done per unit time) pygtk.

    • Jan says:

      2002-ish?
      They released Qt 3.0 in October 2001, so maybe you were still using 2.x at the moment. 3.0 and especially 4.0 changed a lot of stuff, including the theming. Qt 4.0 and onwards now use the native rendering and theming of the OS/Windowing system they’re on, instead of just emulating it. You should not be able to distinguish an app using Qt from one that uses just the native OS toolkit, based on the toolkit alone. That’s not to say that the toolkit might influence you toward some basic design choices, but the “looks like crap” argument for the toolkit (I say nothing about anything built on top of it) is not valid anymore since at least 2005, when Qt 4.0 was released.

      • Bryan says:

        “Native theming”? What is this “native theming” you speak of? :-)

        There *is* no native theme on Linux. Only in a particular desktop environment (or potentially a particular window manager, though having any kind of theming setup gets your WM placed closer to a DE in my — admittedly extremely limited — book).

        Also see above.

  37. Neko says:

    Yesssss, come to the Qt side, it is Cuuutee.

    Seriously, though, I hope you find it interesting enough to get you back into the programming. And if it’s not for you, no big deal. Incidentally, I mostly use my text editor and a few open terminals for C++ && Qt code. I only pull out the Qt IDE when there’s someone else’s code I need to dig through, or I just feel like bashing something out quickly.

    Look for the ,a href=”http://doc.qt.nokia.com/4.7-snapshot/demos.html”qt-demos app that I assume gets installed with your SDK bundle. I’ve always found it a great source of inspiration for “oh hey, I could put something like this in my own program!”

  38. DrMcCoy says:

    Ah, while we’re at the topic of 3D and Perlin noise.
    Partly OT, but might be interesting to you / others here. The demogroup farbrausch (of the product, kkrieger and Debris fame) has released their sources: https://github.com/farbrausch/fr_public

    • Simon Buchan says:

      Oh hey! I’ve been waiting for that! Their “lego block” procedural textures seemed pretty neat, and I’ve wanted to put that in front of a “real” designer and see what they thought.

  39. Scott (Duneyrr) says:

    Does anyone have a video of the Escapist Movie Night?

  40. MadTinkerer says:

    If comparisons to Minecraft bother you, reduce the scale by four. The big “mistake” most other voxel terrain based games make is working on the same scale as Minecraft. One of the big things Cube World has done right is reduce the scale by two and thereby increase the “resolution” of the world (some of which is wasted making the characters have cartoony instead of realistic proportions, but you can still tell if you look closely).

    The number one thing you can do to make your project stand out from the Minecraft “clones” (only a few of which are actually remotely similar) is to adjust the scale another twofold and have even smoother terrain, more detailed plants, thinner walls that could look like two different materials on opposite sides without being two meters thick, more roundish arches, etc.

    Basically, instead of 1 meter^3 cubes, you should have 25 cm^3 cubes.

  41. [...] now. With inspiration found in Shamus Young’s Project Octant and The Real Katie’s Roguish PyGame, I (in the space of an afternoon) made this, a rough but [...]

  42. Sam says:

    Thought you would want to know that the link to the ‘novels’ section under your programming section is broken.
    Before I become a videogame pundit and novelist“… links to page id 6 vs 16453.

One Trackback

  1. By Shabavh » APATTAZ: Intro on May 9, 2012 at 2:51 am

    [...] now. With inspiration found in Shamus Young’s Project Octant and The Real Katie’s Roguish PyGame, I (in the space of an afternoon) made this, a rough but [...]

Leave a Reply

Comments are moderated and may not be posted immediately. Required fields are marked *

*
*

Thanks for joining the discussion. Be nice, don't post angry, and enjoy yourself. This is supposed to be fun.

You can enclose spoilers in <strike> tags like so:
<strike>Darth Vader is Luke's father!</strike>

You can make things italics like this:
Can you imagine having Darth Vader as your <i>father</i>?

You can make things bold like this:
I'm <b>very</b> glad Darth Vader isn't my father.

You can make links like this:
I'm reading about <a href="http://en.wikipedia.org/wiki/Darth_Vader">Darth Vader</a> on Wikipedia!