{"id":15980,"date":"2012-05-25T04:31:31","date_gmt":"2012-05-25T09:31:31","guid":{"rendered":"http:\/\/www.shamusyoung.com\/twentysidedtale\/?p=15980"},"modified":"2012-05-25T13:13:28","modified_gmt":"2012-05-25T18:13:28","slug":"project-octant-12-fix-all-the-things","status":"publish","type":"post","link":"https:\/\/www.shamusyoung.com\/twentysidedtale\/?p=15980","title":{"rendered":"Project Octant 12: Fix All The Things"},"content":{"rendered":"<p>The last several posts have been a sort of lead-up to this one.  It probably doesn&#8217;t look that way, but all the fussing I&#8217;ve been doing is about to pay off. Moving from Qt to Visual Studio broke things. Moving to marching cubes broke things. Adding shaders broke things.  <\/p>\n<p>Since then I&#8217;ve been adding technology and fixing things, and as those pieces come together we&#8217;re finally getting something worth looking at. Bit we&#8217;re getting ahead of ourselves. First, I need to backtrack and explain a few items that I glossed over in earlier entries.<\/p>\n<p><table   class=\"\" cellpadding='0' cellspacing='0' border='0' align='center'><tr><td><img src='https:\/\/www.shamusyoung.com\/twentysidedtale\/images\/octant12_2.jpg' class='insetimage'   alt='octant12_2.jpg' title='octant12_2.jpg'\/><\/td><\/tr><\/table><\/p>\n<p>The big problem is that it is taking crazy, super, insano, bonkers long to generate terrain. It&#8217;s about a second per node. Several people, including fellow code-hobbyist <a href=\"http:\/\/www.sea-of-memes.com\/\">Michael Goodfellow<\/a> &#8211; suggested that the octree is really expensive to use. So let&#8217;s disable that and see if we get any speed gains.<\/p>\n<p>And by &#8220;disable&#8221; I mean, &#8220;write an entirely different, simpler storage system for my data&#8221;.  No trees. No dynamic allocation. Just one huge, honkin&#8217; block of memory where you can get anywhere in a single lookup. And let&#8217;s see how the new system performs&#8230;<\/p>\n<p><!--more--><table   class=\"\" cellpadding='0' cellspacing='0' border='0' align='center'><tr><td><img src='https:\/\/www.shamusyoung.com\/twentysidedtale\/images\/octant12_1.jpg' class='insetimage'   alt='octant12_1.jpg' title='octant12_1.jpg'\/><\/td><\/tr><\/table><\/p>\n<p>Wow. Just over a tenth of a second. Almost ten times faster. <\/p>\n<p>Click on octree.cpp. Hit delete. <\/p>\n<p>Now my project is probably a thousand lines smaller and runs ten times faster.  And of course now I have a project called &#8220;octant&#8221; that is no longer related to octrees. So that&#8217;s hilarious.   I think I should make this into a game called &#8220;Shoot Guy Online&#8221;, which is a single-player game about a woman who fights with a sword. <\/p>\n<p>I&#8217;m not really that sad to see the octree go.  The octree made it super-fast to do things that were rare, and super-slow to do things that were common. This is like a sprinter buying shoes that make her walk three times as fast and run ten times slower. <\/p>\n<p>I&#8217;d wanted to use it so that I could fit a really huge data set into memory, and see how far out I could get my draw distance.  It became clear a few days ago that memory wasn&#8217;t going to be my chief hurdle.  I was going to run into throughput problems <strong>long<\/strong> before I ran into memory problems, even though every doubling of distance results in eight times as much memory usage.  A kilometer view distance is just <em>not<\/em> going to happen within the context of this project.  (Oh, it&#8217;s <em>possible<\/em>, but it would require super-long loading times, or multi-pass block generation where the map would fill with increasing levels of detail, or some other trick to manage the sheer volume of stuff I have to process.  That could take days to investigate and it might not pan out.)<\/p>\n<p>Moving on&#8230;<\/p>\n<p>People have been asking about the surface normals. If you remember, those are vectors (think of them like arrows) that say which way a vertex is pointed.<\/p>\n<p><table   class=\"\" cellpadding='0' cellspacing='0' border='0' align='center'><tr><td><img src='https:\/\/www.shamusyoung.com\/twentysidedtale\/images\/octant12_3.png' class='insetimage'   alt='octant12_3.png' title='octant12_3.png'\/><\/td><\/tr><\/table><\/p>\n<p>Now that normals are working again, I can use lighting. <\/p>\n<p><table   class=\"\" cellpadding='0' cellspacing='0' border='0' align='center'><tr><td><img src='https:\/\/www.shamusyoung.com\/twentysidedtale\/images\/octant12_4.jpg' class='insetimage'   alt='octant12_4.jpg' title='octant12_4.jpg'\/><\/td><\/tr><\/table><\/p>\n<p>I&#8217;ve also got a super-cheap shadowing system that will darken a block if there is another block anywhere above it.  It&#8217;s strictly pass\/fail &#8211; the &#8220;sunlight&#8221; doesn&#8217;t spread. I&#8217;m not sure I want to change it. This looks pretty good as-is.<\/p>\n<p>Now that we have some free CPU cycles, let&#8217;s see if we can&#8217;t make more interesting terrain. Right now I&#8217;m just making rolling hills with a maze of random tunnels underneath. The above screenshot is from a rare location where the terrain dips below the tunnels and exposes them.  I&#8217;m not even using the noise system very much yet. No overhangs, no mountains. Cliffs are massive boring walls of smeared pixels. Yawn.<\/p>\n<p>So let&#8217;s just generate some random pockets of noise and use it to remove volumes of terrain.  How does that look?<\/p>\n<p><table   class=\"\" cellpadding='0' cellspacing='0' border='0' align='center'><tr><td><img src='https:\/\/www.shamusyoung.com\/twentysidedtale\/images\/octant12_5.jpg' class='insetimage'   alt='octant12_5.jpg' title='octant12_5.jpg'\/><\/td><\/tr><\/table><\/p>\n<p>That&#8217;s a lot more interesting. This more complex topography is really exposing the shortcomings of my texture mapping system. We&#8217;ll deal with that in a bit.<\/p>\n<p>Let&#8217;s add some different types of terrain.  Maybe some dirt would look good. <\/p>\n<p><table   class=\"\" cellpadding='0' cellspacing='0' border='0' align='center'><tr><td><img src='https:\/\/www.shamusyoung.com\/twentysidedtale\/images\/octant12_6.jpg' class='insetimage'   alt='octant12_6.jpg' title='octant12_6.jpg'\/><\/td><\/tr><\/table><\/p>\n<p>I like that.  And the tunnels look good too.  They looked very mechanical in the cube world, but here in blob world they look like natural caves.  It&#8217;s fun to run through them to go from one open area to the next. <\/p>\n<p>You know we need next? Let me grab the grass-scattering code I used in <a href=\"?p=11893\">Part 3 of Project Frontier<\/a>.<\/p>\n<p><table width='600'  cellpadding='0' cellspacing='0' border='0' align='center'><tr><td><a href='images\/octant12_7.jpg'><img src='https:\/\/www.shamusyoung.com\/twentysidedtale\/images\/octant12_7.jpg' class='insetimage' width='600' alt='octant12_7.jpg' title='octant12_7.jpg'\/><\/a><\/td><\/tr><\/table><\/p>\n<p>Yeah. Okay, enough fun. Now that we have a shader, let&#8217;s fix this texture mapping problem.<\/p>\n<p>The problem: I don&#8217;t just want to smear texture around randomly.  If I put a brick texture down, I want to be able to map it all over these lumpy hills and still have the bricks align properly. But how can we do this? We&#8217;ve got a 2D square that we&#8217;re trying to project onto complex 3D shapes. <\/p>\n<p>Well, there are three ways to apply patterned textures to a scene so that you can be sure they&#8217;ll line up. First, I could project the texture along the x-axis, so that walls facing east \/ west look good: <\/p>\n<p><table   class=\"\" cellpadding='0' cellspacing='0' border='0' align='center'><tr><td><img src='https:\/\/www.shamusyoung.com\/twentysidedtale\/images\/octant12_8.jpg' class='insetimage'   alt='octant12_8.jpg' title='octant12_8.jpg'\/><\/td><\/tr><\/table><\/p>\n<p>You can see this makes the x-facing walls look great, at the expense of making everything else look like a big smear.<\/p>\n<p>Also: I should note that these square trenches are part of my testing grounds, not my terrain system.  I created these for exactly this sort of situation.<\/p>\n<p>Another way to map things is to project the texture along the y-axis.<\/p>\n<p><table   class=\"\" cellpadding='0' cellspacing='0' border='0' align='center'><tr><td><img src='https:\/\/www.shamusyoung.com\/twentysidedtale\/images\/octant12_9.jpg' class='insetimage'   alt='octant12_9.jpg' title='octant12_9.jpg'\/><\/td><\/tr><\/table><\/p>\n<p>I don&#8217;t think you&#8217;d call this an improvement, unless you happened to be looking at a north-facing wall and you didn&#8217;t have any plans to turn your head.<\/p>\n<p>Finally, I could project from above:<\/p>\n<p><table   class=\"\" cellpadding='0' cellspacing='0' border='0' align='center'><tr><td><img src='https:\/\/www.shamusyoung.com\/twentysidedtale\/images\/octant12_10.jpg' class='insetimage'   alt='octant12_10.jpg' title='octant12_10.jpg'\/><\/td><\/tr><\/table><\/p>\n<p>But again, this just changes who gets screwed. What we need is to switch between these systems. If only there was some way to know which one we needed&#8230;<\/p>\n<p><table   class=\"\" cellpadding='0' cellspacing='0' border='0' align='center'><tr><td><img src='https:\/\/www.shamusyoung.com\/twentysidedtale\/images\/octant12_11.jpg' class='insetimage'   alt='octant12_11.jpg' title='octant12_11.jpg'\/><\/td><\/tr><\/table><\/p>\n<p>Oh right! Those surface normals we just got done fixing.  To make the above image, I just had it color the walls according to which way the normals were pointing. (Programmer talk: Actually, I&#8217;m using the absolute values of the normals and not <code>(vector \/ 2) + 0.5<\/code> like you do when converting normal maps to color data, because this is closer to how I&#8217;ll be using them.)<\/p>\n<p>The more something points along the x axis, the more red it has.  The more along y, the more green. The more up or down, the more blue.  Because our normals are normalized (remember from above when we resized them?) we know that if you add up red, green, and blue the sum of the values will always be 1.  So, red=1, green=1 is impossible. This means we have a nice, clear indication of which way a particular bit of wall is pointing.<\/p>\n<p>So all I need to do is pick my projection method based on which axis is the predominant one.  (Which color has the highest value.) <\/p>\n<p>Perhaps you&#8217;re wondering why we needed shaders for this.  Consider:<\/p>\n<p><table   class=\"\" cellpadding='0' cellspacing='0' border='0' align='center'><tr><td><img src='https:\/\/www.shamusyoung.com\/twentysidedtale\/images\/octant12_12.jpg' class='insetimage'   alt='octant12_12.jpg' title='octant12_12.jpg'\/><\/td><\/tr><\/table><\/p>\n<p>That vertical edge would be a bad spot.  On the left edge of those polygons they&#8217;re using one projection system, and on the opposite edge they&#8217;re using another system.  The texture would go crazy at this point.  Maybe it would repeat sixteen times to try to reconcile the discrepancy. Maybe it wouldn&#8217;t repeat at all, leaving a smear. The corner above would be even worse: Every vertex in that triangle would be using a different projection system.  That triangle is doomed. <\/p>\n<p>(We could also solve this if we didn&#8217;t have the triangles share verts.  If I wanted to pump three times as much data through our graphics pipeline I could give each and every triangle its very own copy of the verts to use its own projection system so that it didn&#8217;t have to share.  Also, I could do ten times as much programming and make it only duplicate verts when it&#8217;s required according to the shape of <em>who am I kidding it&#8217;s not even worth typing the rest of this sentence.<\/em>)<\/p>\n<p>This happens because texture mapping values are usually set at the corners of a triangle and then averaged across the face. But with a pixel shader we can use whatever projection system we like, at any time.  The projection can change on a per-pixel level, instead of per-vertex.<\/p>\n<p>So let&#8217;s see how it looks.  I&#8217;ll put a brick texture on the world to make sure the corners all match&#8230;<\/p>\n<p><table   class=\"\" cellpadding='0' cellspacing='0' border='0' align='center'><tr><td><img src='https:\/\/www.shamusyoung.com\/twentysidedtale\/images\/octant12_13.jpg' class='insetimage'   alt='octant12_13.jpg' title='octant12_13.jpg'\/><\/td><\/tr><\/table><\/p>\n<p>That texture is pretty much our worst-case scenario, and it seems to hold up pretty well, even on those round edges. <\/p>\n<p>So that&#8217;s the week. Going forward: There&#8217;s an infuriating bug that I can&#8217;t nail down that&#8217;s resulting in seams between nodes. (You might see really bright lines in some of the above sceenshots.)  It&#8217;s a bit random, but I need to nail that one down before it drives me nuts.<\/p>\n<p><table width='600'  cellpadding='0' cellspacing='0' border='0' align='center'><tr><td><a href='images\/octant12_14_large.jpg'><img src='https:\/\/www.shamusyoung.com\/twentysidedtale\/images\/octant12_14.jpg' class='insetimage' width='600' alt='octant12_14.jpg' title='octant12_14.jpg'\/><\/a><\/td><\/tr><\/table><\/p>\n<p>Also, I&#8217;ve got speed issues again.  Dropping the octree freed up a ton of horsepower, and then this new terrain system ate it all.  At a full sprint, the program can just barely generate stuff fast enough to keep polygons under my feet.  It takes a good twenty seconds to fill in a scene like this.  I need to climb down into the guts of this thing and see where the cycles are going and if there&#8217;s anything I can do to help that. I&#8217;m going to be working more on terrain, and I don&#8217;t want to have to wait twenty seconds every time I make a change. <\/p>\n","protected":false},"excerpt":{"rendered":"<p>The last several posts have been a sort of lead-up to this one. It probably doesn&#8217;t look that way, but all the fussing I&#8217;ve been doing is about to pay off. Moving from Qt to Visual Studio broke things. Moving to marching cubes broke things. Adding shaders broke things. Since then I&#8217;ve been adding technology [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[66],"tags":[233],"class_list":["post-15980","post","type-post","status-publish","format-standard","hentry","category-programming","tag-octant"],"_links":{"self":[{"href":"https:\/\/www.shamusyoung.com\/twentysidedtale\/index.php?rest_route=\/wp\/v2\/posts\/15980","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.shamusyoung.com\/twentysidedtale\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.shamusyoung.com\/twentysidedtale\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.shamusyoung.com\/twentysidedtale\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.shamusyoung.com\/twentysidedtale\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=15980"}],"version-history":[{"count":0,"href":"https:\/\/www.shamusyoung.com\/twentysidedtale\/index.php?rest_route=\/wp\/v2\/posts\/15980\/revisions"}],"wp:attachment":[{"href":"https:\/\/www.shamusyoung.com\/twentysidedtale\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=15980"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.shamusyoung.com\/twentysidedtale\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=15980"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.shamusyoung.com\/twentysidedtale\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=15980"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}