{"id":16009,"date":"2012-05-29T04:01:10","date_gmt":"2012-05-29T09:01:10","guid":{"rendered":"http:\/\/www.shamusyoung.com\/twentysidedtale\/?p=16009"},"modified":"2012-05-29T04:15:45","modified_gmt":"2012-05-29T09:15:45","slug":"project-octant-part-13-bug-hunt","status":"publish","type":"post","link":"https:\/\/www.shamusyoung.com\/twentysidedtale\/?p=16009","title":{"rendered":"Project Octant Part 13: Bug Hunt"},"content":{"rendered":"<p>And now it is time to point and laugh at all the silly things that dum-dum Shamus has done over the past week or so. Let&#8217;s start with noise. Remember the image I ended with last week?<\/p>\n<p><table width='600'  cellpadding='0' cellspacing='0' border='0' align='center'><tr><td><a href='images\/octant12_14_large.jpg'><img src='https:\/\/www.shamusyoung.com\/twentysidedtale\/images\/octant12_14.jpg' class='insetimage' width='600' alt='octant12_14.jpg' title='octant12_14.jpg'\/><\/a><\/td><\/tr><\/table><\/p>\n<p>There&#8217;s something wrong here. Don&#8217;t get me wrong, that&#8217;s a cool canyon and all.  I&#8217;m not knocking the canyon.  The problem is that there is another canyon right next to this one, running in parallel. Defying all odds, there&#8217;s yet another one, very similar, just beyond that one. And another.  And&#8230;<\/p>\n<p>Yeah. My noise-generating system is broken.  It&#8217;s not spewing out duplicate data.  These canyons are unique.  But we&#8217;re seeing patterns when we shouldn&#8217;t.  <\/p>\n<p>Also, the format of the noise makes it really annoying to use.  It&#8217;s supposedly giving me values between zero and one. As I&#8217;ve plugged these numbers into my world-building system, I&#8217;ve noticed that it&#8217;s all kind of homogeneous. The <em>possible<\/em> range might be between zero and one, but the <em>actual<\/em> values it gives me are very, very rarely lower than 0.4 or higher than 0.6.  This isn&#8217;t really a bug.  This is how value noise works, actually.  It&#8217;s just that it&#8217;s not very convenient like this.<\/p>\n<p>So far I&#8217;ve designed my world-building code around this clunky mess, constantly re-scaling the noise and fudging the numbers until things look the way I want.  But this makes for messy code. If I want mountains that range from zero to fifty, I&#8217;ll end up with code that looks like <code>(noise - 0.4) * 250<\/code>.  Those are what programmers call &#8220;magic numbers&#8221;.  They&#8217;re unexplained values floating around in source, and another programmer (probably future-Shamus, programming from the cockpit of his flying car) will look at this and say, &#8220;If you want mountains fifty meters high, why are you multiplying by 250? And subtract point four? What&#8217;s that all about? What idiot wrote this crap!?!&#8221;<\/p>\n<p>So let&#8217;s calibrate:<\/p>\n<p><!--more--><table   class=\"\" cellpadding='0' cellspacing='0' border='0' align='center'><tr><td><img src='https:\/\/www.shamusyoung.com\/twentysidedtale\/images\/octant13_2.jpg' class='insetimage'   alt='octant13_2.jpg' title='octant13_2.jpg'\/><\/td><\/tr><\/table><\/p>\n<p>This is our noise spectrum.  When we generate a page of noise values, we should see a little bit of everything. A bit of low red. A bit of high purple. Things can follow a bell curve, but we need to see the full spectrum.  If, during the course of churning out the 262,144 pixels for one of the upcoming images, I don&#8217;t get a single purple pixel, then it&#8217;s so rare that it&#8217;s not worth having in the spectrum.  If I actually needed something to be that rare, there are better ways to come up with it than sacrificing one-fifth of my noise spectrum. <\/p>\n<p>So let&#8217;s generate a page of noise and see what we get:<\/p>\n<p><table   class=\"\" cellpadding='0' cellspacing='0' border='0' align='center'><tr><td><img src='https:\/\/www.shamusyoung.com\/twentysidedtale\/images\/octant13_3.jpg' class='insetimage'   alt='octant13_3.jpg' title='octant13_3.jpg'\/><\/td><\/tr><\/table><\/p>\n<p>Wow. There are so many things wrong with this image.  <\/p>\n<ol>\n<li>We&#8217;re using less than a fifth of the spectrum.\n<\/li>\n<li>Worse, it&#8217;s not even the middle of the spectrum!\n<\/li>\n<li>We can see a strong pattern of vertical lines.\n<\/li>\n<li>I don&#8217;t know if it will show up in the eventual blog-screenshots, but I can see a couple of pure blue pixels mixed in with the cyan values. These sudden shifts in values should not happen.  The noise generator is for making smooth outputs.  If I needed spikes, I&#8217;d generate them some other way. Random spikes like this will produce&#8230;\n<\/li>\n<\/ol>\n<p><table   class=\"\" cellpadding='0' cellspacing='0' border='0' align='center'><tr><td><img src='https:\/\/www.shamusyoung.com\/twentysidedtale\/images\/octant13_4.jpg' class='insetimage'   alt='octant13_4.jpg' title='octant13_4.jpg'\/><\/td><\/tr><\/table><\/p>\n<p>Yeah.  Man, I&#8217;ve been seeing those telephone poles now and again. I wondered what was causing those.<\/p>\n<p>(Half an hour of furious pondering, which non-programmers mistakenly refer to as &#8220;surfing the web and reading webcomics&#8221;.)<\/p>\n<p>Ah! The input image.  I&#8217;m using a image of pure noise &#8211; a 256&#215;256 image of random color. It&#8217;s a PNG file. I&#8217;m just loading these data values and dumping them into a big &#8216;ol bin of bits.  Can you intuit the problem?<\/p>\n<p>PNG files have an alpha channel. One pixel is a single byte of red, another byte for blue, then green, then the transparency. This image is opaque, which means the forth value is always the same. (Max value.) I&#8217;m sure this problem appeared when I switched from Qt to Visual Studio and changed to using a different set of image-loading tools. <\/p>\n<p>I could modify the image to have random opacity, but it&#8217;s probably safer to just drop that channel.  Let&#8217;s see:<\/p>\n<p><table   class=\"\" cellpadding='0' cellspacing='0' border='0' align='center'><tr><td><img src='https:\/\/www.shamusyoung.com\/twentysidedtale\/images\/octant13_5.jpg' class='insetimage'   alt='octant13_5.jpg' title='octant13_5.jpg'\/><\/td><\/tr><\/table><\/p>\n<p>Much better. The stripes are gone and the noise is nearly centered. (It&#8217;s at 0.53, when a perfect average ought to come up with 0.5. Not sure if I care about that.  That could be an artifact of the noise I&#8217;m using.) <\/p>\n<p>I&#8217;m not so sure about the blue dots, which is where the noise system is giving a value that doesn&#8217;t fall in with the gentle gradients it normally turns out.  Why? Blue is not the bottom of the spectrum. The bad values all appear on the same spot in the range. For example, we&#8217;re not seeing incongruous purple or red dots. I don&#8217;t see any pattern to their placement. <\/p>\n<p>I&#8217;ll have to come back to this one later. <\/p>\n<p>So now I just need to re-scale my noise to fill as much of the spectrum as I can.  Here is what we&#8217;re doing:<\/p>\n<p><table   class=\"\" cellpadding='0' cellspacing='0' border='0' align='center'><tr><td><img src='https:\/\/www.shamusyoung.com\/twentysidedtale\/images\/octant13_6.jpg' class='insetimage'   alt='octant13_6.jpg' title='octant13_6.jpg'\/><\/td><\/tr><\/table><\/p>\n<p>Most of the spectrum is going to waste, which means that I&#8217;m groping around, trying to find the range where the &#8220;interesting&#8221; noise is taking place. Every time I ask for some noise I re-scale it, and then re-normalize it to the zero-to-one scale. It&#8217;s much neater and more sensible to do this at the source, so that I can just blindly query some noise and be sure that the numbers I&#8217;m getting will be useful.  The only danger is this:<\/p>\n<p><table   class=\"\" cellpadding='0' cellspacing='0' border='0' align='center'><tr><td><img src='https:\/\/www.shamusyoung.com\/twentysidedtale\/images\/octant13_7.jpg' class='insetimage'   alt='octant13_7.jpg' title='octant13_7.jpg'\/><\/td><\/tr><\/table><\/p>\n<p>If I narrow the range too much, then occasionally a value will drift out of bounds.  These values are probably very rare, but statistically inevitable.  In fact, this is sort of the whole point of noise.  <\/p>\n<p>If I don&#8217;t tighten the range enough, then my values will be too flat and I&#8217;ll once again clutter up my code with scaling and endless tweaking.  If I tighten it too much, I&#8217;ll end up with clamped values.  <\/p>\n<p>A clamped value might look like a mountain with a perfectly flat top. Or a cave with a flat bottom.  It&#8217;s some point where the numbers drift off to the maximum value and just&#8230; stay there. Suddenly I&#8217;m not getting noise anymore.  I&#8217;m just getting max value, over and over. <\/p>\n<p>I adjust the ranges a bit.  Let&#8217;s see how it looks:<\/p>\n<p><table   class=\"\" cellpadding='0' cellspacing='0' border='0' align='center'><tr><td><img src='https:\/\/www.shamusyoung.com\/twentysidedtale\/images\/octant13_8.jpg' class='insetimage'   alt='octant13_8.jpg' title='octant13_8.jpg'\/><\/td><\/tr><\/table><\/p>\n<p>Lots of variety. If some of those red or purple spots result in a bit of flatness, we can probably play it off as part of the scenery. Much better to have the occasional freak spot than to have it bland everywhere.<\/p>\n<p>Which leads us to our final problem:<\/p>\n<p>On Friday I mentioned that it was taking crazy long to fill in the scene with data. It turns out that problem wasn&#8217;t so much a bug as a design flaw. Consider a grid of nodes:<\/p>\n<p><table   class=\"\" cellpadding='0' cellspacing='0' border='0' align='center'><tr><td><img src='https:\/\/www.shamusyoung.com\/twentysidedtale\/images\/octant13_1.jpg' class='insetimage'   alt='octant13_1.jpg' title='octant13_1.jpg'\/><\/td><\/tr><\/table><\/p>\n<p>Imagine we&#8217;re looking down from above, and the viewer is in A.  Each letter represents a node &#8211; a 16x16x16 block of points.  The order of the letters is the order in which nodes will be filled in.  A, B, C, D, etc.  This builds the stuff close to you first, and works outward in concentric circles.  There are three stages to building a node:<\/p>\n<ol>\n<li>Generation &#8211; The noise system is used to create a bunch of points and decide what they&#8217;re made of. (Grass, dirt, what-have-you.)\n<\/li>\n<li>Lighting &#8211; The system looks at each block and figures out if its in direct sunlight or not.  Eventually this stage will probably include some sort of other light sources, so that underground areas can be lit.\n<\/li>\n<li>Mesh construction &#8211; We take the data from the previous two steps and turn it into a big pile of polygons.\n<\/li>\n<\/ol>\n<p>I had a bit of code in here that would cause new nodes to force their neighbors to update.  When A is built, nothing in B exists yet. So it can&#8217;t make these blocks match up with whatever will eventually appear in B.  When B does appear, there will be a gap or a seam or a bunch of extra polygons or whatever. <\/p>\n<p>To correct for this, as soon as B is done building it would nudge A and tell it to update. So then A rebuilds, repeating step #3. Then C is created, and it nudges A, and A does step 3 again. Then D, and A is built again. Then E, then A <em>again<\/em>. Then it gets really, really stupid&#8230; <\/p>\n<p>F is created, and it causes both B and C to be rebuilt. Oh, and because it shares a single corner cube with A, then A must be built again.<\/p>\n<p>The upshot is that in the process of filling in the nine nodes in the center, A will be rebuilt nine times. <\/p>\n<p>Note that I couldn&#8217;t see this happening. I mean, if it rebuilds a node and the node looks exactly the same, then the process is invisible to me.  This is a case of me just not thinking things through.  I put in that bit of nudge code and moved on without considering the implications.  <\/p>\n<p>The fix for this was to just have it be a lot less stupid about when it builds meshes. A doesn&#8217;t do step #3 until the other 8 nodes have finished the first two steps.  It means you wait a little bit longer up front, but in the long run it saves a ton of time. <\/p>\n<p>Let&#8217;s crank up the draw distance:<\/p>\n<p><table   class=\"\" cellpadding='0' cellspacing='0' border='0' align='center'><tr><td><img src='https:\/\/www.shamusyoung.com\/twentysidedtale\/images\/octant13_13.jpg' class='insetimage'   alt='octant13_13.jpg' title='octant13_13.jpg'\/><\/td><\/tr><\/table><\/p>\n<p>Here we have a draw distance of 384 meters.  It&#8217;s a bit bland because we&#8217;re only making these swiss-cheese canyons.<\/p>\n<p>I&#8217;d wanted to go for higher, but it seems sort of silly now.  This feels pretty far. It took almost a minute to generate this. (Although over half of that time is spent filling the last few rows on the horizon.  Every doubling of the view distance is quadruple the time cost.)  That&#8217;s not too bad for rough, un-optimized code.  I&#8217;m content that we don&#8217;t have any serious design flaws or obvious structural problems that would prevent us from creating scenery at an interactive rate. Moving at a full sprint, I can&#8217;t get anywhere near the edge of the terrain as it&#8217;s being built. <\/p>\n<p>Let&#8217;s do one last test.  Let me double the draw distance again. I&#8217;m also going to disable the fog, since I think it makes things look far away and kind of screws with your perception of how far you&#8217;re seeing.<\/p>\n<p><table   class=\"\" cellpadding='0' cellspacing='0' border='0' align='center'><tr><td><img src='https:\/\/www.shamusyoung.com\/twentysidedtale\/images\/octant13_14.jpg' class='insetimage'   alt='octant13_14.jpg' title='octant13_14.jpg'\/><\/td><\/tr><\/table><\/p>\n<p>That&#8217;s over half a kilometer. Took ages to generate the terrain, and I don&#8217;t think it really looks any more impressive. And on the ground, you can almost never see that far because there&#8217;s always something in the way.  Maybe this would change if we had some mountains or other large-scale features to look at.<\/p>\n<p>At any rate, the framerate is still over 40fps, even with this aggressive draw distance.  Again, it&#8217;s too early to start patting myself on the back, but it does mean I don&#8217;t have any serious problems in what I&#8217;ve made so far.<\/p>\n<p>I&#8217;m looking at this endless expanse of Swiss cheese and thinking the next step should be to add some variety. <\/p>\n","protected":false},"excerpt":{"rendered":"<p>And now it is time to point and laugh at all the silly things that dum-dum Shamus has done over the past week or so. Let&#8217;s start with noise. Remember the image I ended with last week? There&#8217;s something wrong here. Don&#8217;t get me wrong, that&#8217;s a cool canyon and all. I&#8217;m not knocking the [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[66],"tags":[233],"class_list":["post-16009","post","type-post","status-publish","format-standard","hentry","category-programming","tag-octant"],"_links":{"self":[{"href":"https:\/\/www.shamusyoung.com\/twentysidedtale\/index.php?rest_route=\/wp\/v2\/posts\/16009","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.shamusyoung.com\/twentysidedtale\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.shamusyoung.com\/twentysidedtale\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.shamusyoung.com\/twentysidedtale\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.shamusyoung.com\/twentysidedtale\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=16009"}],"version-history":[{"count":0,"href":"https:\/\/www.shamusyoung.com\/twentysidedtale\/index.php?rest_route=\/wp\/v2\/posts\/16009\/revisions"}],"wp:attachment":[{"href":"https:\/\/www.shamusyoung.com\/twentysidedtale\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=16009"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.shamusyoung.com\/twentysidedtale\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=16009"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.shamusyoung.com\/twentysidedtale\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=16009"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}