{"id":21302,"date":"2013-10-09T02:02:59","date_gmt":"2013-10-09T07:02:59","guid":{"rendered":"http:\/\/www.shamusyoung.com\/twentysidedtale\/?p=21302"},"modified":"2015-07-01T04:47:48","modified_gmt":"2015-07-01T09:47:48","slug":"project-good-robot-22-text-files","status":"publish","type":"post","link":"https:\/\/www.shamusyoung.com\/twentysidedtale\/?p=21302","title":{"rendered":"Project Good Robot 22: Text Files"},"content":{"rendered":"<p>No pretty screenshots for you today, just walls of text and some confused ramblings. I&#8217;m a little out of sorts and not really in any mental shape to be accomplishing interesting things. <\/p>\n<p>I&#8217;m feeling a bit un-creative and run down today. So I want to get away from the stuff that requires a lot of creativity (gameplay, writing, art) and just work on something straightforward and mechanical. I&#8217;m trying to make sure I do something on the project every day, but it&#8217;s probably a bad idea to do anything tricky while I&#8217;m all muddle-brained. I check out my miles-long to-do list and find something really dull: reading ini files.<\/p>\n<p>You know ini files. Stuff like this:<\/p>\n<p><!--more--><\/p>\n<pre lang=\"ini\">\r\n[Settings]\r\nSetup=1\r\nmusic=0.40\r\nsound=1.00\r\nfullscreen=0\r\n[Window]\r\nWidth=1456\r\nHeight=887\r\n<\/pre>\n<p>This is important because right now my ini reader is the only part of my program that&#8217;s still tied to Windows. To read and write ini files I&#8217;m calling <code>GetPrivateProfileString ()<\/code> and <code>WritePrivateProfileString ()<\/code>. Boo. If I ever want to do a Linux port then I need to replace those with my own code. <\/p>\n<p>You might remember that <a href=\"?p=18188\" title=\"Coding a Parser\">last Christmas I wrote an ini file parser<\/a>. The good news is that the parser seemed to turn out okay. The bad news is that I was using Qt at the time. Which means I was using the much more useful and powerful Qt string class and not the bog-standard std::string. <\/p>\n<p>Crud. <\/p>\n<p>std::string is how you&#8217;re supposed to use strings in C++. The problem is that it sucks. It doesn&#8217;t have a built-in way to strip whitespace off the beginning and end of a string, even though I have never ever written a parser of any sort that didn&#8217;t need that. Also, being able to compare two strings in a case-insensitive way is pretty fundamental to most text-processing problems. I need to find the [settings] section of the file even if it&#8217;s called [Settings] or [SETTINGS]. Also, I need to convert between numerical values and strings. Qt strings do all of these. std::string does none of them. <\/p>\n<p>So what happens? Everyone needs this stuff, so everyone writes their own string-manipulating tools to cover all these deficiencies. Then everyone has their own little set of string tools, which end up in all the code they write.  So now if I want to share my ini reading code with somebody they also need my string tools. They likely have the exact same string tools, and now they have duplicates hanging off my project, cluttering up my code, duplicating their code, causing name collisions, and just generally making life less easy.<\/p>\n<p>I moaned about this on Twitter yesterday and got this reply:<\/p>\n<p><center><\/p>\n<blockquote class=\"twitter-tweet\">\n<p><a href=\"https:\/\/twitter.com\/shamusyoung\">@shamusyoung<\/a> &quot;case&quot; and lexical sorting are very complicated in I18N. C++ stdlib will never do it well because it can&#39;t marry unicode.<\/p>\n<p>&mdash; James Iry (@jamesiry) <a href=\"https:\/\/twitter.com\/jamesiry\/statuses\/387682191280336896\">October 8, 2013<\/a><\/p><\/blockquote>\n<p><script async src=\"\/\/platform.twitter.com\/widgets.js\" charset=\"utf-8\"><\/script><\/p>\n<blockquote class=\"twitter-tweet\">\n<p><a href=\"https:\/\/twitter.com\/shamusyoung\">@shamusyoung<\/a> As one example, unicode strings can sort differently based on how they&#39;ve been normalized. But std::string has no &quot;normalize&quot;<\/p>\n<p>&mdash; James Iry (@jamesiry) <a href=\"https:\/\/twitter.com\/jamesiry\/statuses\/387682508977872897\">October 8, 2013<\/a><\/p><\/blockquote>\n<p><script async src=\"\/\/platform.twitter.com\/widgets.js\" charset=\"utf-8\"><\/script><br \/>\n<\/center><\/p>\n<p>I don&#8217;t even know what that means. Unicode still confuses me. I have no idea what &#8220;normalizing&#8221; a STRING would do. Now, normalizing a vector? That I get. But what does normalizing text do, make all the letters the same height? (I kid.)  <\/p>\n<p>I do wonder how Python, Java,  and C# do these things, since they don&#8217;t have this problem. Regardless of the cause, it&#8217;s still a terrible problem that ultimately undermines the strengths of the language. I&#8217;ve seen a lot of string tool sets in my lifetime. Most of them were good, a few were great, none of them were complete and all of them were incompatible with each other. Stuff like this is one of the reasons that <a href=\"?p=9557\" title=\"Object-Disoriented Programming\">C++ isn&#8217;t nearly as portable as it should be<\/a>. <\/p>\n<div class=\"dmnotes\">Hey you: Language evangelist. Yeah you. Don&#8217;t tell me to learn your favorite language because it doesn&#8217;t have this problem. Just don&#8217;t. That&#8217;s like telling someone to learn to play the the trumpet because they&#8217;re getting blisters playing the guitar. You&#8217;re not solving my problem, you&#8217;re giving me a new, larger problem: &#8220;Learn an entirely new language.&#8221; I&#8217;m trying to finish a game here, and switching languages mid-project would be idiotic even if I could be guaranteed that the new language could do everything I need it to do. (And that is not a guarantee by any stretch.) <\/p>\n<p>It takes time to learn to use a language well. Sure, you can get pretty far in a couple of weeks, but if you&#8217;re trying to make a playble, portable, stable, smooth-running game then it requires a depth of knowledge you just can&#8217;t get noodling around with example programs. And even if the learning curve wasn&#8217;t an issue, the simple conversion time to translate these 40k lines of code would be completely unreasonable.<\/p>\n<p>Don&#8217;t be <a href=\"http:\/\/www.youtube.com\/watch?v=cqkI691dxNg\" title=\"Captain Hindsight\">that guy<\/a>. I&#8217;m telling you. That guy is obnoxious.<\/div>\n<p>&#8220;Hey Shamus, just use <a href=\"http:\/\/en.wikipedia.org\/wiki\/Boost_(C%2B%2B_libraries)\">boost<\/a>!&#8221; Yeah, yeah. Of course, I&#8217;ve already got my own file-handling code, threading code, a bunch of math, and stuff for juggling strings. Should I re-write all my own stuff to use the more standardized boost library, or maybe just some of them? I mean, I hate to import a massive thing like boost SINCE ALL I WANTED TO DO WAS STRIP SOME SPACES OFF A STRING FOR THE LOVE OF KONRAD ZUSE, ARE YOU KIDDING ME?<\/p>\n<p>Anyway.<\/p>\n<p>So I&#8217;ve got this ini-reading code from last year.  I pull out the Qt strings, add std:strings, and then plug the holes left by the transition.  I run the program and it crashes.<\/p>\n<p>Or at least, I assume it crashes. I hit the run button and the window fails to appear. I assume it silently bombed on startup? A couple of minutes later I realize the thing is still running, it just never got to the part of the program where it created the window. I kill it and then try again, but this time stepping through the code manually. It turns out the problem is that this new code is now the slowest code I have ever written in my entire life. Like, I&#8217;ve never seen anything this deeply screwed. <\/p>\n<p>The file I&#8217;m reading looks like this:<\/p>\n<pre lang=\"ini\">[Map1]\r\nTitle=Chapter One\r\nSubtitle=The Freezing Caves\r\nFog=0\r\nWallColor=89f\r\nLightColor=444\r\nSkyColor=47a\r\nStoryPoints=5 6 7\r\nMusicTrack=steady-climb.ogg\r\nXp=150\r\nTileset=3\r\nStars=0\r\nPages=10\r\nSwarmSize=5\r\nPatterns=cave cave cave donut\r\nRobotSwarm=cutter1\r\nRobotMooks=worm1\r\nRobotNormal=worm2\r\nRobotBadass=worm3\r\n<\/pre>\n<p>That is the entire description for level one. The level description file itself is under 300 lines and is ~5k in size. For some danged reason, it takes ONE FULL SECOND to read the above data. That&#8217;s like taking eighteen months to make coffee. It&#8217;s so far beyond what&#8217;s sane or reasonable that I want to delete the parser to punish it for sucking so bad. <\/p>\n<p>The same parser worked reasonably under Qt, and the only change I made was the aforementioned switch to std:string. I&#8217;ve heard complaints that std::string is slow, but there&#8217;s no way I can blame it for this insane dawdling. I don&#8217;t know <em>what<\/em> to blame, actually. Really, in a single second I should be able to read in the whole file, put every single character in alphabetical order, <a href=\"http:\/\/en.wikipedia.org\/wiki\/ROT13\">ROT-13<\/a> encode the whole thing, remove all instances of the letter &#8220;F&#8221;, save it back to disk, and still have nine-tenths of a second left over. What is going on here?<\/p>\n<p>In a fit, I just rip the entire parser apart and re-write it from scratch. I make a fancy version that loads in the file, parses it, and keeps it cached in memory with key values already forced to lowercase for fast lookups. This solves the speed problem and now I can read ini files without using any Microsoft code. <\/p>\n<p>This is actually kind of unfortunate. If I had my head in any kind of working order, I would have run some tests to figure out where the bottleneck was coming from. As it stands, I probably did more work than I needed to, and I destroyed the problem instead of studying it. <\/p>\n<p>So that was my day. I did a ton of work to make a system that was probably overkill for my needs, all to replace a system that used to be a single line of Microsoft-specific code. I have no new features to show for my efforts. I&#8217;m not feeling too good about myself right now. <\/p>\n<p>Since today is <em>Post Huge Blocks Of Text Until People Fall Asleep<\/em> day, let me hit you with another one. I have a reason for all of this text-parsing, and the reason is this:<\/p>\n<p>(Protip: I don&#8217;t actually expect you to read this. Just <a href=\"#bottom\">skip to the bottom<\/a> and I&#8217;ll explain.<\/p>\n<pre lang=\"ini\">\r\n#Robot template\r\n#First is the name of the robot, in brackets. This is the name used to spawn\r\n#the robot in the console or in the levels file.\r\n\r\n[Template]\r\n\r\n#Ai core defines what kind of logic the bot will use to move. Options are:\r\n#* beeline - the bot heads directly for the player in a straight line.\r\n#* pounce - the bot attempts to circle around the player, spiraling inward until close, where it \"pounces\".\r\n#* tunnel - the bot loops around the player, passing through level geometry if needed.\r\n#* sentry - bot will move until close enough to attack, then root in place.\r\n#* orbit - the bot will circle-strafe the player.\r\n#* walk - moves along ground.\r\n#* hitnrun - Bot heads towards the player when it's ready to attack and away when waiting to refire.\r\n\r\nAiCore=walk\r\n\r\n#determines if this is a boss or not. Bosses have their name overhead and special music.\r\n\r\nBoss=1\r\n\r\n#This is the name shown to the user. Only applies to bosses.\r\n\r\nName=Walter\r\n\r\n#BodyParts defines what sprites will make up the robot's body.\r\n#If it's a boss, it will use the parts Boss0 through Boss7.\r\n#Otherwise it will use parts Robot0 through Robot29\r\n#Use -1 to end the group.\r\n#See sprite.ini \r\n\r\nBodyParts=9 24 23 24 23 21 -1\r\n\r\n#This controls how the body parts are arranged. Options are:\r\n#* fixed - Single-sprite body.\r\n#* snake - Sprites are chained together, with the ending ones chasing after the head.\r\n#* squid - Main body has 3 limbs of 2 segments each.\r\n#* jelly - Fixed body with one additional sprite hanging below it. Intended to look sort of \"jellyfish\"-ish.\r\n#* worm - Like snake, except the body segments will fall with gravity, collide with ground, and move up and down as the bot moves laterally.\r\n#* turret - Uses three sprites: A fixed main body with a rotating arm on either side. \r\n\r\nBodyType=worm\r\n\r\n# Size of the head in game units. For reference, player torso is about 0.1\r\n\r\nBodySize=0.3\r\nBodyColor=fff\r\n\r\n#Movement speed\r\n\r\nSpeed=1.9\r\n\r\n#How close the player must be before this robot will switch to alerted state.\r\n\r\nSpotDistance=4\r\n\r\n#Eye is the index of the eye used on the head. \r\n\r\nEye=1\r\n\r\n#This determines how the eye moves around. Options:\r\n#sweep, scan, player, heading\r\n\r\nEyeMovement=player\r\n\r\n#color of the iris\r\n\r\nEyeColor=000\r\n\r\n#offset of eye from center of head \r\n\r\nEyeOffset=0 0\r\n\r\n#Size of eye relative to head. 0.5 = half head size. \r\n\r\nEyeSize=0.4\r\n\r\n#This is the overpower level of all attacks: Lasers, melee, and missiles.\r\n#They correspond to the player levels. Bounce is the number of bounces each laser should do.\r\n\r\nAttackPower=1\r\n\r\n#The distance at which the robot can \/ should \/ try to use ranged attacks.\r\n#How this number is actually used depends on which AICore is being used.\r\n#Defaults to 3\r\n\r\nAttackRange=6\r\n\r\n#This controls if the robot will lead the player when aiming at them. (Makes them much more dangerous.)\r\n#0 = Never\r\n#1 = Always\r\n#N = Every Nth shot is predicted.\r\n\r\nAttackPredict=0\r\n\r\n#This is used by lasers. Number of wall reflections before the bullet stops.\r\n\r\nShotBounce=0\r\n\r\n#Number of shots in a single volley of laser fire. 0 to disable lasers.\r\n#Refire is time between volleys, in milliseconds.\r\n\r\nVolleyLaser=5\r\nRefireLaser=2500\r\n\r\n#Launcher is the kind of thing to fire. Options are:\r\n#basic, homing, cluser, and robot <robot name>\r\n\r\nLauncher=basic\r\n\r\n#Volley is how many missiles to fire in a single attack.\r\n#Refire is time between volleys, in milliseconds.\r\n\r\nVolleyMissile=2\r\nRefireMissile=1500\r\n\r\n#This is how often (milliseconds) it can hit with melee attack. Set to 0 to disable melee\r\n\r\nRefireMelee=150\r\n\r\n#The MAXIMUM number of each powerup to drop. Defaults to 0.\r\n#Actual drop rate is determined by dice rolls and game logic.\r\n#If DropMissiles is non-zero, then it will always drop at least 1.\r\n\r\nDropMissiles=10\r\nDropShields=10\r\n\r\n#Voce bank to play when seeing the player, being hit, or dying\r\n\r\nVoice=2\r\n\r\n#Armor is simple damage reduction. \r\n\r\nArmor=1\r\nHitpoints=10\r\n\r\n# Legs controls how many legs the bot has. (Only work PROPERLY for \"walk\" AiCores)\r\n# WalkHeight is the distance from the ground to the knee. WalkStride is the \r\n# lateral distance from the body to the knee. WalkCrouch is how high off the ground\r\n# the HEAD is in crouching stance. WalkStand is how high when standing. These values\r\n# are expressed in BODY LENGTHS, not world units. \r\n\r\nLegs=2\r\nWalkHeight=3\r\nWalkStride=3\r\nWalkCrouch=2\r\nWalkStand=5\r\n<\/pre>\n<p><A name=\"bottom\"><\/a><\/p>\n<p>Okay, the point of all that is that I wanted to move a bunch of the game systems into the config files.  It&#8217;s now possible to add new robots to the game just by editing the text file. You can then fire up the game and spawn your creation using the console.  Or you can open up the level file I showed you earlier and add the bot to the game. <\/p>\n<p>I originally opened up the game like this because a few people said they wanted to play around with modding. Once I was done I realized it was actually a lot nicer to make content for the game like this. I can mess around with game balance, difficulty, and pacing by altering the robot properties and without touching the code. <\/p>\n<p>This also had some odd side-effects.  At one point I had a special boss that was just hard-coded to create minions. When I generalized all robot properties I had to include this functionality. So now you can make any bot create other bots. (Instead of firing missiles, they fire robots.) I don&#8217;t even know what kind of trouble you can make with that.  For fun, I made a robot that &#8220;attacks&#8221; by creating another robot of the same type. This creates an interesting powers-of-two problem that&#8217;s basically unsolvable if you let it go on too long. The robots can&#8217;t hurt you, but they can make copies of themselves until your computer runs out of memory, which I think is a kind of victory state for them.<\/p>\n<p>So that&#8217;s how today went: I accomplished nothing, but I worked very hard on it.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>No pretty screenshots for you today, just walls of text and some confused ramblings. I&#8217;m a little out of sorts and not really in any mental shape to be accomplishing interesting things. I&#8217;m feeling a bit un-creative and run down today. So I want to get away from the stuff that requires a lot of [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[498],"tags":[],"class_list":["post-21302","post","type-post","status-publish","format-standard","hentry","category-good-robot"],"_links":{"self":[{"href":"https:\/\/www.shamusyoung.com\/twentysidedtale\/index.php?rest_route=\/wp\/v2\/posts\/21302","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.shamusyoung.com\/twentysidedtale\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.shamusyoung.com\/twentysidedtale\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.shamusyoung.com\/twentysidedtale\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.shamusyoung.com\/twentysidedtale\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=21302"}],"version-history":[{"count":0,"href":"https:\/\/www.shamusyoung.com\/twentysidedtale\/index.php?rest_route=\/wp\/v2\/posts\/21302\/revisions"}],"wp:attachment":[{"href":"https:\/\/www.shamusyoung.com\/twentysidedtale\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=21302"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.shamusyoung.com\/twentysidedtale\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=21302"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.shamusyoung.com\/twentysidedtale\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=21302"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}