T w e n t y S i d e d

So there are thousands of webpages that have information we want. When faced with this problem, ancient civilizations used to go to these pages using Internet Explorer 6 and copy the data into Notepad. We don’t know what they did with it after that, because they got eaten by Woolly Mammoths or conquered by Mongols or whatever. I’m not a historian so I might be slightly off with my timeline, but you get the basic idea: The past was hard.

But now we have these newfangled web scrapers that can surf the web for you and harvest whatever data you like. The problem is that putting the data into Notepad isn’t terribly helpful. Great, now you have an enormous text file of random facts. Are you going to sit down and read it manually? Probably not. So what do we do? Write another program to read that file? You need to turn this text into data sooner or later, and to do that we need to put it into a database.

Continue reading ⟩⟩ “Scraping Part 2: Full Control”

Shamus Young is a programmer, an author, and nearly a composer. He works on this site full time. If you'd like to support him, you can do so via Patreon or PayPal.

You might remember a couple of months ago I posted a bunch of charts of video game data. The obvious question that went unanswered in those postsTo the genuine annoyance of some. was, “Where did this data come from?” So let’s talk about that.

Actually, before we talk about that I should make it clear that this is a programming project. I should note that that this project pre-dates that crazy stuff I was doing with BSP loading a couple of weeks ago, but I’m posting them in the opposite order. For some reason.

Maybe reading yet another programming project sounds fun, but this isn’t a game-focused project with cool screenshots to show off my project. This is pretty dry and you’ve already seen the end result. I’d talk you out of reading more, but we both know you’re going to read this stupid thing no matter what I say. So Let’s just get this over with.

Continue reading ⟩⟩ “Scraping Part 1: Easy Mode”

Shamus Young is a programmer, an author, and nearly a composer. He works on this site full time. If you'd like to support him, you can do so via Patreon or PayPal.

In the previous entry I ended with an abrupt half-joke where I said I got raytracing working. The idea was that I spent days struggling to get simple old technology working properly, but then casually mastered cutting-edge tech in a single sentence. Sadly, it’s not totally true. I got access to raytracing, but I think it’s a stretch to say it’s “working”.

When I left off, I presented you with an image that looked more or less like this:

That’s what the program looks like when you leave it alone, but if you move the camera even slightly then it looks something like this: Continue reading ⟩⟩ “Adventures in Raytracing”

Shamus Young is a programmer, an author, and nearly a composer. He works on this site full time. If you'd like to support him, you can do so via Patreon or PayPal.