Technical Difficulties

  By Shamus   Feb 7, 2008   30 comments

please_stand_by.jpg
If you attempted to visit the site this morning you may have noticed that the thing was very nicely broken. Sometime yesterday this site started putting a “huge load” on the server. The load increased overnight, during the normally low-traffic hours, and by this morning it was pretty much shutting down, along with all the other websites on the same server.

My web host was a very good sport about this, all things considered, although I’m still not clear on where this load was coming from. I wasn’t flooded with newcomers or lots of comments, so the “visitors” were most likely not people. Perhaps an attack, perhaps just some very stupid spammer. It’s hard to tell.

I had this problem a couple of months ago, and then my host went in and blocked huge ranges of IP’s. Most of them from Russia and China. I wasn’t crazy about this solution, since I wasn’t consulted, but it did indeed fix the problem. Over the next few weeks I’d get emails from people saying they could reach me at work but not at home, or vice versa. I unblocked some of the IP ranges and the frustrated readers came back. I figured we were all good until it happened again today.

Sigh.

I’m going to leave the IP blocks in place this time. I take no joy in this. I know there are readers out there who won’t be able to get to my site, won’t be able to read this message, and won’t be able to reach me and tell me about the problem. Right now I don’t see any way around it. Someone out there – or more than one person – is pounding away at this site in such a way that they use up more resources than all of my legit visitors combined. The bandwidth costs are unacceptable, my website can’t function, and there is always the risk that my host will ask me to take my business elsewhere. (Note that they have never threatened this, I’m just saying.)

The “attack” – intentional or the result of inept spamming attempts – is particularly pointless since none of it makes it through. Between Akismet, Bad Behavior, and Peter’s Custom Anti-Spam, less than one in ten thousand spams make it through the nets, and I’m pretty sure the ones that DO make it through are entered by hand and not automated.

Sorry to all the users who are locked out, not that you can hear me. What a stupid, pointless shame.

201030 comments. (A twenty-sided die has 30 edges.)


  1. Rhykker says:

    Can’t the unjustly blocked readers access your webpage through a proxy?

    Sad to hear about the possible denial-of-service attack.

  2. Kleedrac says:

    Hate to say this but you know you’ve arrived on the web when you get DDOS’ed ;)

  3. Adam says:

    That sucks to the global asthmar.

  4. Thom says:

    Fortunately, I can still enter my favorite website where I can spend minutes reading and enjoying your stories Shamus :)
    By the way… I don’t know what your standings are on MMORPGs, but, having read your DM of the Rings comics, I’ve been wondering whether you would like LotRo or not. There’s a 7-day free trial available, so you could always try it out and give some comments ;)

  5. scragar says:

    I wrote a PHP script to fix this problem myself a while ago, basicly the script logs every IP’s last request, and if it recives too many requests too quicky it automaticly adds their IP to IPtables (firewall)(using the exec function), IP tables then redirects them to a simple text page on a free host that explains the situation.

    it’s not a perfect code, but it more than meets my requirements, maybe you should addopt a similar system for your site shamus. :P

  6. Cadamar says:

    Damn script kiddies.
    Since you are not a big corporate entity with global aspirations or some other type of organization that people can get politically or religiously upset with then my guess is that you are being hacked by some script kiddy. Possibly one that disagreed with one of your game reviews?
    Unless you are pushing an agenda then there is no reason to attack a site with a DOS other then for simple juvenile obnoxiousness. You haven’t received any strange spam emails threatening to take your site hostage have you?

    The only thing worse then script kiddies are spammers. Ohhh… if I ever got my hands on one of them… I’d make the atrocities of Vlad the Impaler look kind and gentle.

  7. Phlux says:

    There are two events from my youth that I “remember where I was”.

    The OJ verdict: 8th grade social studies class, taking a test, one student finished earlier and went into the hallway with a radio and came in to announce it to the class.

    The day the simultaneous denial of service attacks shut down ebay, amazon, yahoo and many others: My high school “Global Studies” class…we were doing a current events thing, and the teacher stopped class to watch the news coverage.

    I remember feeling very vulnerable. Nobody knew what was happening or how these sites just “disappeared” from the internet. DDOS wasn’t a new concept, but it had never been so massively orchestrated and gotten so much publicity. People weren’t set up to deal with it. It’s pretty much less than a footnote now. I can’t find a single article referencing the event anymore.

  8. roxysteve says:

    This DDOS tactic is the thin end of the badger in my humble opinion.

    Steve.

  9. Ryan says:

    I feel sorry for all those that can’t enjoy this great site. I wish I could help remedy the problem but I lack the tech skills to do so.

  10. Zaghadka says:

    They were probably trying to dictionary hack your devious CAPTCHA script. ;^)

  11. McNutcase says:

    Some Halo fanboy has found a ddos script. That’s my theory…

  12. Avaz says:

    I second McNutcase. It makes the most sense. :)

  13. Alden says:

    This is why I wish WordPress would add a static publishing option. I don’t like the fact it has to do extra work to re-render pages every time someone views them, especially when the pages don’t generally change all that often (at least once the original rush of comments has died down).

  14. Davesnot says:

    We’ll keep you our little secret…

    Phlux… thanks for making me feel extreeeeeemly old..

    speaking of denial of service.. me and a bunch of surfers just denied service to the Toll Troll..

    http://www.pangeanative.wordpress.com

    The little guy can win!

  15. MaxEd says:

    Both my home and work IP are blocked now :( Still, nothing a good public Proxy Server cannot help.

  16. RodeoClown says:

    Home -> No
    Work -> Yes
    (Both in Sydney, Australia)

    I’m currently remote desktop-ing into my work account from home so I can read this.

    I saw there was problems thanks to Google Reader, but because it wasn’t full-text, I couldn’t just read the whole thing :(

    Can you please set your feed to full-text?

    Then I can read your stuff on the train on the way to work.

    Thanks in advance,

    RodeoClown

  17. Miral says:

    I’m in the exact same boat as RodeoClown (can’t get in from home, but can from work, so I’m RDCing in to work to post this). Although I’m in NZ… but I suspect the IP ranges are similar.

    I’m in 203.109.*.*, if that helps. Please loosen your blocks a bit… (maybe after the attacker has gone away again).

    :'(

    (Also, it might not be an attack; could be some script on the site went haywire. I’ve seen that happen before.)

    PS. the banner at the top said “Roses are #ff0000. Violets are #00FF00.” Shouldn’t that be #0000FF? (If you’re following the poem, anyway. Technically “violet” is #8D38C9.)

  18. Ian says:

    @Miral: When you read out the color names as they are given, the poem rhymes:

    Roses are red
    Violets are green
    My HTML is
    All over your screen

    If you do it the right way it doesn’t rhyme.

  19. Redem says:

    I think I can guess what it was. You were added to stumblupon, a social bookmarking tool, by someone. That’s how I got here. Someone liked your “The procedural world” item, reviewed it, and submitted it to the mass of stumble upon users. Although I may be wrong, as I would have thought that would result in unique IP addresses.

  20. David V.S. says:

    I don’t know how to use a proxy server.

    At this point I don’t need one. But in the future it might be me who is blocked.

    Anyone have a link to a good “how to” page? Thanks!

  21. Davesnot says:

    I don’t need a proxy, yet, either.. but here’s some links

    http://www.oreillynet.com/pub/h/4807

    http://askit.uq.edu.au/itanswers/quikit/1_msieprox.html

    seems like an interesting challenge if need be… but those people that can’t see these pages.. what error message do they get?? denied?? what error would one see to know they’re in this boat?

  22. Chosen One says:

    I’m blocked from home, here in Brazil :(

  23. Dihydrogen says:

    While I’m not blocked, a good site for proxies is proxy.org as at least one of the proxies should work. Another solution for people who go through a proxy that blocks most other proxies is to use Tor with foxyproxy or privoxy. The easiest choice for Firefox users is to go with foxyproxy. However, using Tor is overkill for most cases, and hinges on your ability to install software and that ports besides 80 are open.

  24. Miral says:

    @Ian: Ok, that makes more sense now ;)

    @Davesnot: if you’re in a banned IP range, you go to a stock-standard Apache “Forbidden” page. (The same sort of page you get if you don’t know the password to a site, or if you try to browse to a directory that doesn’t have an index page and doesn’t permit directory listings.)

  25. John Hill says:

    Hello, your D20 of the Rings site was pointed out to me by a freind who figured I’d like it since I like LotR and KotDT and Fuzzy KoDT. I’ve been reading a few pages a day, when I get extra time, stopping when I’m laughing to much for one evening. I went today and not of the pictures will load. :( Stupid Hackers! Now I’ll have to find something else to make me laugh till it hurts.

    Anyway, I hope the system gets back up and running in full soon. I hope I can see the rest of the story someday, but in any event, thank you for the laughs.

    Odd, the anti hacking pickture just says d20. That hardly seems right.

  26. ArchU says:

    #25 John Hill: The anti-spam word is just as you saw it. There’s no mistake =)

  27. Justin says:

    I’m in Canberra, Australia, and it works fine at home but not at work (an Australian Government agency). At least I’m not missing out completely, but I’m sure there are plenty of loyal readers who are…

    I second the call to add the full text to the RSS feed, that way you might reduce the number of readers who have been completely blocked.

  28. Andrew says:

    I’m also in Australia, and fortunately I can access the site from home. Unfortunately I can’t access from work, which means I’ll have to spend ten extra minutes working every day.

    Oh well.

  29. Justin says:

    The website is now working again at work. Were some IP addresses removed from the banned list?

  30. SimeSublime says:

    Good to finally be allowed back in. The suggestion above to redirect to a free site explaining what is happening would be nice, as it’s worrying when somebody online just disappears. On the up side, I have plenty of reading to catch up on now.

Leave a Reply

Comments are moderated and may not be posted immediately. Required fields are marked *

*
*

Thanks for joining the discussion. Be nice, don't post angry, and enjoy yourself. This is supposed to be fun.

You can enclose spoilers in <strike> tags like so:
<strike>Darth Vader is Luke's father!</strike>

You can make things italics like this:
Can you imagine having Darth Vader as your <i>father</i>?

You can make things bold like this:
I'm <b>very</b> glad Darth Vader isn't my father.

You can make links like this:
I'm reading about <a href="http://en.wikipedia.org/wiki/Darth_Vader">Darth Vader</a> on Wikipedia!