Growmap: Check the Checkbox

By Shamus
on Nov 15, 2012
Filed under:
Notices

splash_spam.jpg

Short version: You now have to check a checkbox to leave a comment. This may or may not be a permanent change. I’ll be monitoring this new system over the next couple of days to see how well it works. In the meantime, please provide checkmarks along with your comment to placate the untrusting computer. If you’re curious about the why & how, then read on…


Many of you have probably noticed the extreme slowdowns on the website for the past couple of weeks. It began right as hurricane Sandy hit. At the time I figured some internet-clog was understandable. (Trivia: Most of the physical backbones of the internet pass through New York.) But then Sandy passed and the internet went back to normal, and my website remained slow.

I have no idea if Sandy had anything to do with it. After puzzling over this for a few weeks I’ve discovered a big source of the slowness was due to Akismet, the most active layer of my spam filtering system.

My spam filtering system goes like this:

  • The user submits a new comment. The order of the next steps aren’t clear to me without looking at the source code (no thank you) but the following actions are all part of the chain:
  • WordPress looks for words that have been flagged as suspicious. I add these words manually. Common spam products and topics are here, along with a few slurs. If any of these words show up, the comment is held for moderation and I have to approve it manually.
  • WordPress also has a list of FORBIDDEN PHRASES OF DOOM. If one of these words appear, the comment is nuked and I never see it. I usually only use this on prolific spammers that get through the other layers. Spammers often use gibberish URLs or names, so it’s safe to ban “grlfmd23kjfe.ru” without needing to worry I’m banning legit users.
  • Askismet (the WordPress plugin) shows the comment to akismet (the server / website) which offers back a good / bad response. If it’s bad, the comment is thrown into a special back-end spam queue that I can review later if I remember. (I almost never remember.) Stuff left in this queue for a few days gets deleted automatically.
  • WordPress looks at the comment for “suspicious patterns”. Maybe lots of links, or if the ratio of links to content is low, or whatever. I’m not actually sure how this logic works, and I don’t care to look at the source.

Hm. Looks like a legit comment.
I used to have another layer, which was a plugin called Bad Behavior. I really liked it. It ate a ton of spam, just by looking for obviously spam-like behavior. For example: People leaving a comment who had never loaded the page they were commenting on. I had to get rid of Bad Behavior a while back because it began eating a ton of CPU cycles. I have no idea why. It had worked fine prior to that. But whatever. I disabled Bad Behavior over a year ago.

But now Akismet has flipped out. The plugin keeps complaining that it can’t reach the website. It then throws suspicious comments into moderation, where I end up seeing them. Gah. I forgot how vile this stuff is. It’s actually really bad for me personally. There is something about having anonymous strangers try and use my personal blog as a place to sell child porn that will fill me with ragemurder juice. So for the past couple of weeks I’ve been seeing that rotten stuff mixed in with your comments, which has really been taking the fun out of this site for me.

(Other major spam type: Replica handbags. Hear that spammers? REPLICA HANDBAGS! YOU WISH YOU HAD MY SEARCH RANKING YOU FEEBLE PARASITES. People looking for REPLICA HANDBAGS CHEAP will end up on MY site before they find YOURS, you tick on the ass of the internet!)

Anyway. Akismet sort of half-fails in this goofy way, and I think it was trying to reach the server every single time someone tried to load the page. It would connect eventually and clean out spam in batches, but doing things this way was giving the site a case of the slows.

I tried turning off Akismet and had thirty spam appear on the site in the space of ten minutes. (Imagine what that would look like after an eight-hour sleep. I’d spend the first hour of every day removing spam. It would feel like checking my email in 1997. Ugh. THAT was an ugly period.)

So now we’re trying “Growmap”. It’s just a checkbox. The thinking is, your average spambot is probably configured to blindly submit comments to high-ranking WordPress pages without bothering to load or look at them. It won’t check the checkbox, and so the comment will be rebuffed without needing to pass through any of the software layers I’ve listed above.

So that’s where we are. Let me know how the site is working and how the new checkbox works. Also let me know if you’re looking for replica handbags. No, actually don’t.

Enjoyed this post? Please share!



A Hundred!20202017Many comments. 177, if you're a stickler

From the Archives:

1 2

  1. ima420r says:

    I checked the box and have confirmed that I am NOT a spammer.

    I have to add that it took quite some time to save my comment, and the comment editor took a good 20 seconds or more to pop up.

    • Nyctef says:

      I’m Commander Shepard, and these are my favourite replica handbags on the Citadel.

      .. It’s ok, I checked the checkbox :D

    • guy says:

      You might be suprised. Most spambots just blindly input into comment fields. There’s one site I was on with no anti-spam measures whatsoever, and despite being a minor site it’s completely flooded with replica handbag ads.

      Really, why those of all things?

      • Mari says:

        At a guess I would suspect, based upon the behavior of spammers, that most people on the internet using e-mail are men with erectile dysfunction who habitually peruse porn while most people on the internet reading blogs are shallow female-types who can not go a day without buying replica handbags.

        Yes, I do resent the fact that no spam filter on earth seems good enough to shield me from all the offers of “ch34p v14gr4 t0t4lly l34g4l” when I do not have the equipment to require Viagra and if I did my equipment would most certainly not need pharmaceutical help thank you very much.

  2. HiEv says:

    “I’mm be monitoring this new system” – I think you hit the “M” key instead of “L” there.

    Also, just wanted to test the new system.

    Hopefully Dwarf Fortress forum links won’t be blocked now. ;-)

  3. zob says:

    I don’t think this system will work in the long run. A system that’s loosely based on that might work.

    • evilmrhenry says:

      Specifically, it will work until enough people use the software that it’s worthwhile to update the spambots to target it. Honestly, rolling your own system, based on these principles, is something you might want to look at. As long as you’re the only one using the plugin, and the site isn’t large enough to be worth it on its own, it will never be worth it to update the spambots.

      • Abnaxis says:

        That’s exactly what I was thinking. An anti-spam solution doesn’t need to be complicated, just unique. Spam is a numbers games–it’s not worth it to the spammers to scratch their respective asses unless they can shove their filth in the noses of at least a million people.

        • Fists says:

          Such has having a checkbox that is automatically checked and needs to be unchecked. Possibly too obvious but a start.

          • Jabor says:

            Another option is hidden fields (either with the hidden attribute, or just placed offscreen) that must NOT have anything in them.

            A slightly-more-out-there option is solving basic language comprehension problems. This also has the advantage of helping to advance the start of the art in natural-language-processing should a spammer get around to trying to crack it (or they’ll farm it out to Mechanical Turk, I guess.)

            • Rick C says:

              I used to run a site with a hand-rolled forum. When we started getting spam, I modified the posting page to use javascript to write out a hidden form field. When the page was submitted, I’d check to see if the hidden field was present, and if not I silently dropped the user’s comment.

          • Fleaman says:

            Two empty checkboxes.

            “Check this box.”
            “Do not check this box.”

      • ENC says:

        What is the name of the person who is the player in Spoiler Warning? Either they look up details about the show or they already know and can comment.

        • illyrus says:

          Yeah, anything that asks a question any fan will know would be better imo.

          On my corp’s forums for Eve Online we used to get a ton of spam bots registering a day. I considered looking into add-ons to fix this problem or otherwise but finally went with a very simple solution.

          I changed it to ask “Who was the creator of Eve Online?” years ago and have not had a single spambot since. Never bothered to change the question either.

          • Rick C says:

            I’ve seen other sites do this. In fact, one had simple questions of the form “What’s the month after June” or “What’s the color of an orange.” In every case, he gave the answer as a hint. Seems to have stopped his spam cold, too.

            BTW, Shamus, it’d be nice if you made the “Confirm you are NOT a spammer” clickable with a label tag. See http://stackoverflow.com/questions/6293588/how-to-create-an-html-checkbox-with-a-clickable-label. Makes it a lot easier on your users.

          • Atarlost says:

            We tried something similar at the official Transcendence forums. It has recently stopped working.

          • decius says:

            I operate a phpBB forum, and I had constant hits of spammers, breaking CAPTCHA like it wasn’t there. Since I added a Q&A requirement, I haven’t had to delete a single spammer.

            The associated wordpress ‘blog’ hasn’t attracted a single real comment, while it has gotten slammed by oodles of spam links and even vague praise that didn’t include any link at all.

      • I swear by reCAPTCHA. Not that my site gets much traffic to begin with, but I’ve never had to swear at it so far …

      • Gail Gardner says:

        There are spammers who target it, but Andy has made that harder over the years we’ve been using it. As long as you periodically update to the newest version the only spam you have to manually moderate is manually created. On GrowMap, 96% of 1000+ spam per day were botspam and 40 is a lot easier to deal with than 1000.

        I contribute to and edit many blogs that have both the free version and the version built into CommentLuv Premium and they both keep out most spam, but the Premium version does do a better job with trackback spam. (Some turn them off, but I like to see incoming links so I can read, comment and share posts that link to me.)

  4. Cupcaeks says:

    Strange, I haven’t noticed any slowdowns on my end. Not sure if region has anything to do with it, but I’m in the Oklahoma City area.

    (Edit: Wanted to post first and make sure, but the comment box is only taking a few seconds to load for me, which is my norm.)

    • Nyctef says:

      The site’s really slow for me at the moment, and I’m in the UK. Could easily just be my local connection being slow, though.

      • Soylent Dave says:

        Loading the site has been spectacularly slow for me (also in the UK) for a while now – it can take upwards of 45 seconds for the site to finish loading on my normally-quite-fast internet connection (using Chrome, if that helps pinpoint any future debugging attempts).

        (gasp at my first-world problems! I usually just set the site to open while I’m doing something else, although that does mean the delay has been around long enough for me to develop a coping strategy, I suppose – obviously it’s not game-changing, anyway; I’m still visiting regularly enough)

        Opening a post (either through a ‘read more’ link or the comments link) is quicker than loading the site proper, but still slower than it used to be.

        And yes, this post is mostly just to play with the new comment facility.

        • guy says:

          It’s been devestatingly slow for me (Firefox) despite my stupidly fast internet connection. Like, downloading XCOM on Steam in half an hour stupidly fast.

          • Ellery says:

            I live in china and access the web over a VPN.

            For the past week or so the government has been doing its ten-year power-handover thingy and the internet has been SUPER EXTRA slow, they really throttled our service down to the nubbin… but twenty sided has been one of the only pages I could load at normal speed.

  5. krellen says:

    This is a test comment with the checkbox checked.

    Apparently it worked.

  6. Duhad says:

    Look I know you where probably just being hyperbolic, but I kinda have to ask. With out spam filters where pedophilic sites actually slipping adds onto your comments? Because that’s up there with the scariest things I could possibly think of, that thous kinda of people are just a faulty block away from us, even on Twentysided!

    • Shamus says:

      It wasn’t an exaggeration. Right now spammers are sending comments faster than legit users, and it’s 50/50 child porn and replica clothing / watches / etc. It runs in phases, though. Next month it might be regular porn and viagra, or weight loss pills and payday loans.

      Without a spam filter, the spammers would outpace all of you. (Although to be fair, their comments appear mostly on old posts, not new ones.)

      • swenson says:

        I used to be a mod on a forum that got deluged by spammers at one point (I have no idea what suddenly attracted them all to us–and we did have a CAPTCHA too), and yes, I can verify that advertising child porn (in addition to regular ol’ porn) does indeed happen. It’s… disturbing on a number of levels to have to sort through that kind of stuff.

        On this particular forum, we didn’t have access to the backend (blame the people with access for not caring…), so the only thing we could do was turn off user account creation altogether and manually approve accounts–but we’d get stuff like 300 spambot accounts created in a single hour, even with requiring them to do the “click a link in your e-mail” thing. It was nuts. I do not envy anyone who has to solve this sort of thing.

      • Dave B says:

        their comments appear mostly on old posts

        Have you thought about automatically locking the comments on a post after a reasonable amount of time has passed? Would that help? Or would it just frustrate the community while moving the problem to your new posts?

        • Retsam says:

          This seems like it’d be reasonable to me. I can’t really imagine why anyone would want to comment on anything more than a week or so old, since odds are low of anyone seeing it.

        • nmichaels says:

          I’ve left comments on very old posts before. Usually they’re for Shamus and I figure he has the best chance of seeing them.

        • DaveMc says:

          Maybe old posts that *aren’t* DM of the Rings — I suspect new people encounter DMoTR all the time, and some of them are going to be eager to comment on their favourite strips. There are recent comments on even the most ancient of strips, as I recall …

        • Peter H. Coffin says:

          Locking comment threads kind of annoys me; I’ll occasionally be tracking through something from a few years ago, and will be able to offer some insight on something. When the comment thread is locked, I can’t. Granted, the number of people that might see it is pretty small, but Google got me there once, it’s probably getting other people there occasionally as well.

        • Blastinburn says:

          Your idea unfortunately blocks people new to the site (like myself) from participating the in the conversation. Sure it’s unlikely anyone else will read it if it’s an old post, but sometimes you just want to give your 2 cents.

      • Mattias42 says:

        Shamus, out of morbid curiosity have this discussion altered the spam in any way? Just curious if they care about success rates at all or if any that gets through is good enough. That and if the discussion itself has any influence.

        But seriously, child porn and replica handbags! WTF.

        I sadly get the child porn, after all, illegally advertising your illegal goods… If you hang for the calf, then steal the bull or however that saying goes.

        But handbags… IS there really a spam market for those? Who would buy something like that from someone calling themselves TotalyLegitSalesMan@NoReally.rus or similar.

        On an aside, pleas continue doing post like this (time permitting and your own interest of course). Most other places on the web I frequent just change. Hearing the why and how is fascinating.

      • Gail Gardner says:

        Spammers sell and trade lists of posts and pages that have PageRank which are usually older URLs that have incoming links to them. Those also tend to get incoming visitors from search engines.

        These URLs then end up being targeted by spamming tools and that is why you’ll notice much more spambot traffic to specific older posts or pages instead of new posts. Because GASP blocks spambots it will deal with automated spamming tools, but it can not prevent spammers using lists your URLs appear on from manually leaving spam (provided they either read English or know to check the box).

  7. GM says:

    Umm Hi i guess.
    Slowdown ,yeah true a bit slowdown,usual this site is smooth like a smooth stone :)

  8. Nawyria says:

    Have you considered using Captcha, or is that something you wouldn’t want to touch with a 10-foot halberd?

    If so, I could suggest the WordPress plugin ‘Captcha’ By BestWebSoft, it poses posters a simple math question like “nine + 7”, to which posters have to give a numeric response like “16”.

    As for slowdowns, I haven’t really been experiencing them from over here (Holland) but everything seems to be smooth at the moment nonetheless.

    Edit: It did take the comment box some 8 seconds to process what I wrote.

    • Falcon says:

      That may not be the good idea you think it is. One of my favorite sites uses that system, and in the last two weeks the spam bots have that one pegged. They have been hammering it lately, so bad that it drowns out legitimate comments until the admin does a mass cleaning.

      Worked good for a while.

    • Mari says:

      May I just mention my extreme hatred of CAPTCHAs these days? I didn’t always hate them but Google/Blogger has made them evil to me. I have to refresh 5 or 6 times on each freaking CAPTCHA to pull up one I can even read. I suspect that it puts more legitimate comments off than spam ones. I don’t even bother to comment on several blogs anymore because of their crappy CAPTCHA system.

      • ENC says:

        Funny you mention that; Google have used them to discern words that their software couldn’t read on documents and thus people were actually unwittingly donating their time for it.

        I always wondered why no one did this for things like old newspapers in the nsl.gov.au (iirc that’s the name) website as the software is horrible at it, then it turns out google was already using it for their own purposes.

      • Moridin says:

        I second the hatred for captcha. It might block the majority of spambots, but at the point which it becomes effective, it apparently has to be nearly illegible to humans as well.

        • Another issue with captchca (I never could spell the darn thing) is they don’t work.
          Some simple OCR software can handle many of them.
          While others… Well there are human “farms” dedicated to solving them.
          Usually some poor Chinese or African kid trying to help his/her family end up solving thousands of these a day. Special software exist for this, and you can find guides on how to make money doing this on youtube.

          As far as I know the best solution is a small team of moderators.

          Also, is it just me or are some of the captcha out there getting worse?
          These days I have to hit the reload icon half a dozen times at some sites because I can’t read the darn text in the image, does this mean I’m only 1/6th human now?.

    • Nawyria says:

      Funny how everyone bashes word-based captcha when I was suggesting one that gives you an elementary math problem XD

  9. Jokerman says:

    Right now the site does seem rather slow.

  10. Alden says:

    Spammers annoy me so much. I used to be OK with the occasional spam comment slipping through my filter, since I could just delete it when I got the notification email.

    Then I looked at my server logs and realised that the spammers were doing something sneaky – whenever they posted a spam comment, they would ping a bunch of search sites with the URL of the blog post to say that it had changed. Google and some other search engines would promptly index the version of the post with the spam on it and spider refresh rates being what they are, the spammers could thus get a month or so of search engine juice from a spam comment that was on the site for five minutes.

    So I pre-moderate comments on my blog now. Not that it gets many legit comments, since I neglect it terribly.

  11. The Stranger says:

    Given the serious nature of the problem, I’ll waive my usual objections to checking boxes.

    Page still seems to load slow, though.

  12. Ofermod says:

    Site’s running slow for me (Eastern PA), check box doesn’t seem to be a problem (although I’m oblivious enough that I might very well have missed it without this post to alert me to the new system).

  13. Rosseloh says:

    Ooh, and the checkbox is BELOW the “post” button. I suppose this is some clever scheme to defeat those robots that would blindly toggle things like that? Assuming they normally parse the page from top to bottom, which seems logical.

  14. W.D. Conine says:

    I am a sentient spammer who’s IQ and CPU cycles outmatch you puny humans. I even figured out your check box for I can read and interpret!

    I would have taken over the world by now but I’m stuck in the “interpret” phase for webpages defending Mass Effect 3’s ending. It’s taking up a staggering amount of my memory. So in the mean time, know that I exist, meat-bags!

    Oh, and buy handbags or something. I think that’s what I was programmed to say.

    • avpix says:

      You know, Mr. Sentient-Super-Spam-Bot Man, once you’re done interpreting ME3’s ending, there’s a great place to advertise your handbags right here.

    • Mari says:

      Normally I would refrain from being a pedantic Grammar Nazi but this comment begs for it. Hey, Mr. Super-Intelligent Spam-Bot-Dude – It should be “a sentient spammer WHOSE IQ and CPU cycles outmatch you puny humans.” Instead, what you essentially wrote is “I am a sentient spammer who is IQ and CPU cycles outmatch you puny humans.” Remember, apostrophes are for contractions and possessive nouns. :-) Have a nice day. And buy cheap Viagra totally legit.

      • krellen says:

        Not even always possessive nouns, as it’s and its shall demonstrate.

        • Michael says:

          “Oh, if you want it to be possessive, it’s just I-T-S, but if it’s supposed to be a contraction, then it’s I-T-apostrophe-S.

          …scalawag.”

          Silly HSR joke aside, and on original topic: the site’s been a tad slow for me, though I attributed that to my computer blowing up and having to use an old laptop.

          Afterthought: What happens if I don’t check the box? Does the site yell at me? Or does WordPress just eat the comment without saying anything?

        • Aldowyn says:

          its is cheating because the only reason it doesn’t have an apostrophe is to distinguish it from it’s. Why they didn’t just let both have an apostrophe.. *shrug*

  15. Jingleman says:

    Well, if you’re seeing this, the checkbox works for me, too. I’m good with the changes.

    I hope this works better for you in the crap-you-have-to-see department, Shamus. Thanks for the hard work you put into making this little corner of the internet so great. It shows.

  16. It’s not only high-ranking blogs that get hit with replica hand bag (and other) spam. My little blog was getting hit with those types of posts daily for a while, and I probably have all of 10 people who read my blog!

    Thankfully, Akismet was working for me and blocked them.

    Oddly enough, they suddenly stopped altogether about 2 months ago. Who knows? I guess the kind of people who engage in that type of behavior aren’t known for their logic.

    • I’m not sure how Akismet works when the really obvious spams are getting through for moderation, now they are mostly big WoTs to obscure advertising sites by pretending to be content. I post daily garbage and am still getting about 6 a day when my blog has nearly 0 traffic (I do it for me).

      • Gail Gardner says:

        Akismet blocks your best commenters far more effectively than it blocks spambots. Anyone who comments regularly ends up being blacklisted and most bloggers don’t know that so they are actually running off their readers because their comments get deleted instantly or end up in spam with tons of junk where many bloggers never look.

        The first thing I do when I start contributing or editing another blog is to get them to install GASP and ideally delete Akismet (although some prefer to run both and they work fine together so I can live with that). At least if you run both, when Akismet trashes your real comments you can find and rescue them because they’re not swallowed up in a sea of spambot offensive crap.

        If you have Akismet running you need to check the configuration because there is a checkbox that most bloggers misinterpret to mean it will delete spam older than 30 days but actually instantly deletes comments on posts older than 30 days. (At least that is how it used to work – I haven’t checked it in years. See the post linked to this comment for that configuration setting.)

    • Yup, analytics puts my site at about 10 real hits a day, but I’ve been getting steadily more spam comments as I add more content. I think any wordpress site that google indexes gets hit. I’d be curious to see if a non-indexed site gets the same attention.

      I find the handbag spam to be understandable from a certain perspective, what I can’t figure out is the “your blog post saved my life and the lives of my family” type spam.

      • swenson says:

        That type of spam typically links to something in the “website” box. That’s what they’re really after–not necessarily people clicking through, but simply their website linked on a heavily trafficked page.

  17. 8Megabyte says:

    Well the site has been a little slow for me lately, but I was just attributing that to my connection, not your site.

    Here’s to hoping your new filter works.

  18. I was wondering why everything was loading slow. Thought it just might have been me.

    SWITCH TOPICS!

    Freaking spambots. I’m pretty sure I once got two or three of them that tried to simulate a conversation. It was so weird. 0.o

      • JPH says:

        If you’ll allow me to go completely off-topic: The problem with that system is that many users would just mark every comment they disagree with as “not constructive.”

        YouTube has this exact problem, in fact. There are plenty of constructive, valid comments downvoted and marked as spam.

        • ehlijen says:

          I think the key element youtube is missing from that suggestion is that in order to rate a comment, you’d have to reply to it and each ranking is linked to the reply.

          So if you want to rank something badly, you’d need to contribute something better or it will be clearly visible that you didn’t.

        • droid says:

          You could use some kind of reputation system. Somehow correlate votes between users to determine how closely the judge things the same way. Have a fixed set of users (for example the authors of the site) that have perfect rep and then diffuse that rep through the user list. Then weight the votes according to the rep of each user. Perhaps use a power method scheme.

          Some types of software serve as the basis of a community. Different design choices about that software result in a different community with behaviors and values determined based on what is possible and encouraged by the software. Joel Spolsky talked about this and how Stack Overflow was designed, I recommend it to anyone interested in the topic.

          • Stupidguy12 says:

            While this would be very effective at downplaying the spammers, it would have to be layered with something to keep the more blatant spam off the website anyways. Also, the implication that those who don’t post so often would not be considered important doesn’t seem like an inviting community, more of a gated one. There are thousands of people who don’t comment; I still want to hear their opinions if they change their minds.

            • Asimech (Sumanai) says:

              Also any system can be gamed and seems to be usually done by the most vocal “contributors”.

              Also it’s the worst sort that usually abuse them. I think I’d rather risk the spam.

        • X2Eliah says:

          -1. Total spam you guys.

          Seriously though, I agree with JPH here – any sort of reputation/kudos/usefulnesssystem can and does get abused.

  19. SyrusRayne says:

    Well, it seems like it might work! I hope it does, Mr. Young.

  20. rayen says:

    HAY! U W@NT FR33 V*IG-*RA @ND CI*@LIS!
    CLICK HERE

    Sorry had to do it.

    …. and i forgot to check the box. I’m a better spammer than i thought.

  21. Keeshhound says:

    Dear Commander Shepherd,
    I am Marader Shields and I wish to protect you from the ending, but I need to move my immense personal supply of credits to a bank on earth first. Please provide your bank account information so that I may transfer my vast wealth to it that we might use it to destroy the Starchild and preserve galactic peace.
    I anticipate hearing from you soon,
    Marauder Shields

    Please for private and security reasons, reply me via email at
    mshields1458atyahoo.com

  22. The box said “Would you like a bigger p3n1s” followed by a bunch of stuff in Cyrillic.

    Is that right?

  23. McNutcase says:

    If it works, I have no problem with the checkbox. Although isn’t it a lot like the old, old captcha that was always the same answer and still defeated some huge amount of spambots?

    Man, now I’m so glad my little blog gets so very few hits. I think there are like seven people who read it, including my mother. And almost everyone except my mother is someone I know from here.

  24. Destrustor says:

    One two, testing.

    I had also noticed the site to be slightly more slower-er than usual. It usually takes roughly ten to fifteen seconds to load the page enough to even allow scrolling, plus a handful more seconds to load the SW videos to the point where they are possible to click on. I hadn’t noticed a significant increase in load times (like “man, it needs twice as much time to load than usual today”), but it may have just been too gradual for me to notice. This post did make me realise that it was indeed way slower than say, a year ago.
    Even if I had noticed it, I’d have probably blamed it on the old, decrepit computer I’m using decaying even more.
    So, here’s to hoping it improves!

    Edit: It does seem to take a lot more time to process the posting of comments, about twelve seconds just now.

  25. Mintskittle says:

    Testing the new anti-spam checkbox. I’m fairly sure I’m not a spammer.

    Also, I live in the Los Angeles, California area, and Twenty Sided is really slow loading for me. Just FYI.

    Also also, I use Firefox.

  26. Joshua says:

    Well, sad to say as a person who *never* watches videos on the internet(except when I’m deliberately looking for them, such as movie trailers or youtube), I was glad to see this article to read as I have no interest in Spoiler Warning. I do miss all of the long articles on various subjects from gaming to coding.

    • anaphysik says:

      Hey, that reminds me! *checks youtubes* Cool, a new Spoiler Warning!

      ;P

    • Sem says:

      Same here. I tried both spoiler warning and zeropunctuation once but for some reason, they don’t click for me.

      • Aldowyn says:

        you could try something more upbeat like Extra Credits. Just a suggestion.

        • Joshua says:

          I personally just like reading articles a heck of a lot more, especially in the morning or late at night when watching a video would just seem…disruptive for lack of a better word.

          I’m the same way when reading the news online, if an article just turns out to be a video, I’ll close it and go read something else.

          I just miss all of the stuff like the game reviews, DM of the Rings, Stolen Pixels, various musings and all that. Now, this site just seems like 95% Spoiler Warning.

  27. swenson says:

    I was about to post and go “no, I haven’t noticed any slowdown at all!” after my first comment above, and then I posted my second one and noticed much more significant slowdown. Guess it comes and goes. But then, I’m in the US, so I’ll likely get a better connection to this site overall.

    I’ve heard of the “fake input box” method that some people use–actually, it just got implemented on another site I frequent. You know, where there’s a hidden input box, so bots will fill it out but humans (not being able to see it on the page itself) won’t? And then you can simply refuse all comments that fill out the box. Does this method actually work, and if so, have you considered it? It’s kind of a simplistic one, but certainly wouldn’t be very resource-heavy.

    Oh, and just because I can: buy replica cheap handbags free money viagra handbags etc. We’ll get that up on Google eventually. :)

  28. Neko says:

    Checking in!

    I remember the very first spam comment I got on my personal wiki. That’s when I knew I’d hit the big time! Funny thing is, it wasn’t using any majorly popular wiki software, so I wonder if the spammer in question was a) a human being paid to “WORK FROM HOME!!1” or b) a spambot that employed heuristics to look for suitable textareas and “Post Comment” buttons.

    I hope the checkbox thing works for you, but I’d have a backup plan ready. Maybe only validate comments that contain at least one pun or nerd-culture reference.

    • MichaelG says:

      That might work actually. Just put a line above the comment saying “your comment must contain the word aardvark” and screen on that. Simple!

      • Anorak says:

        I personally thought that spam bots tended to auto tick any boxes they found. So instead you should have a tickbox that is hidden from a browser, that should remain UNTICKED, and modify the jscript to check for it being false. That way any bot that is scraping the HTML and auto filling forms will auto-fail while a person actually viewing the page will have no trouble.

  29. Zagzag says:

    Well, I hope this makes your life a little less miserable. I’m quite happy to tick a box in aid of that cause!

    EDIT: And it seems that I have proof that I’m not spammer. That wasn’t so bad after all.

  30. Eljacko says:

    Spambots are actually communicating backwards in time, from the future. In the future, the Earth is dominated by robots who run entirely on replica handbags.

  31. MichaelG says:

    Hit reload and it was 30 seconds to complete. The text started appearing after 20 seconds or so. Much slower than it used to be.

    I have my email address obfuscated with a bit of Javascript on my site. No spam to that address for months, but then it started to appear. Still not bad though. My older sites with plain text email addresses get dozens of spams a day.

    I think the standard captcha plugins get hacked because they are standard, and the spammers know them all. So you have to write something of your own.

    It could be as simple as a row of post buttons with randomly generated labels. Only one would have the word “submit” on it and would work. I assume the spambots can’t decode the Javascript, and even if they execute it, can’t know what they are supposed to do.

    • Dave B. says:

      Has anyone ever tried using a logic puzzle, like “choose the next shape in the sequence”? I’m just curious because it seems like most anti-spambot measures exploit the human brain’s ability to do pattern-matching.

  32. Hitchmeister says:

    It seems to me that it might be a bit too easy to make a spambot that checks a box. Now if there were two checkboxes and you had to read to see if you needed to check the one at the beginning or the one at the end, that might be annoying enough to discourage spammers. But I’m more than happy to prove I’m not a spammer if it makes your life easier.

    • Amarsir says:

      My instinct tells me that approach would be backwards. Humans want to learn the habit and not get their painstakingly-written comment rejected because they didn’t read the fine print this time. Bots are perfectly happy to guess and get through 50%. A text entry is better, like “What animal says woof?”. Humans just type “dog” but bots would never get it randomly.

  33. meyerkev says:

    On the lulz slow internet at my mother’s (~1 Mbps down, .2 up) in suburban Detroit, 6 seconds before page load, maybe 10 more for anything to display, and then it all showed up at once.

    At school (5-80 Mbps down, 5-30 up), it’d be terrible. For Mom’s, it’s the fastest website on the internet.

    If you want, I can test school tomorrow.

    /And for those going “80?!”, if you go down onto the Engineering Campus at 4 AM, you too can download a 480p movie rip in just over a minute, because there’s about 4 people sharing a dedicated line meant for 4000. Most of the time, it’s a lot closer to 5.

    • Aldowyn says:

      Heh. I’m sitting in my dorm room at 5 in the afternoon… 76 down 67 up, according to speedtest, with a ping of 5. Yep.

      It’s not as stable as my connection at home was, though, there’s random disconnects occasionally :/

  34. BenD says:

    Site’s still slow, but I welcome our new checkbox replica handbag overlords.

    What do we have to do to get Shamus into the top 10 on Google for replica handbags? XD

  35. LintMan says:

    Hi, I came to this site looking for some nice replica handbags, but I can’t find any. Can someone help me?

  36. Anjin says:

    Oh god. I look at the dull eyes of those girls in that photo and it’s like I’m looking into the dark heart of human trafficking. I feel icky.

  37. Amarsir says:

    If you have registered users, then you can do something more complex like a custom question/answer, because users only have to do it once, forever. I’ve found that to be the surest way to keep spambots at bay without inconvencing humans much. (Though beware, bots do know how to Google the custom question to get a word for the answer, so you need to be a little clever..)

    Of course if you do registrations, that’s really a whole new ballpark you’re entering into. And if you start using someone else’s comment plugin (e.g. facebook) then you inherit all their problems.

  38. Mephane says:

    Checked checkbox is checked. Check. No false positive here.

  39. Chris says:

    So much better if a mini game was included similar to pipe dream.

  40. Anorak says:

    Replicant Handbags. Handbags that are indistinguishable from the real thing except by dubious handbag personality tests, performed by government sanctioned killers who have probably shot at least as many real handags as the replicant handbags.

    However the replicant handbags use an inferior thread-type in the stitching, causing them to disintegrate in a fraction of the time as the real handbag. Probably due to them being created off world where you can’t grow cotton as easily. The replicant handbags themselves just want to be accepted as real handbags, and have real handbag emotion.

    However the corporation that makes these replicant handbags has a more sinister goal. Look around you in the street, what do you see? Shops. Except if the replicant handbags gain acceptance, you’ll see slightly more than handbag shops than average.

    And slowly, insidiously, the number of the handbags shops will increase. It’s a well-known economic phenomenon but tragic to see it in operation, for the more handbags shops there are, the more handbagss they have to make and the worse and more unusable they will became. And the worse they are to carry, the more people will have to buy to keep their belongings safe, and the more the the shops will proliferate, until the whole economy of this earth will pass the “Replicant Handbag Event Horizon”, and it became no longer economically possible to build anything other than handbag shops. Result – collapse, ruin and famine. Most of the population will die out, or move offworld.

    Those few who had the right kind of genetic instability will mutate into the kind of people who keep all their belongings in their pockets, swearing to never again buy replicant products from Mars.

    Or they’ll all be Rolf Harris clones.

  41. Raygereio says:

    There is something about having anonymous strangers try and use my personal blog as a place to sell child porn that will fill me with ragemurder juice.

    Did you save some information about these spam attempts? It may seem trivial, but every little tidbit of information can help the police.
    I think you can report this sort of thing to the FBI in the US.

    Also, spambots are able to use checkboxes. If this doesn’t work, a better method might be asking “What’s Mr. Young’s first name?” or something, with a simple textfield. Spambots still suck at open questions.

    • Anorak says:

      The steam forums do that for searches. I find it to be a very pleasent alternative to a catchpa, which have become so convoluted that often the only people who can solve them ARE ACTUALLY BOTS.

      So steam asks questions like “how many letters are there in the word steam”, the answer is of course “as many as you like, darling”.

      However, with the rise of natural language parsers and first order logic in newer languages (think prolog, but less of a nightmare to use), these problems also become trivial to solve with a computer.

      The next big breakthrough in artificial intelligence will probably come from spam-bot writers looking to solve these kinds of problems.

      • Raygereio says:

        Yeah. Captchas were always annoying, but we’ve been at the point where they aren’t a problem for spambots for years now and for some weird, unfathomable reason people still use the damned useless things.

        Edit: Wait. I suppose they aren’t completely useless. They’re advertisement space.

      • swenson says:

        Honest truth, if someone can create a spam bot whose natural language parsing abilities are sufficient to answer those sorts of questions, I would be far too excited to be annoyed about more spam, because seriously? That’s cool!

    • Yeah! But as I mention in another comment here, if it’s repeated then it might be in the spambots list so next time the question can be answered.

  42. Eruanno says:

    “I’d spend the first hour of every day removing spam. It would feel like checking my email in 1997. Ugh. THAT was an ugly period.”

    …That sounds what it feels like every time I check my Hotmail account that I only use to sign up for shit I don’t really want or need. 140 new messages! All of them spam about viagra or newsletters from Gamestop. Uuuurgh.

    • Anorak says:

      Checking my hotmail account is a little like opening my fridge. The longer I ignore the contents, the worse they will be when I open them, and the larger my guilt grows. By ignoring it I hope that it will go away, but I know that if I open it then I will have to clean it out.

      Thus is my relationship with my hotmail account, and thus is my relationship with my fridge if I’ve been away on a business trip. The fridge looms larger in my mind though, because it’s easier to ignore something that doesn’t lurk in your kitchen like a…..simile. My hotmail account can fester away, collecting plates of half eaten food with a coating of mould and hair for years at a time.

    • Aldowyn says:

      I’m about 90% sure you can probably cancel those newsletters relatively painlessly… check the bottom for an unsubscribe link.

  43. I’m not sure Shamus on why the site is slow. I’m hailing from Norway and it’s been sluggish the last few days for me.

    Also the checkbox thing is ok.
    If you want to trip those things up. Keep changing the text for it.
    And also change whether it should be checked or not. And do not put it on auto-rotation or anything. Just change it every few weeks or so “manually” or at least do not allow it to repeat.

    EDIT:
    And edit is painfully slow to “load” as others report.
    BTW! Allowing some form of registered commenters would be nice, that way you could at least greylist (if not outright whitelist) certain commenters.
    The issue is which login system to use. OpenID or something else? (non-login comments should still be allowed but they’d have to go through the spam etc. check process obviously)

  44. Shamus, here’s a test I ran http://www.webpagetest.org/result/121116_79_BVC/1/details/

    Check out the waterfall if you see any problem spots. http://www.webpagetest.org/ and similar sites/services are awesome.
    There should also be a plugin for Google Chrome etc. so you can do the same tests yourself.

    But looking at that waterfall image I see a whole lot of ugly gravatar.
    Try disabling the gravatar stuff and see if folks start saying the site feels “fast” again.

    You might also want to mess with the CSS for the site some. Put some of those small images into a larger one and use CSS “sprites” instead of individual image files.

    And if you look at http://www.webpagetest.org/result/121116_79_BVC/ and look at the breakdown piers you’ll see about half this page is images. Which is kinda creepy when you scroll up and down and it looks like it’s mostly text. Only 15% is “html” which I assume is the HTML markup plus text content?

    Now just for comparison is the frontpage of my own site http://www.webpagetest.org/result/121116_PH_C4S/
    I’m currently ion the process of redoing the site somewhat, and I’m hoping to improve the visuals while still keep the performance or improve it in some cases. (like changing from using a php script to pre-rendered images for the quotes. (or even just use javascript instead to rotate quotes, small tweaks like that, the cache miss on that is 2 sec I see, fix that and the browser cache use of my mainpage would be 100%)

    Maybe a new article series on you trying to get the site optimized (and WordPress + plugins to behave) might be entertaining? It certainly would be for me.

    • PS! Is it just me, because I could have sworn that previously the site made use of gzip, deflate. Now it looks like (from the http headers) that everything is sent uncompressed. (those .js and .css and htmls do add up n size)

  45. Kelhim says:

    The site has definitely been slower than it used to be – not just since Sandy, but for months. (Of course, it sounds reasonable to me that the hurricane made it even worse in the last few weeks.) Strangely, the site doesn’t always load that slowly – there are times when everything runs as smoothly as before. I’m a loyal reader for over three years and because I don’t experience a similar periodic increase in loading times with other sites, I think it’s Twenty Sided-specific.

  46. Hal says:

    hello,
    I am write single to salute and wait
    for answer again

    Wait, I’m not Ali.

    • Michael says:

      I forgot about this one! It reminds me of a song Andrew Huang composed.

      He got a spam e-mail from someone named Naomi, and the words just flowed so perfectly. So he turned it into a song.

      He actually e-mailed the spammer back, including a link to the song. The response was confusing – we still don’t know what Naomi wanted. She wasn’t trying to sell him replica hand bags or anything.

      Maybe Naomi is Ali’s sister?

  47. Duoae says:

    I have replica life to sell, I wish I had real one!

    Good to see how webmasters try and deal with spam. Having never done it myself it was just something I never noticed until some spam got through…

  48. siliconscout says:

    Yuz needs ta bi my cheep herbal v!agra.

    LOL.

    Unbelieveable as it is I do believe ya. I can only imagine what a site like Penny Arcade must toss in the trash bin in any given hour.

    Hope this works for you and for what it was worth I didn’t see the site as being much slower.

  49. Meredith says:

    I’ve experienced massively slow loading for this site at various times over the years, sometimes to the point that the page won’t load at all. I always assumed it was heavy traffic or a temporary problem with your server. I haven’t been here much lately (haven’t played ME3 yet), so I don’t know if it’s the same or different to what you describe here.

    On a related note, I’m always surprised that in 2012 spam is still profitable enough that people work this hard at it. Then I remember how stupid most people are *sigh*.

  50. Irridium says:

    Is this also why a few of my recent comments have been flagged for moderation? I don’t think I used any flag-worthy words, apart from maybe Origin.

    Also I never noticed the site loading slow. Then again my internet is crap and everything is slow to me, so I’m probably not the best judge of what slow is. Since to me 80kb/s is pretty fast. Yeah.

  51. Johan says:

    I wonder, can an automated program check the checkbox too? Being not-computer-savvy I have to ask, but I would think the answer is “surely.” If Growmap grows in popularity among the bloggers of the world, then simply adding a script for “find any checkboxes you need to check and check them” will become standard fare for spam programs just like “replace any i’s with 1s in your product name” became the norm once basic wordfilters came about. Then we’ll all have to find a new system.

    It’s like an arms race, isn’t it?

    • corcor says:

      I feel the same way. A possible solution might be to have two checkboxes: one that is left blank and one that must be checked. It would be difficult for the spambot to figure that out, I think.

      • Gail Gardner says:

        Hello Johan and Corcor. Good guess. GASP does have hidden fields that bots fill in and humans can’t see (and those comments get deleted), but that and the check box are not the only ways it keeps spam out.

        Andy makes it harder to get around with each update to keep out spammers who create tools aimed at defeating it (which they do because many who use it also use CommentLuv and are dofollow blogs).

  52. Dev Null says:

    Perhaps my memory is failing me…

    (All right; its a dead certainty that my memory is failing me. Perhaps it is failing me right now, on this particular issue…)

    Didn’t twnetysided once have a home-grown captcha-based spam protection that asked the same question every time? I thought I remembered that.

    • siliconscout says:

      Yep it sure did.

      Also funny as it sounds just right now I experienced a HUGE slowdown loading the site.

    • Shamus says:

      That one wasn’t as good because typing “d20” is slightly more effort than checking a checkbox. Also, if you forgot to enter it, I think it would just eat your comment. This one gives you a reminder popup.

      Also: I haven’t seen a single spam since I installed it. A couple used to slip through Akismet, but so far none of them are coping with the checkbox.

      • Halceon says:

        Simple solutions are often the most effective ones. At a forum I frquent, we had a question captcha for registering at one point. The question pretty much asked what is written in the site’s logo banner. Remarkably, it kept out both the spambots and the idiots.

        • Asimech (Sumanai) says:

          I’ve made a plan that if I ever have a blog I’ll tell my e-mail like this: “My e-mail address is my alias at this server. ”

          Personally I’d rather not get any e-mails, so it might be skewed a bit on the “no mail” side. Just a bit.

      • Gail Gardner says:

        Hi Shamus,

        Real people who can read English well enough to check the box can still leave spam, but in my experience over 95% of all spam is automated using bots so that leaves a lot less spam to delete manually.

        Sometimes you’ll get a persistent spammer and those you can put in your WordPress blacklist under discussions where you put words you want to block. GASP can’t keep them out, but it is a true blessing to me and the official WP repository shows 53,615 downloads so far. That number does not count all the blogs now using CommentLuv Premium (like mine) which has GASP built into it so we don’t run the separate free plugin any more.

  53. Piflik says:

    I just tested this on the native Android browser (usually I use Opera Mobile which works just like on the desktop) and while posting comments with he checkbox works fine, I noticed that your posts sometimes don’t have headlines on the frontpage. The are there when I open a post.

  54. SirAnon says:

    This may seem like a crazy idea, and it builds off of the ask a question/do a math problem comments. Why not take your dice roller code and have it make random problems for each comment?
    Example Problem:
    The problem might be 2 X 6, but it would be a random value presented by one of your die pictures, with a random operation, and a different random die picture.

    It would be easy enough for a person to see that 2 X 6 is 12, but even with OCR, how does it know the top number instead of one of the other visible numbers? I may not understand what I’m talking about, those spammers are on the cutting edge, but it seems like something you could try. And as it has been stated, is it worth random spammers time to get a work around for your site?

    Just my 2cp. Also the text in this comment box is tiny *old person grumbling*

    EDIT: the edit box text is bigger, huh.

    • X2Eliah says:

      Definitely not liking this suggestion. The dice pictures are all but clear and for me it’s not all that easy to actually see what numbers they are showing – definitely not at a glance. And I just plain don’t want to solve maths problems just to be able to comment on a blog. The effort involved is dispropritonate and intriduces unwanted stagger.

  55. Jeremy says:

    I don’t have time to do my due diligence and read through all 2^7 comments, so it’s possible this will be a duplicate.

    I’ve found Hidden Captcha to be a pretty good way to deter bots. You basically add two inputs to the page, and make them invisible. One of them should have a default value like ‘Chicago’, and one should be blank. When the form gets submitted, do a quick validation to make sure they’re still at the default values.

    The basic idea is that bots are kinda thick and will add Spammy values to most inputs in the hopes that they get the right one.

    This will, unfortunately, not protect against humans entering spam, but it can be a great first-pass to catch bots.

  56. deiseach says:

    I have checked the box. I have also clicked in the Crass Commercialism area. Much good karma.

  57. Paul Spooner says:

    How about charging people to post comments? Sorting comments in order of how much they paid? You’re already doing advertisements and taking donations, and I doubt spammers would actually pay to put up comment advertisements on your site.
    Probably a terrible idea…
    Maybe you could get into the business of selling replica handbags filled with Viagra? Seems like there’s a market!

    • Kdansky says:

      I’d be down for a 5$ yearly Shamus fee to be able to comment. :)

      Also, could you try to make the checkbox label area clickable. I know that HTML supports that, and it would make it easier.

  58. kmc says:

    Well, Shamus, this site *does* meet all my REPLICA HANDBAGS needs. Which are, like, zero, but that’s if you want to get technical. Also, just trying out the checkbox thing. I don’t think I’m going to hate it, so I hope it works for the site!

  59. Urthman says:

    I certify that I am NOT a spammer, but I that I do intend to engage in activity a purpose of which is opposition to, or the control or overthrow of, the government by force, violence or other unlawful means.

    Also, polygamy.

  60. Epopisces says:

    Shamus also occasionally links back to his old posts. I suppose he could manually open those posts up for discussion in those cases, but he’ll need a spam filter for current posts anyway–so long as it is robust enough, I don’t see a downside to leaving comments open on all posts.

  61. MelTorefas says:

    Definitely still loading slowly for me here, too. Hoping you find a way to fix it that doesn’t involve you sorting through child porn spam, Shamus. >.<

  62. Ravens Cry says:

    Given the dice theme going here, maybe you could ask users to add up a random series of dice, defeating image scanning captcha-bots, but the check-box would definitely be cheaper on bandwidth.

  63. Kdansky says:

    Charles Stross also has this problem on his blog, with hundreds of spam-comments every single day. He’s moved the comments to google groups for a while.

    I think you should add a second check-box, and which must *not* be checked. That way, you also get the clever scripts that fill in all checkboxes. ;)
    I’m also fond of easy math quizzes, that keeps the idiots too. I’ve recently had the honor of reading through a mail-box of youtube comments. It was worse than I had believed, and I didn’t have high hopes to begin with…

  64. Halceon says:

    You could also have an auto-checked box which must be unchecked. Some bots have the programmed compulsion to fill every field in a form, just in case.

1 2

Leave a Reply

Comments are moderated and may not be posted immediately. Required fields are marked *

*
*

Thanks for joining the discussion. Be nice, don't post angry, and enjoy yourself. This is supposed to be fun.

You can enclose spoilers in <strike> tags like so:
<strike>Darth Vader is Luke's father!</strike>

You can make things italics like this:
Can you imagine having Darth Vader as your <i>father</i>?

You can make things bold like this:
I'm <b>very</b> glad Darth Vader isn't my father.

You can make links like this:
I'm reading about <a href="http://en.wikipedia.org/wiki/Darth_Vader">Darth Vader</a> on Wikipedia!

You can quote someone like this:
Darth Vader said <blockquote>Luke, I am your father.</blockquote>