Super Cache

 By Shamus Sep 15, 2009 44 comments

I’m using a WordPress plugin called “super cache”. Every few months I’ll get a link from a big site and the resulting crush of traffic that will flatten the site. Instead of me getting new readers, everyone – existing readers and visitors – is left out. Everyone loses. I eventually determined that it was (probably) a CPU problem, not a bandwidth one. (I can post some huge image that will increase my bandwidth usage for the day by a factor of three with no ill effects. But triple traffic can bring the cite to a crawl.) PHP pages like the one you’re reading now are generated on the fly. It’s cool like that. Except, it can take time to produce. WordPress eats a good bit of CPU, and then my dice rollers and sidebar shuffler add to that.

Super Cache will pre-generate all pages for the site. Since installing it, I’ve absorbed a few decent hits from larger sites without going down. It works. Now, if you leave a comment anywhere on the site you’ll be exempt from Super Cache because your page must be generated: Note how the comments form remembers your name and such. In order to have it do that, the page you’re viewing must be generated just for you.

But for everyone who hasn’t commented, they all view a single common page. Problem: My theme selector. Whoever views a page first will set the page for everyone else. If they’re using the evil theme, then all non-commenting visitors will see the page in black for a while. This explains why I often see comments from people:

Hey, your theme switcher isn’t working.

EDIT: Oops. Looks like it’s working now.

Of course, it’s “working” because they left a comment.

Now, I can tun off super cache and have things work right. I normally don’t need it. But if I get hit, I do need it. And usually I don’t know about the problem until I’m already locked out by the mob and I can’t get in to turn it on. Still, it seems to make more sense to design the site to work properly in most cases, instead of designing it to malfunction all the time so that it can stay up in a rush. Hmm. Anyway, that’s why things are sometimes wonky, in case you’ve ever wondered.

Aside: Ever notice how WordPress blogs always say “Twenty Sided is proudly powered by WordPress” or “World of Wombats is proudly powered by WordPress”, etc. It makes it seem like WordPress doesn’t have any standards. It would be much better if it could say, “World of White Supremacy is shamefully and begrudgingly powered by WordPress”.

20204Feeling chatty? There are 44 comments.


  1. MuonDecay says:

    WordPress is proud of all free expression, even if it is on a subject as unerringly puerile and disgusting as wombats.

  2. Chargone says:

    i don’t suppose you could set it up to pre-do the pages in both themes and treat the one’s in each theme as separate pages?

    for new ones anyway. takes up more space, might take more time, but *shrugs*

    i’m not explaining this well. long day. but maybe you get the idea.

    also, yes, wombats are evil, evil things. wordpress should be ashamed to have anything to do with them.

    them and the dolphins.

    dolphins are evil.

  3. Zem says:

    Well, you could have a little javascript that reads the theme cookie and changes the stylesheet based on that.

    Kind of inelegant and needs javascript to work.
    But it would require only minimal changes.
    Here are two simple examples:
    http://www.spoono.com/javascript/tutorials/tutorial.php?id=18
    http://www.tek-tips.com/faqs.cfm?fid=6093

    Other option would be to include the theme name in the url for each page, so that supercache can differentiate them, But that probably means some bigger changes in the wordpress code to generate links, no idea how hard that would be.

    Or you could make the .css page be generated dynamically, but cache everything else. That would probaby only comsume minimal amounts of processing time server side.
    Might cause problems with browsers that cache the css, when someone switches themes though

  4. nine says:

    Who cares about a theme switcher?! Your website should be a single unified scheme anyway, screw the people who dislike it!

    As far as stopping your website imploding under huge load goes, super cache is pretty decent! Kudos for using it.

  5. SeekerOfThePath says:

    Were this my page, I would omit themes completely and focus on stability. It doesn’t make much sense to me that user is able to customize page design without being logged in (customizing page content).
    But again, Zem’s idea sound doable if you are up to changing source code.

  6. Nihil says:

    Still, it seems to make more sense to design the site to work properly in most cases, instead of designing it to malfunction all the time so that it can stay up in a rush.

    I’d say that having the content up all the time is far, far more important than having customised colour schemes.

    I would suggest disabling the themes until you find a solution – I doubt a single person will stop visiting the site because they can’t get their favourite background (and I do use the ‘evil’ one, personally).

    As for a solution… could you perhaps tweak the themes system so that it generates separate pages for each version? I assume each style is a .css file, so you’d have to make WordPress create three .html home pages with different .css references. Definitely not the most efficient system (it will triple your cache usage!), but it could work, and if SuperCache doesn’t allow even a crude IF tree then it might be the only option.

  7. Zem says:

    Well, you could have a little javascript that reads the theme cookie and changes the stylesheet based on that.

    Kind of inelegant and needs javascript to work.
    But it would require only minimal changes.
    Here is a simple examples:
    http://www.spoono.com/javascript/tutorials/tutorial.php?id=18

    Other option would be to include the theme name in the url for each page, so that supercache can differentiate them, But that probably means some bigger changes in the wordpress code to generate links, no idea how hard that would be.

    Or you could make the .css page be generated dynamically, but cache everything else. That would probaby only comsume minimal amounts of processing time server side.
    Might cause problems with browsers that cache the css, when someone switches themes though

  8. Jordi says:

    Reminds me of a couple of years ago when I was building my own web framework with caching. Pages were built up from components and the framework would decide which to get from the cache and which ones to generate normally. For instance, here it would probably have generated the comments section from scratch and taken the rest from the cache.

    I never really tested it though and it is not entirely unlikely that it wasn’t faster at all then just generating the pages every time.

    I don’t know how your plugin and theme switcher work, because right now I can’t even see it (let’s see what happens when I hit the submit button), but the problem would probably be fixed if you use different URLs for all of the themes, because that will likely trigger the making of different cache pages. That probably won’t happen if you use sessions/cookies. Problem is that doing that would mean that every link on your blog should take this into account.
    Alternatively, it may be possible to put the logic into your CSS file (make it a PHP file and check for the session/cookie there to determine which styles to use).

    Edit: hmm, I see other people already made the same and better suggestions while I was typing this. I think I like the JavaScript suggestion best. And yes, stability is definitely more important than themes.

    I’m still not seeing the switcher by the way.

  9. Henebry says:

    What fraction of your readers are also commenters? If I understand you correctly, if a reader posts even once, your site (not “cite”) uses php to generate pages for that person from then on. So there are two classes of reader, the commenters and the non-commenters?

  10. Lilfut says:

    Hey, your theme switcher isn’t working.

    EDIT: Oops. Looks like it’s working now.

  11. beno says:

    what I noticed is that a while back I stopped seeing the blog in Chaotic Evil – it reverted to Lawful Good. it may have been a browser upgrade that did it – but now I can’t switch back even if I click on Chaotic Evil!? it wasn’t that long ago that I knew how to get my OS and browser to do what it should – now it’s so user friendly that I have no idea!

  12. karln says:

    FWIW I would rather just not have themes on the site than have it keep switching to that awful black one.

  13. zimboptoo says:

    I’m with Seeker on this. It makes a lot more sense to focus on stability and content support for everyone rather than making sure that the pages fit aesthetic specifications of a few. Hardcore followers who care about the way the page looks will have commented before, and the unwashed masses that get sent over when you’re linked won’t care one way or another which theme they’re looking at. What you’ve got isn’t a perfect or elegant solution, but it’s much better than not having it.

  14. Matt K says:

    I would also recommend getting rid of the themes. In fact, the Evil one is mostly unreadable in IE6 (black text on black background). In fact, in IE6 a lot of the colors are different than in Firefox, but the Evil theme is the only one that’s unreadable. And sadly at work I have to use IE6.

  15. Abnaxis says:

    May I submit a test, to see how this is working?

    From my understanding, the whole theme problem only happens when you open a specific thread to view the comments. In my experimenting, I changed to Chaotic Evil (my preferred theme, but it gets switched so much I have decided to just leave it Lawful) and saw all the post overviews as white text on a black background. I came here to look at comments, and it automatically switched me back to Lawful since I haven’t commented in this specific thread yet. Now that I’m commenting…

    EDIT: Yep, the second I commented, I got my Chaotic back.

    Correct me if I am wrong, but all the pages we look at are code-generated, no? Shouldn’t it be simple to put a snippet of code to modify all the link URLs (for all the people who are saying “EWW, you’d have to change all the links!”)? Is this not the purpose of using PHP to generate your content?

  16. Segev says:

    While I would find seeing a strange view jarring, the mere fact that they can fix it by leaving a comment means it’s already served its purpose. In other words, unless you have a simple solution to the problem occur to you, treat it as a non-issue. At least, that’s my suggestion.

    (For what it’s worth, I use whatever the black-text-on-white-background theme is. I think it’s the default and what you had before you really started using themes.)

  17. Shamus, having the site accessible is more important than proper theme switching.

    @Abnaxis – yes, you could put a code snippet to do anything you want. The point is that, like Shamus said, it is CPU intensive. Each snippet takes few cycles and when the traffic spikes up the server will be getting requests faster than it can actually generate the pages. The solution is to cache the pages serve them up as plain HTML. That’s what Super Cache thing is for.

  18. Shamus, sorry to hear about your troubles! I’ve had a much better experience optimizing WordPress by treating it as a normal webapp. Your first step should be to identify where your load problems are occurring – being that this is WordPress, I am suspecting that your database is where the issue is.

    There are several variables you can tune in MySQL (I’m assuming…) that will increase your server’s RAM use, but will overall increase performance. (like query_cache_*, table_cache, tmp_table_size and innodb_buffer_pool_size (if you’re using InnoDB)) I also recommend evaluating your table types, and weighing the differences between MyISAM and InnoDB, and converting the tables over where you see necessary.

    If you find that you’re CPU-bound (which your post indicates), you will vastly benefit from looking at a PHP accelerator (I recommend Xcache), and modifying how you serve your content. Apache by nature is very fat, and will spin off a single process per connection… this is, generally, undesirable, and you would do better to switch to a threaded version of Apache, namely mpm-worker. (Google it.)

    My best experience is with running PHP as CGI, and switching Apache out for LigHTTPd. On RolePlayGateway, we had a very resource-intensive RPG Chat, and we were able to decrease the load by 80% with this switch.

    Your mileage may vary, but definitely worth a weekend of Googling.

    Good luck!

  19. rats says:

    Shamus,

    I recently came up against a similar problem with caching while writing a web app. I used PHPAccelerator to cache my wrtten pages, but this caused problems for me in a similar way. I got around this by addig a variable to the URL. for instance, http://www.shamusyoung.com/twentysidedtale/theme/ce/?p=4972 would always be chaotic evil, or you could do it as a GET variable. This way the page will still be cached, and give you the benefits in performance (at a slight cost to space on your server) while allowing all to use their preferred view)

  20. Henry says:

    Your site is bit slow. I strongly recommend to get better hosting. I have good times with this host:
    Good hosting

    3$ a month will get get you a lot there.

  21. Abnaxis says:

    @16: I think I was unclear. I don’t mean adding a new code snippet, I mean modifying the one that is already there generating the links. If my understanding of the system is correct, the whole point of using code to generate pages is so you can make little changes like that easily…

    Also, even if you added a code snippet, it still would be less CPU intensive than running SQL calls, which is what the caching is supposed to prevent. To me, the whole issue is a grouping problem–you have the themes cached along with everything else, whereas you can just stick them outside of your optimization and everything would be kosher. Yeah, it would take cycles to load everyone’s themes, but not the sort of cycles trolling through the database requires.

    In short, it reduces optimization very slightly to resolve the issue. I still don’t think it would result in bringing down the site when you get flooded with traffic.

    Of course, all of us are blowing hot air without actually seeing any of the code behind this–armchair programming, as it were…

  22. Matt says:

    As an avid supporter of the CE theme, I’m glad to know why things kept switching back to Lawful :)

  23. elias says:

    Most of the ideas with having the different themes as separate pages would be difficult and still not actually achieve the desired result. The JavaScript solution is the simplest and most direct. I had a similar problem on my site when enabling WP-SuperCache: my random quotes would end up getting cached, so they would no longer be random for each page. Ended up having to find a new quotes plugin which used AJAX, and then rejigger it to load quotes when the page loads.

    The stuff Eric Martindale is saying though is interesting; stuff I did not know about.

  24. Daemian Lucifer says:

    Huh,thats interesting.But I never had a problem with the theme switcher(unless everyone that posts first is evil).

  25. Tacoma says:

    “WordPress was initially unwilling to power World of Wombats but Mrs. WordPress convinced her son that sometimes it’s important to be friends with even the spottiest of nerds”

  26. Dio says:

    If the themes are CSS, which they better be, it’s a simple matter of sending them as a final step to the cached version of the page.

  27. Volatar says:

    I recommend that for Super Cache, all pages be put in Lawful Good by default, regardless of how a person views it. Lawful Good is the most basic and universally readable of the 3 themes.

  28. B.J. says:

    Every time I visit your site Shamus it tells me “This page is unavailable” until I hit refresh. I always figured it was just because you were so darn popular. :) I use google chrome.

  29. Daemian Lucifer says:

    @Volatar

    And most eye straining.

  30. Helgi says:

    Not related to this post but I just noticed something that I think is strange, clicking on “Home” at the top of the page just takes you back to the page you are currently viewing but not the front page like I was expecting.

  31. Cuthalion says:

    Honestly, I prefer Lawful Good. I find it much less eye-straining than Chaotic Evil, and the True Neutral theme wigs me out with the horizontal gradient.

    Edit: Trying CE again, it isn’t that it’s more eye-straining, just… I dunno. My roommate’s always complaining when I turn on the ceiling light. Says it’s too bright and wants me to use the floor lamps instead. But that always seems to dim for me. Also, if the lights are off, my eyes feel tired if I look at a computer screen, so I always prefer the lights on. Furthermore, when I’m driving at night, I never use the polarized mirror because I feel like I can’t see anything that way, and the shining headlights on the regular mirror setting never bothers me.

    I can see fine though (I’m 19 years old with something like 19/20 vision and no glasses/contacts). So it’s not as if my eyes don’t work. But for some reason, I’m much more tolerant of bright light and, while I can adjust to the dark like normal people, I hate working in dim light. It’s got to be either bright or dark.

    All this is to say that the CE scheme bugs me, though not as much as those horrible red-text-on-black themes some places have. So just in case you were considering getting rid of LG, please don’t.

  32. Volatar says:

    @Daemian Lucifer:

    Read more books. Black on white is good for you :P

    @Helgi:

    I have noticed that also. I have used the Twenty Sided logo to go to the home page instead in place of it, but I always hit home first before remembering I have to do that :)

  33. BlckDv says:

    I’d agree with Luke that having a site that can be read is more important than one that uses my preferences. Those “surges” are likely your best chances to gain new dedicated readers.. as long as you do not crash.

    Two ideas I had (Disclosure: I have never used WordPress or Supercache)

    1. Have an a scripted “hit” run each new page on your default theme (I’m guessing LG) as that is more likely to fit the expectations of new visitors and encourage them to stay, preventing the odd reality of switching themes as they navigate based on random traffic luck.

    2. Cookies. Yes, they are not the most elegant, but a cookie that sets after an initial visit would let you serve the static only to true first visitors, not just lurkers.

    Good luck.

  34. Cineris says:

    Many/most modern browsers allow client side styles to be used in lieu of the styles used by the site.* So if people really can’t live without white-on-black or silver themed versions of your site they could load their own.

    * Note: I’ve never bothered to use this feature myself, so I could be getting something wrong. But I know I’ve seen this kind of client-side styling.

  35. Uncle Festy says:

    Mmm, wombats. Tasty.
    In other news, your theme switcher isn’t working. :P
    No, but seriously, I don’t know how to switch themes. >.<
    EDIT:
    Ok. Found it.
    But it's not working – it's stuck on LG on the main page and CE or TN (whichever is white fading to black) everywhere else.
    AND I've already posted.
    A little help please?

  36. Stonebear says:

    I guess you guys are more for balance by weighing out both ends of the alignment see-saw. I feel all alone playing the true neutral fulcrum

  37. Bryan says:

    Wait, wait — what? Why are the three different styles not done by simply serving up a different stylesheet — either depending on the cookie (at a single stylesheet URL), or statically, being swapped out via browser controls? (Or javascript, if you must.)

    Is WordPress really too dumb to have figured out that content needs to be separated from presentation, so that you can swap out the presentation at will? If the theme has to be more than just a bunch of extra CSS, then your tools would seem broken to me…

  38. Bryan – I assume that the correct stylesheet is pointed to by a bit of PHP code depending on cookies/session variables etc. Thus, the cached page (which is an html page, not a php script) will be statically linked to that stylesheet.

    Shamus, if that is the case, you could write some javascript to extract the same information and grab the correct stylesheet on page load, I guess.

  39. Well, that explains why I can never get the Evil theme to stick. I always wondered why it was so seemingly random, and now I have the answer.

  40. Dave says:

    Oops, sorry about that hit from reddit a few months back. That was my fault.

    They thought Pixel City was really really cool though.

  41. Henebry says:

    Shouldn’t the themes be simply a question of the .css file applied? I’d think that theme could be decoupled from pre-rendered pages, since the pre-rendered stuff is the .html, yes?

    • Shamus says:

      For those who are asking: No, you can’t change the CSS without re-generating the page. Well, you COULD, but it would require an entirely different version of supercache that generates everything EXCEPT the part that includes the CSS. It could certainly be done, but the time cost is prohibitive since I’d have to re-engineer supercache.

      Including the theme in the URL seems inelegant to me, since it would screw with permalinks.

  42. Brickman says:

    Couldn’t you make the normal theme be the normal URL and make the alternate themes add something to the end of it? That wouldn’t screw with any existing permalinks, and as long as people had the sense to make future permalinks to the normal version of the page rather than the evil version, it wouldn’t screw with future ones even if you decided to change the way you handled it or for some reason threw out the themes altogether.

  43. Bryan says:

    Sure you could change the CSS: Serve up a different stylesheet (depending on the preference cookie thing) at the same URL. Then you can cache the page with the link element in it (point to e.g. stylesheet.php, and be sure to set type=”text/css”), and everything will be happy.

    It *should* even be doable: just have that script serve up a different static file depending on the cookie value. And the simpler this script, the better: no point in trading out a WP slowness problem for a stylesheet-script-slowness problem. Just don’t do this via a DB. :-)

    Of course, you might run afoul of some stupid “we’re smarter than the webmaster” rule in IE: it might see a .php extension on the stylesheet and refuse to treat it as CSS, even if the element has the right type, and the PHP file sets Content-Type: text/css in the response headers. So that might scuttle the whole idea.

    But, I haven’t seen any behavior like that with anything else in IE (just the “oh, this looks like it might be HTML, even though it was text/plain, so I’ll try to render it…” stupidity), so it might be fine…

Leave a Reply

Comments are moderated and may not be posted immediately. Required fields are marked *

*
*

Thanks for joining the discussion. Be nice, don't post angry, and enjoy yourself. This is supposed to be fun.

You can enclose spoilers in <strike> tags like so:
<strike>Darth Vader is Luke's father!</strike>

You can make things italics like this:
Can you imagine having Darth Vader as your <i>father</i>?

You can make things bold like this:
I'm <b>very</b> glad Darth Vader isn't my father.

You can make links like this:
I'm reading about <a href="http://en.wikipedia.org/wiki/Darth_Vader">Darth Vader</a> on Wikipedia!