As many of you noticed, “Alas” was offline for about 16 hours, from late yesterday afternoon to early this morning. We’re back online now, obviously. The rest of this post is a detailed discussion of what went wrong and what I’m planning to do about it.<\/p>\n
<\/p>\n
What happened was a server load problem – a problem which has plagued this blog. But now I think we finally know what the problem is – on a blog that gets as many page views as “Alas” does, you just can’t have a 20-post-long “recently commented” list. As Robert <\/a>wrote me via email:<\/p>\n Diagnosing your problem: It’s your most-recent-comment plugin.<\/p>\n That query is extremely computationally expensive. (Expense means how much disk and CPU activity it needs in order to run) You’re saying “give me every field in this big honkin, table, and also figure out which record has the latest date, from this huge ass database and with a bunch of logic to process for every row…oh, and run this on two big-tables in conjunction with each other…and then when you’re done with that look at the whole damn result set and put it in order and then subgroup each row by this index field.”<\/p>\n I’m tired just reading it; your plugin is probably asking for this every time someone loads any page on your system. I ran into a similar problem on BNN<\/a>: the architecture was very programmatically attractive (this is a nice query) but it didn’t scale up well. This query is fantastic for a blog with 2 posts a day and 10 comments apiece but it will put a bullet in the brain of a machine asked to do it 8000 times a day on posts with 300+ comments. (Particularly because inexpensive web hosting computers tend to be cheap POS rack boxes that don’t have a ton of hardware because 99% of the sites don’t need any real processing power to deliver their trickle of hits.)<\/p>\n Easy fix: kill the plugin. Hard fix: Learn or hire WordPress expertise and have someone write you an efficient plugin for big comment sites.<\/p><\/blockquote>\n For now, I’ve taken Robert’s “easy fix,” because it was the quickest way to get “Alas” back online.<\/p>\n Here’s a letter I sent to DreamHost support, explaining my future plans:<\/p>\n Hi! I’m working on solving my recent server load problems, which led DreamHost Support to turn my WordPress database off yesterday.<\/p>\n The problem was, I believe, caused by my blog’s “recently commented posts” list. A WordPress expert I talked to suggested to me that such a list, when loaded on every single page view on a popular blog, could cause problems if it calls for more than 10 posts to be listed. My “recently commented posts” list was set to list the 20 posts that comments had recently been left on.<\/p>\n Because my blog emphasizes reader comments and discussion (the blog sometimes get hundreds of reader comments a day), the “recently commented posts” list is essential – without it, posts that still have active discussions will “fall off” the front page and be too hard for readers to find. What I’d like to do is experiment a little, to find a way to restore the “recently commented posts” list that won’t cause an unfair strain on DreamHost’s server load.<\/p>\n My proposal is to restore the “recently commented posts” list on the sidebar – but this time with only 7 posts listed, rather than 20. That should reduce the server load enormously, hopefully eliminating the problem. I’ll also create a separate page with a larger, more detailed listing – but since it’s a separate page, I expect it’ll only be loaded by the relatively small number of very dedicated readers who follow the discussions very closely.<\/p>\n What I’d like is your help in making it possible for me to run this experiment. I could start this at any time that works for you (I suggest 9am pacific time Thursday). What I’d like from DreamHost Support is, if the server load gets too high (and I don’t think it will), rather than shutting me off, email me letting me know there’s a problem. I’ll make sure to monitor my email frequently. Once emailed, I’ll turn the offending plugin off immediately and use the new information to figure out the next step.<\/p>\n This will allow me to try to find a solution without fear of having my blog suddenly turned off by the host; and it will provide you with assurance that if there is a server load issue caused by the plugin, it will be turned off quickly.<\/p>\n The other thing I’d like, if things go wrong and I have to shut the plugin off, is a detailed report from the database regarding server load. This will help my consulting expert figure out what is wrong and how to run my blog while avoiding the problem in the future.<\/p>\n I hope that we can work together to let my blog have the functionality it needs, without causing server load problems. I’ll look forward to your response.<\/p><\/blockquote>\n So that’s the plan; I should hear back from DreamHost Support by sometime tomorrow. Let me know what y’all think.<\/p>\n On another subject, I have no idea why the comment preview stopped working once I’d moved to DreamHost. If anyone has a clue, please enlighten me. :-)<\/p>\n","protected":false},"excerpt":{"rendered":" As many of you noticed, “Alas” was offline for about 16 hours, from late yesterday afternoon to early this morning. We’re back online now, obviously. The rest of this post is a detailed discussion of what went wrong and what … Continue reading