Digg: A Web Host’s Dream… Then Nightmare
Posted by Kevin Hazard, November 19th, 2006
So… I was sitting at my computer at about 10:30pm PST tonight, and I noticed in a Digg-watching widget on my homepage that a story was called “$5 = 55GB (web space) + 5TB (transfer) + 55 (domains),” and it was at the top of Digg’s front page. I thought to myself, “Self (because that’s what I call myself when I think to myself), Site5 has a deal just like that one… That’s funny…” I voyaged over to the Digg page about the story and saw that it was, in fact, talking about our “Totally Ridiculous $5 Web Hosting Deal.” It was awesome to see the attention our plan got when someone (Digg user d0rmsl0b, actually) dugg the deal and it garnered the attention of the all-seeing, all-knowing Digg monster.
Recently, our corporate blog was dugg to the front page when we posted an article about “Apple’s Time Machine, Site5’s Flashback, and a 1981 DeLorean,” so we were anticipating the throngs of Digg fanatics checking out our hosting deal… PHP on Site5’s Corporate Site was not anticipating those throngs, though. About 15 minutes into the digg, we were still at the top of the front page, but the tides turned… Our site’s Digg changed from being “A Web Host’s Dream” to being “A Web Host’s Nightmare” in the blink of an eye (and destruction of a server).
You can read the comments from the Digg users about how horrible of a web host we are for not surviving the Digg Effect (called “The Slashdot Effect” in the Wikipedia article for the same reason, referring to a similar traffic spike coming from Slashdot.org), and you can bet that we were both frustrated and embarrassed by our corporate site’s server going offline as a result of the tidal wave of traffic. Coincidentally for us (and perhaps an act of the internet gods), at this time, the story dropped off of Digg’s front page either because some kind soul at Digg wanted to hide the web hosting company’s site crashing or some of the often-hesitant Digg users thought that we submitted the deal to Digg ourselves as some kind of advertising spam. While we did not submit that story ourselves, it is a bit of a blessing in disguise that the story was buried when it was, given the server’s crash… Speaking of servers crashing, if you haven’t watched our Server Toss in a while, you should go check it out again… it’s always good times.
What happened?
Essentially, PHP PHPwned us.
Our site predominantly consists of .php pages, and each one of those pages dynamically connects to our server/databases to pull our site’s index content. This process is exceptionally useful to maintain a dynamic site, but with a flood of traffic, PHP essentially ate several GBs of RAM and closed off all connections when the server ran out of available memory. The result: www.site5.com is inaccessible, and every user on Digg smiles to him/herself with the intention of writing a witty “Man, they should probably get a new web host that can handle the traffic” comment. These jabs are certainly well-deserved, and we can’t do anything but take them in stride.
How did we fix it?
While we are not immediately throwing down the gauntlet for Digg with the message, “Do Your Worst,” the Digg incident brought about immediate changes in the way our site is hosted (following an immediate wake-up call to a few of the folks working for Site5 on the East coast). We have cached several of our highest-trafficked dynamic .php pages into .html files that do not communicate as extensively with the page’s server, and we are using this traffic-induced crash to re-evaluate our entire site sub-structure.
Thank you to everyone that dugg the $5 Deal. We hope we can be a bit more accommodating to your traffic the next time the elite of digital world notice Site5’s deals.
Now we know how Costco.com felt after being taken down for several hours following their initial sale offer of Sony’s Playstation 3 yesterday. 
P.S. People continue to try and call us out for overselling when they see our $5 hosting deal. While I tried covering the topic in a previous post and several other hosting companies have done the same, I will write a post to explain the “Why?” and “How?” of Site5’s plan structure and hosting prices along with several overselling myths and truths within the next week or so. Until then, please understand that Site5’s goal has always been to maximize the productivity of each server and provide an excellent hosting experience for each customer.
delicious
newsvine
blinklist
reddit
fark
6 Responses to “Digg: A Web Host’s Dream… Then Nightmare”
November 20th, 2006
Very cool post and explanation. Sorry about crashin’ your shit.
November 21st, 2006
I admire you for following up on this and lasting as long as you did. I guess no publicity is bad publicity. Unless its bad
November 21st, 2006
Thank you for your comments Dave and Mike. It is almost a welcomed crash… Obviously, we’d rather have taken a full wave of traffic without a hitch, but this made us focus on improving the infrastructure of our site in addition to the content/design.
Mike’s Digg was great publicity. Our site’s response to that great publicity was bad publicity. I don’t know what that means in the end… I guess we’ll see if Site5 ever shows up around the Digg water cooler again: I’d rather be “That host that got Dugg under and explained why/how they were fixing it” than just “That host that got Dugg under.”
November 21st, 2006
I just want to add here that our website wasn’t coded with this kind of traffic in mind. You don’t generally think of web hosting as “newsworthy” (although we most certainly will henceforth). The server hosting the Site5.com corporate site was one of our older admin machines, and definitely not at all in the same class as even our oldest shared hosting systems (it had been online for over 400 days at the time of this Digg, if that tells you anything). The system and the website’s code simply weren’t optimized or designed to handle massive traffic surges, as there was previously no need.
A well-written site on one of our production class servers will be able to handle a Digg without breaking a sweat. In fact, a well-written site on an unimpressive server would probably have no trouble either. A Flashback article on this blog, which resides on one of our other administrative servers (one which is definitely not a very powerful system hardware-wise and also hosts several other sites/resources), was Dugg a few weeks ago and the server handled it marvelously. What’s the difference? Our blog is powered by Wordpress, which was no doubt designed to withstand high-stress situations, whereas the antiquated PHP “framework” I wrote several years back to power our corporate site (www.site5.com) has been patched onto for the last 3+ years without any kind of rewrite or performance tuning. As I said: it works fine for our normal 100 or so simultaneous browsers, but when you get surges of several thousand within minutes of one another, the lack of performance consideration really starts to show. Better coding practices are a great (and often less expensive) substitute for more hardware–let that be a lesson to all our customers out there. Don’t be like your web host: optimize your site’s code!
We had a short-term caching solution implemented within minutes of the Digg, as described above, but I’ll also be working on a more long-term solution in the coming weeks and months as we work toward more closely integrating our corporate website with our services. In the meantime, we’re ready. I dare you to Digg us again
-Matt
December 3rd, 2006
Hey guys great story and explanations.
As you implement the long term restructuring solution I hope you’ll share your long term solution as an example of how to properly code sites for extreme traffic/hits. Just as I hope you can provide more detail in what NOT to do to avoid getting overrun when faced with extreme traffic.
Perhaps as mentioned the main issue was due to incompetent hardware.
March 26th, 2007
well written great post, well your right about some kind soul at Digg wanted to hide the web hosting company’s site crashing or some of the often-hesitant Digg users thought that we submitted the deal to Digg ourselves as some kind of advertising spam.
Yup!!
Stephen