Look into the Pink Eye!

Here's where we publish news about Pinkeye Graphics
as well as other matters which we consider worthy of your attention.

« How we reduced incoming spam by 99%The art of MIME: decoding email images »
 

Using CoralCDN to accommodate peak demands on a static website

NB: CoralCDN appears to have stopped working since we wrote this article. So you probably shouldn't use it any more.

We run quite a few websites from our fairly modest servers which we host at the highly-recommended eUKHost. Most of these are websites that are well used throughout the year but don't get huge jumps in traffic. However we do have a few clients that have occasional big spikes in demand, for example when there's a product launch or big annual event. This can cause problems with the server, as when it is working hard and slows down for one website, it slows down for all the others on that particular machine.

This graph shows traffic in GB throughout the month of September. Guess which day the BWPA winners were announced!

Up to now we've juggled server resources by hand and managed to keep things going most of the time. But this year we knew we faced a bigger challenge with the announcement of the winners of the British Wildlife Photography Awards. This major national competition is entirely web-based, and we manage and host the website. Last year the announcement of the winners nearly - but not quite - toppled our server. This year we knew it was a much more popular competition, with many thousands more images uploaded. So we were expecting a big rush. It would never do for the BWPA website to fail just at its moment of glory. So we put in place a contingency plan that we hadn't tried before, the use of a service called CoralCDN, also known as Coral Cache. It worked like a charm. The website continued live throughout the two days with a 1500% spike in traffic when it was featured on the front page of BBC Online, National Geographic, Bild, The Times, the Guardian and many many more. Read on to see how we did it.

Follow up:

CoralCDN, or The Coral Content Distribution Network, is a method for caching static web content. It is a distributed system that transparently serves content simply by adding to the URL of the original content. You can read more about it on the CoralCDN website. The documentation is a bit sparse (or broken) so we thought we'd do our bit, as there are some great reasons to use this system. Here's the for and against:

FOR CDN
CoralCDN is free. Yep, absolutely free, and seemingly unlimited. No ads, no landing pages, no usage limits.
CoralCDN needs no change to your own server setup at all (Although you can change it if you want, to make things easier).
Deploying or deactivating CoralCDN is as simple as editing a webpage.
COralCDN is all but invisible to the end user. Most will never notice it was there.

AGAINST CDN
Some filters (including OpenDNS) can be configured to block CoralCDN URLs as it can be used as a web proxy. Corporate filters are also likely to block it in some circumstances.
COralCDN is a caching service and so will not work with any dynamic content. If users need to upload files or submit comments, or any other interactive service, they won't be able to do so. The pages will still load but that functionality will almost certainly not work.
If users bookmark a page with the CoralCDN URL on it they will always be taken to the cached page. This isn't too much of a problem for them as it won't ever expire. But it might be for you as you won't ever get them coming to your native uncached page.
Hits to the cached page are not recorded by stats packages or your own internal server logs. So you can't really tell how many hits you get on the cached pages.

How to use CoralCDN
So if you want to start using CoralCDN, how do you go about it? Using CoralCDN is simply adding the extension nyud.net to the end of the URL you want to retrieve, before any subfolders/page addresses. There are many ways to use it, from the simple to the complex. Here are the two easiest.

ULTRA-EASY: Just tell people the Coralised address
Firstly, the simplest use of all is just to publicise a link to the resource which has the CoralCDN addition in it. No server change needed at all. So if the website you're now reading goes down at any time, I could send you a link to the Coralised version - in fact, why not try it now?

NORMAL LINK: http://pinkeyegraphics.co.uk/blog

CORALISED LINK: http://pinkeyegraphics.co.uk.nyud.net/blog

If you click on those two links you should get the same result, pretty much. But notice the difference - the first one comes from the Pinkeye server as normal. The second is a copy from the Coral cache. So you'll have spotted that you can use this to retrieve copies of websites you don't even own - very useful when a busy site has just gone down with that vital picture you wanted to see! This kind of use is good if you want to give a reliable backup address, for example in a press release or for an important client. Another example: if you have a really successful post on a social networking site like Reddit or Facebook (or SlashDot - remember that?) that is hitting your server hard, make the link go to the Coralised address instead of the normal one, then it won't put extra pressure on your server.

STILL PRETTY EASY: edit your website to take pressure off high-bandwidth pages
This is what we did with BWPA. On the BWPA website, we knew that lots of people would want to see the same page at the same time, the winning photos of course! So we publicised the home page, not the direct link to the winners' page. And on the home page was a big, inviting link to take visitors to see the winning photos. All the time the server was holding out, we left it alone. Then when the usage was in the red zone, we changed that single very obvious link to a Coralised version. Immediately all traffic going via that link was taken away from our server and handled by the Coral network. More was still coming in from elsewhere of course, there were other links on the site and also via social media. But the pressure was off... for a while. As people got home from work there was a big evening peak on its way. When even this didn't stop the traffic growth, we changed a few more links on the site so even fewer visitors got through to our own page. Even more were served by Coral - the vast majority would not have noticed.

This graph shows traffic in KB throughout the day of 26 September. Notice how the traffic climbs from 0600 onwards (with a small peak at lunchtime). At 1715 we introduced one Coralised link. At 1900 we introduced a few others. Overnight the server ran quietly, and by the next evening it was all returned to normal.

There are other more complex and powerful uses of CoralCDN, for example using .htaccess to redirect an entire domain or subdomain. That kind of thing is beyond the scope of this article but there are other resources elsewhere online to tell you more.

So in short, we'd strongly recommend use of CoralCDN if you've got a short-term traffic problem to solve. It's quick, easy to reverse, transparent to users and best of all, its free!

External link: http://www.coralcdn.org/


 Published: 4th October 2011 by Matt.
36744 views
Categories: From the Engine Room

No feedback yet

Leave a comment


Your email address will not be revealed on this site.
  
(For my next comment on this site)
(Allow users to contact me through a message form -- Your email will not be revealed!)
Captcha question:
What is two plus three?
Please answer on question above.
Pinkeye Graphics Limited is a small and lively company which specialises in producing the distinctive collaborative design work of Matthew Chatfield and Cat James. Here's the blog where they publish news about Pinkeye Graphics as well as other matters which they feel worthy of drawing to your attention.

Search