How do I use Amazon S3 with local fallback in case of S3 outage?

Question

I need to wrap my head around this...
With Assets I can upload images directly to S3. With CE Image I can store the manipulated images on S3. Is it easy to switch to local images in case S3 is not available? Maybe via the fallback image parameter of CE image?

Answer 1

Let me give you a different spin on this that maybe a much easier situation for you to implement as it should work regardless of the add-ons being used your using.

Rather than have individual addons upload the assets to S3, keep them on the local server, but setup a CloudFront distribution, but set it up to reverse proxy from your server.

Let's say the address of your CloudFront distribution is d24532452345.cloudfront.net.

You set that distribution up to reverse proxy from your server, so d24532452345.cloudfront.net/assets/image1.jpg is pulled once from www.yourserver.com/assets/image1.jpg, then cached on cloudfront for future requests. Requests for that file, until it expires will be pulled from CloudFront.

You simply setup a folder in ExpressionEngine that has the path pointing to the local server but the URL pointing to CloudFront.

If you should run into an issue with CloudFront, you simple update that URL to point to your local server, requests are instantly pointed back to your server again working round any issues.

You can further automate this by setting up a cname for the CloudFront distribution, so d24532452345.cloudfront.net has a cname of cdn.yourdomain.com.

There are then a few DNS services out there that you could easily use to test for the availability of an asset that should always be present, if you get a failure for any reason, it automatically updates the cname to point back to your server rather than CloudFront.

This will speed up delivery of your assets. The downside to this approach is that your not offloading the assets from your server, only offloading the delivery of them to the visitor. It's a very flexible approach though that does not require heavy customisation of any one addon to achieve the desired result.

If this sounds like it would work for you, let me know and I'll get some more info up here to better explain how you would set it up.

Update:

Here's a good article that gives more detail on how to setup CloudFront to reverse proxy your assets:

http://www.newvem.com/enable-cloudfront-for-your-applications-non-dynamic-content/

The section two thirds of the way down called Serving Javascript and CSS files is using the reverse proxy concept. In essence though all we are doing is telling CloudFront not to fetch the content from S3 instead fetch it from our own server.

For the DNS side of things, I think gunning for automatic failover maybe a little overkill, but it's doable without question. The simple solution I think would be to do monitoring with any web monitoring service to notify you if it falls over, you set the TTL low on the DNS record, switching it should there ever be an issue, the low TTL will ensure DNS does not cache the record for too long.

If automatic failover is a requirement, then someone like DNS Made Easy would work. That's who I use. You setup monitoring to check for a single text string in a file, that shoudl always be present, if in any check that text string is not present a failure can be assumed, you then have it switch the cname record to your server rather than CloudFront.

Answer 2

Disclosure: I'm the author of CE Image.

Unfortunately, I have not found a good solution to automate this so that CE Image would serve local assets if AWS is offline. In order to do that, I would have to first check if the service was online with PHP, and then serve it up. While it is possible to do that, it is very slow, which is the opposite of what using AWS is trying to accomplish to begin with. :)

Enabling AWS for CE Image

However, it is quite easy to turn off AWS for CE Image. As a little bit of background, you enable AWS for CE Image in your config (the following code is from the CE Image Advanced Configuration Page):

/*
| Amazon S3 settings and optional headers. For use with the option AWS
| extension. See the installation instructions for more details on how
| to install the extension.
*/
$config['ce_image_aws_key'] = '';
$config['ce_image_aws_secret_key'] = '';
$config['ce_image_bucket'] = '';
$config['ce_image_aws_request_headers'] = array();
$config['ce_image_aws_request_headers']['Cache-Control'] = 'max-age=' . (30 * 24 * 60 * 60);
$config['ce_image_aws_request_headers']['Expires'] = gmdate("D, d M Y H:i:s T", strtotime('+1 month') );
$config['ce_image_aws_storage_class'] = 'STANDARD';
/*
| To use cloudfront with your AWS setup, simply add your cloudfront
| subdomain (in addition to the above AWS info) to the following config
| item. If the cloudfront URL is d38tz0dcx54k85.cloudfront.net, then
| the subdomain would be 'd38tz0dcx54k85'.
*/
$config['ce_image_aws_cloudfront_url'] = '';
// END CE Image advanced config items

Disabling AWS for CE Image

If you set the 'ce_image_bucket' setting, CE Image will use Amazon S3 (or Cloudfront if you have it configured), for each image that is manipulated. If you want to disable AWS in one fell swoop, simply set the bucket setting back to an empty string, like so:

$config['ce_image_bucket'] = '';

If you are only using S3 on certain images in your site (by specifying the bucket= parameter in your EE tags), or if you are using different buckets for different content, then you should be able to simply comment out the credentials in the config to turn it off:

/*  
$config['ce_image_aws_key'] = '';
$config['ce_image_aws_secret_key'] = '';
*/