Using Amazon Simple Storage Service (S3) to backup your files.
About a week ago I wrote an article on how to use Gmail as a simple backup solution. All though I’ve used this solution for many years sometimes it’s not the right choice for the job. Since Gmail was built as an email solution and not a backup/file solution there are many limitations that we can’t overcome. For instance, you can’t backup a file larger than 20mb and currently there is a storage limit of around 6GB’s. In this article I’m going to introduce you to another backup solution that I’m fond of that, although it’s not free, it works where Gmail leaves off.
For those of you that have been living under a rock for the last few years, Amazon Simple Storage Service is a scalable data storage solution created by Amazon (yes the online book store). Amazon offers a bunch of web services and their S3 solution is probably one of the, if not the most, popular services they offer. To put it in non-technical terms, S3 allows you to upload data to the Amazon servers where you can access it later either using their API or one of the many other tools that have been created for it. It’s like having an external hard drive that never runs out of space and can be accessed from anywhere at anytime. Before you get too excited it is a pay service but considering what you’re getting it’s pretty cheap. To check out the latest prices please see their site but last time I checked it was only around $0.10/GB per month.
If you’re not a programmer Amazon’s services can be pretty confusing. They don’t offer much in the way of applications to access their services but instead release an API (Advanced Program Interface) with the hope that developers will pick it up and create the applications for them. In my opinion this is a great approach since I would rather see Amazon spend their time securing my data and keeping the services up to date rather than wasting their time answering support questions. There are a ton of applications out there that use S3 for storage; I won’t dive into all of them here because I’ve only tested a select few but do a Google search to see what’s out there.
For this article we’re going to focus on using S3 for backup, but there are many other uses for this great service. Since Amazon does not provide any direct access to their servers other than the API we’re going to use a free (shareware) Firefox plugin that does the hard work for us. I stumbled upon this application one day and I’ve been using it ever since, I have it installed on all of my computers so I have access to all of my data no matter where I am. The program is called S3Fox and you can download a copy here. In order to use this though you will have to have Firefox an an Amazon Web Services account so shoot over there first and get one.
There are two ways to use this plugin; I use both. The first is you simply use it like an FTP client and drag your files into it; they will then be added to a queue and uploaded one by one. I use this method when uploading single files. The other method is to use the Synchronize Folders option which allows you to keep a local copy and the copy on their servers in sync. This is what I primarily use for backups because when I want to back up a file I simply drag it to that folder and then sync them up at the end of the day. However, S3Fox does not auto-sync your files but as long as you can remember to sync them up this method words great.
To set this feature up make sure you have the plugin installed in Firefox. In the bottom right corner click on the S3Fox icon and open up the window. Then click on the Synchronize Folders then Add Folder. Name this task and tell it what folder on your local computer you want to backup and at which location on the Amazon servers it should be stored. To start the synchronization process simple click on the Synchronize Folders button again and choose the folder name you want to sync up. When you sync the folder any files that are on the server that are not on your local machine will be downloaded and any files on your local computer that are not on the Amazon server will be sent over. Simple isn’t it?
Some Tips:
- Most Internet connections have a slow upload speed so sending large files over can take a while. Knowing this if you’re uploading large files give yourself enough time for it to complete because there is no resume feature.
- Amazon does not have folders and instead they have what they call buckets. Buckets have to be unique amongst ALL USERS that use the service so for this reason I would suggest you name your buckets(folders) something with your domain so you’re sure it’s always unique. For example my backup files bucket for The Vacant Cubicle is /thevacantcubicle.com/backups. For more information on how their service works please read this.
- Make multiple folders for certain areas; in other words don’t backup all of your files in the same place because it gets confusing. For example you might want to create a bucket ‘/backups/documents’ for your word and excel documents and a bucket named ‘/backups/pictures’ for your images and pictures.
- Files will be overwritten and there is no way to rollback to an older version so be careful.
- Delete old files you no longer need. Since you’re charged every month for the space you are using you’re wasting money storing things you no longer need backed up.
My average monthly bill is around $5, where else can you get this kind of service this cheap?
Comments, Questions, Suggestions...
S3Fox as far as I can tell does not use a secure connection BUT the amazon servers are pretty locked down. By default you can only access your files with your Amazon user keys so your data is pretty safe. You can change permissions on files or whole buckets so they can be seen by anyone..for example if you want your images to be public so anyone can access them you can change the permissions on that bucket.
The S3 service is pretty advanced, if you’re really curious on how it works go through the ‘getting starting’ section on their site, they give a pretty good rundown of how it works.
[…] received an email this afternoon asking me if it would be okay if they reproduced my post on the Amazon Simple Storage solution on their site. Well, I didn’t see an immediate problem with this but after asking […]
Wow, I had no idea Amazon offered data storage services. Where have I been!
Ryan, is the SCFox extension totally secure?