How to backup your Drupal database to Amazon S3

We all know the importance of backing up the database for each Drupal site we build and maintain. But it is not uncommon for this to be put on the back burner and never actually implemented. Fortunately, it is really easy to setup with a combination of Amazon S3 and Backup and Migrate.

Why Amazon S3?

Some people backup to the same server that the database is on using a cron job. While backing up to the same server is better than not backing up at all, it is better to backup to an external location. Amazon S3 is a popular choice for this.

Create Amazon S3 account

If you don't have an Amazon S3 account, you can create one here: http://aws.amazon.com. Amazon offer a lot as part of AWS and you will be presented with all of them. All you need is S3.

Find the S3 option in the AWS Management Console

You will need to create a bucket for your database backups. Bucket names must be unique, so no other account can have the same bucket name. You might want to prepend your site name to the bucket name to ensure it is globally unique. You will find more information on about buckets in the AWS documentation.

When you create the bucket, you will need to select the region. It is generally a good idea to chose one that is in the same region as you, or as close as possible. This will reduce latency and also (probably more importantly for backups) ensure you comply with local regulations. For example, if you are inside the EU and your database contains user data, that data should be stored within the EU (or a safe harbour if outside the EU). Therefore, EU based residents or businesses might want to chose Ireland, which is Amazon's location within the EU.

Select a region

Now that you have your Amazon S3 account and bucket, you can integrate it with Drupal.

Setup Backup and Migrate

The easiest way to do this is using the wonderful Backup and Migrate module.

Here is how:

1. Install Backup and Migrate

Install the Backup and Migrate module

2. Backup and Migrate - Destinations tab

Go to the Backup and Migrate settings page. You will find it at Configuration > System > Backup and Migrate.
Go to the Destinations tab and click on Add Destination and then Amazon S3 Bucket.

Fill in the Backup and Migrate S3 settings

You are going to need the following from your Amazon Webservices account:

  • Host: s3.amazonaws.com
  • S3 Bucket: The bucket name you created above
  • Access Key ID: Go to Security Credentials (click on your name in the top navigation), expand Access Keys (Access Key ID and Secret Access Key) and create new access key.

Find the Security Credentials link

Expand the Access Key group

3. Create a Schedule

You can create a schedule, which is the frequency the backup will run. You can create a new schedule from the Schedules tab in the Backup and Migrate settings. Decide on a frequency that makes sense for your site. If you publish content daily, or multiple times per day, you might choose to backup daily. If you publish weekly, then a weekly backup is probably fine.

You can also set the number of backups to keep. Leave this as 0 if you want to keep all of them. But remember, you are paying Amazon to store these backups, so you might want to set a limit.

In order for backups to run, you need to have cron running. The easiest way to do this is to head over to Configuration > System > Cron. Set the frequency under Run cron every.

Drupal cron settings

Cron is used to run a variety of tasks and the frequency that you run it will depend on your specific needs. For my site, I run cron once a day. It is not uncommon for cron to be run hourly for sites with more frequently added content.

With the cron frequency set, it will get triggered to run automatically when people visit the site. This can slow the site down, and you may elect to run cron overnight instead. This is outside the scope of this tutorial, so head over to the Drupal handbook for more information.

The backup source will normally be the Default Database, and the Settings Profile the Default Settings.

Restoring

Restoring a database is a one click job. Beware though, you will lose any content that has not yet been included in a backup. To see a list of all of the database currently held in S3, go to Destinations and click on List files next to S3 Backup. You can either download locally, restore to the site right away or delete them from here.

This will actually show all files that are currently stored in your S3 Bucket. If you only want to see database files from the current site in this list, then you'll need to create a dedicated S3 Bucket for it.

Using the database locally

One of the advantages of using Backup and Migrate is that you can easily pull down a database from the live site and restore it to your local development site. That way, you get the same content that the live site has.

Wrapping up

Having a robust backup strategy is a critical element in running a website. As you have been shown in this tutorial, there are tools to make this process relatively pain free to setup and run on auto pilot.

If you liked this, you'll love my book, Master Drupal Module Development.

"..the must have drupal developers book"


Feeling stuck with Drupal 8 module dev?

Get the free 7 lesson course that will help you get started today without feeling overwhelmed.

  • Create Drupal modules with just a few commands using the Drupal Console
  • Create custom pages
  • Create custom blocks
  • Create admin forms
  • Demystify routers and controllers
  • Bonus material

Find out more


Comments

Thanks for the information about this, I have this setup for all of my clients. Is there a Drupal tool/way for backing up the whole site and then incrementally adding to the backup? I am using rysnc and the s3cmd tool on the server side for a few, but some of my clients are on shared servers and I can't technically add those tools.

very nice and much helpful mate.

It is not recommended to use your root key on aws to do backup. Please make sure the access keys you are using is f.ex for a "backup" type user (i.e it can't start up ec2 instances ).

Else you will be very sorry if someone gets their hand on your key ;)

Nice article, just a comment to say you can set up a backup profile under the settings tab.

In a backup profile you can set up things like backup file name. Which is useful if you want to restore via a script on your local development environment.

I have a number of sites on a dedicated server. The databases on two of these sites are nearly 1GB and I can't do a reliable backup using Backup and Migrate to any destination, either manually or scheduled. I have cleaned up the databases to the minimum size possible even to the point of eliminating content revisions but even during the caching cycle when the size is down to 600 or 700MB it still won't work. (Sometimes, yes, but mostly no.) I have extended the php maximum_execution_time to 900 seconds but to no avail. I am running Ultimate Cron, but it doesn't work on the drupal 7 core cron either. Any clues?

Blair Wadman's picture

Are you getting any errors in watchdog when cron has run that could indicate the problem?

Add new comment