Migrating your Drupal site using Drush make and Drush rsync

In a previous blog post I showed you how to generate a quick drupal deployment script using Drush make (if you're not already familiar with Drush make, review that article first). But that is not all you can do, oh no that is not all.  In the following tutorial I will show you how to migrate an entire website.

1. Plan

This is the part of the conversation where we get into terms like "workflow" and "architecture."  Drupal is a cms and sometimes you can't just jump in; you need to think about what you're doing on a conceptual level.  Here's what you'll need:

  1. Have access to your current server.  You'll need ssh.
  2. Prepare your new server.  You'll need:
    1. SSH access (unless it's a local environment)
    2. A fresh mysql database
    3. Drush
    4. Set up Drush aliases
    5. Sync database & files
  3. Decide whether you're doing it right.  There are lots of ways to migrate a Drupal site from one server to another.  This method has pros and cons, just like any other.  We'll go into them at the end.

2. Pull files from old server

First, ssh into your server, move to your drupal directory, and roll a new makefile. (This assumes you already have ssh/shell access and Drush installed on the server.)

# ssh -p 2222 koppieyoursite.com
# cd www
# drush generate-makefile make2.make
  Some of the properties in your makefile will have to be manually     [warning]
  edited. Please do that now.
  Wrote .make file make2.make                                          [ok]

Great, you have your makefile.  Download / transfer it to the root location of where your site will be migrated to.

3. Prepare new server

Start by logging in to your new server.  If you're on a shared / managed server, you may have to request ssh access.  If you're using a local environment (eg. mamp), open a terminal and navigate to your drupal directory.  Obviously you'll need to make sure you have the full lamp stack set up.  (Or mamp, or wamp if you're out of your mind.)

3a. Create fresh mysql database

Next you'll have to set up a fresh mysql database. If you're on a shared / managed server, there might be a gui like phpmyadmin to do this, otherwise set it up on the command line:

# mysql -u root -p
# mysql> create database make2;
Query OK, 1 row affected (0.12 sec)

Make sure you have a user with adequate permission to access the new databse.  I used root because it was most convenient, if you're on a shared host you'll probably need to consult with your hosting provider.

3b. Set up Drush

If you haven't already installed Drush on your new server the easiest way is:

# pear channel-discover pear.drush.org
# pear install drush/drush

If that doesn't work, do it manually.  It's not that hard. Download the Drush project from Drupal.com and unzip:

# wget http://ftp.drupal.org/files/projects/drush-7.x-5.8.tar.gz
  --2012-11-26 12:41:56--  http://ftp.drupal.org/files/projects/drush-7.x-5.8.tar.gz
  Resolving ftp.drupal.org (ftp.drupal.org)...
  Connecting to ftp.drupal.org (ftp.drupal.org)||:80... connected.
  HTTP request sent, awaiting response... 200 OK
  Length: 409642 (400K) [application/x-gzip]
  Saving to: `drush-7.x-5.8.tar.gz'
  100%[======================================>] 409,642      406K/s   in 1.0s    
  2012-11-26 12:41:57 (406 KB/s) - `drush-7.x-5.8.tar.gz' saved [409642/409642]

# tar -xzf drush-7.x-5.8.tar.gz

Then you will have to add the following to .bashrc on both servers, replace [username] with the correct path:

// User specific aliases and functions
# alias drush='php /home/[username]/drush/drush.php'
# alias php='/usr/bin/php-cli'

// Fix to allow Drush aliases to work
# PATH=$PATH:$HOME/bin:/home/[username]/drush
# export PATH

Now that we've set up Drush, we can deploy the makefile.  First, create a new directory for your new Drupal installation:

# cd /var/www
# mkdir make2
# cd make2

Download the makefile into your new directory.  Use sftp, wget, or whatever method works best for you.  Once you've got the file, you're ready to deploy.  Drush will report its progress to you as it downloads all the packages to make your site:

# drush make make2.make
  Make new site in the current directory? (y/n): y
  drupal-6.x-dev downloaded.                                           [ok]
   >> iui-6.x-1.0 downloaded.                                          [ok]
   >> admire_grunge-6.x-1.5 downloaded.                                [ok]
   . . . . .
   >> backup_migrate_files-6.x-1.x-dev downloaded.                     [ok]
   >> Project admin_menu contains 3 modules: admin_menu_toolbar, admin_views, admin_menu.
   >> admin_menu-6.x-3.0-alpha4 downloaded.                            [ok]
   >> addanother-6.x-1.6 downloaded.                                   [ok]

Now you've got all the packages for your site, but we're not done yet, you still need to pull your files and mysql database.  Before you do that, we need to create some aliases.

3c. Set up Drush aliases

There are different ways to do this, depending on what you want to do.  If you'll be hosting multiple websites from the same server, or want to manage multiple remote sites from the local environment, you can set up a global alias list and store it in /etc/drush, $HOME/drush, or $HOME/.drush .  For our purposes, we'll just set up an alias file that applies to this site only.  First, create the file.  There's a Drush command that will take most of the sting out:

# drush site-alias --with-db --show-passwords --with-optional @self
Take the output and put it in a file, like yoursite.alias.drushrc.php.  The file will need the following touch-up:
  1. Add <?php to the top and ?> to the bottom
  2. Change the name of the alias from @self to something creative, like @mysite

Or, you can write the alias file by hand, like so:

// New server
$aliases['local'] = array(
  'root' =--> '/var/www/make',
  'uri' => 'localhost/make', // I'm using my local environment

// Old server
$aliases['remote'] = array(
  'root' => '~/www',
  'uri' => 'koplowicz.com',
  'remote-host' => 'koplowicz.com',
  'remote-user' => 'koppie',
  'db_url' => 'mysql://user:password@oldserver/database',
  'path-aliases' => array(
    '%files' => 'sites/default/files', // So you can grab your files directory too

3d. Sync database & files

Now the fun part!  We've got all the infrastructure set up, so we can run the actual sync in 2 easy steps:

# drush rsync @koplowicz @self
# drush sql-sync @koplowicz @self


If you want to monitor a large transfer, you can add the --progress flag:

drush rsync @koplowicz @self --progress
By default, rsync will skip settings.php.  This is a good idea because you'll want to use a different database for your dev site.  If you want to include settings.php, add --include-conf, like so:
drush rsync @koplowicz @self --include-conf

Warning: This will transfer the entire root directory.  You can specify sub-directories if you want.  For example:

drush rsync @koplowicz:sites/default @self:sites/default

On the other hand, you may want to sync the entire directory.  For example, if you have custom modules or themes.  In that case, you can skip the drush make steps above; drush rsync will make a carbon copy of your site files.

4. Why This Is A Good Idea (Or Not)

There are lots of different ways to migrate a drupal site from one server to another.  Here are some alternatives:

  • Simple download using ftp (or sftp)
  • svn or git for version control (bonus: use the Features module to control settings too)
  • zip and wget
  • Pantheon has built-in staging features
  • Document your work and recreate the features by hand
  • Use an IDE with built-in version control, like Aptana

So, why would you want to do it this way?  Here are a few reasons:

  • You already have aliases set up, or intend to rely heavily on them.  For example, maybe you're responsible for a long list of sites spread across the internet.  You can control them all in a few minutes using aliases.  If you have all that set up already, you'll save time logging into remote servers by using drush rsync and drush sql-sync.
  • You don't want or need the added features of version control
  • You have to make incremental changes.  Rsync is much better at this than simple file transfers like sftp or wget
  • You can't afford Pantheon or have a specific reason not to use it (eg. multisite)

Here are a few reasons why you wouldn't want to use this method:

  • Using drush make assumes you haven't hacked any contrib modules, themes, or core.  If you have, then you'll have to use drush rsync to import your changes anyway.
  • If you have other sites sharing the same directory (ie. subdirectories), or if you have very large files in your web directory.  You can customize your rsync command to exclude certain directories or files, but it gets cumbersome.  For more information, see the documentation at drush.org.
  • Drupal.org supports git.  So does Pantheon.  There are good reasons for this.  Rsync is essentially the same thing, but without version tracking.  You should think about why you don't want version tracking.
  • Pantheon is free for development even if you can't afford them for your production site.  Not only that, but they make it very easy to export your site when you're ready to go live.

Bottom line: Using drush make/rsync/sql-sync is a good way to migrate a site from one server to another.  Is it right for you?  You'll have to decide that for yourself.