10

Every time I have to update a site, I need to download it so I can back it up and to place it in a dev environment. But a site with a lot of add-ons/themes/images/media can take forever to download and subsequently re-upload when updates are complete.

What's the fastest way to handle this file transfer process? I've heard of "persistent connections" like with Git, but I don't know how to implement that on my computer, if that's even the best route to take.

Also, do you download the entire site before backup, or just the critical files like templates and config?

Improve this question

8 Answers 8

Reset to default
12

Having used Git to manage updates and deploys of a fairly major EE-based site since around June, I have to say that from my own experience, it is by far the best method I've found for handling this process.

The need to "download and back up" is essentially removed, since git handles versioning and archiving for you, and it speeds the process up immensely for both getting the latest production version onto a local/dev machine and pushing your changes from local/dev out to production because it only transmits the files that actually changed.

A big part of what makes this all work is utilizing Focus Labs' multi-environment configuration setup, which takes a tiny bit of time to set up, but will save you a massive headache in the long run. The same goes for git; it looks intimidating but is actually pretty easy to learn and use, especially since you only need to know a few basic command line codes to operate it. If you're not comfortable with the command line, Git is a great excuse to start using it; it's what forced me to finally start using it and getting familiar with it a few years back.

My vote/recommendation in this situation is definitely in favor of Git, provided your hosting provider supports it.

Improve this answer
4
  • Beat me to it. Git and Focus Labs' config all the way!
    – Stephen Callender
    Commented Nov 29, 2012 at 17:56
  • 4
    To complement this, you can have a look at deployhq.com or beanstalkapp.com for hosted deployment solution. That automatically deploys your site when you push your changes in ftp or sftp.
    – pvledoux
    Commented Nov 29, 2012 at 18:15
  • @pvledoux: It won't go in reverse though will it? You can't bring new files added on the server back down to your local environment with DeployHQ or Beanstalk can you?
    – Tyssen
    Commented Nov 29, 2012 at 22:02
  • @Tyssen there isn't a perfect solution (yet?) Really the only thing that should be added server side is new content via the CP which will be in the database or in say your uploads folder for media; assuming you keep ALL uploads there. If you have DB backup in place via Backup Pro for example or via your server itself (manual export even) then the only thing you'd need to be downloading is the most recent DB and or uploads folders. I've never needed a one to one clone locally to dev though, It sure would be nice and am hoping someone addresses the issue for all our sake.
    – Natetronn
    Commented Nov 29, 2012 at 22:38
8

Take a look at Backup Pro from Eric Lamb (mithra62). It can backup an entire file system, making your job quite easy. Otherwise using command-line in terminal (shell access) is pretty easy as well. But not as easy as Eric's module.

http://devot-ee.com/add-ons/backup-pro

Improve this answer
1
  • 3
    Second this. You can also try Safe Harbor add-on. Command line (shell) can be daunting, but as long as you use the right command (which will be just one line), you can zip 50,000 files at once and then download to your PC. Not all accounts have shell access, though, check with your host.
    – 4midori
    Commented Nov 29, 2012 at 18:06
3

You can use Updater from DevDemon make the whole process automatic.

http://www.devdemon.com/updater

Brad

Improve this answer
2
  • 1
    Yup, but there are still many add-ons that are incompatible with Updater, am I correct?
    – kmgdev
    Commented Nov 29, 2012 at 19:31
  • Some addons might be incompatible. But simply rezipping them with the correct file structure will make it work. Everyday more and more are zipping their files with the correct structure.
    – Parscale
    Commented Dec 20, 2012 at 19:23
3

I haven't learned Git yet, so still do it the old fashioned way. I go into the hosting CP and create a zip file and then download that as well as export the database. Both CPanel and Plesk have the ability to create zip files on the server for you (I've done this with both)

I also like @Webodans suggestion for backup-pro for which I wrote a review here. If you have backup pro installed you can manually run the backup in addition to scheduling it. Then just download your zip and database backup.

Improve this answer
3

If you want to keep things simple and you have SSH access, there's two things you need to do:

  1. Dump the database
  2. Archive the files

So if you were to SSH in to your box, and change directories to where your index.php file is, you'd first want to dump the database:

mysqldump -h DBHOST -u DBUSER -p DBNAME > data.sql

Replace the DB* variables accordingly. Your DBHOST is typically "localhost" (sans quotes). It will prompt you for the database user's password, so enter that.

Once that finishes, archive the files:

tar cvf backup.tar * .htaccess

Then you can download it:

http://yourdomain.com/backup.tar

To extract the data:

tar xvf backup.tar

And then import the database dump:

mysql -h DBHOST -u DBUSER -p DBNAME < data.sql

You'll be prompted for the database user's password again, so enter that.

Improve this answer
2

When I can't use git on the prod server (new client or client doesn't use git and edits files on the seerver) I use rsync to grab things. Makes life much easier.

Be very careful with the --delete flag though. Use --exclude="path/or/file/or/*.ext" to exclude things or put a list of excludes in a text file and use --exclude-from '/path/to/exclude.txt'

rsync -ravc --delete --verbose --progress --stats --compress --exclude="error_log" --exclude="cache/*" [email protected]:/path/to/docroot/ /local/path/to/your/dev/docroot/
Improve this answer
2

Use git for version control.

Use capistrano to deploy your sites ('upload'). You can also write tasks to, for example:

  • Download your DB (mysqldump, gzip, download, unzip and pipe into mysql)
  • Sync the uploads directory with rsync

Checkout some advice on this here: How can I use capistrano deployment with ExpressionEngine?

Improve this answer
2

I find that working through SSH makes uploading and downloading much easier because you can remotely zip the file, download it, re-upload, and unzip even the most bulky sites much faster.

for instance if you wanted to zip the html directory on a remote machine. once you ssh into your site cd till you can see the whole site you would like to zip. once you are there the terminal ssh command would be:

> ls
bootstrap-master html
> zip -r html.zip html/

You can then download via what ever method you are comfortable, but instead of getting an entire site of files your just getting one zip.

Improve this answer
4
  • I've heard zipping files on servers can be extremely buggy, though.
    – kmgdev
    Commented Nov 29, 2012 at 21:37
  • @kgrote tar -cvfz archive.tgz backThisDirUp works great
    – Paulo Elias
    Commented Nov 29, 2012 at 22:14
  • Would be more helpful if you updated your answer to include the commands for the actions you listed.
    – Natetronn
    Commented Nov 29, 2012 at 22:18
  • Late comment, but, if your command line zipping a file on remotes it works great. I haven't had a missing file on some very extensive sites (±3gigs).
    – Allan Kukral
    Commented Dec 19, 2012 at 21:18

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.