CREATE BLOG dbo.SqlServer

If you're a hacker, a consultant, or just a developer/admin enthusiast, one problem likely to be high on your list is, "How can I make backups of all my stuff regularly?" I have about 200GB of data that I want backed up weekly and about 50GB that I need backed up somewhere between hourly and daily - and I'm a single developer on a budget which means tape drives are not attractive to me ATM. DVDs are out (even dual layer) since I want 250GB backed up weekly...

After doing a bit of research, I've found what I think is a good solution for anyone using Windows Vista/XP/Windows 2003/2008 to maintaining current file backups for free using nothing but freeware. I'll tell you about my solution below but one caveat first: this works for me but I don't know whether it work for you so you need to do your due diligence on what other options are available before committing to this solution. Also I'm sure that I've failed to mention some good freeware tools so please don't hesitate to comment on what else works for you.

N.B. Just want to see what tools I use? Click here if you want to skip the preamble and get to the software.

My Three-Fold Backup Problem

Problem #1: Maintaining current backups of development environment

I'm currently developing the videobooks.com website as well as maintaining all of the LearnItFirst.com websites) so it is very important to me to have super-current backups of every file/folder/whatever. What I wanted was two things: (1) a point and click solution that allowed me to take a backup of the entire website project (*everything including supporting files not in the VS2005 project such as documentation, PSDs, databases, etc), and (2) a scheduled backup that took backups at predefined intervals. Why the point and click solution? I like the idea of taking the backup immediately before starting development and immediately after finishing a section of the project - it just seems "clean". All told, the project files are about 5GB currently and will not grow significantly over the next year. One challenge here is that I wanted my point-and-click backups to be fast - very fast. When inspiration strikes, I don't want to have to wait 10 minutes for a backup to complete...

Problem #2: Maintaining current backups of 50GB of data on my laptop

On my laptop - the machine I use for active development - I have another 50GB of stuff that I need backed up; some of it daily and most of it weekly. The challenges here are (1) how to perform incremental backups in Windows XP/Windows 2003 since there's probably only 5GB or less of changes each week, and (2) that this 50GB today will be 100GB within six months. This needs to be a combination of point and click and scheduled backups.

Problem #3: Maintaining current backups of 200GB of data on external drives

External hard drives are the way to go for the one-man team, right? We store everything on them: mp3s, movies, backups, application installation files, etc. I have three that I actively use:

1. A 180GB hard drive for backups of my laptop, applications, mp3s and backups of other machines ("Drive 1")
2. A 250GB hard drive to store the LearnItFirst.com and VideoBooks.com videos ("Drive 2")
3. A 500GB hard drive that is a backup of the first two ("Drive 3")

The challenge here is how to perform incremental backups on Windows XP (which manages the external drives). Drive 2 only changes about 500MB a week but Drive 1 changes anywhere from 1GB to 5GB a week depending on whether I've downloaded new stuff from MSDN or whatever - with that little bit of change, the incremental backup is definitely the way to go. This needs to be scheduled since point and click would take too long.

The Freeware That I Choose to Use for My Backups

SyncToy Is a Great Choice for Point and Click Backups and for Scheduled Backups

My friend, Grant Moyle (author of our Exchange/Windows video books), turned me on to SyncToy in October 2006 when he made a great video extolling it's virtues (titled Using SyncToy to Synchronize Directories/Machines). I'm slow on the uptake though so it took me almost two years before I used it...

Let me first give you an idea of what SyncToy does: SyncToy synchronizes files and folders between "A" and "B" with "A" and "B" being either local folders or remote folders. It does whatever you wish: want to delete all files in B and then copy A over? It can do that. Want to copy over only the changed files? Check. Want to <insert description here> from A/B to B/A? Check. SyncToy also covers both bases: it features both point-and-click backup options as well as the ability to schedule it to run.

So like I said earlier, I'm developing the videobooks.com website project and maintaining the LearnItFirst.com websites. SyncToy uses "Folder Pairs" to determine "A" and "B" so, when you define your synchronization, you define "A" and "B" as local or remote folders. Check out an example screenshot:

The Run button is the point and click option and, as you might be able to see, I've set mine up to perform a differential with the Left Folder (what I'll call "A") being the master and the Right Folder ("B") being the target. This is what SyncToy calls an "Echo". When I click Run after modifying one file at the master, here's what I see:

So, with SyncToy, I've got a great tool that performs all of my point-and-click incremental backups - for free. Next stop: how do I schedule backups with SyncToy? Easy - launch SyncToy and then go to Help-> Learn How to Schedule SyncToy... as shown here:

I'll leave it to you to read the instructions on setting up your scheduled tasks since each OS will have different actions. I will also say that I prefer to write batch files to do this type of thing instead of creating an "action"-based task... It has a super-easy syntax - here's an example batch file that I use called SyncToy.bat that synchronizes all folders:

cd "C:\Documents and Settings\ScottW\Local Settings\Application Data\SyncToy\"
SyncToy.exe -R

That's it - and it could even be simplified to a single line if you wish. Next - just create a scheduled task to execute the batch file and you're done! Note: if you wanted only to synch a single folder pair, change the -R to something like -R"My Folder Pair" (substituting the name of your folder pair of course).

Okay - so with SyncToy I've solved most of my problems - I can do point-and-click backups as well as scheduled incremental backups. For free :)

Download SyncToy or read an article on using it here.

Next Tools: Robocopy and Robocopy GUI

I'll wrap up here since Robocopy has been around forever and will require you to dig into the documentation and figure it out (unlike SyncToy which is super easy). There are also tons of links on the web that you can use to get started (and I'm tired of writing about backups). Here's how I would describe Robocopy: Robocopy synchronizes folders, not files, in every way that you will likely ever need to do so. Simply put, Robocopy does what SyncToy does and a whole lot more (such as continuing after a network outage and even failover). In fact, SyncToy was written as a direct result of user feedback about Robocopy (people wanted a simpler interface).

Robocopy originally shipped with the Windows Server Resource Kit(s) and, for those of you on Windows XP or Windows Server 2003, you can download it here as part of the Windows Server 2003 Resource Kit tools. Robocopy is included in Windows Vista so you do not need a separate download if you are on Vista.

You can do point-and-click backups with Robocopy via batch files/command line or you can schedule it by creating a batch file and scheduling a task. Here's a sample Robocopy command:

ROBOCOPY \\sourceserver\share \\remoteserver\share /MIR

The above command uses a source ("\\sourceserver\share") and a destination (\\remoteserver\share) and, when you use the /MIR switch, it will delete all files in \\remoteserver\share and then copy all files from the source to the destination (making the destination a "mirror"). Here's a good example of how to use Robocopy as a scheduled task to create such a mirror: http://www.simplefailover.com/kb.aspx?kbid=1013 

Robocopy GUI was also written in response to the user feedback that requested an easier interface for Robocopy. Robocopy GUI is nothing more than a wrapper around Robocopy.exe, the tool mentioned above (so make sure you download the Windows 2003 Resource Kit and install it otherwise Robocopy GUI won't work). Robocopy GUI's most important feature, in my opinion, is the option to create a batch file featuring your choices/options. Once you have the batch file you can then schedule it as a task!

Robocopy allows for both point and click as well as scheduled backups but, if I had to guess, you'll likely only use it for scheduled backups (otherwise you'll have to memorize all switches at the command line each time you run it).

Summary

Both SyncToy and Robocopy do the same thing but I use SyncToy to synchronize folders for point and click and Robocopy for anything that I schedule. SyncToy's visual options just seem easier for me to use than batch files so I tend to have only a few folder pairs and the rest is managed on a schedule by RoboCopy.

Download Links:

• Download SyncToy 
• Robocopy for Windows XP and Windows Server 2003 (included in Vista/2008): Windows Server 2003 Resource Kit tools

# posted by Scott Whigham @ 5:33 AM