Exchange 2007 and Windows 2008: Online Exchange Backup (part 6 of 7)

In the first five parts of this series, I've given you the background to understand how Exchange backup works when using VSS and how to acquire the necessary information from your Exchange server to know what you should back up. Today, I present to you a full-blown working script that will generate a full-backup of your Exchange 2007 Server on Windows Server 2008, verify that the backup is good using ESEUTIL, and flush the transaction logs for the Exchange storage groups if the backup is good.

The first five parts of the series were:

Part 1: Getting a List of Storage Groups in a PowerShell Script

Part 2: Getting a List of Stores in a PowerShell Script

Part 3: Exchange 2007 and Windows 2008: Offline Exchange Backup

Part 4: Volume Shadow Copy Services (VSS) and Exchange - The Basics

Part 5: Exchange 2007 and Windows 2008: Using Diskshadow for Online Exchange Backup

Now, before you ask - is this a supported backup tool? The answer is yes and no. VSS backups are a supported way to back up an Exchange server's databases. Diskshadow is a supported tool on Windows Server 2008. Is my script supported? No. Only so far as I find the time, energy, and effort to provide support for it. I can't warrant that it will work in your environment. It's worked everywhere I've tested it, that's all I can tell you. If you find a problem, let me know and I'll try to help, but there are no guarantees.

You won't find anything new in this script (from the prior postings in this series), except that the Diskshadow script is generated within the PowerShell script. This makes it easier when you run into a situation that you are using multiple volumes in your Exchange environment (which is a best practice for performance reasons).

The script takes three parameters:

$backupLocation - This is the volume and directory (or mountpoint) where the backup should go. It defaults to C:\Backups. You will probably need to change this for your environment.

$startLetter - This is the first letter that should be used by the script for exposing shadow copies as drive letters for the backup scripts. This defaults to g.

$startScript - This is a switch parameter. When set, the PowerShell script will initiate the backup using diskshadow.exe as soon as the script is built. The switch defaults to unset.

The #1 limitation of this script is that it backs up all storage groups on an Exchange server. I have plans to address that in a future revision.

The #2 limitation of this script is that it's "an ugly command line tool". I have plans to address that in a future revision.

This script will create a metadata file named (join-path $backupLocation "online-backup.cab") (That is C:\Backups\online-backup.cab by default). This file is used by diskshadow.exe for storing information required for restores. We will cover basic restores in part 7 of this series.

The cmd.exe script is stored as (join-path %TEMP% "online-backup.cmd"). The Diskshadow.exe script is stored as (join-path %TEMP% "online-backup.dsh").

On my server, I store PowerShell scripts in the c:\scripts folder, and name this particular script MBS-online-backup.ps1. A typical invocation is:

join-path "F:\ExchangeBackups" (get-date -uFormat "%Y-%m-%d-%H-%M-%S") |% { md $_ |% { ./MBS-online-backup.ps1 -backupLocation $_.FullName -startScript } }

This causes a unique directory to be created for each invocation of the backup script and for the script to be automatically run.

Granted, that's a dense for a beginner to understand. You can separate that into multiple lines quite easily:

$backupSubDir = get-date -uFormat "%Y-%m-%d-%H-%M-%S"
## As specified, the uFormat string means: yyyy-mm-dd-hh-mm-ss
## where the first 'mm' is the month number, and
## the second 'mm' is the minute number.
$backupDir = join-path "F:\ExchangeBackups" $backupSubDir
md $backupDir
./MBS-online-backup.ps1 -backupLocation $backupDir -startScript

And that is much easier to understand. Without further ado, here is the script:

##
## MBS-online-backup.ps1
##
## Michael B. Smith
## January, 2009
##
## This program generates an online VSS-based backup of an Exchange server
## (Exchange related files only) to a specified remote disk location.
##
## No warranties, express or implied, are available. It works for me. If
## you find errors or have problems, please feel free to let me know, but
## I can't guarantee that I can fix them.
##
## Feel free to use this in your own scripts. I would appreciate attribution.
##
Param(
	[string]$backupLocation = "C:\Backups",
	[string]$startLetter    = "g",
	[switch]$startScript    = $false
)

## $backupLocation is where the files and metadata go.

## $startLetter will contain the first letter we use to remap
## the volume letters contained in $volumes.

## $nl is the DOS newline character string

$nl = "`r`n"

## $volumes will contain the volume letters used by all named
## files and directories.

$volumes = @{}

## any storage group will contain:
## a] a system file directory
## b] a log file directory
## c] a filename for each database within the SG
##
## $pathPattern contains the dos patterns of files in the storage group

$pathpattern = @{}		### Exx.chk, Exx*.log, *.edb

## $storeList contains the filenames of the Exchange databases that need
## to be checked.

$storeList = @{}

## $letters contains the mapping between the original drive letter
## and the exposed driver letter in the shadow copy.

$letters = @{}

## $computerName contains the local computer's name

$computerName = $env:ComputerName

function buildRobocopyString($collection)
{
	$str = ""

	foreach ($filepath in $collection)
	{
		$file = split-path $filepath -leaf
		$path = split-path $filepath -parent
		#
		# the destination path is the source path appended to
		# the backup folder location.
		#
		$destpath = join-path $backupLocation $path.SubString(3, $path.Length - 3)
		#
		# the source path is the true path modified by the
		# letter of the exposed shadow copy
		#
		$letter = $letters.($path.SubString(0, 1))
		$subpath = $path.SubString(1, $path.Length - 1)
		$srcpath = "$letter$subpath"
		$str += "echo Copying " + $filepath + "..." + $nl
		$str += "robocopy " + '"' + $srcpath + '" "' + $destpath + 
		        '" "' + $file + '" /copyall /ZB >nul' + $nl
		$str += "if not errorlevel 0 goto :abort" + $nl
	}

	return $str
}

function buildESEUTILString($collection)
{
	$str = ""

	foreach ($path in $collection)
	{
		#
		# the destination path is the source path appended to
		# the backup folder location.
		#
		$path = $path.ToString()
		$destpath = join-path $backupLocation $path.SubString(3, $path.Length - 3)

		$str += "echo Checking " + $destpath + "..." + $nl
		$str += "call :checkit " + '"' + $destpath, '"' + $nl
		$str += "if not errorlevel 0 goto :abort" + $nl
	}

	return $str
}

function buildCMD
{
	$script = "@echo off" + $nl

	$script += buildRobocopyString $pathPattern.keys
	$script += $nl
	$script += buildESEUTILString $storeList.keys
	$script += $nl
	$script += "exit 0" + $nl
	$script += ":abort" + $nl
	$script += "exit 1" + $nl
	$script += $nl
	$script += ":checkit" + $nl
##	$script += "echo Checking %1" + $nl
	$script += "eseutil /k %1 >nul" + $nl
	$script += "if not errorlevel 0 exit 1" + $nl
	$script += $nl

	$scriptFile = join-path $env:temp "online-backup.cmd"
	$script | out-file $scriptFile -encoding ascii

	return $scriptFile
}

function writerOptimizationGarbage
{
	$script  = ""

	$script += "# verify presence of Exchange Writer" + $nl
	$script += "writer verify {76fe1ac4-15f7-4bcd-987e-8e1acb462fb7}" + $nl
	$script += "# exclude system writer" + $nl
	$script += "writer exclude {e8132975-6f93-4464-a53e-1050253ae220}" + $nl
	$script += "# exclude IIS config writer" + $nl
	$script += "writer exclude {2a40fd15-dfca-4aa8-a654-1f8c654603f6}" + $nl
	$script += "# exclude ASR writer" + $nl
	$script += "writer exclude {be000cbe-11fe-4426-9c58-531aa6355fc4}" + $nl
	$script += "# exclude BITS writer" + $nl
	$script += "writer exclude {4969d978-be47-48b0-b100-f328f07ac1e0}" + $nl
	$script += "# exclude WMI writer" + $nl
	$script += "writer exclude {a6ad56c2-b509-4e6c-bb19-49d8f43532f0}" + $nl
	$script += "# exclude registry writer" + $nl
	$script += "writer exclude {afbab4a2-367d-4d15-a586-71dbb18f8485}" + $nl
	$script += "# exclude iis metabase writer" + $nl
	$script += "writer exclude {59b1f0cf-90ef-465f-9609-6ca8b2938366}" + $nl
	$script += "# exclude com+ regdb writer" + $nl
	$script += "writer exclude {542da469-d3e1-473c-9f4f-7847f01fc64f}" + $nl
	$script += "# exclude shadow-copy optimization writer (does not apply to exchange)" + $nl
	$script += "writer exclude {4dc3bdd4-ab48-4d07-adb0-3bee2926fd7f}" + $nl
	$script += $nl

	return $script
}

function buildDSH([string]$cmdfilename)
{
	write-host "Building backup script"

	$script  = ""
	$script += "# Diskshadow backup script." + $nl
##	$script += "set verbose on" + $nl
	$script += "set context persistent" + $nl
	$script += "set metadata " + (join-path $backupLocation "online-backup.cab") + $nl
	$script += $nl
	$script += writerOptimizationGarbage
	$script += "begin backup" + $nl
	$script += $nl

	foreach ($drive in $volumes.keys)
	{
		$script += "add volume " + $drive + ": alias shadow_" + $drive + $nl
	}

	$script += $nl + "create" + $nl + $nl

	foreach ($drive in $volumes.keys)
	{
		$script += "expose %shadow_" + $drive + "% " + $letters.$drive + ":" + $nl
	}

	$script += $nl
	$script += "exec " + $cmdfilename + $nl

	#
	# If the batch file from exec fails, diskshadow terminates without
	# executing any more commands.
	#
	$script += "end backup" + $nl

	foreach ($drive in $volumes.keys)
	{
		## remove the temporary shadow copy and unexpose the letter
		$script += "delete shadows exposed " + $letters.$drive + ":" + $nl
	}

	$script += $nl
	$Script += "exit" + $nl

	$scriptFile = join-path $env:temp "online-backup.dsh"

	$script | out-file $scriptFile -encoding ascii
	write-host "Diskshadow script file $scriptFile"
	return $scriptFile
}

function getStores
{
	## locate the databases, both mailbox and public folder

	$colMB = get-MailboxDatabase -server $computername
	$colPF = get-PublicFolderDatabase -server $computername

	## parse them for volumes too

	foreach ($mdb in $colMB)
	{
		if ($mdb.Recovery)
		{
			write-host ("Skipping RECOVERY MDB " + $mdb.Name)
			continue
		}
		write-host ($mdb.Name + "`t " + $mdb.Guid)
		write-host ("`t" + $mdb.EdbFilePath)
		write-host " "

		$pathPattern.($mdb.EdbFilePath) = 1
		$storeList.($mdb.EdbFilePath)   = 1

		$vol = $mdb.EdbFilePath.ToString().SubString(0, 1)
		$volumes.$vol += 1
	}

	foreach ($mdb in $colPF)
	{
		## a PF db can never be in a recovery storage group
		## which is why the Recovery check isn't done here

		write-host ($mdb.Name + "`t " + $mdb.Guid)
		write-host ("`t" + $mdb.EdbFilePath)
		write-host " "

		$pathPattern.($mdb.EdbFilePath) = 1
		$storeList.($mdb.EdbFilePath)   = 1

		$vol = $mdb.EdbFilePath.ToString().SubString(0, 1)
		$volumes.$vol += 1
	}

	return
}

function getStorageGroups
{
	$count = 0
	#
	# locate the storage groups and their log files and system files
	#
	$colSG = get-StorageGroup -server $computername
	if ($colSG.Count -lt 1)
	{
		write-host "No storage groups found on server $computername"
		return 1
	}

	## parse the pathnames for each SG to determine what
	## volumes it stores data upon and what directories are used

	foreach ($sg in $colSG)
	{
		if ($sg.Recovery)
		{
			write-host ("Skipping RECOVERY STORAGE GROUP " + $sg.Name)
			continue
		}

		$count++

		$prefix  = $sg.LogFilePrefix
		$logpath = $sg.LogFolderPath.ToString()
		$syspath = $sg.SystemFolderPath.ToString()

		write-host $sg.Name.ToString() "`t" $sg.Guid.ToString()
		write-host "`tLog prefix:      $prefix"
		write-host "`tLog file path:   $logpath"
		write-host "`tSystem path:     $syspath"

		## E00*.log
		$pathpattern.(join-path $logpath ($prefix + "*.log")) = 1

		$vol = $logpath.SubString(0, 1)
		$volumes.$vol += 1

		## E00.chk
		$pathpattern.(join-path $syspath ($prefix + ".chk")) = 1

		$vol = $syspath.SubString(0, 1)
		$volumes.$vol += 1

		write-host " "
	}

	if ($count -lt 1)
	{
		write-host "No storage groups found on server $computername"
		return 1
	}

	return 0
}

function validateArrays
{
	$drives = $volumes.keys
	if ($drives.Count -lt 1)
	{
		write-host "No disk volumes were found. Aborting."
		return 1
	}

	write-host ("There were " + $drives.Count.ToString() + " disk volumes for Exchange server $computername. They are:")
	foreach ($drive in $drives)
	{
		write-host "`t$drive"
	}

	write-host " "

	$paths = $pathPattern.keys
	if ($paths.Count -lt 1)
	{
		write-host "No paths were found. Aborting."
		return 1
	}

	write-host ("There are " + $pathPattern.Count.ToString() + " directories to be backed up. They are:")
	foreach ($directory in $pathPattern.keys)
	{
		write-host "`t$directory"
	}
	write-host " "

	$letter = $startLetter.Chars(0)

	foreach ($drive in $volumes.keys)
	{
		$letters.$drive = $letter
		$letter = [char]([int]$letter + 1)
	}

	return 0
}

	##
	## Main
	##

	if ((getStorageGroups) -eq 0)
	{
		getStores
		if ((validateArrays) -eq 0)
		{
			$scriptFile = buildCMD
			$scriptFile = buildDSH $scriptFile
			if ($startScript -and ($scriptFile.Length -gt 0))
			{
				diskshadow.exe -s $scriptFile
			}
		}
	}

Until next time...

As always, if there are items you would like me to talk about, please drop me a line and let me know!

Published Monday, January 26, 2009 10:56 AM by michael

Comments

Friday, January 30, 2009 8:40 AM by djwells71

# re: Exchange 2007 and Windows 2008: Online Exchange Backup (part 6 of 7)

Growing impatient with waiting for MS to release the promised plug-in to backup Exchange 2007 on Server 2008, this script is a great find!

I have a couple of questions, you might be able to help with, before i decide to run script in live environemnt.

Will the script run just be using powershell command:

   MBS-online-backup.ps1

Can you confirm the correct ESEutil switches to verify backup copy /m?

Also, what is the correct switch to flush the log files?

Thanks

David (London)

Friday, January 30, 2009 2:30 PM by michael

# re: Exchange 2007 and Windows 2008: Online Exchange Backup (part 6 of 7)

The eseutil command is "eseutil /k <filename>" where <filename> is the name of a EDB file.

The logfiles are flushed by the Exchange Writer when the backup script returns a zero (success) value. If the backup script returns a non-zero value (failure) then the logfiles are not flushed.

Saturday, January 31, 2009 11:05 PM by knet09

# re: Exchange 2007 and Windows 2008: Online Exchange Backup (part 6 of 7)

hi

I tried your wonderful script - it works through successfully, but though the "end backup" is issued and the exposed shadow is deleted - the .log files do not go away...

any ideas here?

kindest regards

harald

Sunday, February 01, 2009 9:41 AM by subject: exchange

# Weekend reading

Tutorial: Secure journaling in Exchange Server 2007 Crash Course: Microsoft Exchange Server 2007 Unified

Wednesday, February 25, 2009 4:06 PM by Exchange 2007 su WS2008 - Backup e file log... ?? | hilpers

# Exchange 2007 su WS2008 - Backup e file log... ?? | hilpers

Pingback from  Exchange 2007 su WS2008 - Backup e file log... ?? | hilpers

Wednesday, March 18, 2009 1:57 PM by leso03

# re: Exchange 2007 and Windows 2008: Online Exchange Backup (part 6 of 7)

Hi , I have just tested your script , It apparently works but I saw : $colSG.Count is undefined and all the time empty. SO I remove the if statement. And when I launch the script :

Key must not null.

Name : key

    $pathPattern.( <<<< $mdb.EdbFilePath) = 1

Key must not null.

Name : key

$storeList.( <<<< $mdb.EdbFilePath)   = 1

    $vol = $mdb.EdbFilePath.ToString( <<<< ).SubString(0, 1)

.

Have you an idea to help me? My Powershell is in french so I removed few lines...

Monday, March 30, 2009 7:00 PM by lucidblue

# re: Exchange 2007 and Windows 2008: Online Exchange Backup (part 6 of 7)

Just wanted to say thanks, a lot!

Shy of changing the backup location, this script worked perfect out of the box.  This has alleviated a lot of issues with 3rd party VSS providers not properly backup our our Exchange 2007 Organization.

Tuesday, March 31, 2009 2:31 PM by Peter

# re: Exchange 2007 and Windows 2008: Online Exchange Backup (part 6 of 7)

Hello and thank you for this script.

I get this error when I run the script:

Key cannot be null.

Parameter name: key

At C:\Scripts\MBS-online-backup.ps1:249 char:16

+         $pathPattern.( <<<< $mdb.EdbFilePath) = 1

Key cannot be null.

Parameter name: key

At C:\Scripts\MBS-online-backup.ps1:250 char:14

+         $storeList.( <<<< $mdb.EdbFilePath)   = 1

You cannot call a method on a null-valued expression.

At C:\Scripts\MBS-online-backup.ps1:252 char:35

+         $vol = $mdb.EdbFilePath.ToString( <<<< ).SubString(0, 1)

Do you now what that means? The script contimues to run, but I just want to make sure that everything is ok.

Thanks again.

Friday, April 10, 2009 10:48 AM by jdinatal

# re: Exchange 2007 and Windows 2008: Online Exchange Backup (part 6 of 7)

This is a great script! I saw your talk on this at DevConnections in the fall and you mentioned a restore script. Is this available for download anywhere?

Thanks

Saturday, April 11, 2009 4:08 PM by Michael's meanderings...

# Backing Up Exchange 2007 on Windows 2008

It has been suggested to me that my series on backing up Exchange Server 2007 on Windows Server 2008

# &raquo; ?????????????????? ?????????????????????? Microsoft Exchange Server 2007 ?????????????????????? ???????????????????? Windows 2008 ??? ExchangeRUS - ???????? ?? Microsoft Exchange Server ?? ?????????????????????? ??????????

Pingback from  &raquo; ?????????????????? ?????????????????????? Microsoft  Exchange Server 2007 ?????????????????????? ????????????????????  Windows 2008 ??? ExchangeRUS - ???????? ?? Microsoft Exchange Server ?? ?????????????????????? ??????????

Monday, June 15, 2009 10:00 AM by konddor

# re: Exchange 2007 and Windows 2008: Online Exchange Backup (part 6 of 7)

Hi

This Script is great, can someone help me how to Schedule it??

Thanks