Robocopy File Migration

Introduction

Migrating storage data has never been so easy with methods like, vMotion and 3PAR’s online import tool. I recently had to do a NAS data transfer from an old to new device and neither of the above options were available and neither was Rsync, so I had to fail back to good old Robocopy. It certainly isn’t the fastest method but when you are short on options it is a useful one. There are lots of switches you can use so I wanted to look at the most common ones and some gotchas.

Useage

Robocopy is short for robust file copy and is included with Windows Server 2008, Windows 7, Windows Server 2008 R2, Windows Server 2012, Windows 8

The basic usage is

robocopy \\source server\source path \\destination server\destination path <switches>

Here are some of the most common switches:

/e Copy subdirectories including empty ones

/zb Enables restartable mode. This is useful if the link you are copying across is unreliable as it will retry the file when the connection is established again. It also allows access to files using backup mode you be otherwise be unable to access

/copy:all Copy all the properties of the file

/copy:DATSOU Like above allows you to copy the properties of the file. But this allows the granular selection of which properties to copy

D Data

A Attributes

T Time stamps

S NTFS access control list (ACL)

O Owner information

U Auditing information

The default value for CopyFlags is DAT (data, attributes, and time stamps).

/log:c:\Robocopy.log Output the results to a log file

/log:c:\Robocopy.log /V /NPCommands can be used in conjunction with the log command to specify what exactly is logged. For example the following writes the results to a log file in verbose mode and does not show the percentage progress

Gotchas

/r:1 Specifies how many times copying a file will be retried. Setting this to something sensible is key or it will default to 1 million

/t:1 Specifies in seconds time to wait between retries. Again worth setting or it will default to 30 seconds

You can see the full list of switches here

Putting it all together

robocopy \\Server1\Y$ \\Server2\Y$ /e /zb /copy:DAT /r:1 /w:1 /log:c:\Scripts\Robocopy.log /V /NP

The above command will copy the data from \\Server1\Y$ to \\Server2\Y$ . Copying subfolders including empty ones, in restartable mode, copying the date, time and file properties. A retry and wait of 1 second each and outputting in verbose mode with no progress percentage to c:\Scripts\Robocopy.log

If you are using Robocopy to migrate data you can run it several times to pick up new files or modified files. A standard strategy would be to run it several times to copy as many files as possible, then to have an outage for the final copy to make sure that you have everything.

 

 

 

 

 

 

EMC Unity Logs Collecition

If you need to do a logs collection from an EMC Unity storage system follow this procedure:

1 Enter the management IP address of your Unity system into a web browser and login.  From the left-hand bar select service

1-service

2 On the top bar select service tasks

2

3 Ensure that Collect Service Information is selected and click on the Execute button

3

4 In the Collect Service Information window click the plus arrow near the top to initiate a logs collection

4

5 Once the SP collect is complete you will see a save file dialog.  Save the files locally, then upload them to EMC as requested.

3PAR OS 3.3.1 Performance Enhancements

Follow my blog with Bloglovin

There are a bunch of performance and under the hood enhancements in 3PAR OS 3.3.1 which I wanted to take a deeper look at today, I have covered an overview of all the new features in a previous post.  Some of them are brand new features others are enhancements of old ones. Let’s look at each one in turn

Adaptive flash cache (AFC) – allows SSD’s to be used as an extension to the controllers onboard DRAM memory to accelerate reads.  Analysis of the 3PAR install base has demonstrated that there is regularly capacity free in the AFC and to maximise its benefit the cache should be as full as possible. In 3PAR OS 3.3.1 AFC remains for reads only but extends the type of requests allowed further to include: large sequential I/O >64K and data read from snapshots.  Having more data in cache will of course increase the chances of a hit, these new types of request take a back seat to those previously defined in AFC i.e. if AFC is full with small reads it will not flush these out to allow a large sequential read.

Express Writes – Express writes aims to deliver lower latency by delivering lower CPU interrupts per IO.  This is achieved by sending the data along with the command rather than waiting for the target to request it. Previously this was only available with FC protocol, this is now extended to iSCSI for the 8000 and 20,000 systems. This will be enabled automatically at upgrade and can result in up to 40% improvements in latency for iSCSI writes

Multi Queue – Is another option that will automatically be turned on and self-optimised in 3Par OS 3.3.1. Previously each SAS or FC port was locked to a processor core, this worked well if all ports were utilised but left cores idle if all ports were not fully utilised.  Multi-queue cores can be shared between ports allowing for greater utilisation

Persistent Checksum – Ensures the integrity of data by performing a checksum on the data from the HBA to the disk.  The current implementation of Persistent Checksum is proprietary, hence the requirements for specific HBAs.  The new implementation switches to using standard T10 diff which is reliant on the host OS not the HBA’s and therefore widens the support.

To make sure that you don’t miss any updates, you can get e-mail updates, or follow via Face Book, LinkedIN and Twitter.

OTHER 3PAR OS 3.3.1 POSTS

3PAR Dedupe + Compression Deep Dive

3PAR Mega News Bundle – Including Compression