RAID Rebuilding

S/A Daniel Dickerman

Technical Advisor to the Director, Electronic Crimes Program IRS - Criminal Investigation daniel.dickerman@ci.irs.gov

Objectives
Brief introduction to RAID technology and the issues you need to be aware of to properly perform the acquisition and rebuilding of data stored on a RAID array, for subsequent analysis. What is a RAID? Hardware vs. Software RAID RAID Attributes RAID Levels

Objectives (cont.)
RAID rebuilding 101 Rebuilding Tools RAID Reconstructor X-Ways Forensics/WinHex (Specialist or Forensic license) Encase SMART

What is RAID?
Redundant Array of Inexpensive/Independent Disks Multiple disks functioning as one for:
Fault Tolerance (Data Protection) Increased Performance Increased Capacity

Hardware RAID
Hardware RAID is controlled by a RAID controller. The OS is typically unaware that it is writing/reading to/from multiple disks.

Hardware RAID
What the forensic examiner sees (physically).

Hardware RAID
What the OS seesa 273GB primary disk and two 2,235 GB Disks

Hardware RAID
The physical drives that are actually present3-136GB array disks and 1-136Gb hot spare, plus 14 400GB IDE disks in an Apple X-Serve RAID (not shown in screenshot).

Hardware RAID
What your imaging tool might see

* The above screenshot is for the sole purpose of demonstrating examples of RAID volume detection and does not necessarily depict the RAID volume detection capabilities of all versions of the above shown tool. The disks and volumes detected will vary depending on the version of your imaging tool and the controller drivers incorporated into your bootable disk.

Hardware RAID
What your imaging tool might see

* The above screenshot is for the sole purpose of demonstrating examples of RAID volume detection and does not necessarily depict the RAID volume detection capabilities of all versions of the above shown tool. The disks and volumes detected will vary depending on the version of your imaging tool and the controller drivers incorporated into your bootable disk.

Hardware RAID
What your imaging tool might see

* The above screenshot is for the sole purpose of demonstrating examples of RAID volume detection and does not necessarily depict the RAID volume detection capabilities of all versions of the above shown tool. The disks and volumes detected will vary depending on the version of your imaging tool and the controller drivers incorporated into your bootable disk.

Hardware RAID
What your imaging tool might see

* The above screenshot is for the sole purpose of demonstrating examples of RAID volume detection and does not necessarily depict the RAID volume detection capabilities of all versions of the above shown tool. The disks and volumes detected will vary depending on the version of your imaging tool and the controller drivers incorporated into your bootable disk.

Software RAID
Software RAID is controlled by the OS or software running in the OS.
On a PC, the bootable system drive is not part of the Software RAID, but usually contains the information required to load/access the software RAID. Many multi-drive external storage devices are actually Linux software RAIDs behind the scenes, where the device has a Linux OS on its firmware that controls disk read/write operations to the multiple disks.

Software RAID
Notice the X: drive is a 4471 GB Windows Server 2003 striped volume made up of two 2235 GB physical diskswhich are actually each made up of 7 400GB IDE disks set up as RAID 5 hardware RAID volumes. (a software
RAID 0 striped across two hardware RAID 5 volumes = RAID 50.)

RAID Attributes
Disk Order Stripe Size RAID Header Parity
Dedicated vs. Distributed Parity Type/Rotation Parity Delay

RAID Attributes
Disk Order
The order of the disks that make up the array This may seem like a very simple one, but when pulling individual drives from a RAID, it is easy to get them out of order or mislabel the image names for each disk image. Always double check yourself, especially when putting the disks back into the server to ensure they are in the correct order.

RAID Attributes
Stripe Size How much data is written to each disk before moving to the next disk to write the next block of data. Typical stripe sizes:
8,16, 32, 64, and 128 kilobytes per stripe you may occasionally see other sizes

RAID Attributes
RAID Header
Static block of data at the beginning of each array disk. May be identical (or nearly identical), making you initially think its a mirror Usually has a byte that identifies the disk # for the array, which gives you your Disk Order Header size and disk # usually found by performing a comparison of the disks. Compaq/HP servers usually = 1088 sector header size

RAID Attributes
Parity
Rebuilding information created by XORing together bytes from each disk containing RAID data, the result of which gets stored as a parity value on the parity disk. The drive on which this calculated parity data is stored will depend on the type of Parity Rotation used.
Parity Rotation described in more detail later in presentation

RAID4 = Dedicated parity disk RAID5 = Distributed parity disk

RAID Levels
RAID 0 (Striping) RAID 1 (Mirroring/Duplexing) RAID 5 (Striping w/ Distributed Parity) Multi-RAID levels RAID 1+0 (a stripe of mirrors) RAID 0+1 (a mirror or stripes) RAID 1+5, 5+1, 0+5, 5+0, etc. Other non-RAID multi-disk setups: Disk Spanning JBOD (Just a Bunch Of Disks)

RAID 0
No fault tolerance
Single disk failure = array failure

Fastest performance Capacity of array = total capacity of individual disks combined Items needed for rebuilding:
Disk Order Stripe Size RAID header size*
* Not all RAIDs have a RAID header

RAID 1
Fault tolerance (via data replication) Increased read performance, same write performance as writing to single disk 50% of disk capacity used for data redundancy Items needed for rebuilding:
Typically no rebuilding necessary unless RAID header exists*

* Not all RAIDs have a RAID header

RAID 5
Fault tolerance (via parity data) Increased read and write performance 1/Nth reduction in disk capacity, used for parity, where N = # of array disks.
Minimum of 3 array disks needed for any RAID level with parity

RAID 5
Rebuilding components:
Disk order Stripe size RAID header size* Parity rotation Parity delay**

* Not all RAIDs have a RAID header ** Only used in Backward Delayed Parity

RAID 5
Parity Rotation
Backward Delayed Parity (Compaq/HP)*

* Example shown using a parity rotation delay of 4, meaning parity stays on its current disk for 4 stripes, then moves for the next 4 stripes and so on.

RAID 5
Parity Rotation
Backward Dynamic Parity (AMI)
Probably the most common type

RAID 5
Other Parity Rotations
Backward Parity
(Adaptec)

Forward Parity

RAID Rebuilding 101

The goal in RAID rebuilding it to put back together the data that has been spread out across multiple disks and may include parity information, depending on the RAID level. This is done by re-pasting the striped data back together into one disk/image and removing the parity as you go.

Individual RAID 5 disks/images Disk 0 Stripe1 T Stripe2 A Stripe3 R Stripe4 ! Disk 1 H S A Parity Parity I Disk 2 I Disk 3 S Parity A D ! Disk 4 Parity W

RAID 5 rebuilt into single disk Disk 0 THIS WAS A RAID!!

RAID Rebuilding 101

The more you document about the RAID onsite, the less you have to manually try to figure out later!
Boot RAID server into RAID Controller BIOS configuration utility during Power On Self Test (POST)

View array configuration and write down the RAID level, disk order, stripe size, disk & array configuration, controller type, etc!!!

RAID Rebuilding 101

RAID Rebuilding 101

RAID Rebuilding 101

Any of the information you are unable to determine onsite during the imaging of the RAID disks will have to be either manually determined or possibly via some guesswork. Manual interpretation of the striped data on RAID disks is not difficult if you have an in-depth understanding of how data structures are laid out on a non-RAID disk, including: MBR and Partition Table Boot Sectors/Records FAT tables, Root Dirs, etc. MFT records, INDX entries, etc. Unfortunately, it is not possible to cover manual data interpretation in this one hour presentation.

RAID Rebuilding Tools

RAID Reconstructor (Runtime Software)
http://www.runtime.org/raid.htm

X-Ways Forensics/WinHex (X-Ways Software

Technology AG) http://www.x-ways.net/forensics/index-m.html

Encase (Guidance Software)

http://www.guidancesoftware.com/products/ef_index.aspx

SMART (ASRData)
http://www.asrdata2.com/ ***There are a few other RAID rebuilding tools out there but as of the writing of this presentation, the above tools were the only ones I had available to include.

RAID Reconstructor
Step #1 chose RAID type, number of drives, add drives images (in correct order), select block size and parity rotation.

RAID Reconstructor
Step #2 analyze data to attempt to determine correct RAID parameters.

RAID Reconstructor
Step #3 - write out a new rebuilt single image from the multiple images.

RAID Reconstructor
Pros Tests numerous combinations of RAID parameters to try and Guess settings using entropy testing. Useful when you dont know the parameters. Works with up to 14 RAID disks for RAID 5. Will rebuild RAID 5, from parity, with one missing disk/image. Cons Can only do a 2-disk RAID 0 Doesnt do Backward Delayed Parity RAIDs Requires you to actually rebuild a new image before you can check to see if you actually have the correct settings. Only after the rebuild can you open the new image in your forensic tools. Does not recognize .e01 or other image formats, must convert images to raw bit.

X-Ways Forensics/WinHex
Step #1 Open each individual disk image and Interpret Image File as Disk from the Specialist menu.

X-Ways Forensics/WinHex
Step #2 Select Assemble RAID system from the Specialist menu. Open each disk component in the correct order, enter the header size, select the parity rotation type and stripe size and click OK.

X-Ways Forensics/WinHex
If you entered the correct RAID parameters, the RAID volume is virtually reconstructed, allowing you to map out the file system.

X-Ways Forensics/WinHex
Pros Performs a virtual rebuild in RAM to allow you to see the results right away. File system mapping errors indicate if you have the wrong parameters. Works with up to 10 RAID disks for RAID 5 or RAID 0. Will rebuild RAID 5, from parity, with one missing disk/image. The only tool that does Backward Delayed Parity (Compaq/HP). Reads .e01 or raw bit images. Cons Does not use entropy or do any guesswork for you.

EnCase (Software RAID)

EnCase (Software RAID)

EnCase (Hardware RAID)

EnCase
Pros Can be used to virtually reconstruct Windows Software RAIDs and some hardware RAIDs. Reads .e01 and raw bit images. Can rebuild RAID 5, from parity, with a missing image. Cons Only rebuilds Right or Left handed stripe RAIDS. (Not sure what Parity rotation types these refer to, but they are not in line with the correct industry terminology used by other vendors.) Lacks features for RAID headers and Delayed Parity.

SMART

1 3 4 2

SMART

1 2 3

1 2

SMART
Pros Can be used to virtually reconstruct RAIDs. The only tool that does RAID4. Allows removal of RAID header when importing images (prior to RAID rebuilding steps). Reads .e01 and raw bit images. Guesses using entropy to try to determine settings for you. Cons Only rebuilds Right Symmetric or Left Symmetric parity RAID5 (no Backward Dynamic or Backward Delayed). Relies on Linux OS it is running on for driver support (i.e. MD raid driver). Device detection may be more complex and require more user interaction or configuration. Linux drivers are not available for all controller cards. Requires Linux knowledge/familiarity.

The End

Questions??? Concerns??? Confusion???