Backup using Rsync to NTFS about:reader?url=https://ubuntuforums.org/showthread.php?

t=820425

ubuntuforums.org

Backup using Rsync to NTFS

HOWTO: Backup using Rsync to NTFS

Problem: You have all your data in ubuntu in ext3 file format. Using
standard rsync tools do not work effectively because permissions of
files/folders cannot be carried over to ntfs systems. Also ntfs
systems have various "glitches" that cause backing up an ext3
system a problem. I have managed to find the ideal solution to
backup:
a) COMPLETE hard drive to NTFS
b) COMPLETE hard drive (minus massive music folder) to EXT3
c) Essential documents to NTFS (pendrive)
For this tutorial I will show you the essential elements of backing up
for a)

Rsync basically can check the destination data and see if any
changes have been made in the source data so that ONLY the
source data that has been changed is updated. VERY useful.

BackUp all folders to external NTFS Hard Drive.

For the purposes of this tutorial I assume the following:

1. User name: bob (so home folder is /home/bob/)

2. External hard drive is mounted at: /media/harddrive/
So, first of all its always a good idea to create a folder inside of your
external hard drive before backing up:

Code:

mkdir /media/harddrive/backup

So, now we are going to be making the backup bash script. I prefer
making a script as the command is quite long. First we need to
make a folder to store the script in (so it is easier to access).
Then we must create the bash script:
Code:

1 de 7 08/09/2016 09:44
Backup using Rsync to NTFS about:reader?url=https://ubuntuforums.org/showthread.php?t=820425

cd bash
gedit backupBIG.sh

I call it backupBIG.sh so as to differentiate it from another backup for

c)
In this text file put the following in:

Code:

#!/bin/bash

sudo rsync -rltDvu --modify-window=1 --progress

--delete --delete-excluded --exclude-from=/home
/bob/Bash/BackupBigExclude.txt / /media/harddrive
/backup

Explanation of commands:
rsync:this is the program used to sync the data. Rsync basically
can check the destination data and see if any changes have been
made in the source data so that ONLY the source data that has been
changed is updated. VERY useful.
-r: copies directories recursively

-l: copies symlinks as symlinks

-t: preserves modification times
-D: preserves device and special files
-v: shows output (verbose)
-u: skips files that are newer at the destination

--modify-window=1: this is ESSENTIAL. Basically in windows

filesystems time is kept in even numbers (or some such problem).
This command tells rsync to ignore filechanges that are only 1
second in difference from the original. It is almost impossible that
you will create a file, sync it, and in ONE second make a change
and want to sync it again. So it is safe to use this option and it
means that rsync will not back up everything every time simply
because of a one second change.

--progress: simply shows the progress

--delete: this is important. Basically rsync syncs files that have been
changed. But what if you delete a file from the source directory?
This command deletes it from your backed up hard drive as well.

2 de 7 08/09/2016 09:44
Backup using Rsync to NTFS about:reader?url=https://ubuntuforums.org/showthread.php?t=820425

--delete-excluded: this deletes folder/files that you have specifically

excluded.
--exclude-from=/home/bob/Bash/BackupBigExclude.txt: this is
the other crucial command. This basically tells rsync to exclude the
files/folders found in the list BackupBigExlude.txt. To create this list
do the following:

Code:

gedit /home/bob/Bash/BackupBigExclude.txt

Now in the text editor it is ENTIRELY upto you what you want to
exlcude.
This is my list:

Code:

/home/bob/MyDownloads
/home/bob/.VirtualBox
/lost+found
/media
/mnt
/proc
/sys
/tmp

Since I am backing up from my root directory, I am not backing up

the transient or temporary folders (/media, /mnt, /proc, /tmp, /sys,
/lost+found). I am not backing up my /MyDownloads directory as I
download a lot of files/data some of which I do not want to backup.
and I am not backing up my .VirtualbBox directory as its a massive
2gb file that will update every time I log into VB, a waste of
time/resources.
Here are some other hints/tips for the exclude command file:

1. To exclude hidden directories (all the directories that start with .)

do: /home/bob/.*/
2. To exclude hidden files (all the files that start with .) do:
/home/bob/.*
3. If you are only backing up from your home directory (that is,
/home/bob/) and NOT as we are backing up here from the root
directory (that is, /), the exclude command files must start from the

3 de 7 08/09/2016 09:44
Backup using Rsync to NTFS about:reader?url=https://ubuntuforums.org/showthread.php?t=820425

/home/bob/. So for example to exclude the MyDownloads folder you

would simply put this in the exclude file: MyDownloads. (WITHOUT
the complete link /home/bob/MyDownloads). This is complicated but
if you need further assistance please comment.
I am no expert with the exclude command but these settings worked
to exclude those files for me.

locations: finally the / and /media/harddrive/backup tells the script

where to backup from (/) and where to backup to.
There are numerous other settings/commands you can use,
including compression, etc. My method is simply to have a place
where I can access all my data with minimal hassle (I find
uncompressing a hassle, especially if I have enough space for full
data backup).

Suffice to say these commands are quite technical, but they all work
for me. Notice nowhere here do I tell it to preserve permissions as
these will not work if you are copying to an ntfs system.

For more information on rsync commands see http://www.samba.org

/ftp/rsync/rsync.html.
Save the files.

Now open a terminal and type the following:

Code:

cd /home/bob/Bash
sh BackupBig.sh

This should run the backup, it may take a long time depending on
how much data you have.
NB: Make sure you take care when changing any settings. If
you are unsure please ask. In fact don't even change the
location without first confirming.

For example, You want to backup /home/bob/ to backup /media

/harddrive/ and you have lets say data x.txt, y.txt, z.txt on /media
/harddrive/. If you do not create a separate folder all data on /media
/harddrive will be erased. SO BE CAREFUL!

4 de 7 08/09/2016 09:44
Backup using Rsync to NTFS about:reader?url=https://ubuntuforums.org/showthread.php?t=820425

Re: HOWTO: Backup using Rsync to NTFS

Moved to Tutorials and Tips.

My blog Poetry and More Free Ubuntu Magazine

Re: HOWTO: Backup using Rsync to NTFS

Thanks, I'm just always in the Absolute Beginner forum and I keep
posting there!

Re: HOWTO: Backup using Rsync to NTFS

Just a suggestion:
add 'p' to the flags to preserve permissions.

Sanus|artificium - FREEVST plug-ins | Music | Web design |

Graphics | Sound engineering | Software | Tutorials | PHP scripts
and more ...

Re: HOWTO: Backup using Rsync to NTFS

Originally Posted by sanus|art
Just a suggestion:
add 'p' to the flags to preserve permissions.

Thanks for the suggestion, but like I have already mentioned, this is
SPECIFICALLY for backing upto NTFS, which will NOT support
permissions. When I backup to my EXT3 I simply use -a (which
includes most of the useful flags).

Re: HOWTO: Backup using Rsync to NTFS

Originally Posted by abhiroopb
Thanks for the suggestion, but like I have already mentioned, this is
SPECIFICALLY for backing upto NTFS, which will NOT support
permissions. When I backup to my EXT3 I simply use -a (which
includes most of the useful flags).

I am not sure, but I am using it with '-a and -z' for archive and
compress mode to NTFS (mounted trough smbmount) and I think
permissions are preserved as I recovered '/etc/apache2' at Sunday.
Here are my comand:
Code:

sudo rsync --force --log-file=/var/log/rsync-

system.log --progress --delete
--exclude=/lost+found/ --exclude=/media/

5 de 7 08/09/2016 09:44
Backup using Rsync to NTFS about:reader?url=https://ubuntuforums.org/showthread.php?t=820425

--exclude=/cdrom/ --exclude=/mnt/
--exclude=/proc/ --exclude=/root/.thumbnails/
--exclude=/root/.Trash/ --exclude=/sys/
--exclude=/home/ --exclude=/tmp/ --delete-
excluded -avzlp / /media/XP/BACK_SYSTEM

Sanus|artificium - FREEVST plug-ins | Music | Web design |

Graphics | Sound engineering | Software | Tutorials | PHP scripts
and more ...

Re: HOWTO: Backup using Rsync to NTFS

Originally Posted by sanus|art
I am not sure, but I am using it with '-a and -z' for archive and
compress mode to NTFS (mounted trough smbmount) and I think
permissions are preserved as I recovered '/etc/apache2' at Sunday.
Here are my comand:
Code:

sudo rsync --force --log-file=/var/log/rsync-

system.log --progress --delete
--exclude=/lost+found/ --exclude=/media/
--exclude=/cdrom/ --exclude=/mnt/
--exclude=/proc/ --exclude=/root/.thumbnails/
--exclude=/root/.Trash/ --exclude=/sys/
--exclude=/home/ --exclude=/tmp/ --delete-
excluded -avzlp / /media/XP/BACK_SYSTEM

Ah I see. The thing is if you compress it permissions can be

preserved, but then you are stuck with (in my case) a massive
30-50gb file. I'd rather have all the files, as I have an EXT3 backup
which preserves all the permissions anyway. Again to reiterate, if
you backup with the -a or -p flags (or any other permission related
flags) on an NTFS drive you'll be getting errors after while backing
up. And permissions will not be preserved.

Re: HOWTO: Backup using Rsync to NTFS

Originally Posted by abhiroopb
... then you are stuck with (in my case) a massive 30-50gb file ...

Why is that? The compression is zlib!

Sanus|artificium - FREEVST plug-ins | Music | Web design |

Graphics | Sound engineering | Software | Tutorials | PHP scripts
and more ...

6 de 7 08/09/2016 09:44
Backup using Rsync to NTFS about:reader?url=https://ubuntuforums.org/showthread.php?t=820425

Re: HOWTO: Backup using Rsync to NTFS

Well I backup my 40gb music collection which does not compress at
all (mp3 is already a compressed format).

Re: HOWTO: Backup using Rsync to NTFS

I'm not sure what you mean, but my real '/home' is 1..6GB of size,
while rsynced '/home' on the backup is 756MB. zlib is 'per-file'
compression - so you can't see it visually like tar or zip (at least for
my knowledge).

Sanus|artificium - FREEVST plug-ins | Music | Web design |

Graphics | Sound engineering | Software | Tutorials | PHP scripts
and more ...

7 de 7 08/09/2016 09:44