Clone root partition 

Newsgroups: comp.os.linux.misc
Date: 31 Dec 2006 04:37:07 GMT

I'm trying to compile a comprehensive document on cloning root partitions. My immediate goal is to clone my current working Linux to external USB HD, so that I can use it wherever I go.

By comprehensive I mean it should not be as simple minded as

dd if=/dev/hda2 of=/dev/sda2

or

cp -a / /mnt/point

or

tar -p -m cf - / | (cd /mnt/point; tar xf - )

I know they work, but there are so many things have been left out. By comprehensive, I mean I want to know all relevant things that need to be considered.

For example for dd, let alone its rigid limitation, if you use it, at least the 'conv=sync,noerror bs=4k' options should be used: sync,noerror just means continue and zero fill any error blocks, bs=4k just writes 4k at a time which will speed things up a lot. For cp, at least 'cp -ax' should be used.

But there are still much more to it.

First, directories that don't need to copy over, like /tmp, /proc. With modern Linux that uses udev, the /dev and /sys don't need to be copied either. Anything else (besides distro specifics like /var/cache/apt/archives)?

2nd, the clone partition should be made bootable, by grub or lilo.

Anything else? Like the concerns of /etc/fstab…

Last, with all the above concerns, how to achieve them with various tools?

*Tags*: tar rsync find cpio dd

Clone root partition 

On Sat, 30 Dec 2006 16:25:25 -0600, Dances With Crows wrote:

>>> the clone partition should be made bootable, by grub or lilo.
>
> No, really?  Are you having trouble with this step?  Explain the trouble
> you're having.

First of all, thank you all who replied. No I don't have problems with that. Actually, as said in OP, I was looking for *concerns* over cloning root partitions.

> I'd say it's a better idea to use GRUB here, since
> booting from things that aren't IDE can cause weird rearrangements in
> BIOS boot order depending on the x86 you're using.  (Boot from IDE
> CD-R*, SATA disk is (hd2).  Boot from SATA disk, SATA disk is (hd0).
> BTDT.)

So I read. GRUB is not very reliable for booting usb. I read that the most reliable way is to use SYSLINUX, but having played with SYSLINUX for the whole day, I think I'll rule out SYSLINUX and use back to GRUB.

> GRUB gives you a shell you can use to try to recover.  LILO
> doesn't.

yep, that'd be the last resort to refer to. On the contrary, using SYSLINUX would *just work*.

>>> For example for dd
>
> dd is not a backup method.  Forget it.

I think so too.

> cp -a all directories that aren't /proc, /sys, or /dev (if using
> udev/devfs), edit /etc/fstab to reflect new position of drive, reinstall
> bootloader.  3 steps--which one are you having trouble with?

Actually I want to rule out 'cp' as well, since "cp -a all directories that aren't /proc, /sys, or /dev" is easy to say than done in script. rsync should be a much better option.

T

Clone root partition 

Newsgroups: gmane.linux.debian.user
Date: Sat, 30 Dec 2006

I did this only yesterday - but in my case I wanted a mirror image of the entire system, not just the root partition.

The simplest most bullet proof procedure I could come up with was:

  1. dd if=/dev/hda of=/dev/sda
  2. vi /etc/fstab in the copy and 1,$s/hda/sda/
  3. either

    1. edit /boot/grub/menu.lst on the internal drive to add a boot obtion passing the USB root to the kernel, or
    2. if your host supports USB booting, update the boot sector on sda to look for the stage 2 boot in the USB partition.

I used for first option, so am not sure if I have covered everything required for a direct USB boot…

I booted using the USB copy and everything looked to be working fine.

Didn't use 'sync,noerror' in the dd operation because I count on having perfect media, and if I don't I want to know about it!

Digby Tarvin

Clone root partition 

> For cp, at least 'cp -ax' should be used.

I've never used option flags for this purpose, and I'm not sure what the -ax options could mean in this context, since copying from device to device ignores not only the filesystem but partition data as well. Used in this way, cp streams data in byte order from device to device, copying partition tables, boot loaders and filesystem indexing tables as well as the filesystems, regardless of the target device geometry. Since I am unsure of the implications of this, I use this method only between identical devices, and only when there are no bad blocks on either device. In practice this means that I only rarely use this approach, but I have been considering it for cloning backup pc archives.

> But there are still much more to it.
>
> First, directories that don't need to copy over, like /tmp, /proc. With
> modern Linux that uses udev, the /dev and /sys don't need to be copied
> either. Anything else (besides distro specifics like /var/cache/apt/archives)?
>
> 2nd, the clone partition should be made bootable, by grub or lilo.
>
> Anything else? Like the concerns of /etc/fstab...
>
> Last, with all the above concerns, how to achieve them with various tools?
>
> Keywords: tar rsync find cpio dd

You don't mention rsync, which is currently my preferred method. It's simpler to use than tar, provides the option of incremental updates, and unlike cp -a it works around linux' lack of an "lutimes" system call, taking care of time stamps of symbolic links. For cloning root filesystem drives I use a small script that performs an rsync backup followed up by fixups to the /dev directory, /etc/fstab, and /etc/lilo.conf and then runs lilo -r to make the backup bootable. (A similar approach could be used for grub.) The script runs in a few minutes if I disable rsync checksumming.

Here is the entire script, which uses /dev/hda1 as my backup root drive:

mount /dev/hda1 /mnt/hda1
time rsync -vxaHD --delete / /mnt/hda1/
rsync -xaHD --delete /dev/ /mnt/hda1/dev/
cp -a /etc/fstab.hda1 /mnt/hda1/etc/fstab
cp -a /etc/lilo.conf.hda1 /mnt/hda1/etc/lilo.conf
rm -rf /mnt/hda1/dev/.udevdb
lilo -r /mnt/hda1
umount /dev/hda1

This script presumes that udev is installed. For it to work, I have to manually maintain /etc/fstab.hda1 and /etc/lilo.conf.hda1.

Disclaimer for newbies: please do not use this script unless you completely understand how it works, and are not using /dev/hda1 as your root filesytem!

The backup drive can be selected through the BIOS boot menu if the main root drive craps out. The only complication is that one my applications (mythtv) uses mysql to keep track of files on an non-root filesystem, which creates the possibility that the database could be out of sync with the mythtv data files in the event of a main filesystem crash.

Marty

Clone root partition 

> For cloning root filesystem drives I use a small script that
> performs an rsync backup followed up by fixups to the /dev directory,
> /etc/fstab, and /etc/lilo.conf and then runs lilo -r to make the backup
> bootable.  (A similar approach could be used for grub.)  The script runs in a
> few minutes if I disable rsync checksumming.
>
> Here is the entire script, which uses /dev/hda1 as my backup root drive:
>
>   mount /dev/hda1 /mnt/hda1
>   time rsync -vxaHD --delete / /mnt/hda1/
>   rsync -xaHD --delete /dev/ /mnt/hda1/dev/
>   cp -a /etc/fstab.hda1 /mnt/hda1/etc/fstab
>   cp -a /etc/lilo.conf.hda1 /mnt/hda1/etc/lilo.conf
>   rm -rf /mnt/hda1/dev/.udevdb
>   lilo -r /mnt/hda1
>   umount /dev/hda1
>
> This script presumes that udev is installed.  For it to work, I have to manually
> maintain /etc/fstab.hda1 and /etc/lilo.conf.hda1.

Thanks Marty, your rsync switch is the most comprehensive one I've seen.

Just FYI, when cloning the root partition, there are directories that don't need to copy over, like /tmp, /proc. So you may want to add —exclude switches to your script next time.

Further, with modern Linux that uses udev, like what you are using, the /dev and /sys don't need to be copied either. You can —exclude them as well.

Moreover, you may want to reconsider whether you want to clone /var/cache/apt/archives next time.

The point is that I don't want to clone more than necessary.

T

Clone root partition 

> Further, with modern Linux that uses udev, like what you are using, the
> /dev and /sys don't need to be copied either. You can --exclude them as
> well.

Found that my Debian (grml to be exactly) can't boot up with an empty /dev directory, kernel panic about no console device found.

T

documented on: 2007-09-16