$ yes | tar -cvf /tmp/txtHist -M -L 5 . ./ ./i/ ./i/IiDev.doc Prepare volume #2 for `/tmp/txtHist' and hit return: Prepare volume #3 for `/tmp/txtHist' and hit return: Prepare volume #4 for `/tmp/txtHist' and hit return: Prepare volume #5 for `/tmp/txtHist' and hit return: ./i/IiMain.doc [...] ./CgFiles.doc Prepare volume #700 for `/tmp/txtHist' and hit return: Prepare volume #701 for `/tmp/txtHist' and hit return:
$ dir /tmp/txtHist*
— only one file created!
$ tar tvf /tmp/txtHist tar: This does not look like a tar archive tar: Skipping to next header tar: Error exit delayed from previous errors
— Failed!?. Specifying -M won't help.
TODO:
investigate why above failed
need to organizate the archive into a sequence of files —info-script=SCRIPT-NAME
better add some label? —label="SOME-PREFIX `date +SOME-FORMAT`"
Newsgroups: comp.os.linux.misc Date: Sat, 08 Feb 2003 12:42:59 -0500
>what is the parameter to create multiple volumes with tar on disk - not on any >tape drive. >I want backup some directories and limit the filesize to 500mb for storing >them on cdrw.
The multi-volume with tar kind of assumes you will be swapping tapes out; so what you need to do is create a script to "simulate" a tape switch by renaming files. You can manually rename the file to file1, etc, and let tar "think" you changed tapes.
Below is an example. The following example only counts to 10 correctly, but you can get the idea. The problem with tar multi-volume is that if you want to compress, you have to compress afterwards, and the results are not predictable to know how to fill a cd, etc. There is a backup solution called "dar", (do a google search); which will let you set a "compressed multi-volume size". It is worth checking out.
I have a better set of scripts which does this with perl, but I hate do distribute them because they are suited to my system, but if you want a copy, email me. My scripts count as high as you need, and break each directory into it's own backup, in case you need to restore just one directory. There are restore scripts too, which do the opposite function to reassemble the tar segments.
#!/bin/sh
#backup
if [ -z "$1" ]; then
echo 'Usage: backup dir'
exit 1
fi
PREFIX="mybackup"
tar -c -X EXCLUDE_FILE -M -L 500 -F backup-rotate.sh -f $PREFIX.tar $1
#this section tests for the last incomplete archive and renames it to
the last number
##################################################################################
#LASTNUM=`ls $PREFIX-*.tar | tail -1 | awk -F. '{print $1}' | awk -F-
'{print $2}'`
LASTNUM=`ls $PREFIX-*.tar | awk -F. '{print $1}' | awk -F- '{print $2}'|
sort -g | tail -1`
if [ X$LASTNUM = X ] ; then
NEWNUM="1"
else
NEWNUM=`expr $LASTNUM + 1`
fi
mv $PREFIX.tar $PREFIX-$NEWNUM.tar
#################################################################################
#this section gunzips all the tar files
#gzip *.tar
exit
And here is the backup-rotate.sh
#!/bin/sh
PREFIX="mybackup"
#LASTNUM=`ls $PREFIX-*.tar | tail -1 | awk -F. '{print $1}' | awk -F-
'{print $2}'`
LASTNUM=`ls $PREFIX-*.tar | awk -F. '{print $1}' | awk -F- '{print $2}'|
sort -g | tail -1`
if [ X$LASTNUM = X ] ; then
NEWNUM="1"
else
NEWNUM=`expr $LASTNUM + 1`
fi
mv $PREFIX.tar $PREFIX-$NEWNUM.tar
zentara
> > I have a better set of scripts which does this with perl, but > >hi, could you mail a copy to me please (and the restore script too)? >That's something I've been looking for for a long time. thanks.
this is how they work. You run "do-backup root" or "do-backup home". They will tar and gzip all directories, and break them into small pieces, for easier copying. /usr is broken into it's sub-dirs since it's so big.
The homedirs are also broken into subdirs, AND all the hidden files and directories are saved into a separate directory "rootbacktemp", so when you restore, remember to copy them back to your homedir root.
When the cd's are make, 3 programs are copied to each cd, "restore,restore-rotate.pl, and microperl". microperl is a small version of perl, that allows you to restore from scratch, without a perl installation.
To restore, just copy all the files you want into a directory, along with "restore,restore-rotate.pl, and microperl". Then run restore. Everything will be automatic after that. These cd's are written as ext2, so if you boot with a rescue floppy, and try to mount the cd's to copy them in, use "mount -t ext2 /dev/hdc /mnt" (or whatever is right for you.)
The "cdwrite" script , way down at the bottom, has the setting for cdrecord. Use "cdrecord -scanbus" to get your dev, and put it in there. The dummy switch is for testing.
If you use the dummy switch, and keep inserting a blank cdr when prompted, you won't write any cd's but all the iso's will be created, in /.
This script isn't perfect, it requires that you have a good bit of free space on the hard drive for the iso's. The iso's are saved in /, but you could modify the script to delete each iso after it is written to cdr, if you are tight on space. There is a line in cdwrite which is commented out "#unlink image.iso". Just uncomment that line to delete isos after they are written.
I suggest using the dummy switch first, and see how it goes, if the iso's are created, then you are doing it right.
I tried to do some parallel processing in this script, so it will write and process the next iso at the same time. This is very cpu intensive, so it is best to do this when nothing else is happening.
The scripts run fine with a scsi cdwriter, I don't know if they behave well with ide-scsi.
Good luck, feel free to modify and use the scripts as you desire.
P.S. If you don't feel comfortable with running the binary "microperl" which I've include (I wouldn't run any binary sent to me); you can make your own or you can change the restore scripts to run with #!/usr/bin/perl
Microperl can be made from the perl5.8 distibution, with "make -f Makefile.micro".
These scripts run good once you understand them.
zentara
Newsgroups: comp.os.linux.alpha, comp.os.linux.misc Date: 1999/11/17
> What backup utility program do you use? Is BRU free to use in academic
You can do incremental backups with tar. Here is my tar script for backing up everything which changed since the last backup. In my case, I do it to a zipdisk, but you could modify this to do it to a tape. You would have to store the date of the last incremental backup somewhere; I do it on the zipdisk. You would also have to use mt commands to position yourself on the tape. And I would strongly recommend using the eof option of mt to add an EOF after each record. Then you can use mt fsf skipping twice for each record to position the tape.
mount /zipdos DAY=`date +%j-%H-%M` #day of the year-hour-min FILE=/zipdos/incr_"$DAY".tgz cd / DIRS="./home ./etc ./boot ./root ./usr/local ./var" /bin/tar --exclude cache --exclude opt --exclude magma -czlf $FILE -N \ "`cat /zipdos/tar.date`" $DIRS date > /zipdos/tar.date du -s /zipdos umount /zipdos
Leonard Evens
> requirements include
>
>1) should do "incremental", ("tar" cannot)
tar -G/-g
-G, --incremental handle old GNU-format incremental backup
-g, --listed-incremental=FILE
handle new GNU-format incremental backup
>2) should do it under cron ("taper" cannot) for incremental.
tar
>3) should do multi-volume in the forground for full (rhbackup cannot)
tar -M (look also -F)
>4) it would be nice if it can do compression (dump cannot)
tar -Z/-z/—use-compress-program PROG
marco
Newsgroups: comp.os.linux.hardware, comp.os.linux.misc, comp.os.linux.setup, redhat.servers.general Date: 2000/05/06
>i try to do an incremental backup with tar on an 4mm DAT.
>first i backed up all files on an multiple volume (3 tapes) now i want to >backup every night only the files that are new or have changed. is that >possible with tar/cron without using an expensive backup-application? >after the backup an email should sent to the admin with a list of the >backuped files (tar tf /dev/st0 >backup.log). the problem is how to send >this log-file to the admin/root?
If you are using GNUtar there are several options to control which files are archived, the best of which is "—listed-incremental filename". If 'filename' does not exist, it is created and you get a full backup. If it does exist, you get all files newer than the last run and everything (including old files) under renamed directories. Additional information is included in the archive to optionally allow you to delete files during the restore that were not present at the time the backup was taken (there is no way to do this with cpio or non-GNU tar). The —listed-incremental file is modified in place after an incremental run.
If you want to base all incrementals from the initial full, you should copy the file before each incremental and replace it with the original afterwards.
If you want to create a 'chain' of incrementals that must be restored in sequence, leave the modified file each time. A simple script can do this, including mailing the output to the admin.
Or, if you have more than a few filesystems you might want to install the free 'amanda' system that not only performs this scripting across machines but also does the scheduling of full/incrementals to match the tape space you have.
Les Mikesell
An archive containing all the files in the file system is called a "full backup" or "full dump". It is more efficient to do a full dump only occasionally. To back up files between full dumps, you can a incremental dump. A "level one" dump archives all the files that have changed since the last full dump.
A typical dump strategy would be to perform a full dump once a week, and a level one dump once a day. This means some versions of files will in fact be archived more than once, but this dump strategy makes it possible to restore a file system to within one day of accuracy by only extracting two archives—the last weekly (full) dump and the last daily (level one) dump.
GNU `tar' comes with scripts you can use to do full and level-one dumps. The name of the restore script is `restore'. The names of the level one and full backup scripts are, respectively, `level-1' and `level-0'. The `level-0' script also exists under the name `weekly', and the `level-1' under the name `daily'
Before you use these scripts, you need to edit the file `backup-specs', which specifies parameters used by the backup scripts and by the restore script.
`tar' will normally automatically cross file system boundaries in order to archive files which are part of a directory tree. You can change this behavior by running `tar' and specifying `—one-file-system' (`-l').
`--one-file-system'
`-l'
Prevents `tar' from crossing file system boundaries when
archiving.
If a file in a directory is not on the same filesystem as the directory itself, then `tar' will not archive that file. If the file is a directory itself, `tar' will not archive anything beneath it; in other words, `tar' will not cross mount points. The mount point is is archived, but nothing under it.
This option only affects files that are archived because they are in a directory that is being archived; `tar' will still archive files explicitly named on the command line or through `—files-from=FILE-OF-NAMES' (`-T FILE-OF-NAMES'), regardless of where they reside.
To create an archive that is larger than will fit on a single unit of the media, use the `—multi-volume' (`-M') option in conjunction with the `—create' (`-c') option (*note create::). A "multi-volume" archive can be manipulated like any other archive (provided the `—multi-volume' (`-M') option is specified), but is stored on more than one tape or disk.
You can read each individual volume of a multi-volume archive as if it were an archive by itself. For example, to list the contents of one volume, use `—list' (`-t'), without `—multi-volume' (`-M') specified. To extract an archive member from one volume (assuming it is described that volume), use `—extract' (`—get', `-x'), again without `—multi-volume' (`-M').
If an archive member is split across volumes (ie. its entry begins on one volume of the media and ends on another), you need to specify `—multi-volume' (`-M') to extract it successfully. In this case, you should load the volume where the archive member starts, and use `tar —extract —multi-volume' — `tar' will prompt for later volumes as it needs them.
Multi-volume archives can be modified like any other archive. To add files to a multi-volume archive, you need to only mount the last volume of the archive media (and new volumes, if needed). For all other operations, you need to use the entire archive.
Often you might want to write a large archive, one larger than will fit on the actual tape you are using.
Use `—multi-volume' (`-M') on the command line, and then `tar' will, when it reaches the end of the tape, prompt for another tape, and continue the archive. You can also use the `—tape-length=1024-SIZE' (`-L 1024-SIZE') option to specify the volume size.
`-M'
`--multi-volume'
Create/list/extract multi-volume archive.
This option causes `tar' to write a "multi-volume" archive--one that may be larger than will fit on the medium used to hold it.
`-L NUM'
`--tape-length=NUM'
Change tape after writing NUM x 1024 bytes.
This option might be useful when your tape drivers do not properly detect end of physical tapes. By being slightly conservative on the maximum tape length, you might avoid the problem entirely.
Each tape will have an independent archive, and can be read without needing the other. (As an exception to this, the file that `tar' was archiving when it ran out of tape will usually be split between the two archives; in this case you need to extract from the first archive, using `—multi-volume' (`-M'), and then put in the second tape when prompted, so `tar' can restore both halves of the file.)
The volume number used by `tar' in its tape-change prompt can be changed; if you give the `—volno-file=FILE-OF-NUMBER' option, then FILE-OF-NUMBER should be an unexisting file to be created, or else, a file already containing a decimal number. That number will be used as the volume number of the first volume written. When `tar' is finished, it will rewrite the file with the now-current volume number.
GNU `tar' multi-volume archives do not use a truly portable format. You need GNU `tar' at both end to process them properly.
When prompting for a new tape, `tar' accepts any of the following responses:
`y'
Request `tar' to begin writing the next volume.
(You should only type `y' after you have changed the tape; otherwise `tar' will write over the volume it just finished.)
If you want more elaborate behavior than this, give `tar' the `—info-script=SCRIPT-NAME' (`—new-volume-script=SCRIPT-NAME', `-F SCRIPT-NAME') option. The file SCRIPT-NAME is expected to be a program (or shell script) to be run instead of the normal prompting procedure. When the program finishes, `tar' will immediately begin writing the next volume.
`-F FILE'
`--info-script=FILE'
`--new-volume-script=FILE'
Execute `file' at end of each tape. If `file' exits with nonzero
status, exit. This implies `--multi-volume' (`-M').
if you give `tar' multiple `—file=ARCHIVE-NAME' (`-f ARCHIVE-NAME') options, then the specified files will be used, in sequence, as the successive volumes of the archive. Only when the first one in the sequence needs to be used again will `tar' prompt for a tape change (or run the info script).
When `tar' writes an archive to tape, it creates a single tape file. If multiple archives are written to the same tape, one after the other, they each get written as separate tape files.
When extracting, it is necessary to position the tape at the right place before running `tar'. To do this, use the `mt' command. For more information on the `mt' command and on the organization of tapes into a sequence of tape files, see *Note mt::.
To give the archive a name which will be recorded in it, use the `—label=ARCHIVE-LABEL' (`-V ARCHIVE-LABEL') option. This will write a special block identifying VOLUME-LABEL as the name of the archive to the front of the archive which will be displayed when the archive is listed with `—list' (`-t'). If you are creating a multi-volume archive with `—multi-volume' (`-M'), then the volume label will have `Volume NNN' appended to the name you give, where NNN is the number of the volume of the archive. (If you use the `—label=ARCHIVE-LABEL' (`-V ARCHIVE-LABEL') option when reading an archive, it checks to make sure the label on the tape matches the one you give.
People seem to often do:
--label="SOME-PREFIX `date +SOME-FORMAT`"
or such, for pushing a common date in all volumes or an archive set.
The `—gzip' (`—gunzip', `—ungzip', `-z') option does not work with the `—multi-volume' (`-M') option, or with the `—update' (`-u'), `—append' (`-r'), `—concatenate' (`—catenate', `-A'), or `—delete' operations.
The difference is that the whole archive is compressed. With `zip', archive members are archived individually. `tar''s method yields better compression. On the other hand, one can view the contents of a `zip' archive without having to decompress it. As for the `tar' and `gzip' tandem, you need to decompress the archive to see its contents.
About corrupted compressed archives: `gzip''ed files have no redundancy, for maximum compression. The adaptive nature of the compression scheme means that the compression tables are implicitly spread all over the archive. If you lose a few blocks, the dynamic construction of the compression tables becomes unsynchronized, and there is little chance that you could recover later in the archive.
`—incremental' (`-G') is used in conjunction with `—create' (`-c'), `—extract' (`—get', `-x') or `—list' (`-t') when backing up and restoring file systems. An archive cannot be extracted or listed with the `—incremental' (`-G') option specified unless it was created with the option specified. This option should only be used by a script, not by the user.
`—incremental' (`-G') in conjunction with `—create' (`-c') causes `tar' to write, at the beginning of the archive, an entry for each of the directories that will be archived. The entry for a directory includes a list of all the files in the directory at the time the archive was created and a flag for each file indicating whether or not the file is going to be put in the archive.
Note that this option causes `tar' to create a non-standard archive that may not be readable by non-GNU versions of the `tar' program.
