rsync source/dir1/ dest/dir2/
rsync source/dir1/ dest/dir2/
where the '/dir1/' and '/dir2/' on source and destination matches each other.
When synchronizing two directories (local or remote), 1st determine the match point (the name of 'dir1' and 'dir2' may or may not be the same), then trail both directories on source and destination with a slash. In short, always end the directories with a slash. E.g.:
rsync -nvvua /mnt/tmp1/home/tong/ /home/tong/
When the match point doesn't exist on the destination directory, meaning that we will do just a (remote) copy, do not use trailing slash on both source and destination directory names. To memories, "no exist", "no slash". E.g.:
rsync -nvvua /mnt/tmp1/home/tong /home
If you want to rsync *into* a symbol-link directory, use the trailing '/' format. Otherwise, the symbol-linked directory will be overwritten as a normal directory.
But the problem is that the symbol-link directory has to be exist.
cd /tmp mv var ttt
ln -s test1 var cd ttt rsync -nva var/ /tmp/var/ -- /tmp/var is still a symbol link
.. cd var dirdel cpan .. cd ttt/ rsync -va var /tmp ..
— redo without trailing '/', symbol link turns into normal directory. NB, merely a -vr instead of the whole -va (rsync -vr var /tmp) will yield the same result.
$ rsync -nva var/ /tmp/var/ building file list ... done created directory /tmp/var/ rsync: push_dir /tmp/var/: No such file or directory
rsync -nvvua webapps/expresso/ /java/expresso/webapps/ ^nv^^
rsync -vua --exclude-from=/home/tong/s/scripts/tdat/rsync.excl.rc --exclude='tmp' --exclude='tdRaw' --exclude='**/.cpan/**' --exclude='**/gens' --exclude='**/try' --exclude='**/dl/**' /mnt/tmp/home/tong /export/images/
rsync -vuaC --exclude='*.ps*' --exclude='*.pdf*' --exclude='*.dvi' --exclude='*.aux' --exclude='*.log' --exclude='*.out' --exclude='*.chk' --exclude='*.lo*' --exclude='*/rcs/*' thesis ~+1/
— You may use as many —exclude options on the command line as you like to build up the list of files to exclude.
find . -name '*.pdf*' -o -name '*.dvi' -o -name '*.aux' -o -name '*.log' -o -name '*.out' -o -name '*.chk' -o -name '*.lo*'
rsync -vua --existing /etc /usr /export/histories/sysbaks/sconf rsync -vua --update
rsync -vuaxC --delete --delete-excluded --exclude-from=/home/tong/s/scripts/tdat/rsync.excl.rc /export/home/ /export.img/home/
cdd /export.img/home/tong/
is determined by the "trailing slash on the source":
rsync -nvvua /mnt/tmp1/home/tong /home/
— tx directory
rsync -nvvua /mnt/tmp1/home/tong/ /home/tong/
— tx all files under the directory
Only the trailing slash in the source is important.
Trailing slash in the destination dir does not affect anything.
The files (output of "building file list") means to be under destination dir! If you are not sure whether you are doing the right thing, use rsync -n and concat the destination dir and the output file name to verify if it is there.
this is irrelevant if -R is specified.
Of the following two, the ~ version will give error:
failed to open exclude file ~/s/scripts/tdat/rsync.excl.rc: No such file or directory rsync error: error in file IO (code 11) at exclude.c(230)
--exclude-from=~/s/scripts/tdat/rsync.excl.rc --exclude-from=/home/tong/s/scripts/tdat/rsync.excl.rc
-a, --archive archive mode, recursive & preservative. -R, --relative preser full source path names
-n, --dry-run show what would have been transferred -x, --one-file-system don't cross filesystem boundaries
rsync [OPTION]... LocalSrc [LocalSrc]... [USER@]RemoteHost:RemoteDest
rsync [OPTION]... [USER@]RemoteHost:RemoteSrc LocalDest
rsync [OPTION]... LocalSrc [LocalSrc]... LocalDest
rsync [OPTION]... [USER@]RemoteRsSvr::RemoteSrc [LocalDest]
rsync [OPTION]... LocalSrc [LocalSrc]... [USER@]RemoteRsSvr::RemoteDest
Note that in all cases (other than listing) at least one of the source and destination paths must be local.
rsync is a program that behaves in much the same way that rcp does, The rsync remote-update protocol allows rsync to transfer just the differences between two sets of files across the network link, using an efficient checksum-search algorithm.
Note that rsync must be installed on both the source and destination machines.
You use rsync in the same way you use rcp. You must specify a source and a destination, one of which may be remote.
You can also specify an alternative to rsh, by either using the -e command line option, or by setting the RSYNC_RSH environment variable.
Perhaps the best way to explain the syntax is some examples:
rsync -e ssh *.c foo:src/
this would transfer all files matching the pattern *.c from the current directory to the directory src on the machine foo. If any of the files already exist on the remote system then the rsync remote-update protocol is used to update the file by sending only the differences. See the tech report for details.
rsync -avxz foo:src/bar /data/tmp
this would recursively transfer all files from the directory src/bar on the machine foo into the /data/tmp/bar directory on the local machine. The files are transferred in "archive" mode, which ensures that symbolic links, devices, attributes, permissions, ownerships etc are preserved in the transfer. Additionally, compression will be used to reduce the size of data portions of the transfer.
rsync -avxz foo:src/bar/ /data/tmp
a trailing slash on the source changes this behavior to transfer all files from the directory src/bar on the machine foo into the /data/tmp/. A trailing / on a source name means "copy the contents of this directory". Without a trailing slash it means "copy the directory". This difference becomes particularly important when using the —delete option.
You can also use rsync in local-only mode, where both the source and destination don't have a ':' in the name. In this case it behaves like an improved copy command.
It is also possible to use rsync without using rsh or ssh as the transport. In this case you will connect to a remote rsync server running on TCP port 873.
-v, --verbose
This option increases the amount of information you are given during the transfer. By default, rsync works silently. A single -v will give you information about what files are being transferred and a brief summary at the end. Two -v flags will give you information on what files are being skipped and slightly more information at the end.
-q, --quiet
This option decreases the amount of information you are given
during the transfer, notably suppressing information messages
from the remote server. This flag is useful when invoking
rsync from cron.
-a, --archive
This is equivalent to -rlptgoD. It is a quick way of saying
you want recursion and want to preserve everything.
-R, --relative
Use relative paths. This means that the full path names specified on the command line are sent to the server rather than just the last parts of the filenames.
rsync -R foo/bar/foo.c remote:/tmp/
then a file called /tmp/foo/bar/foo.c would be created on the remote machine. The full path name is preserved.
-W, --whole-file
With this option the incremental rsync algorithm is not used and
the whole file is sent as-is instead. This is the default when
both the source and target are on the local machine.
-n, --dry-run
This tells rsync to not do any file transfers, instead it will
just report the actions it would have taken.
-x, --one-file-system
This tells rsync not to cross filesystem boundaries when
recursing. This is useful for transferring the contents of
only one filesystem.
-u, --update
This forces rsync to skip any files for which the destination
file already exists and has a date later than the source file.
--existing
This tells rsync not to create any new files - only update
files that already exist on the destination.
--delete
This tells rsync to delete any files on the receiving side
that aren't on the sending side.
-C, --cvs-exclude
This is a useful shorthand for excluding a broad range of
files that you often don't want to transfer between systems.
It uses the same algorithm that CVS uses to determine if a
file should be ignored.
The exclude list is initialized to:
RCS SCCS CVS CVS.adm RCSLOG cvslog.* tags TAGS .make.state .nse_depinfo *~ #* .#* ,* *.old *.bak *.BAK *.orig *.rej .del-* *.a *.o *.obj *.so *.Z *.elc *.ln core
then files listed in a $HOME/.cvsignore are added to the list and any files listed in the CVSIGNORE environment variable (space delimited).
Finally in each directory any files listed in the .cvsignore file in that directory are added to the list.
--exclude-from=FILE
This option is similar to the --exclude option, but instead it
adds all filenames listed in the file FILE to the exclude
list. Blank lines in FILE and lines starting with ';' or '#'
are ignored.
--compare-dest=DIR
This option instructs rsync to use DIR as an additional direc-
tory to compare destination files against when doing trans-
fers. This is useful for doing transfers to a new destination
while leaving existing files intact, and then doing a flash-
cutover when all files have been successfully transferred (for
example by moving directories around and removing the old
directory, although this requires also doing the transfer with
-I to avoid skipping files that haven't changed). This option
increases the usefulness of --partial because partially trans-
ferred files will remain in the new temporary destination
until they have a chance to be completed. If DIR is a rela-
tive path, it is relative to the destination directory.
-z, --compress
With this option, rsync compresses any data from the source
file(s) which it sends to the destination machine. This
option is useful on slow links. The compression method used
is the same method that gzip uses.
Note this this option typically achieves better compression ratios that can be achieved by using a compressing remote shell, or a compressing transport, as it takes advantage of the implicit information sent for matching data blocks.
/etc/rsyncd.conf
*N*:, the default —cvs-exclude option will exclude files in the version control directories, but not ,v files that reside in the same directories as the source files.
$ cat ~/s/scripts/tdat/rsync.excl.rc + RCS/** # tkdesk **/.trash/** # netscape/brx **/cache/** **/*cache/**
rsync builds a ordered list of include/exclude options as specified on the command line. When a filename is encountered, rsync checks the name against each exclude/include pattern in turn. The first matching pattern is acted on. If it is an exclude pattern than that file is skipped. If it is an include pattern then that filename is not skipped. If no matching include/exclude pattern is found then the filename is not skipped.
Note that the —include and —exclude options take one pattern each. To add multiple patterns use the —include-from and —exclude-from options or multiple —include and —exclude options.
The patterns can take several forms. The rules are:
if the pattern starts with a / then it is matched against the start of the filename, otherwise it is matched against the end of the filename. Thus /foo would match a file called foo at the base of the tree whereas foo would match any file called foo anywhere in the tree.
if the pattern ends with a / then it will only match a direc- tory, not a file, link or device.
if the pattern contains a wildcard character from the set *?[ then expression matching is applied using the shell filename matching rules. Otherwise a simple string match is used.
if the pattern contains a / (not counting a trailing /) then it is matched against the full filename, including any leading directory. If the pattern doesn't contain a / then it is matched only against the final component of the filename. Furthermore, if the pattern includes a double asterisk "**" then all wildcards in the pattern will match slashes, other- wise they will stop at slashes.
if the pattern is a single exclamation mark ! then the current exclude list is reset, removing all previous exclude patterns.
if the pattern starts with "` " (a plus followed by a space) then it is always considered an include pattern, even if specified as part of an exclude option. The "` " part is discarded before matching.
if the pattern starts with "- " (a minus followed by a space) then it is always considered an exclude pattern, even if specified as part of an include option. The "- " part is discarded before matching.
The +/- rules are most useful in exclude lists, allowing you to have a single exclude list that contains both include and exclude options.
The exclude patterns actually shortcircuit the directory traversal stage when rsync finds the files to send. If a pattern excludes a particular parent directory, it can render a deeper include pattern ineffectual because rsync did not descend through that excluded section of the hierarchy. This is particularly important when using a trailing '*' rule. For instance, this won't work:
+ /some/path/this-file-will-not-be-found + /file-is-included - *
This fails because the parent directory "some" is excluded by the '*' rule, so rsync never visits any of the files in the "some" or "some/path" directories. One solution is to ask for all directories in the hierarchy to be included by using a single rule: "+ */" (put it somewhere before the "- *" rule), and perhaps use the —prune-empty-dirs option. Another solution is to add specific include rules for all the parent dirs that need to be visited. For instance, this set of rules works fine:
+ /some/ + /some/path/this-file-is-found + /file-also-included - *
Newsgroups: gmane.linux.debian.user Date: Thu, 25 Jan 2007 04:05:25 +0000 (UTC)
> Is there any way to tell rsync to backup all the . files under my home but > no any . directories? E.g., .profile in, but .gconf/ & .gnome/ etc out?
Don't use the recursive option?
Roberto C. Sanchez
> Is there any way to tell rsync to backup all the . files under my home but > no any . directories? E.g., .profile in, but .gconf/ & .gnome/ etc out? > > FYI, I tried with the following exclude rules but it didn't work, . directories > are still included. > > rsync -vua --exclude-from=exclude.list > > File exclude.list: > + /home/tong/.* > /home/tong/.*/
I created a little test tree under /tmp/home/:
$ find /tmp/home/ /tmp/home/ /tmp/home/paul /tmp/home/paul/.foo /tmp/home/paul/.dir1 /tmp/home/paul/.dir1/bar /tmp/home/paul/dir2 /tmp/home/paul/dir2/.file /tmp/home/paul/dir2/.dir3 /tmp/home/paul/dir2/.dir3/last /tmp/home/mary /tmp/home/mary/.foo /tmp/home/mary/.dir1 /tmp/home/mary/.dir1/bar /tmp/home/mary/dir2 /tmp/home/mary/dir2/.file /tmp/home/mary/dir2/.dir3 /tmp/home/mary/dir2/.dir3/last
With .dir1 in each user directory representing (I think) what you want to exclude. The following exclude pattern seems to work:
$ rsync -azvn --exclude=home/*/.*/ /tmp/home /tmp/rsync2 building file list ... done created directory /tmp/rsync2 home/ home/mary/ home/mary/.foo home/mary/dir2/ home/mary/dir2/.file home/mary/dir2/.dir3/ home/mary/dir2/.dir3/last home/paul/ home/paul/.foo home/paul/dir2/ home/paul/dir2/.file home/paul/dir2/.dir3/ home/paul/dir2/.dir3/last
sent 370 bytes received 98 bytes 936.00 bytes/sec total size is 0 speedup is 0.00
For some reason, .dir1/ is not excluded if the option is given as —exclude=/tmp/home/*/.*/ or —exclude=tmp/home/*/.*/.
Ken Irving
> > > For some reason, .dir1/ is not excluded if the option is given as > > > --exclude=/tmp/home/*/.*/ or --exclude=tmp/home/*/.*/. > > > > Expansion of unquoted globs in keyword arguments > > seldom gives the intended result. e.g. > > > > rsync --exclude=/home/*/.*/ /home/ /tmp/ > > > > might expand to: > > > > rsync --exclude=/home/a/.x/ /home/a/.y/ /home/b/.x/ /home/b/.y/ /home/ /tmp/ > > > > which copies a lot more than presumably intended. > > > > Mike Bird > > Duh... Thanks, I needed that. The one that worked wouldn't match > anything, so the globbing characters would pass through as is, but > otherwise would expand as you say. I've already deleted the test > files, but I'm confident (well, sort of...) that those forms would work > if quoted.
Well, tried that but they still don't work. This is probably because rsync is not dealing with the full paths (from /), so the longer patterns just don't match.
Ken Irving
>>> Is there any way to tell rsync to backup all the . files under my home but >>> no any . directories? E.g., .profile in, but .gconf/ & .gnome/ etc out? >>> >> Don't use the recursive option? > > unfortunately that's not an option. I have to use it. The OP is for > illustration purpose only.
That's contradicting what you said before. You want to rsync .files, so just drop the recursive option.
If you want to transfer all non-dot directories, maybe the easiest solution is to do rsync twice: once recursively without .files and once non-recursively for .files.
Johannes Wiedersich
> I should have said I was trying to backup the /home directory, > although the exclude list hinted so. I was trying to cut the craps, > not bring along a life long story, but apparently I trimmed too > much. :-) > > Ok, fair enough. two pass is a solution. But I have to add every > single user in my /home directory, and if there is change in account, > i.e. more or less people, I'll be in trouble.
You could script that part. A trivial example:
cd /home for FILE in $(ls); do file $FILE; done
> Anyway, if that's the only solution. fine.
I understand your frustration. A solution which allows you to distinguish between dot directories and other directories would be neater.
Liam O'Toole
There are seldom any dot dirs at other levels. If it's safe to ignore dot dirs at all levels you can use:
rsync -a --exclude='.*/' /home/ target
Mike Bird
WebHostingBuzz provides SSH access, except it uses a non-standard port (not port 22). So when I do rsync to transfer files, I had to use the following form:
rsync [OPTION]... rsync://[USER@]HOST[:PORT]/SRC [DEST]
Problem is, rsync kept returning the following error message:
rsync: server sent "SSH-2.0-OpenSSH_3.9p1" rather than greeting rsync error: error starting client-server protocol (code 5) at main.c(1046)
. . I figured out that you have to specify the remote shell with the "-e" switch. So the final command that worked is
rsync [OPTION] -e "ssh -p PORT" ... [USER@]HOST:SRC [DEST]
documented on: Wed, 22 Aug 2007, Chieh Cheng
My rsync failed to detect many updated files. For example:
-rw-rw-r-- 1 tong tong 283238 Jan 17 10:55 /java/expresso/webapps/ROOT/WEB-INF/src/com/jcorporate/expresso/core/dbobj/DBObject.java -rw-rw-r-- 1 tong tong 283754 Jan 20 23:30 com/jcorporate/expresso/core/dbobj/DBObject.java
We can see that DBObject.java in cvs is defenately news than that in java/expresso/. But rsync thinks not.
Now I realize that it is the 'Change' time's problem:
File: '/java/expresso/webapps/ROOT/WEB-INF/src/com/jcorporate/expresso/core/dbobj/DBObject.java' Size: 283238 Blocks: 568 IO Block: 4096 Regular File Device: 1645h/5701d Inode: 85379 Links: 1 Access: (0664/-rw-rw-r--) Uid: ( 9999/ tong) Gid: ( 1001/ tong) Access: Tue Jan 21 15:22:45 2003 Modify: Fri Jan 17 10:55:18 2003 Change: Tue Jan 21 15:22:24 2003
$ stat /java/expresso/webapps/ROOT/WEB-INF/src/com/jcorporate/expresso/core/dbobj/DBObject.java com/jcorporate/expresso/core/dbobj/DBObject.java File: "/java/expresso/webapps/ROOT/WEB-INF/src/com/jcorporate/expresso/core/dbobj/DBObject.java" Size: 283238 Blocks: 568 IO Block: 4096 Regular File Device: 1645h/5701d Inode: 344960 Links: 1 Access: (0664/-rw-rw-r--) Uid: ( 9999/ tong) Gid: ( 1001/ tong) Access: Tue Jan 21 19:08:43 2003 Modify: Thu Nov 11 11:11:00 1971 Change: Tue Jan 21 19:09:05 2003
File: "com/jcorporate/expresso/core/dbobj/DBObject.java" Size: 283754 Blocks: 568 IO Block: 4096 Regular File Device: 1645h/5701d Inode: 950973 Links: 1 Access: (0664/-rw-rw-r--) Uid: ( 9999/ tong) Gid: ( 1001/ tong) Access: Tue Jan 21 19:11:43 2003 Modify: Tue Jan 21 19:11:43 2003 Change: Tue Jan 21 19:11:43 2003
— everything newser
$ rsync -nvuaC /export/cvswork/expresso/expresso-web/WEB-INF/src/ /java/expresso/webapps/ROOT/WEB-INF/src/ | grep DBObject com/jcorporate/expresso/ext/xml/dbobj/XMLDBObject.java com/jcorporate/expresso/services/dbobj/RegistrationDBObject.java com/jcorporate/expresso/services/dbobj/SecurityDBObject.java
— no BObject.java
/export/cvswork/expresso/expresso-web/WEB-INF/src$ rsync -nvuaC com/jcorporate/expresso/core/dbobj/DBObject.java /java/expresso/webapps/ROOT/WEB-INF/src/com/jcorporate/expresso/core/dbobj/DBObject.java building file list ... done DBObject.java wrote 82 bytes read 24 bytes 212.00 bytes/sec total size is 283754 speedup is 2676.92
— will do. but why not previously?
oh, damn, the 'core' that is in the default rsync —exclude rule. Specify them explicitly can fix the problem:
$ rsync -nvuaC /export/cvswork/expresso/expresso-web/WEB-INF/src/com/jcorporate/expresso/core/ /java/expresso/webapps/ROOT/WEB-INF/src/com/jcorporate/expresso/core/ | grep DBObject dbobj/AuditedSecuredDBObject.java dbobj/AutoDBObject.java dbobj/DBObject.java dbobj/DBObjectDef.java dbobj/DBObjectImpl.java dbobj/HistAuditSecuredDBObject.java dbobj/JDBCDBObject.java dbobj/MultiDBObject.java dbobj/RowSecuredDBObject.java dbobj/SecuredDBObject.java dbobj/tests/DBObjectTest.java dbobj/tests/MultiDBObjectTest.java