often used commands 

expr 

See command expr.

cmd:echo 

/bin/echo —help Usage: /bin/echo [OPTION]… [STRING]… Echo the STRING(s) to standard output.

-n             do not output the trailing newline
-e             enable interpretation of backslash escapes
-E             disable interpretation of backslash escapes (default)
    --help     display this help and exit

cmd:trap 

Normal usage 

tmpf=tmp.$$
trap "rm -f $tmpf" 0
trap "exit 2" SIGHUP SIGINT SIGQUIT SIGTERM  # 1 2 3 15
cat > $tmpf
cat $tmpf ..

Manipulating 

Trap set & show:

$ trap -p
$ trap 'rm semaphore' 0
$ trap 'rm semaphore' SIGINT
$ trap -p
trap -- 'rm semaphore' EXIT
trap -- 'rm semaphore' SIGINT
$ trap -l
 1) SIGHUP       2) SIGINT       3) SIGQUIT      4) SIGILL
[..]
$ set -x

Press ^C:

$ ++ rm semaphore
rm: cannot lstat `semaphore': No such file or directory

Trap reset:

$ trap - SIGINT
+ trap - SIGINT

Press ^C again, no more traps:

$

cmd:sleep 

Usage 

In unix:

sleep 10

sleep 10 seconds

To execute a command after a certain amount of time:

(sleep 105; command)&
Note sendf can't use this way (when user interaction is needed). Put it in foreground instead.

Linux special 

sleep 2m

sleep 2 minutes

SUFFIX m for minutes, h for hours or d for days.

cmd:rm 

-r Recursively remove directories and subdirectories in the argument list. -f Remove all files (whether write-protected or not) in a directory without prompting the user.

cmd:pidof 

$ pidof emacs
1795 1506 1495
$ [ `/sbin/pidof nono` ] && echo aaa
$ [ `/sbin/pidof X` ] && echo aaa
aaa

cmd:dircmp 

*Tags*: dir compare. compare directory.

cmd:script 

script [-a] [-c COMMAND] [file]

For single command, you can use

script -c COMMAND
-c COMMAND
        Run the COMMAND rather than an interactive shell.  This makes it
        easy for a script to capture the output of a program that behaves
        differently when its stdout is not a tty.

cmd:most 

Info 

most - browse or page through a text file

Usage 

most -ct4 +82 keymap.c

makes searches case sensitive, sets tabwidth to 4, and displays the file keymap.c starting at line 82.

Help 

MOST version S4.8.1 Usage:
most [-1bckstvw] [+/string] [+line number] [+s] [+d] file...
where: -1:  assume VT100 terminal. (VMS only)
       -b:  Startup in binary mode.
       -c:  Make searches case sensitive.
       -k:  Kanji mode.
       -s:  Squeeze out excess blank lines.
       -t:  Display tabs as ^I.  If this option is immediately followed
              by an integer, the integer sets the tab width.
       -v:  Do not interpret backspace formatting characters.
       -w:  Wrap lines.
       -z:  No gunzip-on-the-fly.
       +/string:
            Search for string
       +line number
            Start up at specified line number.
       +d:  Allow file deletion.
       +s:  Secure Mode-- no edit, cd, shell, and reading files not
              already listed on the command line.

cmd:cp — copy a directory structure 

Usage 

recommended:

cp -pR        (supported for almost all version)
cp -av        (gnu shortcut)

Others

cp -pr [-v]   (lazy version)

Version control 

The  backup  suffix  is  `~',  unless  set  with   --suffix   or   SIM-
PLE_BACKUP_SUFFIX.   The version control method may be selected via the
--backup option or through the  VERSION_CONTROL  environment  variable.
Here are the values:
numbered, t
       make numbered backups
test 
touch aaa ab
cp --backup=numbered ab aaa
cp --backup=numbered ab aaa; cp --backup=numbered ab aaa
$ dir
-rw-r--r--    1 root     root            0 Oct 30 20:42 aaa.~1~
-rw-r--r--    1 root     root            0 Oct 30 20:42 ab
-rw-r--r--    1 root     root            0 Oct 30 20:46 aaa.~2~
-rw-r--r--    1 root     root            0 Oct 30 20:46 aaa.~3~
-rw-r--r--    1 root     root            0 Oct 30 20:46 aaa

Help 

-a, --archive
     same as -dpR
-b, --backup
     make backup before removal
-d, --no-dereference
     preserve links
-f, --force
     remove existing destinations, never prompt
-i, --interactive
     prompt before overwrite
-p, --preserve
     preserve file attributes if possible
-r   copy recursively, non-directories as files
-R, --recursive
     copy directories recursively
-u, --update
     copy only when the SOURCE file is newer than the desti-
     nation file or when the destination file is missing
-v, --verbose
     explain what is being done

documented on: 06-03-99 09:58:32

cmd:basename 

not suitable for returning empty when no fname exist:

url_path=suntong001/
url_fname=`basename $url_path`
  -- url_fname=suntong001
url_fname=`basename /$url_path`
  -- url_fname=suntong001

to remove any extension 

fn=a.b.c.ext
$ expr "$fn" : '\(.*\)\....$'
a.b.c

documented on: 2005.08.05

cmd:dirname 

Usage 

$ dirname /some/path/location/file1.ext
/some/path/location

cmd:less 

Personally, I prefer the keyboard-based scrolling that less offers over scrollbars (I like to keep my hands on the keyboard and away from the mouse as much as possible). It is possible to quickly and precisely move about by lines (j/k), half-screens (d/u), full screens (SPACE/BACKSPACE or f/b), or to the bottom/top of file (G/g). (Line number display is toggled with the -N command within less, and can be enabled at the command-line with the -N option or having an "N" in the LESS environment variable.)

But if you insist on scroll bars, how about "vim -g -R +'set number'"? (vim can be found under http://www.vim.org/ .)

Ken Pizzini

documented on: Wed 07-07-99 18:17:46

what's a good text/binary file viewer? 

> I've been using various text file viewers on my RedHat 7.2 setup, and they
> all work, but also all seem to share the same behavior: when I search for a
> string the viewer always places the line containing it at the very top of
> the screen.  This makes it hard to see the context in which the string was
> used.  The viewers I tried were the internal one in Midnight Commander,
> less, and lv.

You can hit the "u" key to go up a half-page in less to see the previous text, or..

> Is there a configurable combination text/binary viewer that would let me
> adjust this line positioning, as well as the line overlap when page scrol-
> ling?

Based on tony's posting, I think 'less -j-6' might suit you most.

Tong

cmd:top 

SORTING of task window 

If you forget which field is being sorted the <x> command can serve as a quick visual reminder.

For compatibility, this top supports most of the former top sort keys. Since this is primarily a service to former top users, these commands do not appear on any help screen.

command   sorted field                  supported
  A         start time (non-display)      No
  M         %MEM                          Yes
  N         PID                           Yes
  P         %CPU                          Yes
  T         TIME+                         Yes

Before using any of the following sort provisions, top suggests that you temporarily turn on column highlighting using the 'x' interactive command. That will help ensure that the actual sort environment matches your intent.

The following interactive commands will only be honored when the cur- rent sort field is visible. The sort field might not be visible because:

  1. there is insufficient Screen Width

  2. the 'f' interactive command turned it Off

<<> :Move_Sort_Field_Left Moves the sort column to the left unless the current sort field is the first field being displayed.

<>> :Move_Sort_Field_Right Moves the sort column to the right unless the current sort field is the last field being displayed.

The following interactive commands will always be honored whether or not the current sort field is visible.

<F> or <O> :Select_Sort_Field These keys display a separate screen where you can change which field is used as the sort column.

If a field is selected which was not previously being displayed,
it  will  be forced On when you return to the top display.  How-
ever, depending upon your screen width and  the  order  of  your
fields, this sort field may not be displayable.
This  interactive command can be a convienent way to simply ver-
ify the current sort field, when running top with  column  high-
lighting turned Off.

<R> :Reverse/Normal_Sort_Field_toggle Using this interactive command you can alternate between high- to-low and low-to-high sorts.

SORTING of task window 

>the output of top shows %WCPU and %CPU.
>what exactly is the difference between the two?
>i read man top. doesn't tell me.

%WCPU is a weighted average that includes past CPU time, but gives higher weight to recent activity. I think %CPU is just the percentage of CPU time used by the process since the previous display update.

>Also, how do i find out what is the
>memory limit that my program can use?
>is this configurable?
limit datasize

or

ulimit -d

The system administrator can configure the default, as well as the hard limit.

documented on: 08-11-99 18:01:58

ls sort order 

Newsgroups:  gmane.linux.debian.user
Date:        Thu, 14 Sep 2006 13:01:25 -0400

I just noticed on a brand new install of testing that ls has begun to sort alphabetically, but mixes uppercase and lowercase together i.e. 'Pearl' comes before 'pearl' but after 'otter'.

otter
Pearl
pearl

A stable Debian version I maintain behaves the good old-fashioned way:

Pearl
otter
pearl

Is this some utf-8 mutation?

Eric d'Alibut

ls sort order 

> > Is this some utf-8 mutation?
>
> Yup.  If you want the old behavior, use this:
>
> export LC_COLLATE=C

Bingo. Thank you sir!

Eric d'Alibut

ls sort order again 

Newsgroups:  gmane.linux.debian.user
Date:        Tue, 17 Oct 2006 04:23:44 +0200
> I think there're unicode.

This is not directly related to Unicode (this also happens with ISO8859-1), but to a language.

> export LC_COLLATE=C

and this will also fix the hyphen-minus problem:

vin:~> printf '%s\n' 1-2 1-3 12 13 | LC_COLLATE=en_US.UTF-8 sort
12
1-2
13
1-3
vin:~> printf '%s\n' 1-2 1-3 12 13 | LC_COLLATE=en_US.ISO8859-1 sort
12
1-2
13
1-3
vin:~> printf '%s\n' 1-2 1-3 12 13 | LC_COLLATE=C sort
1-2
1-3
12
13

Alternatively, one can use LC_COLLATE=POSIX.

Vincent Lefevre

ls sort order again 

Newsgroups:  gmane.linux.debian.user
Date:        Tue, 17 Oct 2006 17:31:00 -0400
> >> >I want that behavior. How can I do that, instead of the traditional order?
> >>
> >> I'm using testing, and it's doing this for me by default.
> >
> > I'm also using testing, and mine also does this by default....
>
> Mine (Debian testing) isn't:
>
> $ ls -1
> Pearl
> otter
> pearl
>
> Hence I want to know how to do it.

The environment variables LANG and LC_COLLATE control sort order and regex pattern matching expansion.

See files: /etc/environment and possibly /etc/profile (if edited…) LC_COLLATE is set via LANG (or LANGUAGE) unless overridden via the shell. Use the command "locale" to see the current values.

I have always used LANG="C" or "POSIX" which pays attention to the case:

Readme
SRC
bin
palm
sco

If you want the other rules use LANG=en_US (or just for the sort order, LC_COLLATE="en_US") to get:

bin
palm
Readme
sco
SRC

Larry Irwin

3 lines to 1 line ? Help Please 

> I would like to concatenate every 3 lines from a file
> to 1 line in a new file.

3 lines to 1 line ? Help Please 

while read line1 do read line2 read line3 done exit 0

Also, if you know you are going to use a posix shell, try using read -r instead of read by itself. Otherwise, the shell will silently eat backslashes.

chris

3 lines to 1 line ? Help Please 

> > I would like to concatenate every 3 lines from a file
> > to 1 line in a new file.
>
>       sed 'N;N;s/\n//g' oldfile > newfile

If the number of lines in "oldfile" is two more than a multiple of three, then you will discover that the way that sed handles "end-of-file hit while trying to execute the N command" can be annoying. To work around it use: sed 'N;$! N;s/\n//g' oldfile > newfile

Ken Pizzini

3 lines to 1 line ? Help Please 

>Somebody please explain why we can't use xargs:
>$ jot 10 | xargs -n 3
>1 2 3
>4 5 6
>7 8 9
>10

Because it won't work if the input contains any space, tab, backslash, single-quote or double-quote characters.

Geoff Clare

3 lines to 1 line ? Help Please 

Interesting idea, but it does not work, e.g.:

$ echo 1 2 3 4 5 6 '7 8' | xargs -n 3 1 2 3 4 5 6 7 8

"xargs" handles (white space separated) tokens, not lines. "xargs -l3" is a bit better, but still no cigar.

$ /bin/echo "1 2\n 3 4 5 6 7 8" | xargs -l3 1 2 3 4 5 6 7 8

Heiner Marxen

documented on: 1999.09.29 Wed 19:04:20

cmd:paste 

*Tags*: cmd:tmpf, cmd:jot

$ jot -w sortt%d 5 12 - -1 | paste -d-  - `tmpf jot -c 5 A`
sortt12-A
sortt11-B
sortt10-C
sortt9-D
sortt8-E

paste -sd 

Date: 12/18/04
> $ seq 10 | paste -sd "   \n"
> 1 2 3 4
> 5 6 7 8
> 9 10
>
> >From the man page:
>
> -d, --delimiters=LIST
> reuse characters from LIST instead of TABs
>
> But I don't understand why more space in delimiter list would produce
> different results. Please explain.

Maybe these examples will make it clear:

$ seq 10 | paste -sd "ABC"
1A2B3C4A5B6C7A8B9C10
$ seq 10 | paste -sd "ABC\n"
1A2B3C4
5A6B7C8
9A10

`paste' always uses ONE character as delimiter. When you specify more than one character `paste' will cycle through them. In your last example, it uses a space as the delimiter for the first three columns and a newline for the fourth.

Maurits van Rees

file date format 

> When i use the command ls -l to list the files in the
> directory, I get the following type of output for the files
> that are newer than 6 months
>
> -rw-r----- 1 abcdef 18 Sep 26 19:00 testfile
>
> but for the files that are older than 6 months, i get
> following type of output
>
> -rw-r----- 1 abcdef 621 Oct 6 1997 testfile1
>

Get GNU ls, it has an option to let you specify the date format. Standard ls always shows the time if the date is within 6 months, the year if it's outside that range.

Barry Margolin,

file date format 

Get the pre-compiled gnu file utilites package (yes, it's in Sun Package format, even) at sunfreeware.com. You may want to follow the link from there to the "Original Solaris 2.5 site" to avoid the Alpha version that's up currently. Oh, and get Gnu Zip from the same site to unzip the package. The use the "—full-time" option to it to display a verbose date.

documented on: 1999.09.30 Thu 11:40:01

Can u help this simple Unix Pattern Match 

> I have 2 ascii files.
> One file contains a record of 300 characters, and another 10 characters.
> I have to compare the first one with the second file and exclude the
> ones that match.

Take a look at egrep:

philip@joker:~/tmp% cat one
1
3
5
8
9
10
philip@joker:~/tmp% cat two
3
8
philip@joker:~/tmp% egrep -v -f two one
1
5
9
10

Can u help this simple Unix Pattern Match 

you should be able to use:

fgrep -v -f SecondFile FirstFile

to do what you want. If the files are guaranteed to be in the same order, you can also use:

comm -23 FirstFile SecondFile

Barry Margolin

documented on: 1999.10.10 Sun 16:36:14

cut 

>Can the UNIX cut command work for a file with multi-bytes delimeter(i.e.
>||)?
>
>Below is what I got:
>%cut -f2-32 -d"||" <filename>
>%cut: invalid delimiter

I think the command told you the answer. If that delimiter were allowed, it wouldn't have said that it's invalid.

awk doesn't complain if you try to use -F'||', but it only uses the first character. nawk allows this to be a regular expression, so you can use:

nawk -F'\|\|' ...

Perl's split() function also allows arbitrary regular expressions as the delimiter.

Barry Margolin

documented on: 1999.10.15 Fri 16:05:47

merge two text files in unix 

> How can i merge two text files in unix
> for example
>
> file a                        file b
> a                                e
> b                                f
> c                                g
> d                                h
>
> output
> a
> e
> b
> f
> c
> g
> d
> h
paste -d '\n' a b

works under FreeBSD. maybe the -d option exists under other OSes ?

Cyrille.

reverses the text 

flip 

flip is a Perl script that reverses the text in a given file line-by-line. That is, the first line in the file switches position with the last, the second line in the file switches with the next-to-last, and so on

tail 

flip IS LIKE tail -r, BUT IT HANDLES MUCH BIGGER FILES

gnu tail don't have -r /bin/tail does

gnu, use tac instead.

documented on: 1999.10.26 Tue 11:14:15

cmd:screen 

Usage 

Launch 

Launch screen to enter screen session.

screen -h 800 Starts a new screen session and sets the number of lines in the scrollback buffer to 800. The default is 100 lines.

New screens 

C-a c     (screen)    Create new windows.

Switch screens 

C-a SPACE (next)      Advance to next window (with wraparound).
C-a C-a   (other)     Toggle between the current and previously
              displayed windows.
C-a 0     (select n)  Switch to window n=0 ... 9.
 ...
C-a 9

screens listing 

C-a w     (windows)   Show a list of window names in the status line.

quiting 

logout from shell will close the very screen. screen session is terminated if all screen session have been log out.

more help 

C-a ?     (help)      Show all keybindings.
C-a l     (redisplay) Redraw this window.
C-a H     (log)       Log stdout of window n to screenlog.n.
C-a C-[   (copy)      Start copy mode.  Move cursor with h,j,k,l.
              Set 2 marks with SPACE or y.  Abort with ESC.
              (C-[ is ESC.)  Preceeding second mark with
              an a appends the text to the copy buffer.
C-a C-]   (paste)     Output copy buffer to current window's stdin.
C-a <     (readbuf)   Read the copy buffer from /tmp/screen-exchange.
C-a >     (writebuf)  Write the copy buffer to /tmp/screen-exchange.
screen -list
screen -ls
screen -wipe

Show all available sessions and their status. Use -wipe to remove DEAD sessions.

documented on: 2000.06.09 Fri 16:43:18

cut and paste with the screen program 

Newsgroups: gmane.linux.debian.user
Date: Fri, 20 Jun 2008
> > How can I cut and paste between screen terminals? (using the keyboard)
>
> ctrl-a ]

more specifically, use "ctrl-a [" to switch to copy mode. Then use the cursor keys to move to the text you want to select. Use the spacebar to mark the beginning of the selection, cursor keys to move to the end of the selection, and spacebar to mark the end of the selection. The selected text will be copied into the buffer. To paste this buffer, at the current insertion point in whatever screen you choose, use "ctrl-a ]".

Andrew Sackville-West @farwestbilliards.com

freeze rows in terminal screen 

Newsgroups: comp.unix.shell
> Does anyone know how to freeze the top part of a
> vt100 type login session?

If your terminal supports scrolling regions and your terminfo database is functional, this should do:

{
  read -r line; tput clear; echo "$line"
  lines=$(tput lines)
  tput sc; tput csr 1 $[lines-1]; tput rc
  cat
  tput sc; tput csr 0 $[lines-1]; tput rc
}
See: man terminfo

Martin

documented on: 2000.06.14 Wed 13:59:19

cmd:touch 

Usage 

touch -t 198612310000 oldfile

no parameter for recursive touching.

ddate=200104150000
find -print0 | xargs -0 touch -c -t $ddate

Help 

touch [OPTION] FILE...
-t STAMP               use [[CC]YY]MMDDhhmm[.ss] instead of current time
-r, --reference=FILE   use this file's times instead of current time
-c, --no-create        do not create any files

change file time in selected directories 

Usage 

cd /usr
touch 0101000000 * .*; find . -name '*' -type d -print | xargs -i echo touch 0101000000 '{}/* {}/.*' | sh -x

Help 

touch MMDDhhmm[YY] FILES...

no recursive option for touch!

Trying History 

Note should not just change on /, 'cause all files including mounted volumes will be affected. What's worse, it will dive into directories such as /dev, /proc which is not supposed to be touched. 
find . -name '*' -type d -print | xargs -ti touch 0101000000 '{}/*'

shows ok, but don't work. well, I just found out that actually it create the file '*' !

delete them 

set -f dir * * find / -name '*' !! | xargs rm

Link's time can't be changed! 

Symptom 

# touch 0101000000 libvga.so.1
# dir !$
dir libvga.so.1
lrwxrwxrwx   1 root     root           16 Sep 25 14:01 libvga.so.1 -> libvga.so.1.2.13*
# touch -am 0101000000 !$
touch -am 0101000000 libvga.so.1
# !di
dir libvga.so.1
lrwxrwxrwx   1 root     root           16 Sep 25 14:01 libvga.so.1 -> libvga.so.1.2.13*

Command:touch and irregular file names 

Date: 15 Aug 2007
> How to let command touch to work with irregular file names?
>
> FYI, I tried this but didn't work:
>
> $ touch -r -- "-test.file1" "-test.file2"
> touch: invalid date format `est.file1'

That's nearly right; put the — immediately following the last option, e.g.

$ touch -r '-test.file1' — '-test.file2'

Philip Rowlands @doc.ic.ac.uk

cmd:tr 

removing CR-LF with tr 

tr -d '\015' < $OLDFILE > $NEWFILE
tr -d '\r' < $OLDFILE > $NEWFILE

Usage 

$ xset -q | grep bell
  bell percent:  50    bell pitch:  400    bell duration:  100
$ xset -q | grep bell | tr '[^0-9]' ' '
  bell percent:        bell pitch:         bell duration:
$ xset -q | grep bell | tr ':A-z' ' '
                 50                 400                    100
$ xset -q | grep bell | tr -s ':A-z' ' '
 50 400 100

Limitation 

: how to use tr to
: replace the following text:
:
: "thank%
: you"
:
: with "thank you".
> tr -s '%\n' ' '
>

Um, no. That will remove all but the first % and newline in the file, and collapse all multiple spaces to one space.

The *correct* answer is that it cannot be done with tr; the correct tool is sed:

sed '/^thank%$/{
N
s/^thank%\nyou$/thank you/
}'

Chris Mattern

NB, -d removes a set, not a sequence:

echo aabbccabcd | tr -d 'abc'

The result is not 'aabbccd', but merely 'd'

documented on: 2003.12.12 Fri

cmd:kill 

> ps -ef | grep $1 | grep -v grep | nawk '{print "kill -9 " $2}' | sh

No no no. Don't use kill -9.

It doesn't give the process a chance to cleanly:

  1. release IPC resources (shared memory, semaphores, message queues)

  2. clean up temp files

  3. inform its children that it is going away

  4. reset its terminal characteristics

and so on and so on and so on.

Generally, send 15, and wait a second or two, and if that doesn't work, send 2, and if that doesn't work, send 1. If that doesn't, REMOVE THE BINARY because the program is badly behaved!

That would be "regular" kill sig (TERM), then INT, then HUP. Incidentally, with certain irritating programs at our site, I have to use USR1. Most programmers forget to block out USR1, while it is common to stupidly block out HUP.

Don't use kill -9. Don't bring out the combine harvester just to tidy up the flower pot.

Randal

cmd:kill 

Back in 92 when I was working for Telebit, a co-worker asked me how to stop and start getty on SunOS 4.x. I told him "kill -1 1", but he lost track and typed "kill 1 -1". His init process went badda-boom.

After that, I started telling people the command was "kill -HUP 1", so it would be harder for them to get them out of order.

Greg

cmd:kill 

Another exciting story: I enjoy Linux "killall" command. I once worked on Solaris as root. I was very new in Solaris, and I wondered if "killall" existed there, too. So I foolishly typed "killall" just to check it. It turned out that "killall" is present on Solaris but it does something quite different than on Linux. Hard reboot was all I could do.

Stop/suspend, then resume with cmd:kill 

Newsgroups:  comp.os.linux.misc
Date:        Mon, 15 Mar 2004 23:08:59 -0500
> Is there a way to stop/suspend an application temporarily, then
> resume its execution later on?
>
> What I want is to suspend wget process while I am temporarily
> off-line. I looked up man pages, and thought the following should
> work:
>
> killall SIGSTOP wget
> killall SIGCONT wget
>
> But actually it won't:
>
> 'killall SIGSTOP wget' told me:
> SIGSTOP: no process killed

The error message tells you that killall interpreted SIGSTOP as a process name, not a signal; the wget processes were killed with the default SIGTERM.

killall -SIGSTOP wget
killall -STOP wget

Bill Marcum

cmd:su 

problem 

why it's PATH not set properly?

Usage 

example% su bin

To become user bin while retaining your previously exported environment, execute:

example% su - bin

To become user bin but change the environment to what would be expected if bin had originally logged in, execute:

Help 

SYNOPSIS
     su [ - ] [ username [ arg ... ] ]
DESCRIPTION
        If the first argument to su is a ' - '  (dash),  the
environment  is passed along unchanged, as if the user actu-
ally logged  in  as  the  specified  user.   Otherwise,  the
environment  is  passed  along, with the exception of $PATH,
which is controlled by PATH and SUPATH in etc/default/su.

cmd:sort 

Usage 

file * | sort -t: -b -k 2       # FS=':' ignore blank, sort on the 2nd field
file * | sort -t: -b +1         # ditto, sort on the 2nd field, deprecated
Note the -b is most important for -k !!!, without it, it would be very confusing. 
cat passwd | sort -u -t : -k 5

sort by the name field, uniq

Help 

-f     fold lower case to upper case characters in keys
-n     compare according to string numerical value, imply -b
-r     reverse the result of comparisons
Field Separator Options
   The treatment of field separators can be altered  using  the
   following options:
-b     ignore leading blanks in sort fields or keys
-t char   Use char as the field separator  character.
Sort Key Options
   Sort keys can be specified using the options:
+POS1 [-POS2]
       start  a key at POS1, end it *before* POS2 (obsolescent) field
       numbers and character offsets are numbered starting with  zero
       (contrast with the -k option)
-k POS1[,POS2]
       start a key at POS1, end it *at* POS2 field numbers and
       character offsets are numbered starting with one (contrast
       with zero-based +POS form)

Fields & blanks 

Try to sort on the device.

$ cat $tf.3  | sort -b -k2.8,2.8 -k2.9n
cache   /dev/hda13      /lfs/cache
export.img      /dev/hda10      /export.img
os1     /dev/hda6       /lfs/os1
os3     /dev/hda8       /lfs/os3
rh8     /dev/hda3       /lfs/rh8
toBurn  /dev/hda12      /lfs/toBurn
bp2     /dev/hdd2       /lfs/bp2
export  /dev/hdd7       /export
vars1   /dev/hdd5       /vars

— wrong order.

$ cat $tf.3 | cut -f 2 | sort -k1.8,1.8 -k1.9n
/dev/hda3
/dev/hda6
/dev/hda8
/dev/hda10
/dev/hda12
/dev/hda13
/dev/hdd2
/dev/hdd5
/dev/hdd7

— right order, meaning the .8 and .9 are right.

$ cat $tf.3  | sort -k2.9,2.9 -k2.10n
rh8     /dev/hda3       /lfs/rh8
os1     /dev/hda6       /lfs/os1
os3     /dev/hda8       /lfs/os3
export.img      /dev/hda10      /export.img
toBurn  /dev/hda12      /lfs/toBurn
cache   /dev/hda13      /lfs/cache
bp2     /dev/hdd2       /lfs/bp2
vars1   /dev/hdd5       /vars
export  /dev/hdd7       /export

— right order, why there is one more shift for field 2 than 1?

$ cat $tf.3 | sort -t"$t" -k2.8,2.8 -k2.9n
rh8     /dev/hda3       /lfs/rh8
os1     /dev/hda6       /lfs/os1
os3     /dev/hda8       /lfs/os3
export.img      /dev/hda10      /export.img
toBurn  /dev/hda12      /lfs/toBurn
cache   /dev/hda13      /lfs/cache
bp2     /dev/hdd2       /lfs/bp2
vars1   /dev/hdd5       /vars
export  /dev/hdd7       /export

— right order

Any way to sort this? 

: for \t delimiter, has to specify explicitly!
Newsgroups: comp.unix.shell
 > > Is there any way to sort the following files into the right order?
 > > thanks
 > >
 > > syntaxref11.html
 > > syntaxref1110.html
[...]
 > > syntaxref1115.html
 > > syntaxref112.html
[...]
 > > syntaxref119.html
 >
 > ls |sort -t. -n +0.9
 > or
 > ls |sort -t. -n -k 1.10
 >
 > The -t. tells sort that the field separator character is a fullstop, the -n
 > specifies an arithmetic sort and +0.9 tells sort that the "key" is plus 0
 > fields (ie the 1st field) and plus 9 characters within the field. Similarly
 > the -k 1.10 tells sort that the "key" is the 1st field and the 10 character
 > within that 1st field.

Here is a summary of the solutions so far:

Perl solution: 

perl -e 'sub d {($_[0] =~ /(\d+)/)[0]} print sort {d($a) <=> d($b)} <>' <infile >outfile

Good for about 50 files, but for more, you'd want to cache the regex part using a Schwartzian Transform or Orcish maneuver or GSR sort.

Randal L. Schwartz

gnu sort, field selection bug? 

I think I have found a bug in sort field selection algorithm. The following are the examples:

$ ls -1 sfa*
sfa1001ext
sfa1002ext
sfa100ext
sfa10ext
sfa1ext
sfa200ext
sfa20ext
sfa2ext
sfa300ext
sfa30ext
sfa3ext

The goal is sort on the number. Before we get into it, let's look at some warm up exercises first:

$ ls sfa* | sort -k 1.4
sfa1001ext
sfa1002ext
sfa100ext
sfa10ext
sfa1ext
sfa200ext
sfa20ext
sfa2ext
sfa300ext
sfa30ext
sfa3ext
$ ls sfa* | sort -n -k 1.4
sfa1ext
sfa2ext
sfa3ext
sfa10ext
sfa20ext
sfa30ext
sfa100ext
sfa200ext
sfa300ext
sfa1001ext
sfa1002ext

— so far so good

$ ls sfa* | sort -n -k 1.4,1.4
sfa1001ext
sfa1002ext
sfa100ext
sfa10ext
sfa1ext
sfa200ext
sfa20ext
sfa2ext
sfa300ext
sfa30ext
sfa3ext

— There is the bug! The output should be the same as previous one.

My sort comes along with RH8:

$ sort --v
sort (textutils) 2.0.21
Written by Mike Haertel and Paul Eggert.

The impat of this bug will make it impossible to sort in certain circumstances. For example:

$ ls -1 sfb*
sfb10-B
sfb10000-A
sfb10001-A
sfb10002-A
sfb11-B
sfb12-B
sfb8-B
sfb9-B
sfb9998-A
sfb9999-A

We need to sort the list first by the characters then by the numbers. There is no way to do it with the current sort program:

$ ls -1 sfb* | sort -t- -k2,2 -n -k1.4,1.4
sfb10-B
sfb10000-A
sfb10001-A
sfb10002-A
sfb11-B
sfb12-B
sfb8-B
sfb9-B
sfb9998-A
sfb9999-A

— this should be the right way, but the result is wrong.

gnu sort, field selection bug? 

> We need to sort the list first by the characters then by the
> numbers. There is no way to do it with the current sort program:
>
> $ ls -1 sfb* | sort -t- -k2,2 -n -k1.4,1.4
> sfb10-B
> sfb10000-A
> sfb10001-A
> sfb10002-A
> sfb11-B
> sfb12-B
> sfb8-B
> sfb9-B
> sfb9998-A
> sfb9999-A
>
>   -- this should be the right way, but the result is wrong.

I think your problem is where you're applying a global "n" option.

Try ls -1 sfb* | sort -t- -k2 -k1.4n

sfb9998-A
sfb9999-A
sfb10000-A
sfb10001-A
sfb10002-A
sfb8-B
sfb9-B
sfb10-B
sfb11-B
sfb12-B

which is, I think, what you want.

documented on: Wed 04-14-99 02:24:46

cmd:join 

Help 

Quick Help 

Usage: join [OPTION]... FILE1 FILE2

For each pair of input lines with identical join fields, write a line to standard output. The default join field is the first, delimited by whitespace. When FILE1 or FILE2 (not both) is -, read standard input.

-a FILENUM        print unpairable lines coming from file FILENUM, where
                    FILENUM is 1 or 2, corresponding to FILE1 or FILE2
-e EMPTY          replace missing input fields with EMPTY
-i, --ignore-case ignore differences in case when comparing fields
-j FIELD          equivalent to `-1 FIELD -2 FIELD'
-o FORMAT         obey FORMAT while constructing output line
-t CHAR           use CHAR as input and output field separator
-v FILENUM        like -a FILENUM, but suppress joined output lines
-1 FIELD          join on this FIELD of file 1
-2 FIELD          join on this FIELD of file 2
    --help     display this help and exit
    --version  output version information and exit

Unless -t CHAR is given, leading blanks separate fields and are ignored, else fields are separated by CHAR. Any FIELD is a field number counted from 1.

FORMAT is one or more comma or blank separated specifications, each being `FILENUM.FIELD' or `0'. Default FORMAT outputs the join field, the remaining fields from FILE1, the remaining fields from FILE2, all separated by CHAR.

Important: FILE1 and FILE2 must be sorted on the join fields.

Problem with command 'join' 

Newsgroups:  comp.unix.shell
Date:        Sun, 14 Dec 2003 17:17:10 GMT
> I have two files listed below, I'm wondering why two lines are
> left out after joined?  Is there something wrong with my file or
> with the command 'join'?
>
> $ join -1 2 -2 1 -e undef jf1 jf2 | tee jf3

The problem is with your files, they are not sorted on the field to be joined.

Icarus Sparry

Newsgroups:  gmane.linux.debian.user
Date:        Fri, 4 Mar 2005 00:03:38 +0000 (UTC)
>My understanding is that hard-links are not reliable. If I modify any of
>the hard links, most probably, the linkage would be broken. Is that so?

That's not because of the hard link, that's because of the application that you are using to modify the file.

If an editor creates a temporary copy of a file to work on, then when you save the file simply moves/renames the temporary file over the existing file, in the case of a hardlink, the link is broken.

Some of the editors can be told to "rewrite the file in-place" instead. That will keep the link.

Miquel van Smoorenburg @ nl