Newsgroups: gmane.linux.debian.user Date: Thu, 02 Mar 2006 10:23:20 -0500
Could someone tell me why the following works in zsh but not in bash/posh/dash?
benjo[3]:~% echo foo bar baz | read a b c benjo[4]:~% echo $a $b $c foo bar baz
If I try the same with bash (or other sh-compatible shells), the variables $a $b and $c are unset. From the bash man page:
> read [-ers] [-u fd] [-t timeout] [-a aname] [-p prompt] [-n nchars] [-d > delim] [name ...] > One line is read from the standard input, or from > the file descriptor fd supplied as an argument to the -u > option, and the first word is assigned to the first name, > the second word to the second name, and so on, with > leftover words and their interven\u2010 ing separators > assigned to the last name.
So "read" claims to read from the standard input, but it doesn't actually seem to happen when a pipe is involved.
Posh and dash behave like bash in this respect, so I guess that this is not a bug, and that what zsh does is actually an extension. So, what is the correct POSIX-compatible way to get "read" to work as I want?
Kevin B. McCarty
> So "read" claims to read from the standard input, but it doesn't > actually seem to happen when a pipe is involved. [...]
Each command in the pipeline gets executed in its own subshell and so the variables are set there and not passed back to the parent process. This is clear for example when you read about the compound command ( list ):
(list) list is executed in a subshell. Variable assignments and builtin commands that affect the shell's environment do not remain in effect after the command completes. [...]
So this would work:
$ echo foo bar baz | ( read a b c; echo $a $b $c) foo bar baz $
Else you can use a 'here document' construct to feed some standard input to "read" in the same environment as the shell doing the "read", as for example:
$ read a b c<<END > foo bar baz > END $ echo $a $b $c foo bar baz $
alfredo
> If I try the same with bash (or other sh-compatible shells), the > variables $a $b and $c are unset. From the bash man page: ... > So "read" claims to read from the standard input, but it doesn't > actually seem to happen when a pipe is involved.
What's happening here is that the pipe is causing a subshell to be spawned, which is then parsing the command "read a b c".
I'm not sure of the POSIX way to use read in this manner, but I found this on Google/A9:
http://linuxgazette.net/issue57/tag/1.html
The example he gives, with the < <() syntax, worked in bash, but not in Debian or FreeBSD's /bin/sh.
David Kirchner
> The example he gives, with the < <() syntax, worked in bash, but not in > Debian or FreeBSD's /bin/sh.
In more recent bashes, the following should work as well
#!/bin/bash read a b c <<<`echo foo bar baz` echo $a $b $c
The <<< ("here strings") are an extension of the "here document" syntax, IOW, the string given after <<< is supplied as stdin to the command.
Then, there's another variant, which is about as ugly as it can get… It should, however, work with most bourne shell compatible shells:
#!/bin/sh eval `echo foo bar baz | (read a b c; echo "a='$a';b='$b';c='$c'" )` echo $a $b $c
To get the variable's values from the subshell back to the main shell, a shell code fragment is written on stdout, captured with backticks, and then eval'ed in the main shell… (this is the moment when I usually switch to some other scripting language — if not before :)
Almut
I think the POSIX way would be
echo foo bar baz | { read a b c; echo $a $b $c; }
Bill Marcum
I wrote:
> Could someone tell me why the following works in zsh but not in > bash/posh/dash? > > benjo[3]:~% echo foo bar baz | read a b c benjo[4]:~% echo $a $b $c > foo bar baz
Thanks everyone for the enlightening answers! So just to summarize, the problem is that the pipeline is treated as a subshell, and so the variables $a $b and $c are defined within the subshell but not the "main" shell.
These seem like the best solutions to my problem:
David Kirchner wrote:
> I'm not sure of the POSIX way to use read in this manner, but I found > this on Google/A9: > > http://linuxgazette.net/issue57/tag/1.html[] > > The example he gives, with the < <() syntax, worked in bash, but not in > Debian or FreeBSD's /bin/sh.
Almut Behrens wrote:
> In more recent bashes, the following should work as well > > #!/bin/bash > read a b c <<<`echo foo bar baz` > echo $a $b $c > > The <<< ("here strings") are an extension of the "here document" syntax, > IOW, the string given after <<< is supplied as stdin to the command.
Bill Marcum wrote:
> I think the POSIX way would be > echo foo bar baz | { read a b c; echo $a $b $c; }
Not too bad if what you want to do inside the { } braces is pretty short.
Almut Behrens wrote:
> #!/bin/sh > eval `echo foo bar baz | (read a b c; echo "a='$a';b='$b';c='$c'" )` > echo $a $b $c > > To get the variable's values from the subshell back to the main shell, a > shell code fragment is written on stdout, captured with backticks, and > then eval'ed in the main shell... (this is the moment when I usually > switch to some other scripting language -- if not before :)
Ugh. Does get the job done though. I guess one has to be a little careful about escaping special characters in this case? Here's the safest version I've found so far — single quotes in the input have to be special cased with the sed command, and the -r flag to "read" keeps it from eating backslashes.
# set some variables to nightmarish values for testing purposes d='"ab\""q"' # literal value is "ab\""q" e='$d' # literal value is $d f="'ba'r'" # literal value is 'ba'r'
# here's the meat of the code result="`echo "$d $e $f" | sed "s/'/\'\\\\\\\'\'/g" | \ ( read -r a b c; echo "a='$a' ; b='$b' ; c='$c'" )`" eval "$result"
# test that $a $b $c have the right values echo "$a $b $c"
Tested on Sarge with zsh, bash, dash and posh :-)
Of course, replace this:
echo "$d $e $f"
with whatever is producing the output that needs to be put into $a $b $c
Personally, I'd rather constrain my script to work only with bash and use <<< or < <() operators than to write something like the above! (N.B. every method above still needs the -r flag to read if the input might contain backslashes.)
Kevin B. McCarty
> # set some variables to nightmarish values for testing purposes > d='"ab\""q"' # literal value is "ab\""q" e='$d' # literal > value is $d > f="'ba'r'" # literal value is 'ba'r' > > # here's the meat of the code > result="`echo "$d $e $f" | sed "s/'/\'\\\\\\\'\'/g" | \ > ( read -r a b c; echo "a='$a' ; b='$b' ; c='$c'" )`" > eval "$result" > > # test that $a $b $c have the right values echo "$a $b $c"
Another, more sane way to do what you want (I think):
set `echo "$d $e $f" | sed "s/'/\'\\\\\\\'\'/g"` a=$1; b=$2; c=$3
which will work if your data have no embedded spaces. Otherwise, pick a character you know is not in the data (+, for example) and:
OIFS=$IFS; IFS="+" set `echo "$d+$e+$f" | sed "s/'/\'\\\\\\\'\'/g"` a=$1; b=$2; c=$3 IFS=$OIFS
I've been writing shell scripts for many years, and I *still* trip over this now and again, curse a time or three, then remember that some things are done in sub-shells or sub-processes, and finally do the IFS and set trick. :)
Neal Murphy