Stupid shell script question about "read" 

Newsgroups:  gmane.linux.debian.user
Date:        Thu, 02 Mar 2006 10:23:20 -0500

Could someone tell me why the following works in zsh but not in bash/posh/dash?

benjo[3]:~% echo foo bar baz | read a b c
benjo[4]:~% echo $a $b $c
foo bar baz

If I try the same with bash (or other sh-compatible shells), the variables $a $b and $c are unset. From the bash man page:

>  read [-ers] [-u fd] [-t timeout] [-a aname] [-p prompt] [-n nchars] [-d
>        delim] [name ...]
>               One  line  is  read  from  the  standard input, or from
>               the file descriptor fd supplied as an argument to the -u
>               option, and  the first word is assigned to the first name,
>               the second word to the second name, and so on, with
>               leftover words and their  interven\u2010 ing  separators
>               assigned  to the last name.

So "read" claims to read from the standard input, but it doesn't actually seem to happen when a pipe is involved.

Posh and dash behave like bash in this respect, so I guess that this is not a bug, and that what zsh does is actually an extension. So, what is the correct POSIX-compatible way to get "read" to work as I want?

Kevin B. McCarty

Stupid shell script question about "read" 

> So "read" claims to read from the standard input, but it doesn't
> actually seem to happen when a pipe is involved. [...]

Each command in the pipeline gets executed in its own subshell and so the variables are set there and not passed back to the parent process. This is clear for example when you read about the compound command ( list ):

(list) list is  executed  in  a  subshell.   Variable
       assignments  and builtin  commands  that  affect the  shell's
       environment do not remain in effect after the command
       completes.  [...]

So this would work:

$ echo foo bar baz | ( read a b c; echo $a $b $c)
foo bar baz
$

Else you can use a 'here document' construct to feed some standard input to "read" in the same environment as the shell doing the "read", as for example:

$ read a b c<<END
> foo bar baz
> END
$ echo $a $b $c
foo bar baz
$

alfredo

Stupid shell script question about "read" 

> If I try the same with bash (or other sh-compatible shells), the
> variables $a $b and $c are unset.  From the bash man page: ...
> So "read" claims to read from the standard input, but it doesn't
> actually seem to happen when a pipe is involved.

What's happening here is that the pipe is causing a subshell to be spawned, which is then parsing the command "read a b c".

I'm not sure of the POSIX way to use read in this manner, but I found this on Google/A9:

http://linuxgazette.net/issue57/tag/1.html

The example he gives, with the < <() syntax, worked in bash, but not in Debian or FreeBSD's /bin/sh.

David Kirchner

Stupid shell script question about "read" 

> The example he gives, with the < <() syntax, worked in bash, but not in
> Debian or FreeBSD's /bin/sh.

In more recent bashes, the following should work as well

#!/bin/bash
read a b c <<<`echo foo bar baz`
echo $a $b $c

The <<< ("here strings") are an extension of the "here document" syntax, IOW, the string given after <<< is supplied as stdin to the command.

Then, there's another variant, which is about as ugly as it can get… It should, however, work with most bourne shell compatible shells:

#!/bin/sh
eval `echo foo bar baz | (read a b c; echo "a='$a';b='$b';c='$c'" )` echo
$a $b $c

To get the variable's values from the subshell back to the main shell, a shell code fragment is written on stdout, captured with backticks, and then eval'ed in the main shell… (this is the moment when I usually switch to some other scripting language — if not before :)

Almut

Stupid shell script question about "read" 

I think the POSIX way would be

echo foo bar baz | { read a b c; echo $a $b $c; }

Bill Marcum

Stupid shell script question about "read" [SOLVED] 

I wrote:

> Could someone tell me why the following works in zsh but not in
> bash/posh/dash?
>
> benjo[3]:~% echo foo bar baz | read a b c benjo[4]:~% echo $a $b $c
> foo bar baz

Thanks everyone for the enlightening answers! So just to summarize, the problem is that the pipeline is treated as a subshell, and so the variables $a $b and $c are defined within the subshell but not the "main" shell.

These seem like the best solutions to my problem:

  • Bash-specific (i.e. not POSIX-compliant) :

David Kirchner wrote:

> I'm not sure of the POSIX way to use read in this manner, but I found
> this on Google/A9:
>
> http://linuxgazette.net/issue57/tag/1.html[]
>
> The example he gives, with the < <() syntax, worked in bash, but not in
> Debian or FreeBSD's /bin/sh.

Almut Behrens wrote:

> In more recent bashes, the following should work as well
>
> #!/bin/bash
> read a b c <<<`echo foo bar baz`
> echo $a $b $c
>
> The <<< ("here strings") are an extension of the "here document" syntax,
> IOW, the string given after <<< is supplied as stdin to the command.
  • POSIX-compliant:

Bill Marcum wrote:

> I think the POSIX way would be
> echo foo bar baz | { read a b c; echo $a $b $c; }

Not too bad if what you want to do inside the { } braces is pretty short.

Almut Behrens wrote:

> #!/bin/sh
> eval `echo foo bar baz | (read a b c; echo "a='$a';b='$b';c='$c'" )`
> echo $a $b $c
>
> To get the variable's values from the subshell back to the main shell, a
> shell code fragment is written on stdout, captured with backticks, and
> then eval'ed in the main shell...  (this is the moment when I usually
> switch to some other scripting language -- if not before :)

Ugh. Does get the job done though. I guess one has to be a little careful about escaping special characters in this case? Here's the safest version I've found so far — single quotes in the input have to be special cased with the sed command, and the -r flag to "read" keeps it from eating backslashes.

# set some variables to nightmarish values for testing purposes
d='"ab\""q"' # literal value is "ab\""q"
e='$d'       # literal value is $d
f="'ba'r'"   # literal value is 'ba'r'
# here's the meat of the code
result="`echo "$d $e $f" | sed "s/'/\'\\\\\\\'\'/g" | \
        ( read -r a b c; echo "a='$a' ; b='$b' ; c='$c'" )`"
eval "$result"
# test that $a $b $c have the right values
echo "$a $b $c"

Tested on Sarge with zsh, bash, dash and posh :-)

Of course, replace this:

echo "$d $e $f"

with whatever is producing the output that needs to be put into $a $b $c

Personally, I'd rather constrain my script to work only with bash and use <<< or < <() operators than to write something like the above! (N.B. every method above still needs the -r flag to read if the input might contain backslashes.)

Kevin B. McCarty

Stupid shell script question about "read" [SOLVED] 

>         # set some variables to nightmarish values for testing purposes
>         d='"ab\""q"' # literal value is "ab\""q" e='$d'       # literal
>         value is $d
>         f="'ba'r'"   # literal value is 'ba'r'
>
>         # here's the meat of the code
>         result="`echo "$d $e $f" | sed "s/'/\'\\\\\\\'\'/g" | \
>                 ( read -r a b c; echo "a='$a' ; b='$b' ; c='$c'" )`"
>         eval "$result"
>
>         # test that $a $b $c have the right values echo "$a $b $c"

Another, more sane way to do what you want (I think):

set `echo "$d $e $f" | sed "s/'/\'\\\\\\\'\'/g"` a=$1; b=$2; c=$3

which will work if your data have no embedded spaces. Otherwise, pick a character you know is not in the data (+, for example) and:

OIFS=$IFS; IFS="+"
set `echo "$d+$e+$f" | sed "s/'/\'\\\\\\\'\'/g"` a=$1; b=$2; c=$3
IFS=$OIFS

I've been writing shell scripts for many years, and I *still* trip over this now and again, curse a time or three, then remember that some things are done in sub-shells or sub-processes, and finally do the IFS and set trick. :)

Neal Murphy