Last month, I began this column by talking about the many different file redirection options available from within the shell, whether you're a bash or csh fan (or some variant thereof). What I deliberately ignored, however, was the fact that there are actually three file descriptors - three I/O channels - that you can control from the command line (and, therefore, from within shell scripts).

This time, let's begin to dig deeper into the three file descriptors that are commonly found on Unix and Linux systems and learn what they are, how they differ, and - most importantly - how to exploit them when you're writing shell scripts.

Input Is Input Is Input

The simplest of the file descriptors is stdin (pronounced, confusingly enough, as "standard in"), which is always file descriptor #0 on a running Linux command. When you type in a command like

wc -l < something.txt

the shell automatically opens up the specified file and reassigns stdin for the wc command to the opened file descriptor of the specified file. You can see this by observing the difference in errors between specifying a redirect when the file doesn't exist and specifying a nonexistent file to the command itself:

$ wc -l < does-not-exist
bash: does-not-exist: No such file or directory
$ wc -l does-not-exist
wc: does-not-exist: open: No such file or directory

The Two Kinds of Output

Since we have a situation that produces an error message, let's use it to explore how errors are sent to stderr (standard error) rather than the more typical stdout (standard output):

$ wc -l does-not-exist > my.output
wc: does-not-exist: open: No such file or directory

What happened here is that when the shell launched the wc command, it assigned three file descriptors, as always, to the utility. The stdin was the keyboard (which is the default for appli-cations unless something else is specified, either by redirection or a pipe); stdout was reassigned to the newly created or re-created file called my.output in the current directory; and stderr remained the screen, the default output for errors.

When the error message was output by wc, the program correctly sent the message to stderr, bypassing the redirection and leaving my.output empty.

But don't despair! There's a pretty simple way to specify that you want the error messages redirected to the same place as stdout, or even to a different location/file. However, there's a catch: the notation for doing this varies between shells, so you need to know what shell you're using before you can figure out which will work best.

Redirecting Stderr the Bash Way

Let's start out by looking at how Bash and the other spawn of Bourne Shell shells expect you to redirect stderr. The formal notation is based on the numbers of the file descriptors (where stdin=0, stdout=1 and stderr=2):

wc -l does-not-exist > my.output 2>&1

This causes file descriptor 2 (stderr) to become a copy of file descriptor 1 (stdout), which has already been assigned to the file my.output. Most important, order counts, so changing to something like

wc -l does-not-exist 2>&1 > my.output

will not cause stderr to be redirected to the file; stderr becomes a duplicate of stdout before the redirection is processed by the shell. There's also a more succinct way of requesting this dual redirection that's worth knowing:

wc -l does-not-exist &>my.output

By the way, to send stdout to stderr you can use

echo "this is an error message" >&2

as a convenient shorthand (it's identical to specifying 1>&2). You can also redirect errors in a pipe by using the 2>&1 notation, as demonstrated here:

wc -l does-not-exist 2>&1 | wc -l

That command will correctly count the number of lines in the output stream, which consists of only one line, the error itself.

Next Month...

Next month we'll consider how Csh deals with file redirection and learn some techniques for ensuring that shell scripts conform to the Linux standard behavior of having errors output to stderr, while regular output is sent to stdout.