Search     or:     and:
 LINUX 
 Language 
 Kernel 
 Package 
 Book 
 Test 
 OS 
 Forum 
 iakovlev.org 
 Books
  Краткое описание
 Linux
 W. R. Стивенс TCP 
 W. R. Стивенс IPC 
 A.Rubini-J.Corbet 
 K. Bauer 
 Gary V. Vaughan 
 Д Вилер 
 В. Сталлинг 
 Pramode C.E. 
 Steve Pate 
 William Gropp 
 K.A.Robbins 
 С Бекман 
 Р Стивенс 
 Ethereal 
 Cluster 
 Languages
 C
 Perl
 M.Pilgrim 
 А.Фролов 
 Mendel Cooper 
 М Перри 
 Kernel
 C.S. Rodriguez 
 Robert Love 
 Daniel Bovet 
 Д Джеф 
 Максвелл 
 G. Kroah-Hartman 
 B. Hansen 
NEWS
Последние статьи :
  Rust 07.11   
  Go 25.12   
  EXT4 10.11   
  FS benchmark 15.09   
  Сетунь 23.07   
  Trees 25.06   
  Apache 03.02   
  SQL 30.07   
  JFS 10.06   
  B-trees 01.06   
 
TOP 20
 Go Web ...802 
 Httpd-> История Ap...503 
 2.0-> Linux IP Networking...419 
 Trees...362 
 MySQL & PosgreSQL...361 
 Secure Programming for Li...349 
 Assembler...292 
 TCP 2...257 
 Mod_perl 1...252 
 Steve Pate 3...249 
 Secure Programming for Li...247 
 Cluster 3...237 
 Максвелл 3...232 
 Advanced Bash Scripting G...229 
 Linux Inline Assembly...225 
 Python...220 
 Rubni-Corbet -> Глав...210 
 Mod_perl 2...210 
 Steve Pate 1...209 
 Rodriguez 8...207 
 
  01.06.2017 : 2215600 посещений 

iakovlev.org

Redirecting Code Blocks

Blocks of code, such as while, until, and for loops, even if/then test blocks can also incorporate redirection of stdin. Even a function may use this form of redirection (see Example 23-11). The < operator at the end of the code block accomplishes this.

Example 16-5. Redirected while loop

#!/bin/bash
 # redir2.sh
 
 if [ -z "$1" ]
 then
   Filename=names.data       # Default, if no filename specified.
 else
   Filename=$1
 fi  
 #+ Filename=${1:-names.data}
 #  can replace the above test (parameter substitution).
 
 count=0
 
 echo
 
 while [ "$name" != Smith ]  # Why is variable $name in quotes?
 do
   read name                 # Reads from $Filename, rather than stdin.
   echo $name
   let "count += 1"
 done <"$Filename"           # Redirects stdin to file $Filename. 
 #    ^^^^^^^^^^^^
 
 echo; echo "$count names read"; echo
 
 exit 0
 
 #  Note that in some older shell scripting languages,
 #+ the redirected loop would run as a subshell.
 #  Therefore, $count would return 0, the initialized value outside the loop.
 #  Bash and ksh avoid starting a subshell *whenever possible*,
 #+ so that this script, for example, runs correctly.
 #  (Thanks to Heiner Steven for pointing this out.)
 
 # However . . .
 # Bash *can* sometimes start a subshell in a *redirected* "while" loop.
 
 abc=hi
 echo -e "1\n2\n3" | while read l
      do abc="$l"
         echo $abc
      done
 echo $abc
 
 # (Thanks, Bruno de Oliveira Schneider, for demonstrating this
 #+ with the above snippet of code.)

Example 16-6. Alternate form of redirected while loop

#!/bin/bash
 
 # This is an alternate form of the preceding script.
 
 #  Suggested by Heiner Steven
 #+ as a workaround in those situations when a redirect loop
 #+ runs as a subshell, and therefore variables inside the loop
 # +do not keep their values upon loop termination.
 
 
 if [ -z "$1" ]
 then
   Filename=names.data     # Default, if no filename specified.
 else
   Filename=$1
 fi  
 
 
 exec 3<&0                 # Save stdin to file descriptor 3.
 exec 0<"$Filename"        # Redirect standard input.
 
 count=0
 echo
 
 
 while [ "$name" != Smith ]
 do
   read name               # Reads from redirected stdin ($Filename).
   echo $name
   let "count += 1"
 done                      #  Loop reads from file $Filename
                           #+ because of line 20.
 
 #  The original version of this script terminated the "while" loop with
 #+      done <"$Filename" 
 #  Exercise:
 #  Why is this unnecessary?
 
 
 exec 0<&3                 # Restore old stdin.
 exec 3<&-                 # Close temporary fd 3.
 
 echo; echo "$count names read"; echo
 
 exit 0

Example 16-7. Redirected until loop

#!/bin/bash
 # Same as previous example, but with "until" loop.
 
 if [ -z "$1" ]
 then
   Filename=names.data         # Default, if no filename specified.
 else
   Filename=$1
 fi  
 
 # while [ "$name" != Smith ]
 until [ "$name" = Smith ]     # Change  !=  to =.
 do
   read name                   # Reads from $Filename, rather than stdin.
   echo $name
 done <"$Filename"             # Redirects stdin to file $Filename. 
 #    ^^^^^^^^^^^^
 
 # Same results as with "while" loop in previous example.
 
 exit 0

Example 16-8. Redirected for loop

#!/bin/bash
 
 if [ -z "$1" ]
 then
   Filename=names.data          # Default, if no filename specified.
 else
   Filename=$1
 fi  
 
 line_count=`wc $Filename | awk '{ print $1 }'`
 #           Number of lines in target file.
 #
 #  Very contrived and kludgy, nevertheless shows that
 #+ it's possible to redirect stdin within a "for" loop...
 #+ if you're clever enough.
 #
 # More concise is     line_count=$(wc -l < "$Filename")
 
 
 for name in `seq $line_count`  # Recall that "seq" prints sequence of numbers.
 # while [ "$name" != Smith ]   --   more complicated than a "while" loop   --
 do
   read name                    # Reads from $Filename, rather than stdin.
   echo $name
   if [ "$name" = Smith ]       # Need all this extra baggage here.
   then
     break
   fi  
 done <"$Filename"              # Redirects stdin to file $Filename. 
 #    ^^^^^^^^^^^^
 
 exit 0

We can modify the previous example to also redirect the output of the loop.

Example 16-9. Redirected for loop (both stdin and stdout redirected)

#!/bin/bash
 
 if [ -z "$1" ]
 then
   Filename=names.data          # Default, if no filename specified.
 else
   Filename=$1
 fi  
 
 Savefile=$Filename.new         # Filename to save results in.
 FinalName=Jonah                # Name to terminate "read" on.
 
 line_count=`wc $Filename | awk '{ print $1 }'`  # Number of lines in target file.
 
 
 for name in `seq $line_count`
 do
   read name
   echo "$name"
   if [ "$name" = "$FinalName" ]
   then
     break
   fi  
 done < "$Filename" > "$Savefile"     # Redirects stdin to file $Filename,
 #    ^^^^^^^^^^^^^^^^^^^^^^^^^^^       and saves it to backup file.
 
 exit 0

Example 16-10. Redirected if/then test

#!/bin/bash
 
 if [ -z "$1" ]
 then
   Filename=names.data   # Default, if no filename specified.
 else
   Filename=$1
 fi  
 
 TRUE=1
 
 if [ "$TRUE" ]          # if true    and   if :   also work.
 then
  read name
  echo $name
 fi <"$Filename"
 #  ^^^^^^^^^^^^
 
 # Reads only first line of file.
 # An "if/then" test has no way of iterating unless embedded in a loop.
 
 exit 0

Example 16-11. Data file "names.data" for above examples

Aristotle
 Belisarius
 Capablanca
 Euler
 Goethe
 Hamurabi
 Jonah
 Laplace
 Maroczy
 Purcell
 Schmidt
 Semmelweiss
 Smith
 Turing
 Venn
 Wilson
 Znosko-Borowski
 
 #  This is a data file for
 #+ "redir2.sh", "redir3.sh", "redir4.sh", "redir4a.sh", "redir5.sh".

Redirecting the stdout of a code block has the effect of saving its output to a file. See Example 3-2.

Here documents are a special case of redirected code blocks.

Here Documents

Here and now, boys.

Aldous Huxley, "Island"

A here document is a special-purpose code block. It uses a form of I/O redirection to feed a command list to an interactive program or a command, such as ftp, cat, or the ex text editor.

COMMAND <<InputComesFromHERE
 ...
 InputComesFromHERE

A limit string delineates (frames) the command list. The special symbol << designates the limit string. This has the effect of redirecting the output of a file into the stdin of the program or command. It is similar to interactive-program < command-file, where command-file contains
command #1
 command #2
 ...

The here document alternative looks like this:

#!/bin/bash
 interactive-program <<LimitString
 command #1
 command #2
 ...
 LimitString

Choose a limit string sufficiently unusual that it will not occur anywhere in the command list and confuse matters.

Note that here documents may sometimes be used to good effect with non-interactive utilities and commands, such as, for example, wall.

Example 17-1. broadcast: Sends message to everyone logged in

#!/bin/bash
 
 wall <<zzz23EndOfMessagezzz23
 E-mail your noontime orders for pizza to the system administrator.
     (Add an extra dollar for anchovy or mushroom topping.)
 # Additional message text goes here.
 # Note: 'wall' prints comment lines.
 zzz23EndOfMessagezzz23
 
 # Could have been done more efficiently by
 #         wall <message-file
 #  However, embedding the message template in a script
 #+ is a quick-and-dirty one-off solution.
 
 exit 0

Even such unlikely candidates as vi lend themselves to here documents.

Example 17-2. dummyfile: Creates a 2-line dummy file

#!/bin/bash
 
 # Non-interactive use of 'vi' to edit a file.
 # Emulates 'sed'.
 
 E_BADARGS=65
 
 if [ -z "$1" ]
 then
   echo "Usage: `basename $0` filename"
   exit $E_BADARGS
 fi
 
 TARGETFILE=$1
 
 # Insert 2 lines in file, then save.
 #--------Begin here document-----------#
 vi $TARGETFILE <<x23LimitStringx23
 i
 This is line 1 of the example file.
 This is line 2 of the example file.
 ^[
 ZZ
 x23LimitStringx23
 #----------End here document-----------#
 
 #  Note that ^[ above is a literal escape
 #+ typed by Control-V <Esc>.
 
 #  Bram Moolenaar points out that this may not work with 'vim',
 #+ because of possible problems with terminal interaction.
 
 exit 0

The above script could just as effectively have been implemented with ex, rather than vi. Here documents containing a list of ex commands are common enough to form their own category, known as ex scripts.
#!/bin/bash
 #  Replace all instances of "Smith" with "Jones"
 #+ in files with a ".txt" filename suffix. 
 
 ORIGINAL=Smith
 REPLACEMENT=Jones
 
 for word in $(fgrep -l $ORIGINAL *.txt)
 do
   # -------------------------------------
   ex $word <<EOF
   :%s/$ORIGINAL/$REPLACEMENT/g
   :wq
 EOF
   # :%s is the "ex" substitution command.
   # :wq is write-and-quit.
   # -------------------------------------
 done

Analogous to "ex scripts" are cat scripts.

Example 17-3. Multi-line message using cat

#!/bin/bash
 
 #  'echo' is fine for printing single line messages,
 #+  but somewhat problematic for for message blocks.
 #   A 'cat' here document overcomes this limitation.
 
 cat <<End-of-message
 -------------------------------------
 This is line 1 of the message.
 This is line 2 of the message.
 This is line 3 of the message.
 This is line 4 of the message.
 This is the last line of the message.
 -------------------------------------
 End-of-message
 
 #  Replacing line 7, above, with
 #+   cat > $Newfile <<End-of-message
 #+       ^^^^^^^^^^
 #+ writes the output to the file $Newfile, rather than to stdout.
 
 exit 0
 
 
 #--------------------------------------------
 # Code below disabled, due to "exit 0" above.
 
 # S.C. points out that the following also works.
 echo "-------------------------------------
 This is line 1 of the message.
 This is line 2 of the message.
 This is line 3 of the message.
 This is line 4 of the message.
 This is the last line of the message.
 -------------------------------------"
 # However, text may not include double quotes unless they are escaped.

The - option to mark a here document limit string (<<-LimitString) suppresses leading tabs (but not spaces) in the output. This may be useful in making a script more readable.

Example 17-4. Multi-line message, with tabs suppressed

#!/bin/bash
 # Same as previous example, but...
 
 #  The - option to a here document <<-
 #+ suppresses leading tabs in the body of the document,
 #+ but *not* spaces.
 
 cat <<-ENDOFMESSAGE
 	This is line 1 of the message.
 	This is line 2 of the message.
 	This is line 3 of the message.
 	This is line 4 of the message.
 	This is the last line of the message.
 ENDOFMESSAGE
 # The output of the script will be flush left.
 # Leading tab in each line will not show.
 
 # Above 5 lines of "message" prefaced by a tab, not spaces.
 # Spaces not affected by   <<-  .
 
 # Note that this option has no effect on *embedded* tabs.
 
 exit 0

A here document supports parameter and command substitution. It is therefore possible to pass different parameters to the body of the here document, changing its output accordingly.

Example 17-5. Here document with parameter substitution

#!/bin/bash
 # Another 'cat' here document, using parameter substitution.
 
 # Try it with no command line parameters,   ./scriptname
 # Try it with one command line parameter,   ./scriptname Mortimer
 # Try it with one two-word quoted command line parameter,
 #                           ./scriptname "Mortimer Jones"
 
 CMDLINEPARAM=1     #  Expect at least command line parameter.
 
 if [ $# -ge $CMDLINEPARAM ]
 then
   NAME=$1          #  If more than one command line param,
                    #+ then just take the first.
 else
   NAME="John Doe"  #  Default, if no command line parameter.
 fi  
 
 RESPONDENT="the author of this fine script"  
   
 
 cat <<Endofmessage
 
 Hello, there, $NAME.
 Greetings to you, $NAME, from $RESPONDENT.
 
 # This comment shows up in the output (why?).
 
 Endofmessage
 
 # Note that the blank lines show up in the output.
 # So does the "comment".
 
 exit 0

This is a useful script containing a here document with parameter substitution.

Example 17-6. Upload a file pair to "Sunsite" incoming directory

#!/bin/bash
 # upload.sh
 
 #  Upload file pair (Filename.lsm, Filename.tar.gz)
 #+ to incoming directory at Sunsite/UNC (ibiblio.org).
 #  Filename.tar.gz is the tarball itself.
 #  Filename.lsm is the descriptor file.
 #  Sunsite requires "lsm" file, otherwise will bounce contributions.
 
 
 E_ARGERROR=65
 
 if [ -z "$1" ]
 then
   echo "Usage: `basename $0` Filename-to-upload"
   exit $E_ARGERROR
 fi  
 
 
 Filename=`basename $1`           # Strips pathname out of file name.
 
 Server="ibiblio.org"
 Directory="/incoming/Linux"
 #  These need not be hard-coded into script,
 #+ but may instead be changed to command line argument.
 
 Password="your.e-mail.address"   # Change above to suit.
 
 ftp -n $Server <<End-Of-Session
 # -n option disables auto-logon
 
 user anonymous "$Password"
 binary
 bell                             # Ring 'bell' after each file transfer.
 cd $Directory
 put "$Filename.lsm"
 put "$Filename.tar.gz"
 bye
 End-Of-Session
 
 exit 0

Quoting or escaping the "limit string" at the head of a here document disables parameter substitution within its body.

Example 17-7. Parameter substitution turned off

#!/bin/bash
 #  A 'cat' here document, but with parameter substitution disabled.
 
 NAME="John Doe"
 RESPONDENT="the author of this fine script"  
 
 cat <<'Endofmessage'
 
 Hello, there, $NAME.
 Greetings to you, $NAME, from $RESPONDENT.
 
 Endofmessage
 
 #  No parameter substitution when the "limit string" is quoted or escaped.
 #  Either of the following at the head of the here document would have the same effect.
 #  cat <<"Endofmessage"
 #  cat <<\Endofmessage
 
 exit 0

Disabling parameter substitution permits outputting literal text. Generating scripts or even program code is one use for this.

Example 17-8. A script that generates another script

#!/bin/bash
 # generate-script.sh
 # Based on an idea by Albert Reiner.
 
 OUTFILE=generated.sh         # Name of the file to generate.
 
 
 # -----------------------------------------------------------
 # 'Here document containing the body of the generated script.
 (
 cat <<'EOF'
 #!/bin/bash
 
 echo "This is a generated shell script."
 #  Note that since we are inside a subshell,
 #+ we can't access variables in the "outside" script.
 
 echo "Generated file will be named: $OUTFILE"
 #  Above line will not work as normally expected
 #+ because parameter expansion has been disabled.
 #  Instead, the result is literal output.
 
 a=7
 b=3
 
 let "c = $a * $b"
 echo "c = $c"
 
 exit 0
 EOF
 ) > $OUTFILE
 # -----------------------------------------------------------
 
 #  Quoting the 'limit string' prevents variable expansion
 #+ within the body of the above 'here document.'
 #  This permits outputting literal strings in the output file.
 
 if [ -f "$OUTFILE" ]
 then
   chmod 755 $OUTFILE
   # Make the generated file executable.
 else
   echo "Problem in creating file: \"$OUTFILE\""
 fi
 
 #  This method can also be used for generating
 #+ C programs, Perl programs, Python programs, Makefiles,
 #+ and the like.
 
 exit 0

It is possible to set a variable from the output of a here document.
variable=$(cat <<SETVAR
 This variable
 runs over multiple lines.
 SETVAR)
 
 echo "$variable"

A here document can supply input to a function in the same script.

Example 17-9. Here documents and functions

#!/bin/bash
 # here-function.sh
 
 GetPersonalData ()
 {
   read firstname
   read lastname
   read address
   read city 
   read state 
   read zipcode
 } # This certainly looks like an interactive function, but...
 
 
 # Supply input to the above function.
 GetPersonalData <<RECORD001
 Bozo
 Bozeman
 2726 Nondescript Dr.
 Baltimore
 MD
 21226
 RECORD001
 
 
 echo
 echo "$firstname $lastname"
 echo "$address"
 echo "$city, $state $zipcode"
 echo
 
 exit 0

It is possible to use : as a dummy command accepting output from a here document. This, in effect, creates an "anonymous" here document.

Example 17-10. "Anonymous" Here Document

#!/bin/bash
 
 : <<TESTVARIABLES
 ${HOSTNAME?}${USER?}${MAIL?}  # Print error message if one of the variables not set.
 TESTVARIABLES
 
 exit 0

Tip

A variation of the above technique permits "commenting out" blocks of code.

Example 17-11. Commenting out a block of code

#!/bin/bash
 # commentblock.sh
 
 : <<COMMENTBLOCK
 echo "This line will not echo."
 This is a comment line missing the "#" prefix.
 This is another comment line missing the "#" prefix.
 
 &*@!!++=
 The above line will cause no error message,
 because the Bash interpreter will ignore it.
 COMMENTBLOCK
 
 echo "Exit value of above \"COMMENTBLOCK\" is $?."   # 0
 # No error shown.
 
 
 #  The above technique also comes in useful for commenting out
 #+ a block of working code for debugging purposes.
 #  This saves having to put a "#" at the beginning of each line,
 #+ then having to go back and delete each "#" later.
 
 : <<DEBUGXXX
 for file in *
 do
  cat "$file"
 done
 DEBUGXXX
 
 exit 0

Tip

Yet another twist of this nifty trick makes "self-documenting" scripts possible.

Example 17-12. A self-documenting script

#!/bin/bash
 # self-document.sh: self-documenting script
 # Modification of "colm.sh".
 
 DOC_REQUEST=70
 
 if [ "$1" = "-h"  -o "$1" = "--help" ]     # Request help.
 then
   echo; echo "Usage: $0 [directory-name]"; echo
   sed --silent -e '/DOCUMENTATIONXX$/,/^DOCUMENTATIONXX$/p' "$0" |
   sed -e '/DOCUMENTATIONXX$/d'; exit $DOC_REQUEST; fi
 
 
 : <<DOCUMENTATIONXX
 List the statistics of a specified directory in tabular format.
 ---------------------------------------------------------------
 The command line parameter gives the directory to be listed.
 If no directory specified or directory specified cannot be read,
 then list the current working directory.
 
 DOCUMENTATIONXX
 
 if [ -z "$1" -o ! -r "$1" ]
 then
   directory=.
 else
   directory="$1"
 fi  
 
 echo "Listing of "$directory":"; echo
 (printf "PERMISSIONS LINKS OWNER GROUP SIZE MONTH DAY HH:MM PROG-NAME\n" \
 ; ls -l "$directory" | sed 1d) | column -t
 
 exit 0

See also Example A-27 for an excellent example of a self-documenting script.

Note

Here documents create temporary files, but these files are deleted after opening and are not accessible to any other process.

bash$ bash -c 'lsof -a -p $$ -d0' << EOF
 > EOF
 lsof    1213 bozo    0r   REG    3,5    0 30386 /tmp/t1213-0-sh (deleted)
 	      

Caution

Some utilities will not work inside a here document.

Warning

The closing limit string, on the final line of a here document, must start in the first character position. There can be no leading whitespace. Trailing whitespace after the limit string likewise causes unexpected behavior. The whitespace prevents the limit string from being recognized.

#!/bin/bash
 
 echo "----------------------------------------------------------------------"
 
 cat <<LimitString
 echo "This is line 1 of the message inside the here document."
 echo "This is line 2 of the message inside the here document."
 echo "This is the final line of the message inside the here document."
      LimitString
 #^^^^Indented limit string. Error! This script will not behave as expected.
 
 echo "----------------------------------------------------------------------"
 
 #  These comments are outside the 'here document',
 #+ and should not echo.
 
 echo "Outside the here document."
 
 exit 0
 
 echo "This line had better not echo."  # Follows an 'exit' command.

For those tasks too complex for a "here document", consider using the expect scripting language, which is specifically tailored for feeding input into interactive programs.

Subshells

Running a shell script launches another instance of the command processor. Just as your commands are interpreted at the command line prompt, similarly does a script batch process a list of commands in a file. Each shell script running is, in effect, a subprocess of the parent shell, the one that gives you the prompt at the console or in an xterm window.

A shell script can also launch subprocesses. These subshells let the script do parallel processing, in effect executing multiple subtasks simultaneously.

Command List in Parentheses

( command1; command2; command3; ... )

A command list embedded between parentheses runs as a subshell.

Note

Variables in a subshell are not visible outside the block of code in the subshell. They are not accessible to the parent process, to the shell that launched the subshell. These are, in effect, local variables.

Example 20-1. Variable scope in a subshell

#!/bin/bash
 # subshell.sh
 
 echo
 
 echo "Subshell level OUTSIDE subshell = $BASH_SUBSHELL"
 # Bash, version 3, adds the new         $BASH_SUBSHELL variable.
 echo
 
 outer_variable=Outer
 
 (
 echo "Subshell level INSIDE subshell = $BASH_SUBSHELL"
 inner_variable=Inner
 
 echo "From subshell, \"inner_variable\" = $inner_variable"
 echo "From subshell, \"outer\" = $outer_variable"
 )
 
 echo
 echo "Subshell level OUTSIDE subshell = $BASH_SUBSHELL"
 echo
 
 if [ -z "$inner_variable" ]
 then
   echo "inner_variable undefined in main body of shell"
 else
   echo "inner_variable defined in main body of shell"
 fi
 
 echo "From main body of shell, \"inner_variable\" = $inner_variable"
 #  $inner_variable will show as uninitialized
 #+ because variables defined in a subshell are "local variables".
 #  Is there any remedy for this?
 
 echo
 
 exit 0

See also Example 31-2.

+

Directory changes made in a subshell do not carry over to the parent shell.

Example 20-2. List User Profiles

#!/bin/bash
 # allprofs.sh: print all user profiles
 
 # This script written by Heiner Steven, and modified by the document author.
 
 FILE=.bashrc  #  File containing user profile,
               #+ was ".profile" in original script.
 
 for home in `awk -F: '{print $6}' /etc/passwd`
 do
   [ -d "$home" ] || continue    # If no home directory, go to next.
   [ -r "$home" ] || continue    # If not readable, go to next.
   (cd $home; [ -e $FILE ] && less $FILE)
 done
 
 #  When script terminates, there is no need to 'cd' back to original directory,
 #+ because 'cd $home' takes place in a subshell.
 
 exit 0

A subshell may be used to set up a "dedicated environment" for a command group.
COMMAND1
 COMMAND2
 COMMAND3
 (
   IFS=:
   PATH=/bin
   unset TERMINFO
   set -C
   shift 5
   COMMAND4
   COMMAND5
   exit 3 # Only exits the subshell.
 )
 # The parent shell has not been affected, and the environment is preserved.
 COMMAND6
 COMMAND7
One application of this is testing whether a variable is defined.
if (set -u; : $variable) 2> /dev/null
 then
   echo "Variable is set."
 fi     #  Variable has been set in current script,
        #+ or is an an internal Bash variable,
        #+ or is present in environment (has been exported).
 
 # Could also be written [[ ${variable-x} != x || ${variable-y} != y ]]
 # or                    [[ ${variable-x} != x$variable ]]
 # or                    [[ ${variable+x} = x ]]
 # or                    [[ ${variable-x} != x ]]
Another application is checking for a lock file:
if (set -C; : > lock_file) 2> /dev/null
 then
   :   # lock_file didn't exist: no user running the script
 else
   echo "Another user is already running that script."
 exit 65
 fi
 
 #  Code snippet by Stephan�Chazelas,
 #+ with modifications by Paulo Marcel Coelho Aragao.

Processes may execute in parallel within different subshells. This permits breaking a complex task into subcomponents processed concurrently.

Example 20-3. Running parallel processes in subshells

	(cat list1 list2 list3 | sort | uniq > list123) &
 	(cat list4 list5 list6 | sort | uniq > list456) &
 	# Merges and sorts both sets of lists simultaneously.
 	# Running in background ensures parallel execution.
 	#
 	# Same effect as
 	#   cat list1 list2 list3 | sort | uniq > list123 &
 	#   cat list4 list5 list6 | sort | uniq > list456 &
 	
 	wait   # Don't execute the next command until subshells finish.
 	
 	diff list123 list456

Redirecting I/O to a subshell uses the "|" pipe operator, as in ls -al | (command).

Note

A command block between curly braces does not launch a subshell.

{ command1; command2; command3; ... }

Complex Functions and Function Complexities

Functions may process arguments passed to them and return an exit status to the script for further processing.

function_name $arg1 $arg2

The function refers to the passed arguments by position (as if they were positional parameters), that is, $1, $2, and so forth.

Example 23-2. Function Taking Parameters

#!/bin/bash
 # Functions and parameters
 
 DEFAULT=default                             # Default param value.
 
 func2 () {
    if [ -z "$1" ]                           # Is parameter #1 zero length?
    then
      echo "-Parameter #1 is zero length.-"  # Or no parameter passed.
    else
      echo "-Param #1 is \"$1\".-"
    fi
 
    variable=${1-$DEFAULT}                   #  What does
    echo "variable = $variable"              #+ parameter substitution show?
                                             #  ---------------------------
                                             #  It distinguishes between
                                             #+ no param and a null param.
 
    if [ "$2" ]
    then
      echo "-Parameter #2 is \"$2\".-"
    fi
 
    return 0
 }
 
 echo
    
 echo "Nothing passed."   
 func2                          # Called with no params
 echo
 
 
 echo "Zero-length parameter passed."
 func2 ""                       # Called with zero-length param
 echo
 
 echo "Null parameter passed."
 func2 "$uninitialized_param"   # Called with uninitialized param
 echo
 
 echo "One parameter passed."   
 func2 first           # Called with one param
 echo
 
 echo "Two parameters passed."   
 func2 first second    # Called with two params
 echo
 
 echo "\"\" \"second\" passed."
 func2 "" second       # Called with zero-length first parameter
 echo                  # and ASCII string as a second one.
 
 exit 0

Important

The shift command works on arguments passed to functions (see Example 33-15).

But, what about command-line arguments passed to the script? Does a function see them? Well, let's clear up the confusion.

Example 23-3. Functions and command-line args passed to the script

#!/bin/bash
 # func-cmdlinearg.sh
 #  Call this script with a command-line argument,
 #+ something like $0 arg1.
 
 
 func ()
 
 {
 echo "$1"
 }
 
 echo "First call to function: no arg passed."
 echo "See if command-line arg is seen."
 func
 # No! Command-line arg not seen.
 
 echo "============================================================"
 echo
 echo "Second call to function: command-line arg passed explicitly."
 func $1
 # Now it's seen!
 
 exit 0

In contrast to certain other programming languages, shell scripts normally pass only value parameters to functions. Variable names (which are actually pointers), if passed as parameters to functions, will be treated as string literals. Functions interpret their arguments literally.

Indirect variable references (see Example 34-2) provide a clumsy sort of mechanism for passing variable pointers to functions.

Example 23-4. Passing an indirect reference to a function

#!/bin/bash
 # ind-func.sh: Passing an indirect reference to a function.
 
 echo_var ()
 {
 echo "$1"
 }
 
 message=Hello
 Hello=Goodbye
 
 echo_var "$message"        # Hello
 # Now, let's pass an indirect reference to the function.
 echo_var "${!message}"     # Goodbye
 
 echo "-------------"
 
 # What happens if we change the contents of "hello" variable?
 Hello="Hello, again!"
 echo_var "$message"        # Hello
 echo_var "${!message}"     # Hello, again!
 
 exit 0

The next logical question is whether parameters can be dereferenced after being passed to a function.

Example 23-5. Dereferencing a parameter passed to a function

#!/bin/bash
 # dereference.sh
 # Dereferencing parameter passed to a function.
 # Script by Bruce W. Clare.
 
 dereference ()
 {
      y=\$"$1"   # Name of variable.
      echo $y    # $Junk
 
      x=`eval "expr \"$y\" "`
      echo $1=$x
      eval "$1=\"Some Different Text \""  # Assign new value.
 }
 
 Junk="Some Text"
 echo $Junk "before"    # Some Text before
 
 dereference Junk
 echo $Junk "after"     # Some Different Text after
 
 exit 0

Example 23-6. Again, dereferencing a parameter passed to a function

#!/bin/bash
 # ref-params.sh: Dereferencing a parameter passed to a function.
 #                (Complex Example)
 
 ITERATIONS=3  # How many times to get input.
 icount=1
 
 my_read () {
   #  Called with my_read varname,
   #+ outputs the previous value between brackets as the default value,
   #+ then asks for a new value.
 
   local local_var
 
   echo -n "Enter a value "
   eval 'echo -n "[$'$1'] "'  #  Previous value.
 # eval echo -n "[\$$1] "     #  Easier to understand,
                              #+ but loses trailing space in user prompt.
   read local_var
   [ -n "$local_var" ] && eval $1=\$local_var
 
   # "And-list": if "local_var" then set "$1" to its value.
 }
 
 echo
 
 while [ "$icount" -le "$ITERATIONS" ]
 do
   my_read var
   echo "Entry #$icount = $var"
   let "icount += 1"
   echo
 done  
 
 
 # Thanks to Stephane Chazelas for providing this instructive example.
 
 exit 0

Exit and Return

exit status

Functions return a value, called an exit status. The exit status may be explicitly specified by a return statement, otherwise it is the exit status of the last command in the function (0 if successful, and a non-zero error code if not). This exit status may be used in the script by referencing it as $?. This mechanism effectively permits script functions to have a "return value" similar to C functions.

return

Terminates a function. A return command [1] optionally takes an integer argument, which is returned to the calling script as the "exit status" of the function, and this exit status is assigned to the variable $?.

Example 23-7. Maximum of two numbers

#!/bin/bash
 # max.sh: Maximum of two integers.
 
 E_PARAM_ERR=-198    # If less than 2 params passed to function.
 EQUAL=-199          # Return value if both params equal.
 
 max2 ()             # Returns larger of two numbers.
 {                   # Note: numbers compared must be less than 257.
 if [ -z "$2" ]
 then
   return $E_PARAM_ERR
 fi
 
 if [ "$1" -eq "$2" ]
 then
   return $EQUAL
 else
   if [ "$1" -gt "$2" ]
   then
     return $1
   else
     return $2
   fi
 fi
 }
 
 max2 33 34
 return_val=$?
 
 if [ "$return_val" -eq $E_PARAM_ERR ]
 then
   echo "Need to pass two parameters to the function."
 elif [ "$return_val" -eq $EQUAL ]
   then
     echo "The two numbers are equal."
 else
     echo "The larger of the two numbers is $return_val."
 fi  
 
   
 exit 0
 
 #  Exercise (easy):
 #  ---------------
 #  Convert this to an interactive script,
 #+ that is, have the script ask for input (two numbers).

Tip

For a function to return a string or array, use a dedicated variable.
count_lines_in_etc_passwd()
 {
   [[ -r /etc/passwd ]] && REPLY=$(echo $(wc -l < /etc/passwd))
   #  If /etc/passwd is readable, set REPLY to line count.
   #  Returns both a parameter value and status information.
   #  The 'echo' seems unnecessary, but . . .
   #+ it removes excess whitespace from the output.
 }
 
 if count_lines_in_etc_passwd
 then
   echo "There are $REPLY lines in /etc/passwd."
 else
   echo "Cannot count lines in /etc/passwd."
 fi  
 
 # Thanks, S.C.

Example 23-8. Converting numbers to Roman numerals

#!/bin/bash
 
 # Arabic number to Roman numeral conversion
 # Range: 0 - 200
 # It's crude, but it works.
 
 # Extending the range and otherwise improving the script is left as an exercise.
 
 # Usage: roman number-to-convert
 
 LIMIT=200
 E_ARG_ERR=65
 E_OUT_OF_RANGE=66
 
 if [ -z "$1" ]
 then
   echo "Usage: `basename $0` number-to-convert"
   exit $E_ARG_ERR
 fi  
 
 num=$1
 if [ "$num" -gt $LIMIT ]
 then
   echo "Out of range!"
   exit $E_OUT_OF_RANGE
 fi  
 
 to_roman ()   # Must declare function before first call to it.
 {
 number=$1
 factor=$2
 rchar=$3
 let "remainder = number - factor"
 while [ "$remainder" -ge 0 ]
 do
   echo -n $rchar
   let "number -= factor"
   let "remainder = number - factor"
 done  
 
 return $number
        # Exercise:
        # --------
        # Explain how this function works.
        # Hint: division by successive subtraction.
 }
    
 
 to_roman $num 100 C
 num=$?
 to_roman $num 90 LXXXX
 num=$?
 to_roman $num 50 L
 num=$?
 to_roman $num 40 XL
 num=$?
 to_roman $num 10 X
 num=$?
 to_roman $num 9 IX
 num=$?
 to_roman $num 5 V
 num=$?
 to_roman $num 4 IV
 num=$?
 to_roman $num 1 I
 
 echo
 
 exit 0

See also Example 10-28.

Important

The largest positive integer a function can return is 255. The return command is closely tied to the concept of exit status, which accounts for this particular limitation. Fortunately, there are various workarounds for those situations requiring a large integer return value from a function.

Example 23-9. Testing large return values in a function

#!/bin/bash
 # return-test.sh
 
 # The largest positive value a function can return is 255.
 
 return_test ()         # Returns whatever passed to it.
 {
   return $1
 }
 
 return_test 27         # o.k.
 echo $?                # Returns 27.
   
 return_test 255        # Still o.k.
 echo $?                # Returns 255.
 
 return_test 257        # Error!
 echo $?                # Returns 1 (return code for miscellaneous error).
 
 # ======================================================
 return_test -151896    # Do large negative numbers work?
 echo $?                # Will this return -151896?
                        # No! It returns 168.
 #  Version of Bash before 2.05b permitted
 #+ large negative integer return values.
 #  Newer versions of Bash plug this loophole.
 #  This may break older scripts.
 #  Caution!
 # ======================================================
 
 exit 0

A workaround for obtaining large integer "return values" is to simply assign the "return value" to a global variable.
Return_Val=   # Global variable to hold oversize return value of function.
 
 alt_return_test ()
 {
   fvar=$1
   Return_Val=$fvar
   return   # Returns 0 (success).
 }
 
 alt_return_test 1
 echo $?                              # 0
 echo "return value = $Return_Val"    # 1
 
 alt_return_test 256
 echo "return value = $Return_Val"    # 256
 
 alt_return_test 257
 echo "return value = $Return_Val"    # 257
 
 alt_return_test 25701
 echo "return value = $Return_Val"    #25701

A more elegant method is to have the function echo its "return value to stdout," and then capture it by command substitution. See the discussion of this in Section 33.7.

Example 23-10. Comparing two large integers

#!/bin/bash
 # max2.sh: Maximum of two LARGE integers.
 
 #  This is the previous "max.sh" example,
 #+ modified to permit comparing large integers.
 
 EQUAL=0             # Return value if both params equal.
 E_PARAM_ERR=-99999  # Not enough params passed to function.
 
 max2 ()             # "Returns" larger of two numbers.
 {
 if [ -z "$2" ]
 then
   echo $E_PARAM_ERR
   return
 fi
 
 if [ "$1" -eq "$2" ]
 then
   echo $EQUAL
   return
 else
   if [ "$1" -gt "$2" ]
   then
     retval=$1
   else
     retval=$2
   fi
 fi
 
 echo $retval        # Echoes (to stdout), rather than returning value.
 
 }
 
 
 return_val=$(max2 33001 33997)
 #            ^^^^             Function name
 #                 ^^^^^ ^^^^^ Params passed
 #  This is actually a form of command substitution:
 #+ treating a function as if it were a command,
 #+ and assigning the stdout of the function to the variable "return_val."
 
 
 # ========================= OUTPUT ========================
 if [ "$return_val" -eq "$E_PARAM_ERR" ]
   then
   echo "Error in parameters passed to comparison function!"
 elif [ "$return_val" -eq "$EQUAL" ]
   then
     echo "The two numbers are equal."
 else
     echo "The larger of the two numbers is $return_val."
 fi
 # =========================================================
   
 exit 0
 
 #  Exercises:
 #  ---------
 #  1) Find a more elegant way of testing
 #+    the parameters passed to the function.
 #  2) Simplify the if/then structure at "OUTPUT."
 #  3) Rewrite the script to take input from command-line parameters.

Here is another example of capturing a function "return value." Understanding it requires some knowledge of awk.
month_length ()  # Takes month number as an argument.
 {                # Returns number of days in month.
 monthD="31 28 31 30 31 30 31 31 30 31 30 31"  # Declare as local?
 echo "$monthD" | awk '{ print $'"${1}"' }'    # Tricky.
 #                             ^^^^^^^^^
 # Parameter passed to function  ($1 -- month number), then to awk.
 # Awk sees this as "print $1 . . . print $12" (depending on month number)
 # Template for passing a parameter to embedded awk script:
 #                                 $'"${script_parameter}"'
 
 #  Needs error checking for correct parameter range (1-12)
 #+ and for February in leap year.
 }
 
 # ----------------------------------------------
 # Usage example:
 month=4        # April, for example (4th month).
 days_in=$(month_length $month)
 echo $days_in  # 30
 # ----------------------------------------------

See also Example A-7.

Exercise: Using what we have just learned, extend the previous Roman numerals example to accept arbitrarily large input.

Redirection

Redirecting the stdin of a function

A function is essentially a code block, which means its stdin can be redirected (as in Example 3-1).

Example 23-11. Real name from username

#!/bin/bash
 # realname.sh
 #
 # From username, gets "real name" from /etc/passwd.
 
 
 ARGCOUNT=1       # Expect one arg.
 E_WRONGARGS=65
 
 file=/etc/passwd
 pattern=$1
 
 if [ $# -ne "$ARGCOUNT" ]
 then
   echo "Usage: `basename $0` USERNAME"
   exit $E_WRONGARGS
 fi  
 
 file_excerpt ()  # Scan file for pattern, then print relevant portion of line.
 {
 while read line  # "while" does not necessarily need "[ condition ]"
 do
   echo "$line" | grep $1 | awk -F":" '{ print $5 }'  # Have awk use ":" delimiter.
 done
 } <$file  # Redirect into function's stdin.
 
 file_excerpt $pattern
 
 # Yes, this entire script could be reduced to
 #       grep PATTERN /etc/passwd | awk -F":" '{ print $5 }'
 # or
 #       awk -F: '/PATTERN/ {print $5}'
 # or
 #       awk -F: '($1 == "username") { print $5 }' # real name from username
 # However, it might not be as instructive.
 
 exit 0

There is an alternate, and perhaps less confusing method of redirecting a function's stdin. This involves redirecting the stdin to an embedded bracketed code block within the function.
# Instead of:
 Function ()
 {
  ...
  } < file
 
 # Try this:
 Function ()
 {
   {
     ...
    } < file
 }
 
 # Similarly,
 
 Function ()  # This works.
 {
   {
    echo $*
   } | tr a b
 }
 
 Function ()  # This doesn't work.
 {
   echo $*
 } | tr a b   # A nested code block is mandatory here.
 
 
 # Thanks, S.C.

Aliases

A Bash alias is essentially nothing more than a keyboard shortcut, an abbreviation, a means of avoiding typing a long command sequence. If, for example, we include alias lm="ls -l | more" in the ~/.bashrc file, then each lm typed at the command line will automatically be replaced by a ls -l | more. This can save a great deal of typing at the command line and avoid having to remember complex combinations of commands and options. Setting alias rm="rm -i" (interactive mode delete) may save a good deal of grief, since it can prevent inadvertently losing important files.

In a script, aliases have very limited usefulness. It would be quite nice if aliases could assume some of the functionality of the C preprocessor, such as macro expansion, but unfortunately Bash does not expand arguments within the alias body. [1] Moreover, a script fails to expand an alias itself within "compound constructs", such as if/then statements, loops, and functions. An added limitation is that an alias will not expand recursively. Almost invariably, whatever we would like an alias to do could be accomplished much more effectively with a function.

Example 24-1. Aliases within a script

#!/bin/bash
 # alias.sh
 
 shopt -s expand_aliases
 # Must set this option, else script will not expand aliases.
 
 
 # First, some fun.
 alias Jesse_James='echo "\"Alias Jesse James\" was a 1959 comedy starring Bob Hope."'
 Jesse_James
 
 echo; echo; echo;
 
 alias ll="ls -l"
 # May use either single (') or double (") quotes to define an alias.
 
 echo "Trying aliased \"ll\":"
 ll /usr/X11R6/bin/mk*   #* Alias works.
 
 echo
 
 directory=/usr/X11R6/bin/
 prefix=mk*  # See if wild card causes problems.
 echo "Variables \"directory\" + \"prefix\" = $directory$prefix"
 echo
 
 alias lll="ls -l $directory$prefix"
 
 echo "Trying aliased \"lll\":"
 lll         # Long listing of all files in /usr/X11R6/bin stating with mk.
 # An alias can handle concatenated variables -- including wild card -- o.k.
 
 
 
 
 TRUE=1
 
 echo
 
 if [ TRUE ]
 then
   alias rr="ls -l"
   echo "Trying aliased \"rr\" within if/then statement:"
   rr /usr/X11R6/bin/mk*   #* Error message results!
   # Aliases not expanded within compound statements.
   echo "However, previously expanded alias still recognized:"
   ll /usr/X11R6/bin/mk*
 fi  
 
 echo
 
 count=0
 while [ $count -lt 3 ]
 do
   alias rrr="ls -l"
   echo "Trying aliased \"rrr\" within \"while\" loop:"
   rrr /usr/X11R6/bin/mk*   #* Alias will not expand here either.
                            #  alias.sh: line 57: rrr: command not found
   let count+=1
 done 
 
 echo; echo
 
 alias xyz='cat $0'   # Script lists itself.
                      # Note strong quotes.
 xyz
 #  This seems to work,
 #+ although the Bash documentation suggests that it shouldn't.
 #
 #  However, as Steve Jacobson points out,
 #+ the "$0" parameter expands immediately upon declaration of the alias.
 
 exit 0

The unalias command removes a previously set alias.

Example 24-2. unalias: Setting and unsetting an alias

#!/bin/bash
 # unalias.sh
 
 shopt -s expand_aliases  # Enables alias expansion.
 
 alias llm='ls -al | more'
 llm
 
 echo
 
 unalias llm              # Unset alias.
 llm
 # Error message results, since 'llm' no longer recognized.
 
 exit 0
bash$ ./unalias.sh
 total 6
 drwxrwxr-x    2 bozo     bozo         3072 Feb  6 14:04 .
 drwxr-xr-x   40 bozo     bozo         2048 Feb  6 14:04 ..
 -rwxr-xr-x    1 bozo     bozo          199 Feb  6 14:04 unalias.sh
 
 ./unalias.sh: llm: command not found

Notes

[1]

However, aliases do seem to expand positional parameters.

List Constructs

The "and list" and "or list" constructs provide a means of processing a number of commands consecutively. These can effectively replace complex nested if/then or even case statements.

Chaining together commands

and list

command-1 && command-2 && command-3 && ... command-n
Each command executes in turn provided that the previous command has given a return value of true (zero). At the first false (non-zero) return, the command chain terminates (the first command returning false is the last one to execute).

Example 25-1. Using an "and list" to test for command-line arguments

#!/bin/bash
 # "and list"
 
 if [ ! -z "$1" ] && echo "Argument #1 = $1" && [ ! -z "$2" ] && echo "Argument #2 = $2"
 then
   echo "At least 2 arguments passed to script."
   # All the chained commands return true.
 else
   echo "Less than 2 arguments passed to script."
   # At least one of the chained commands returns false.
 fi  
 # Note that "if [ ! -z $1 ]" works, but its supposed equivalent,
 #   if [ -n $1 ] does not.
 #     However, quoting fixes this.
 #  if [ -n "$1" ] works.
 #     Careful!
 # It is always best to QUOTE tested variables.
 
 
 # This accomplishes the same thing, using "pure" if/then statements.
 if [ ! -z "$1" ]
 then
   echo "Argument #1 = $1"
 fi
 if [ ! -z "$2" ]
 then
   echo "Argument #2 = $2"
   echo "At least 2 arguments passed to script."
 else
   echo "Less than 2 arguments passed to script."
 fi
 # It's longer and less elegant than using an "and list".
 
 
 exit 0

Example 25-2. Another command-line arg test using an "and list"

#!/bin/bash
 
 ARGS=1        # Number of arguments expected.
 E_BADARGS=65  # Exit value if incorrect number of args passed.
 
 test $# -ne $ARGS && echo "Usage: `basename $0` $ARGS argument(s)" && exit $E_BADARGS
 #  If condition 1 tests true (wrong number of args passed to script),
 #+ then the rest of the line executes, and script terminates.
 
 # Line below executes only if the above test fails.
 echo "Correct number of arguments passed to this script."
 
 exit 0
 
 # To check exit value, do a "echo $?" after script termination.

Of course, an and list can also set variables to a default value.
arg1=$@       # Set $arg1 to command line arguments, if any.
 
 [ -z "$arg1" ] && arg1=DEFAULT
               # Set to DEFAULT if not specified on command line.

or list

command-1 || command-2 || command-3 || ... command-n
Each command executes in turn for as long as the previous command returns false. At the first true return, the command chain terminates (the first command returning true is the last one to execute). This is obviously the inverse of the "and list".

Example 25-3. Using "or lists" in combination with an "and list"

#!/bin/bash
 
 #  delete.sh, not-so-cunning file deletion utility.
 #  Usage: delete filename
 
 E_BADARGS=65
 
 if [ -z "$1" ]
 then
   echo "Usage: `basename $0` filename"
   exit $E_BADARGS  # No arg? Bail out.
 else  
   file=$1          # Set filename.
 fi  
 
 
 [ ! -f "$file" ] && echo "File \"$file\" not found. \
 Cowardly refusing to delete a nonexistent file."
 # AND LIST, to give error message if file not present.
 # Note echo message continued on to a second line with an escape.
 
 [ ! -f "$file" ] || (rm -f $file; echo "File \"$file\" deleted.")
 # OR LIST, to delete file if present.
 
 # Note logic inversion above.
 # AND LIST executes on true, OR LIST on false.
 
 exit 0

Caution

If the first command in an "or list" returns true, it will execute.

# ==> The following snippets from the /etc/rc.d/init.d/single script by Miquel van Smoorenburg
 #+==> illustrate use of "and" and "or" lists.
 # ==> "Arrowed" comments added by document author.
 
 [ -x /usr/bin/clear ] && /usr/bin/clear
   # ==> If /usr/bin/clear exists, then invoke it.
   # ==> Checking for the existence of a command before calling it
   #+==> avoids error messages and other awkward consequences.
 
   # ==> . . .
 
 # If they want to run something in single user mode, might as well run it...
 for i in /etc/rc1.d/S[0-9][0-9]* ; do
         # Check if the script is there.
         [ -x "$i" ] || continue
   # ==> If corresponding file in $PWD *not* found,
   #+==> then "continue" by jumping to the top of the loop.
 
         # Reject backup files and files generated by rpm.
         case "$1" in
                 *.rpmsave|*.rpmorig|*.rpmnew|*~|*.orig)
                         continue;;
         esac
         [ "$i" = "/etc/rc1.d/S00single" ] && continue
   # ==> Set script name, but don't execute it yet.
         $i start
 done
 
   # ==> . . .

Important

The exit status of an and list or an or list is the exit status of the last command executed.

Clever combinations of "and" and "or" lists are possible, but the logic may easily become convoluted and require extensive debugging.
false && true || echo false         # false
 
 # Same result as
 ( false && true ) || echo false     # false
 # But *not*
 false && ( true || echo false )     # (nothing echoed)
 
 #  Note left-to-right grouping and evaluation of statements,
 #+ since the logic operators "&&" and "||" have equal precedence.
 
 #  It's best to avoid such complexities, unless you know what you're doing.
 
 #  Thanks, S.C.

Arrays

Newer versions of Bash support one-dimensional arrays. Array elements may be initialized with the variable[xx] notation. Alternatively, a script may introduce the entire array by an explicit declare -a variable statement. To dereference (find the contents of) an array element, use curly bracket notation, that is, ${variable[xx]}.

Example 26-1. Simple array usage

#!/bin/bash
 
 
 area[11]=23
 area[13]=37
 area[51]=UFOs
 
 #  Array members need not be consecutive or contiguous.
 
 #  Some members of the array can be left uninitialized.
 #  Gaps in the array are okay.
 #  In fact, arrays with sparse data ("sparse arrays")
 #+ are useful in spreadsheet-processing software.
 
 
 echo -n "area[11] = "
 echo ${area[11]}    #  {curly brackets} needed.
 
 echo -n "area[13] = "
 echo ${area[13]}
 
 echo "Contents of area[51] are ${area[51]}."
 
 # Contents of uninitialized array variable print blank (null variable).
 echo -n "area[43] = "
 echo ${area[43]}
 echo "(area[43] unassigned)"
 
 echo
 
 # Sum of two array variables assigned to third
 area[5]=`expr ${area[11]} + ${area[13]}`
 echo "area[5] = area[11] + area[13]"
 echo -n "area[5] = "
 echo ${area[5]}
 
 area[6]=`expr ${area[11]} + ${area[51]}`
 echo "area[6] = area[11] + area[51]"
 echo -n "area[6] = "
 echo ${area[6]}
 # This fails because adding an integer to a string is not permitted.
 
 echo; echo; echo
 
 # -----------------------------------------------------------------
 # Another array, "area2".
 # Another way of assigning array variables...
 # array_name=( XXX YYY ZZZ ... )
 
 area2=( zero one two three four )
 
 echo -n "area2[0] = "
 echo ${area2[0]}
 # Aha, zero-based indexing (first element of array is [0], not [1]).
 
 echo -n "area2[1] = "
 echo ${area2[1]}    # [1] is second element of array.
 # -----------------------------------------------------------------
 
 echo; echo; echo
 
 # -----------------------------------------------
 # Yet another array, "area3".
 # Yet another way of assigning array variables...
 # array_name=([xx]=XXX [yy]=YYY ...)
 
 area3=([17]=seventeen [24]=twenty-four)
 
 echo -n "area3[17] = "
 echo ${area3[17]}
 
 echo -n "area3[24] = "
 echo ${area3[24]}
 # -----------------------------------------------
 
 exit 0

Note

Bash permits array operations on variables, even if the variables are not explicitly declared as arrays.
string=abcABC123ABCabc
 echo ${string[@]}               # abcABC123ABCabc
 echo ${string[*]}               # abcABC123ABCabc 
 echo ${string[0]}               # abcABC123ABCabc
 echo ${string[1]}               # No output!
                                 # Why?
 echo ${#string[@]}              # 1
                                 # One element in the array.
                                 # The string itself.
 
 # Thank you, Michael Zick, for pointing this out.
Once again this demonstrates that Bash variables are untyped.

Example 26-2. Formatting a poem

#!/bin/bash
 # poem.sh: Pretty-prints one of the document author's favorite poems.
 
 # Lines of the poem (single stanza).
 Line[1]="I do not know which to prefer,"
 Line[2]="The beauty of inflections"
 Line[3]="Or the beauty of innuendoes,"
 Line[4]="The blackbird whistling"
 Line[5]="Or just after."
 
 # Attribution.
 Attrib[1]=" Wallace Stevens"
 Attrib[2]="\"Thirteen Ways of Looking at a Blackbird\""
 # This poem is in the Public Domain (copyright expired).
 
 echo
 
 for index in 1 2 3 4 5    # Five lines.
 do
   printf "     %s\n" "${Line[index]}"
 done
 
 for index in 1 2          # Two attribution lines.
 do
   printf "          %s\n" "${Attrib[index]}"
 done
 
 echo
 
 exit 0
 
 # Exercise:
 # --------
 # Modify this script to pretty-print a poem from a text data file.

Array variables have a syntax all their own, and even standard Bash commands and operators have special options adapted for array use.

Example 26-3. Various array operations

#!/bin/bash
 # array-ops.sh: More fun with arrays.
 
 
 array=( zero one two three four five )
 # Element 0   1   2    3     4    5
 
 echo ${array[0]}       #  zero
 echo ${array:0}        #  zero
                        #  Parameter expansion of first element,
                        #+ starting at position # 0 (1st character).
 echo ${array:1}        #  ero
                        #  Parameter expansion of first element,
                        #+ starting at position # 1 (2nd character).
 
 echo "--------------"
 
 echo ${#array[0]}      #  4
                        #  Length of first element of array.
 echo ${#array}         #  4
                        #  Length of first element of array.
                        #  (Alternate notation)
 
 echo ${#array[1]}      #  3
                        #  Length of second element of array.
                        #  Arrays in Bash have zero-based indexing.
 
 echo ${#array[*]}      #  6
                        #  Number of elements in array.
 echo ${#array[@]}      #  6
                        #  Number of elements in array.
 
 echo "--------------"
 
 array2=( [0]="first element" [1]="second element" [3]="fourth element" )
 
 echo ${array2[0]}      # first element
 echo ${array2[1]}      # second element
 echo ${array2[2]}      #
                        # Skipped in initialization, and therefore null.
 echo ${array2[3]}      # fourth element
 
 
 exit 0

Many of the standard string operations work on arrays.

Example 26-4. String operations on arrays

#!/bin/bash
 # array-strops.sh: String operations on arrays.
 # Script by Michael Zick.
 # Used with permission.
 
 #  In general, any string operation in the ${name ... } notation
 #+ can be applied to all string elements in an array
 #+ with the ${name[@] ... } or ${name[*] ...} notation.
 
 
 arrayZ=( one two three four five five )
 
 echo
 
 # Trailing Substring Extraction
 echo ${arrayZ[@]:0}     # one two three four five five
                         # All elements.
 
 echo ${arrayZ[@]:1}     # two three four five five
                         # All elements following element[0].
 
 echo ${arrayZ[@]:1:2}   # two three
                         # Only the two elements after element[0].
 
 echo "-----------------------"
 
 #  Substring Removal
 #  Removes shortest match from front of string(s),
 #+ where the substring is a regular expression.
 
 echo ${arrayZ[@]#f*r}   # one two three five five
                         # Applied to all elements of the array.
                         # Matches "four" and removes it.
 
 # Longest match from front of string(s)
 echo ${arrayZ[@]##t*e}  # one two four five five
                         # Applied to all elements of the array.
                         # Matches "three" and removes it.
 
 # Shortest match from back of string(s)
 echo ${arrayZ[@]%h*e}   # one two t four five five
                         # Applied to all elements of the array.
                         # Matches "hree" and removes it.
 
 # Longest match from back of string(s)
 echo ${arrayZ[@]%%t*e}  # one two four five five
                         # Applied to all elements of the array.
                         # Matches "three" and removes it.
 
 echo "-----------------------"
 
 # Substring Replacement
 
 # Replace first occurance of substring with replacement
 echo ${arrayZ[@]/fiv/XYZ}   # one two three four XYZe XYZe
                             # Applied to all elements of the array.
 
 # Replace all occurances of substring
 echo ${arrayZ[@]//iv/YY}    # one two three four fYYe fYYe
                             # Applied to all elements of the array.
 
 # Delete all occurances of substring
 # Not specifing a replacement means 'delete'
 echo ${arrayZ[@]//fi/}      # one two three four ve ve
                             # Applied to all elements of the array.
 
 # Replace front-end occurances of substring
 echo ${arrayZ[@]/#fi/XY}    # one two three four XYve XYve
                             # Applied to all elements of the array.
 
 # Replace back-end occurances of substring
 echo ${arrayZ[@]/%ve/ZZ}    # one two three four fiZZ fiZZ
                             # Applied to all elements of the array.
 
 echo ${arrayZ[@]/%o/XX}     # one twXX three four five five
                             # Why?
 
 echo "-----------------------"
 
 
 # Before reaching for awk (or anything else) --
 # Recall:
 #   $( ... ) is command substitution.
 #   Functions run as a sub-process.
 #   Functions write their output to stdout.
 #   Assignment reads the function's stdout.
 #   The name[@] notation specifies a "for-each" operation.
 
 newstr() {
     echo -n "!!!"
 }
 
 echo ${arrayZ[@]/%e/$(newstr)}
 # on!!! two thre!!! four fiv!!! fiv!!!
 # Q.E.D: The replacement action is an 'assignment.'
 
 #  Accessing the "For-Each"
 echo ${arrayZ[@]//*/$(newstr optional_arguments)}
 #  Now, if Bash would just pass the matched string as $0
 #+ to the function being called . . .
 
 echo
 
 exit 0

Command substitution can construct the individual elements of an array.

Example 26-5. Loading the contents of a script into an array

#!/bin/bash
 # script-array.sh: Loads this script into an array.
 # Inspired by an e-mail from Chris Martin (thanks!).
 
 script_contents=( $(cat "$0") )  #  Stores contents of this script ($0)
                                  #+ in an array.
 
 for element in $(seq 0 $((${#script_contents[@]} - 1)))
   do                #  ${#script_contents[@]}
                     #+ gives number of elements in the array.
                     #
                     #  Question:
                     #  Why is  seq 0  necessary?
                     #  Try changing it to seq 1.
   echo -n "${script_contents[$element]}"
                     # List each field of this script on a single line.
   echo -n " -- "    # Use " -- " as a field separator.
 done
 
 echo
 
 exit 0
 
 # Exercise:
 # --------
 #  Modify this script so it lists itself
 #+ in its original format,
 #+ complete with whitespace, line breaks, etc.

In an array context, some Bash builtins have a slightly altered meaning. For example, unset deletes array elements, or even an entire array.

Example 26-6. Some special properties of arrays

#!/bin/bash
 
 declare -a colors
 #  All subsequent commands in this script will treat
 #+ the variable "colors" as an array.
 
 echo "Enter your favorite colors (separated from each other by a space)."
 
 read -a colors    # Enter at least 3 colors to demonstrate features below.
 #  Special option to 'read' command,
 #+ allowing assignment of elements in an array.
 
 echo
 
 element_count=${#colors[@]}
 # Special syntax to extract number of elements in array.
 #     element_count=${#colors[*]} works also.
 #
 #  The "@" variable allows word splitting within quotes
 #+ (extracts variables separated by whitespace).
 #
 #  This corresponds to the behavior of "$@" and "$*"
 #+ in positional parameters. 
 
 index=0
 
 while [ "$index" -lt "$element_count" ]
 do    # List all the elements in the array.
   echo ${colors[$index]}
   let "index = $index + 1"
 done
 # Each array element listed on a separate line.
 # If this is not desired, use  echo -n "${colors[$index]} "
 #
 # Doing it with a "for" loop instead:
 #   for i in "${colors[@]}"
 #   do
 #     echo "$i"
 #   done
 # (Thanks, S.C.)
 
 echo
 
 # Again, list all the elements in the array, but using a more elegant method.
   echo ${colors[@]}          # echo ${colors[*]} also works.
 
 echo
 
 # The "unset" command deletes elements of an array, or entire array.
 unset colors[1]              # Remove 2nd element of array.
                              # Same effect as   colors[1]=
 echo  ${colors[@]}           # List array again, missing 2nd element.
 
 unset colors                 # Delete entire array.
                              #  unset colors[*] and
                              #+ unset colors[@] also work.
 echo; echo -n "Colors gone."			   
 echo ${colors[@]}            # List array again, now empty.
 
 exit 0

As seen in the previous example, either ${array_name[@]} or ${array_name[*]} refers to all the elements of the array. Similarly, to get a count of the number of elements in an array, use either ${#array_name[@]} or ${#array_name[*]}. ${#array_name} is the length (number of characters) of ${array_name[0]}, the first element of the array.

Example 26-7. Of empty arrays and empty elements

#!/bin/bash
 # empty-array.sh
 
 #  Thanks to Stephane Chazelas for the original example,
 #+ and to Michael Zick for extending it.
 
 
 # An empty array is not the same as an array with empty elements.
 
 array0=( first second third )
 array1=( '' )   # "array1" consists of one empty element.
 array2=( )      # No elements . . . "array2" is empty.
 
 echo
 ListArray()
 {
 echo
 echo "Elements in array0:  ${array0[@]}"
 echo "Elements in array1:  ${array1[@]}"
 echo "Elements in array2:  ${array2[@]}"
 echo
 echo "Length of first element in array0 = ${#array0}"
 echo "Length of first element in array1 = ${#array1}"
 echo "Length of first element in array2 = ${#array2}"
 echo
 echo "Number of elements in array0 = ${#array0[*]}"  # 3
 echo "Number of elements in array1 = ${#array1[*]}"  # 1  (Surprise!)
 echo "Number of elements in array2 = ${#array2[*]}"  # 0
 }
 
 # ===================================================================
 
 ListArray
 
 # Try extending those arrays.
 
 # Adding an element to an array.
 array0=( "${array0[@]}" "new1" )
 array1=( "${array1[@]}" "new1" )
 array2=( "${array2[@]}" "new1" )
 
 ListArray
 
 # or
 array0[${#array0[*]}]="new2"
 array1[${#array1[*]}]="new2"
 array2[${#array2[*]}]="new2"
 
 ListArray
 
 # When extended as above; arrays are 'stacks'
 # The above is the 'push'
 # The stack 'height' is:
 height=${#array2[@]}
 echo
 echo "Stack height for array2 = $height"
 
 # The 'pop' is:
 unset array2[${#array2[@]}-1]	#  Arrays are zero-based,
 height=${#array2[@]}            #+ which means first element has index 0.
 echo
 echo "POP"
 echo "New stack height for array2 = $height"
 
 ListArray
 
 # List only 2nd and 3rd elements of array0.
 from=1		# Zero-based numbering.
 to=2		#
 array3=( ${array0[@]:1:2} )
 echo
 echo "Elements in array3:  ${array3[@]}"
 
 # Works like a string (array of characters).
 # Try some other "string" forms.
 
 # Replacement:
 array4=( ${array0[@]/second/2nd} )
 echo
 echo "Elements in array4:  ${array4[@]}"
 
 # Replace all matching wildcarded string.
 array5=( ${array0[@]//new?/old} )
 echo
 echo "Elements in array5:  ${array5[@]}"
 
 # Just when you are getting the feel for this . . .
 array6=( ${array0[@]#*new} )
 echo # This one might surprise you.
 echo "Elements in array6:  ${array6[@]}"
 
 array7=( ${array0[@]#new1} )
 echo # After array6 this should not be a surprise.
 echo "Elements in array7:  ${array7[@]}"
 
 # Which looks a lot like . . .
 array8=( ${array0[@]/new1/} )
 echo
 echo "Elements in array8:  ${array8[@]}"
 
 #  So what can one say about this?
 
 #  The string operations are performed on
 #+ each of the elements in var[@] in succession.
 #  Therefore : Bash supports string vector operations
 #+ if the result is a zero length string,
 #+ that element disappears in the resulting assignment.
 
 #  Question, are those strings hard or soft quotes?
 
 zap='new*'
 array9=( ${array0[@]/$zap/} )
 echo
 echo "Elements in array9:  ${array9[@]}"
 
 # Just when you thought you where still in Kansas . . .
 array10=( ${array0[@]#$zap} )
 echo
 echo "Elements in array10:  ${array10[@]}"
 
 # Compare array7 with array10.
 # Compare array8 with array9.
 
 # Answer: must be soft quotes.
 
 exit 0

The relationship of ${array_name[@]} and ${array_name[*]} is analogous to that between $@ and $*. This powerful array notation has a number of uses.

# Copying an array.
 array2=( "${array1[@]}" )
 # or
 array2="${array1[@]}"
 
 # Adding an element to an array.
 array=( "${array[@]}" "new element" )
 # or
 array[${#array[*]}]="new element"
 
 # Thanks, S.C.

Tip

The array=( element1 element2 ... elementN ) initialization operation, with the help of command substitution, makes it possible to load the contents of a text file into an array.

#!/bin/bash
 
 filename=sample_file
 
 #            cat sample_file
 #
 #            1 a b c
 #            2 d e fg
 
 
 declare -a array1
 
 array1=( `cat "$filename"`)                #  Loads contents
 #         List file to stdout              #+ of $filename into array1.
 #
 #  array1=( `cat "$filename" | tr '\n' ' '`)
 #                            change linefeeds in file to spaces. 
 #  Not necessary because Bash does word splitting,
 #+ changing linefeeds to spaces.
 
 echo ${array1[@]}            # List the array.
 #                              1 a b c 2 d e fg
 #
 #  Each whitespace-separated "word" in the file
 #+ has been assigned to an element of the array.
 
 element_count=${#array1[*]}
 echo $element_count          # 8

Clever scripting makes it possible to add array operations.

Example 26-8. Initializing arrays

#! /bin/bash
 # array-assign.bash
 
 #  Array operations are Bash specific,
 #+ hence the ".bash" in the script name.
 
 # Copyright (c) Michael S. Zick, 2003, All rights reserved.
 # License: Unrestricted reuse in any form, for any purpose.
 # Version: $ID$
 #
 # Clarification and additional comments by William Park.
 
 #  Based on an example provided by Stephane Chazelas
 #+ which appeared in the book: Advanced Bash Scripting Guide.
 
 # Output format of the 'times' command:
 # User CPU <space> System CPU
 # User CPU of dead children <space> System CPU of dead children
 
 #  Bash has two versions of assigning all elements of an array
 #+ to a new array variable.
 #  Both drop 'null reference' elements
 #+ in Bash versions 2.04, 2.05a and 2.05b.
 #  An additional array assignment that maintains the relationship of
 #+ [subscript]=value for arrays may be added to newer versions.
 
 #  Constructs a large array using an internal command,
 #+ but anything creating an array of several thousand elements
 #+ will do just fine.
 
 declare -a bigOne=( /dev/* )
 echo
 echo 'Conditions: Unquoted, default IFS, All-Elements-Of'
 echo "Number of elements in array is ${#bigOne[@]}"
 
 # set -vx
 
 
 
 echo
 echo '- - testing: =( ${array[@]} ) - -'
 times
 declare -a bigTwo=( ${bigOne[@]} )
 #                 ^              ^
 times
 
 echo
 echo '- - testing: =${array[@]} - -'
 times
 declare -a bigThree=${bigOne[@]}
 # No parentheses this time.
 times
 
 #  Comparing the numbers shows that the second form, pointed out
 #+ by Stephane Chazelas, is from three to four times faster.
 #
 #  William Park explains:
 #+ The bigTwo array assigned as single string, whereas
 #+ bigThree assigned element by element.
 #  So, in essence, you have:
 #                   bigTwo=( [0]="... ... ..." )
 #                   bigThree=( [0]="..." [1]="..." [2]="..." ... )
 
 
 #  I will continue to use the first form in my example descriptions
 #+ because I think it is a better illustration of what is happening.
 
 #  The reusable portions of my examples will actual contain
 #+ the second form where appropriate because of the speedup.
 
 # MSZ: Sorry about that earlier oversight folks.
 
 
 #  Note:
 #  ----
 #  The "declare -a" statements in lines 31 and 43
 #+ are not strictly necessary, since it is implicit
 #+ in the  Array=( ... )  assignment form.
 #  However, eliminating these declarations slows down
 #+ the execution of the following sections of the script.
 #  Try it, and see what happens.
 
 exit 0

Note

Adding a superfluous declare -a statement to an array declaration may speed up execution of subsequent operations on the array.

Example 26-9. Copying and concatenating arrays

#! /bin/bash
 # CopyArray.sh
 #
 # This script written by Michael Zick.
 # Used here with permission.
 
 #  How-To "Pass by Name & Return by Name"
 #+ or "Building your own assignment statement".
 
 
 CpArray_Mac() {
 
 # Assignment Command Statement Builder
 
     echo -n 'eval '
     echo -n "$2"                    # Destination name
     echo -n '=( ${'
     echo -n "$1"                    # Source name
     echo -n '[@]} )'
 
 # That could all be a single command.
 # Matter of style only.
 }
 
 declare -f CopyArray                # Function "Pointer"
 CopyArray=CpArray_Mac               # Statement Builder
 
 Hype()
 {
 
 # Hype the array named $1.
 # (Splice it together with array containing "Really Rocks".)
 # Return in array named $2.
 
     local -a TMP
     local -a hype=( Really Rocks )
 
     $($CopyArray $1 TMP)
     TMP=( ${TMP[@]} ${hype[@]} )
     $($CopyArray TMP $2)
 }
 
 declare -a before=( Advanced Bash Scripting )
 declare -a after
 
 echo "Array Before = ${before[@]}"
 
 Hype before after
 
 echo "Array After = ${after[@]}"
 
 # Too much hype?
 
 echo "What ${after[@]:3:2}?"
 
 declare -a modest=( ${after[@]:2:1} ${after[@]:3:2} )
 #                    ---- substring extraction ----
 
 echo "Array Modest = ${modest[@]}"
 
 # What happened to 'before' ?
 
 echo "Array Before = ${before[@]}"
 
 exit 0

Example 26-10. More on concatenating arrays

#! /bin/bash
 # array-append.bash
 
 # Copyright (c) Michael S. Zick, 2003, All rights reserved.
 # License: Unrestricted reuse in any form, for any purpose.
 # Version: $ID$
 #
 # Slightly modified in formatting by M.C.
 
 
 # Array operations are Bash-specific.
 # Legacy UNIX /bin/sh lacks equivalents.
 
 
 #  Pipe the output of this script to 'more'
 #+ so it doesn't scroll off the terminal.
 
 
 # Subscript packed.
 declare -a array1=( zero1 one1 two1 )
 # Subscript sparse ([1] is not defined).
 declare -a array2=( [0]=zero2 [2]=two2 [3]=three2 )
 
 echo
 echo '- Confirm that the array is really subscript sparse. -'
 echo "Number of elements: 4"        # Hard-coded for illustration.
 for (( i = 0 ; i < 4 ; i++ ))
 do
     echo "Element [$i]: ${array2[$i]}"
 done
 # See also the more general code example in basics-reviewed.bash.
 
 
 declare -a dest
 
 # Combine (append) two arrays into a third array.
 echo
 echo 'Conditions: Unquoted, default IFS, All-Elements-Of operator'
 echo '- Undefined elements not present, subscripts not maintained. -'
 # # The undefined elements do not exist; they are not being dropped.
 
 dest=( ${array1[@]} ${array2[@]} )
 # dest=${array1[@]}${array2[@]}     # Strange results, possibly a bug.
 
 # Now, list the result.
 echo
 echo '- - Testing Array Append - -'
 cnt=${#dest[@]}
 
 echo "Number of elements: $cnt"
 for (( i = 0 ; i < cnt ; i++ ))
 do
     echo "Element [$i]: ${dest[$i]}"
 done
 
 # Assign an array to a single array element (twice).
 dest[0]=${array1[@]}
 dest[1]=${array2[@]}
 
 # List the result.
 echo
 echo '- - Testing modified array - -'
 cnt=${#dest[@]}
 
 echo "Number of elements: $cnt"
 for (( i = 0 ; i < cnt ; i++ ))
 do
     echo "Element [$i]: ${dest[$i]}"
 done
 
 # Examine the modified second element.
 echo
 echo '- - Reassign and list second element - -'
 
 declare -a subArray=${dest[1]}
 cnt=${#subArray[@]}
 
 echo "Number of elements: $cnt"
 for (( i = 0 ; i < cnt ; i++ ))
 do
     echo "Element [$i]: ${subArray[$i]}"
 done
 
 #  The assignment of an entire array to a single element
 #+ of another array using the '=${ ... }' array assignment
 #+ has converted the array being assigned into a string,
 #+ with the elements separated by a space (the first character of IFS).
 
 # If the original elements didn't contain whitespace . . .
 # If the original array isn't subscript sparse . . .
 # Then we could get the original array structure back again.
 
 # Restore from the modified second element.
 echo
 echo '- - Listing restored element - -'
 
 declare -a subArray=( ${dest[1]} )
 cnt=${#subArray[@]}
 
 echo "Number of elements: $cnt"
 for (( i = 0 ; i < cnt ; i++ ))
 do
     echo "Element [$i]: ${subArray[$i]}"
 done
 echo '- - Do not depend on this behavior. - -'
 echo '- - This behavior is subject to change - -'
 echo '- - in versions of Bash newer than version 2.05b - -'
 
 # MSZ: Sorry about any earlier confusion folks.
 
 exit 0

--

Arrays permit deploying old familiar algorithms as shell scripts. Whether this is necessarily a good idea is left to the reader to decide.

Example 26-11. An old friend: The Bubble Sort

#!/bin/bash
 # bubble.sh: Bubble sort, of sorts.
 
 # Recall the algorithm for a bubble sort. In this particular version...
 
 #  With each successive pass through the array to be sorted,
 #+ compare two adjacent elements, and swap them if out of order.
 #  At the end of the first pass, the "heaviest" element has sunk to bottom.
 #  At the end of the second pass, the next "heaviest" one has sunk next to bottom.
 #  And so forth.
 #  This means that each successive pass needs to traverse less of the array.
 #  You will therefore notice a speeding up in the printing of the later passes.
 
 
 exchange()
 {
   # Swaps two members of the array.
   local temp=${Countries[$1]} #  Temporary storage
                               #+ for element getting swapped out.
   Countries[$1]=${Countries[$2]}
   Countries[$2]=$temp
   
   return
 }  
 
 declare -a Countries  #  Declare array,
                       #+ optional here since it's initialized below.
 
 #  Is it permissable to split an array variable over multiple lines
 #+ using an escape (\)?
 #  Yes.
 
 Countries=(Netherlands Ukraine Zaire Turkey Russia Yemen Syria \
 Brazil Argentina Nicaragua Japan Mexico Venezuela Greece England \
 Israel Peru Canada Oman Denmark Wales France Kenya \
 Xanadu Qatar Liechtenstein Hungary)
 
 # "Xanadu" is the mythical place where, according to Coleridge,
 #+ Kubla Khan did a pleasure dome decree.
 
 
 clear                      # Clear the screen to start with. 
 
 echo "0: ${Countries[*]}"  # List entire array at pass 0.
 
 number_of_elements=${#Countries[@]}
 let "comparisons = $number_of_elements - 1"
 
 count=1 # Pass number.
 
 while [ "$comparisons" -gt 0 ]          # Beginning of outer loop
 do
 
   index=0  # Reset index to start of array after each pass.
 
   while [ "$index" -lt "$comparisons" ] # Beginning of inner loop
   do
     if [ ${Countries[$index]} \> ${Countries[`expr $index + 1`]} ]
     #  If out of order...
     #  Recalling that \> is ASCII comparison operator
     #+ within single brackets.
 
     #  if [[ ${Countries[$index]} > ${Countries[`expr $index + 1`]} ]]
     #+ also works.
     then
       exchange $index `expr $index + 1`  # Swap.
     fi  
     let "index += 1"
   done # End of inner loop
 
 # ----------------------------------------------------------------------
 # Paulo Marcel Coelho Aragao suggests for-loops as a simpler altenative.
 #
 # for (( last = $number_of_elements - 1 ; last > 1 ; last-- ))
 # do
 #     for (( i = 0 ; i < last ; i++ ))
 #     do
 #         [[ "${Countries[$i]}" > "${Countries[$((i+1))]}" ]] \
 #             && exchange $i $((i+1))
 #     done
 # done
 # ----------------------------------------------------------------------
   
 
 let "comparisons -= 1" #  Since "heaviest" element bubbles to bottom,
                        #+ we need do one less comparison each pass.
 
 echo
 echo "$count: ${Countries[@]}"  # Print resultant array at end of each pass.
 echo
 let "count += 1"                # Increment pass count.
 
 done                            # End of outer loop
                                 # All done.
 
 exit 0

--

Is it possible to nest arrays within arrays?

#!/bin/bash
 # "Nested" array.
 
 #  Michael Zick provided this example,
 #+ with corrections and clarifications by William Park.
 
 AnArray=( $(ls --inode --ignore-backups --almost-all \
 	--directory --full-time --color=none --time=status \
 	--sort=time -l ${PWD} ) )  # Commands and options.
 
 # Spaces are significant . . . and don't quote anything in the above.
 
 SubArray=( ${AnArray[@]:11:1}  ${AnArray[@]:6:5} )
 #  This array has six elements:
 #+     SubArray=( [0]=${AnArray[11]} [1]=${AnArray[6]} [2]=${AnArray[7]}
 #      [3]=${AnArray[8]} [4]=${AnArray[9]} [5]=${AnArray[10]} )
 #
 #  Arrays in Bash are (circularly) linked lists
 #+ of type string (char *).
 #  So, this isn't actually a nested array,
 #+ but it's functionally similar.
 
 echo "Current directory and date of last status change:"
 echo "${SubArray[@]}"
 
 exit 0

--

Embedded arrays in combination with indirect references create some fascinating possibilities

Example 26-12. Embedded arrays and indirect references

#!/bin/bash
 # embedded-arrays.sh
 # Embedded arrays and indirect references.
 
 # This script by Dennis Leeuw.
 # Used with permission.
 # Modified by document author.
 
 
 ARRAY1=(
         VAR1_1=value11
         VAR1_2=value12
         VAR1_3=value13
 )
 
 ARRAY2=(
         VARIABLE="test"
         STRING="VAR1=value1 VAR2=value2 VAR3=value3"
         ARRAY21=${ARRAY1[*]}
 )       # Embed ARRAY1 within this second array.
 
 function print () {
         OLD_IFS="$IFS"
         IFS=$'\n'       #  To print each array element
                         #+ on a separate line.
         TEST1="ARRAY2[*]"
         local ${!TEST1} # See what happens if you delete this line.
         #  Indirect reference.
 	#  This makes the components of $TEST1
 	#+ accessible to this function.
 
 
         #  Let's see what we've got so far.
         echo
         echo "\$TEST1 = $TEST1"       #  Just the name of the variable.
         echo; echo
         echo "{\$TEST1} = ${!TEST1}"  #  Contents of the variable.
                                       #  That's what an indirect
                                       #+ reference does.
         echo
         echo "-------------------------------------------"; echo
         echo
 
 
         # Print variable
         echo "Variable VARIABLE: $VARIABLE"
 	
         # Print a string element
         IFS="$OLD_IFS"
         TEST2="STRING[*]"
         local ${!TEST2}      # Indirect reference (as above).
         echo "String element VAR2: $VAR2 from STRING"
 
         # Print an array element
         TEST2="ARRAY21[*]"
         local ${!TEST2}      # Indirect reference (as above).
         echo "Array element VAR1_1: $VAR1_1 from ARRAY21"
 }
 
 print
 echo
 
 exit 0
 
 #   As the author of the script notes,
 #+ "you can easily expand it to create named-hashes in bash."
 #   (Difficult) exercise for the reader: implement this.

--

Arrays enable implementing a shell script version of the Sieve of Eratosthenes. Of course, a resource-intensive application of this nature should really be written in a compiled language, such as C. It runs excruciatingly slowly as a script.

Example 26-13. Complex array application: Sieve of Eratosthenes

#!/bin/bash
 # sieve.sh (ex68.sh)
 
 # Sieve of Eratosthenes
 # Ancient algorithm for finding prime numbers.
 
 #  This runs a couple of orders of magnitude slower
 #+ than the equivalent program written in C.
 
 LOWER_LIMIT=1       # Starting with 1.
 UPPER_LIMIT=1000    # Up to 1000.
 # (You may set this higher . . . if you have time on your hands.)
 
 PRIME=1
 NON_PRIME=0
 
 let SPLIT=UPPER_LIMIT/2
 # Optimization:
 # Need to test numbers only halfway to upper limit (why?).
 
 
 declare -a Primes
 # Primes[] is an array.
 
 
 initialize ()
 {
 # Initialize the array.
 
 i=$LOWER_LIMIT
 until [ "$i" -gt "$UPPER_LIMIT" ]
 do
   Primes[i]=$PRIME
   let "i += 1"
 done
 #  Assume all array members guilty (prime)
 #+ until proven innocent.
 }
 
 print_primes ()
 {
 # Print out the members of the Primes[] array tagged as prime.
 
 i=$LOWER_LIMIT
 
 until [ "$i" -gt "$UPPER_LIMIT" ]
 do
 
   if [ "${Primes[i]}" -eq "$PRIME" ]
   then
     printf "%8d" $i
     # 8 spaces per number gives nice, even columns.
   fi
   
   let "i += 1"
   
 done
 
 }
 
 sift () # Sift out the non-primes.
 {
 
 let i=$LOWER_LIMIT+1
 # We know 1 is prime, so let's start with 2.
 
 until [ "$i" -gt "$UPPER_LIMIT" ]
 do
 
 if [ "${Primes[i]}" -eq "$PRIME" ]
 # Don't bother sieving numbers already sieved (tagged as non-prime).
 then
 
   t=$i
 
   while [ "$t" -le "$UPPER_LIMIT" ]
   do
     let "t += $i "
     Primes[t]=$NON_PRIME
     # Tag as non-prime all multiples.
   done
 
 fi  
 
   let "i += 1"
 done  
 
 
 }
 
 
 # ==============================================
 # main ()
 # Invoke the functions sequentially.
 initialize
 sift
 print_primes
 # This is what they call structured programming.
 # ==============================================
 
 echo
 
 exit 0
 
 
 
 # -------------------------------------------------------- #
 # Code below line will not execute, because of 'exit.'
 
 #  This improved version of the Sieve, by Stephane Chazelas,
 #+ executes somewhat faster.
 
 # Must invoke with command-line argument (limit of primes).
 
 UPPER_LIMIT=$1                  # From command line.
 let SPLIT=UPPER_LIMIT/2         # Halfway to max number.
 
 Primes=( '' $(seq $UPPER_LIMIT) )
 
 i=1
 until (( ( i += 1 ) > SPLIT ))  # Need check only halfway.
 do
   if [[ -n $Primes[i] ]]
   then
     t=$i
     until (( ( t += i ) > UPPER_LIMIT ))
     do
       Primes[t]=
     done
   fi  
 done  
 echo ${Primes[*]}
 
 exit 0

Compare this array-based prime number generator with an alternative that does not use arrays, Example A-16.

--

Arrays lend themselves, to some extent, to emulating data structures for which Bash has no native support.

Example 26-14. Emulating a push-down stack

#!/bin/bash
 # stack.sh: push-down stack simulation
 
 #  Similar to the CPU stack, a push-down stack stores data items
 #+ sequentially, but releases them in reverse order, last-in first-out.
 
 BP=100            #  Base Pointer of stack array.
                   #  Begin at element 100.
 
 SP=$BP            #  Stack Pointer.
                   #  Initialize it to "base" (bottom) of stack.
 
 Data=             #  Contents of stack location.  
                   #  Must use global variable,
                   #+ because of limitation on function return range.
 
 declare -a stack
 
 
 push()            # Push item on stack.
 {
 if [ -z "$1" ]    # Nothing to push?
 then
   return
 fi
 
 let "SP -= 1"     # Bump stack pointer.
 stack[$SP]=$1
 
 return
 }
 
 pop()                    # Pop item off stack.
 {
 Data=                    # Empty out data item.
 
 if [ "$SP" -eq "$BP" ]   # Stack empty?
 then
   return
 fi                       #  This also keeps SP from getting past 100,
                          #+ i.e., prevents a runaway stack.
 
 Data=${stack[$SP]}
 let "SP += 1"            # Bump stack pointer.
 return
 }
 
 status_report()          # Find out what's happening.
 {
 echo "-------------------------------------"
 echo "REPORT"
 echo "Stack Pointer = $SP"
 echo "Just popped \""$Data"\" off the stack."
 echo "-------------------------------------"
 echo
 }
 
 
 # =======================================================
 # Now, for some fun.
 
 echo
 
 # See if you can pop anything off empty stack.
 pop
 status_report
 
 echo
 
 push garbage
 pop
 status_report     # Garbage in, garbage out.      
 
 value1=23; push $value1
 value2=skidoo; push $value2
 value3=FINAL; push $value3
 
 pop              # FINAL
 status_report
 pop              # skidoo
 status_report
 pop              # 23
 status_report    # Last-in, first-out!
 
 #  Notice how the stack pointer decrements with each push,
 #+ and increments with each pop.
 
 echo
 
 exit 0
 
 # =======================================================
 
 
 # Exercises:
 # ---------
 
 # 1)  Modify the "push()" function to permit pushing
 #   + multiple element on the stack with a single function call.
 
 # 2)  Modify the "pop()" function to permit popping
 #   + multiple element from the stack with a single function call.
 
 # 3)  Add error checking to the critical functions.
 #     That is, return an error code, depending on
 #   + successful or unsuccessful completion of the operation,
 #   + and take appropriate action.
 
 # 4)  Using this script as a starting point,
 #   + write a stack-based 4-function calculator.

--

Fancy manipulation of array "subscripts" may require intermediate variables. For projects involving this, again consider using a more powerful programming language, such as Perl or C.

Example 26-15. Complex array application: Exploring a weird mathematical series

#!/bin/bash
 
 # Douglas Hofstadter's notorious "Q-series":
 
 # Q(1) = Q(2) = 1
 # Q(n) = Q(n - Q(n-1)) + Q(n - Q(n-2)), for n>2
 
 # This is a "chaotic" integer series with strange and unpredictable behavior.
 # The first 20 terms of the series are:
 # 1 1 2 3 3 4 5 5 6 6 6 8 8 8 10 9 10 11 11 12 
 
 #  See Hofstadter's book, "Goedel, Escher, Bach: An Eternal Golden Braid",
 #+ p. 137, ff.
 
 
 LIMIT=100     # Number of terms to calculate.
 LINEWIDTH=20  # Number of terms printed per line.
 
 Q[1]=1        # First two terms of series are 1.
 Q[2]=1
 
 echo
 echo "Q-series [$LIMIT terms]:"
 echo -n "${Q[1]} "             # Output first two terms.
 echo -n "${Q[2]} "
 
 for ((n=3; n <= $LIMIT; n++))  # C-like loop conditions.
 do   # Q[n] = Q[n - Q[n-1]] + Q[n - Q[n-2]]  for n>2
 #  Need to break the expression into intermediate terms,
 #+ since Bash doesn't handle complex array arithmetic very well.
 
   let "n1 = $n - 1"        # n-1
   let "n2 = $n - 2"        # n-2
   
   t0=`expr $n - ${Q[n1]}`  # n - Q[n-1]
   t1=`expr $n - ${Q[n2]}`  # n - Q[n-2]
   
   T0=${Q[t0]}              # Q[n - Q[n-1]]
   T1=${Q[t1]}              # Q[n - Q[n-2]]
 
 Q[n]=`expr $T0 + $T1`      # Q[n - Q[n-1]] + Q[n - Q[n-2]]
 echo -n "${Q[n]} "
 
 if [ `expr $n % $LINEWIDTH` -eq 0 ]    # Format output.
 then   #      ^ Modula operator
   echo # Break lines into neat chunks.
 fi
 
 done
 
 echo
 
 exit 0
 
 # This is an iterative implementation of the Q-series.
 # The more intuitive recursive implementation is left as an exercise.
 # Warning: calculating this series recursively takes a VERY long time.

--

Bash supports only one-dimensional arrays, though a little trickery permits simulating multi-dimensional ones.

Example 26-16. Simulating a two-dimensional array, then tilting it

#!/bin/bash
 # twodim.sh: Simulating a two-dimensional array.
 
 # A one-dimensional array consists of a single row.
 # A two-dimensional array stores rows sequentially.
 
 Rows=5
 Columns=5
 # 5 X 5 Array.
 
 declare -a alpha     # char alpha [Rows] [Columns];
                      # Unnecessary declaration. Why?
 
 load_alpha ()
 {
 local rc=0
 local index
 
 for i in A B C D E F G H I J K L M N O P Q R S T U V W X Y
 do     # Use different symbols if you like.
   local row=`expr $rc / $Columns`
   local column=`expr $rc % $Rows`
   let "index = $row * $Rows + $column"
   alpha[$index]=$i
 # alpha[$row][$column]
   let "rc += 1"
 done  
 
 #  Simpler would be
 #+   declare -a alpha=( A B C D E F G H I J K L M N O P Q R S T U V W X Y )
 #+ but this somehow lacks the "flavor" of a two-dimensional array.
 }
 
 print_alpha ()
 {
 local row=0
 local index
 
 echo
 
 while [ "$row" -lt "$Rows" ]   #  Print out in "row major" order:
 do                             #+ columns vary,
                                #+ while row (outer loop) remains the same.
   local column=0
 
   echo -n "       "            #  Lines up "square" array with rotated one.
   
   while [ "$column" -lt "$Columns" ]
   do
     let "index = $row * $Rows + $column"
     echo -n "${alpha[index]} "  # alpha[$row][$column]
     let "column += 1"
   done
 
   let "row += 1"
   echo
 
 done  
 
 # The simpler equivalent is
 #     echo ${alpha[*]} | xargs -n $Columns
 
 echo
 }
 
 filter ()     # Filter out negative array indices.
 {
 
 echo -n "  "  # Provides the tilt.
               # Explain how.
 
 if [[ "$1" -ge 0 &&  "$1" -lt "$Rows" && "$2" -ge 0 && "$2" -lt "$Columns" ]]
 then
     let "index = $1 * $Rows + $2"
     # Now, print it rotated.
     echo -n " ${alpha[index]}"
     #           alpha[$row][$column]
 fi    
 
 }
   
 
 
 
 rotate ()  #  Rotate the array 45 degrees --
 {          #+ "balance" it on its lower lefthand corner.
 local row
 local column
 
 for (( row = Rows; row > -Rows; row-- ))
   do       # Step through the array backwards. Why?
 
   for (( column = 0; column < Columns; column++ ))
   do
 
     if [ "$row" -ge 0 ]
     then
       let "t1 = $column - $row"
       let "t2 = $column"
     else
       let "t1 = $column"
       let "t2 = $column + $row"
     fi  
 
     filter $t1 $t2   # Filter out negative array indices.
                      # What happens if you don't do this?
   done
 
   echo; echo
 
 done 
 
 #  Array rotation inspired by examples (pp. 143-146) in
 #+ "Advanced C Programming on the IBM PC," by Herbert Mayer
 #+ (see bibliography).
 #  This just goes to show that much of what can be done in C
 #+ can also be done in shell scripting.
 
 }
 
 
 #--------------- Now, let the show begin. ------------#
 load_alpha     # Load the array.
 print_alpha    # Print it out.  
 rotate         # Rotate it 45 degrees counterclockwise.
 #-----------------------------------------------------#
 
 exit 0
 
 # This is a rather contrived, not to mention inelegant simulation.
 
 # Exercises:
 # ---------
 # 1)  Rewrite the array loading and printing functions
 #     in a more intuitive and less kludgy fashion.
 #
 # 2)  Figure out how the array rotation functions work.
 #     Hint: think about the implications of backwards-indexing an array.
 #
 # 3)  Rewrite this script to handle a non-square array,
 #     such as a 6 X 4 one.
 #     Try to minimize "distortion" when the array is rotated.

/dev

The /dev directory contains entries for the physical devices that may or may not be present in the hardware. [1] The hard drive partitions containing the mounted filesystem(s) have entries in /dev, as a simple df shows.
bash$ df
 Filesystem           1k-blocks      Used Available Use%
  Mounted on
  /dev/hda6               495876    222748    247527  48% /
  /dev/hda1                50755      3887     44248   9% /boot
  /dev/hda8               367013     13262    334803   4% /home
  /dev/hda5              1714416   1123624    503704  70% /usr
 	      

Among other things, the /dev directory also contains loopback devices, such as /dev/loop0. A loopback device is a gimmick that allows an ordinary file to be accessed as if it were a block device. [2] This enables mounting an entire filesystem within a single large file. See Example 13-8 and Example 13-7.

A few of the pseudo-devices in /dev have other specialized uses, such as /dev/null, /dev/zero, /dev/urandom, /dev/sda1, /dev/udp, and /dev/tcp.

For instance:

To mount a USB flash drive, append the following line to /etc/fstab. [3]
/dev/sda1    /mnt/flashdrive    auto    noauto,user,noatime    0 0
(See also Example A-23.)

When executing a command on a /dev/tcp/$host/$port pseudo-device file, Bash opens a TCP connection to the associated socket. [4]

Getting the time from nist.gov:

bash$ cat </dev/tcp/time.nist.gov/13
 53082 04-03-18 04:26:54 68 0 0 502.3 UTC(NIST) *
 	      

[Mark contributed the above example.]

Downloading a URL:

bash$ exec 5<>/dev/tcp/www.net.cn/80
 bash$ echo -e "GET / HTTP/1.0\n" >&5
 bash$ cat <&5
 	      

[Thanks, Mark and Mihai Maties.]

Example 27-1. Using /dev/tcp for troubleshooting

#!/bin/bash
 # dev-tcp.sh: /dev/tcp redirection to check Internet connection.
 
 # Script by Troy Engel.
 # Used with permission.
  
 TCP_HOST=www.dns-diy.com   # A known spam-friendly ISP.
 TCP_PORT=80                # Port 80 is http.
   
 # Try to connect. (Somewhat similar to a 'ping' . . .) 
 echo "HEAD / HTTP/1.0" >/dev/tcp/${TCP_HOST}/${TCP_PORT}
 MYEXIT=$?
 
 : <<EXPLANATION
 If bash was compiled with --enable-net-redirections, it has the capability of
 using a special character device for both TCP and UDP redirections. These
 redirections are used identically as STDIN/STDOUT/STDERR. The device entries
 are 30,36 for /dev/tcp:
 
   mknod /dev/tcp c 30 36
 
 >From the bash reference:
 /dev/tcp/host/port
     If host is a valid hostname or Internet address, and port is an integer
 port number or service name, Bash attempts to open a TCP connection to the
 corresponding socket.
 EXPLANATION
 
    
 if [ "X$MYEXIT" = "X0" ]; then
   echo "Connection successful. Exit code: $MYEXIT"
 else
   echo "Connection unsuccessful. Exit code: $MYEXIT"
 fi
 
 exit $MYEXIT

Notes

[1]

The entries in /dev provide mount points for physical and virtual devices. These entries use very little drive space.

Some devices, such as /dev/null, /dev/zero, and /dev/urandom are virtual. They are not actual physical devices and exist only in software.

[2]

A block device reads and/or writes data in chunks, or blocks, in contrast to a character device, which acesses data in character units. Examples of block devices are a hard drive and CD ROM drive. An example of a character device is a keyboard.

[3]

Of course, the mount point /mnt/flashdrive must exist. If not, then, as root, mkdir /mnt/flashdrive.

To actually mount the drive, use the following command: mount /mnt/flashdrive

Newer Linux distros automount flash drives in the /media directory.

[4]

A socket is a communications node associated with a specific I/O port. It permits data transfer between hardware devices on the same machine, between machines on the same network, between machines across different networks, and, of course, between machines at different locations on the Internet.

Оставьте свой комментарий !

Ваше имя:
Комментарий:
Оба поля являются обязательными

 Автор  Комментарий к данной статье