UNIX Shell Scripting Basics
UNIX Shell Scripting Basics
The UNIX shell is a command-line interpreter that processes each command or combination
of commands on a command line when you press Enter.
$ sort -r userlist
You can combine multiple commands on the same command line by creating a composite
command.
This example shows a composite command comprising the ls and less commands.
$ ls -l | less
You can use a number of special characters in the command line. A semicolon (;), for
example, allows you to place more than one command statement in the same command
line. When you enter the code, the shell executes the preceding commands only when it
reaches the semicolon.
In this example, the shell executes the cp command statement and then the cat command
statement.
Special characters you can use to manipulate commands in the command line include
backslash (\)
greater than (>)
less than (<)
pipe (|)
ampersand (&)
backslash (\)
The backslash (\) character prevents the shell from treating another character as a special
character through a process called backslash escaping.
This allows you to split a command statement across multiple lines. When you place the
backslash at the end of a line and then press Enter, you can continue the statement on the
next line. The backslash prevents the shell from treating the Enter keystroke – or new line
character – as a special character.
This example shows a long echo statement carried across three lines.
$ echo Long pieces of text may not always fit onto a single \
> line of the command line interface, so it becomes \
> necessary to split them across multiple lines using \
> backslashes.
greater than (>)
The greater-than character (>) allows you to direct the standard output of a command to a
file or a device such as a printer instead of to the terminal screen.
This example will send the output of the ls command to a file called userdirs.
This example sends input from a file called list to the sort command.
This example pipes the output from the cat command as input to the grep command for
further processing.
If you want to use special characters in command-line text without the shell recognizing
them as special characters, you have to enclose them in quotes or precede them with a
backslash (\).
This example shows an echo command in which the echo text contains an ampersand.
There's a backslash in front of the ampersand, which prevents the shell from treating it as a
special character.
Question
How do you think the shell processes the contents of a command line in order to
execute it?
Options:
Answer
When you execute a command line, the shell looks for spaces and special characters
and splits the command line into segments wherever these characters occur. It then
processes each segment in turn.
The segments into which the shell divides a command line are called tokens. To execute a
command line, the shell processes the first token and then each subsequent token in turn.
To begin processing a token, the shell checks whether it's a keyword, an alias, or an
ordinary word.
If the token is a keyword that opens a substructure such as a function, conditional
statement, or bracketed group of commands, the shell processes the substructure before
moving on to the next token.
If a token is an alias, the shell replaces it with the command to which the alias is mapped.
After comparing a token against the list of known keywords and aliases, the shell processes
it using several stages of expansion and substitution.
brace expansion
tilde expansion
parameter substitution
command substitution
arithmetic substitution
word splitting
pathname substitution
brace expansion
In brace expansion, the shell looks for braces ({}) – also called curly brackets – in the token.
If braces are present, it expands their contents.
tilde expansion
In tilde expansion, the shell looks for tildes (~) in the token. If a tilde is present, it replaces
the tilde with the location of the current user's home directory.
For example, depending on the system configuration, the token ~vincep/file2 might
expand into /usr/home/vincep/file2.
parameter substitution
In parameter substitution, the shell checks whether the token is a variable name preceded
by a dollar sign ($). If it is, the shell replaces the token with the current value of the
corresponding variable.
For example, if the value of the SHELL parameter is /bin/ksh, the token $SHELL is
replaced with /bin/ksh.
command substitution
In command substitution, the shell checks whether the token is a command enclosed in
brackets and preceded by a dollar sign ($). If it is, the shell processes the command and
replaces the token with the command's output.
For example, the token $(type username) might be replaced with vincep.
arithmetic substitution
In arithmetic substitution, the shell checks whether the token is an arithmetic expression
enclosed in double brackets and preceded by a dollar sign. If it is, the shell evaluates the
expression and replaces the token with the result.
word splitting
In word splitting, the shell examines those parts of the command line that have resulted
from previous stages of expansion and substitution. If any of these contain spaces or
special characters, it splits them into tokens for processing.
pathname substitution
In pathname substitution, the shell looks for wildcard characters in the token. If it finds
asterisks (*), question marks (?), or double slashes (//), it searches the current directory for
filenames that match these wildcards and substitutes them for the token.
For example, depending on the files in the current directory, the token f*.txt might
expand into fares.txt flights.txt funding.txt.
After performing expansion and substitution, the shell processes subsequent tokens until it
reaches the end of a command, denoted by a semicolon or a new line character.
Then it matches the command against its list of known functions, built-in commands, and
pathnames.
Once the shell has identified which command it needs to execute, it executes the command
to produce output.
It then moves on to the next command, processing its tokens in the same way.
Question
Options:
1. &
2. |
3. ;
4. \
Answer
Option 1 is correct. The ampersand symbol (&) is used at the end of a command, to
run a job in the background. Jobs in the background are assigned a job id number.
You can you can use this number to foreground the job again.
Option 2 is incorrect. The pipe (|) special character allows you to use the output of
the command on the left of the pipe as input for the command on the right of the
pipe.
Option 4 is incorrect. The backslash (\) is used to prevent special characters from
being interpreted in such a way that their literal values are used in stings. Also, you
can continue your command on a new line by typing a backslash before pressing
Enter.
Question
Options:
Answer
2. Command grouping
You can join commands on a command line in such a way that the second command
executes only if the first command has executed successfully.
For example, you can use a first command to check whether a file exists and a second
command to perform an operation on it if it exists.
Question
How do you think the shell knows whether a command has executed successfully?
Options:
Answer
The shell knows that a command has executed successfully when the exit status of
the command is zero.
To make one command conditional on another, you join the commands using a double
ampersand (&&). The command after the && symbols executes only if the command before
the && symbols produces a zero exit status – in other words, if it executes successfully.
In this example, the ls command checks whether the userlist file exists. Because it does
exist, the ls command executes without errors ( so its exit state is zero. This causes the
sort command to execute.
If you delete the userlist file and run the command again, the ls command encounters an
error – so its exit state is non-zero. Because the sort command is conditional, the shell
doesn't attempt to execute it.
You use a double pipe (||) to make a command conditional on the unsuccessful execution of
the previous command.
In such a case, the second command executes only if the first command has a non-zero exit
state.
In this example, the ls command looks for a file called userlist. If it fails to find the file, the
touch command creates it.
If the ls command executes successfully, this means that the file already exists. In this
case, the touch command doesn't execute.
You can group commands using braces ({}). The shell treats any command block enclosed
in braces as if it were a single command.
This allows you to redirect input and output to and from a group of commands.
In this example, the braces group the sort and grep commands into a code block so that
the shell sorts input and then extracts any lines containing the word Mexico.
$ {sort | grep 'Mexico'}
You can redirect input and output to a command block as if it were a single command. In
this example, the code specifies the flights file as input and the mex_flights file as output.
You can group commands using round brackets – often called parentheses – instead of
braces. This causes the shell to spawn a subshell and execute the command block in the
subshell.
Commands that execute in a subshell do not affect what's happening in the main shell.
This allows you to define variables that exist only for the lifetime of the subshell, and to
change the working directory within the subshell without affecting the parent shell.
Question
You want to create a file named hostname and containing the text
easy1.easynomad.com. However, you don't want to overwrite any existing file by that
name.
Options:
Answer
The use of the || ensures that the code that writes the output from the echo command
to the hostname file will only execute if the attempt to list the hostname file fails.
Option 1 is correct. You use the double pipe to make a command conditional on the
unsuccessful execution of a previous command.
Option 2 is incorrect. The && conditional execution symbol ensures that if the
attempt to list the hostname file succeeds, it will get overwritten.
Option 3 is incorrect. The >> redirector is used to append output to a file.
Option 4 is incorrect. The I symbol pipes the output from one command into another
command as input.
Command grouping is useful for executing relatively short command-line code that you need
to run only once.
However, you may need to run larger pieces of code that include several lines or to use the
same piece of code many times.
Question
What do you think distinguishes shell script files from ordinary ASCII text files?
Options:
Answer
Unlike ordinary ASCII text files, shell scripts contain commands, are executable, and
have an introductory line of code that defines them as scripts.
You can read and edit ordinary text files, but you cannot execute them. However, you need
to be able to execute shell scripts.
The first line in any shell script has to be a special line of code that specifies the particular
shell program in which the script must run.
This is necessary because some commands run differently in different shell programs.
The shell identifier at the beginning of a shell script consists of a hash followed by an
exclamation point (#!) – commonly called a shebang – and the absolute pathname of the
shell program.
This example shows the first line of a script that uses the Korn shell.
#! /bin/ksh
This simple example of a script tests whether the directory /usr/shared/tours exists. If it
doesn't, the script creates it. Then it creates a file called tourlist inside this directory and
returns a message.
#! /bin/ksh
ls /usr/shared/tours || mkdir /usr/shared/tours
touch /usr/shared/tours/tourlist
echo tour directory and tourlist file created.
Once you've created a script and made it executable, you can use it as many times as you
like. You can execute it directly from the command line or you can invoke it from inside other
scripts.
Question
Options:
Answer
Shell scripts are ASCII text files that need to have executable permissions set. The
first line of a shell script identifies the command interpreter.
Option 1 is correct. Because shell scripts are simple ASCII text files, you can easily
create them in a text editor such as vi or emacs.
Option 2 is incorrect. Shell scipts are interpreted by the command interpreter, so they
don't contain binary code and aren't compiled.
Option 3 is correct. Because shell scipts are executed, either one or more of the
owner, group, or other executable permissions must be set.
Option 4 is correct. The first line of a shell script consists of a hash symbol followed
by an exclamation mark and the absolute path to the command interpreter that will
be used to execute the script. For example:
#!/bin/bash
Summary
You can use special characters to join commands on a single command line, to redirect
input and output, to run commands in the background, and to continue a command over
multiple lines. You can prevent the shell from recognizing a special character by preceding it
with a backslash. When you execute a command line, the shell splits it into tokens and
processes each token in turn.
You can group commands using braces or brackets, which cause the shell to treat the
commands as a single command. You can join two commands so that the second command
will execute only if the first command executes successfully or only if it executes
unsuccessfully.
You can store blocks of commands in a text file called a shell script and make this file
executable. You can execute shell scripts directly from the command line and reuse them as
often as necessary.
Table of Contents
| Top of page |
| Learning objective |
| Summary |
| Print | Contents | Close |
To create a shell script, you open a text file in a text editor such as vi or emacs.
For example, let's say that you decide to write a shell script for the Korn shell that
automates the repetitive task of creating a file and folder structure for a new travel
package.
You open a new text file in vi and – in this case – name it add_pkg_files.
Question
You need to invoke the Korn shell at the beginning of the script.
Assuming that the path to the Korn shell is /bin/ksh, see if you can complete the
code that does this.
MISSING CODE
Answer
You type #!/bin/ksh to invoke the Korn shell at the beginning of the script.
The names you need to use for the new files and directories depend on the names of the
travel packages. Therefore, the shell script needs to prompt the user to supply the
package name. It then needs to read the name and store it as a variable.
#!/bin/ksh
clear
echo "Please enter the name of the new travel package: "
read PNAME
You need to create a directory for the new package in /usr/shared, and you need to
create "bookings" and "info" subdirectories in the package directory.
#!/bin/ksh
clear
echo "Please enter the name of the new travel package: "
read PNAME
ls /usr/shared/$PNAME || mkdir /usr/shared/$PNAME
cd /usr/shared/$PNAME
mkdir bookings
mkdir info
You need to create three empty text files in the new folder hierarchy – one for flights, one
for accommodation, and one for bookings.
#!/bin/ksh
clear
echo "Please enter the name of the new travel package: "
read PNAME
ls /usr/shared/$PNAME && mkdir /usr/shared/$PNAME
cd /usr/shared/$PNAME
mkdir bookings
mkdir info
touch /bookings/$PNAME_bookings
touch /info/$PNAME_accom
touch /info/$PNAME_flights
Finally, you want the script to display a message to the user that confirms that the files
and folders have been created successfully.
#!/bin/ksh
clear
echo "Please enter the name of the new travel package: "
read PNAME
ls /usr/shared/$PNAME && mkdir /usr/shared/$PNAME
cd /usr/shared/$PNAME
mkdir bookings
mkdir info
touch /bookings/$PNAME_bookings
touch /info/$PNAME_accom
touch /info/$PNAME_flights
echo "Files and folders for $PNAME created in /usr/shared."
After you've saved and closed the text file containing your new script, you can confirm its
existence using the ls -l command. In this example, you've called the script
add_pkg_files.
This command reveals that the file doesn't have execute permissions set for it. Therefore,
you can't run it as a script.
$ ls -l
total 2
-rw-r--r-- 1 vincep vincep 326 Feb 11 11:56 add_pkg_files
$
When you specify a set of permissions, you need to specify the permissions that apply to
In symbolic form, you use the letters r, w, and x to denote read, write, and execute
permissions respectively.
To make a script executable using symbolic form, you use the chmod +x command.
$ chmod +x add_pkg_files
$
In numeric form, you specify permissions using a three-digit octal code. The numbers
from 0 to 7 represent increasingly open permissions, with 0 specifying no access and 7
specifying read, write, and execute permissions.
The first digit applies to the file's owner, the second to the owner's group, and the third to
all other users.
Let's say that you need to allow everyone to execute a script, but you don't want anyone
else to be able to write to the script.
Let's say that you're a member of a group of developers and you want other members of
the group to be able to edit your script. On the other hand, you don't want general users
to be able to use the script at all.
In this case, you would need to specify the script's permissions as 770.
Question
If the octal mode value of a shell script's permissions is 740, who will be able to
run the shell script?
Options:
1. Any user
2. Only member of the group that owns the file
3. Only the user that owns the file
4. Only users who are logged on locally
Answer
In this example, only the file owner has the executable permission assigned – so
only they can run the shell script.
Option 3 is correct. The file owner's permissions are represented by the 7, so they
have full permissions to the file and can execute it.
Option 4 is incorrect. File system permissions aren't affected by whether users are
logged in locally or remotely.
Question
Options:
1. Everyone
2. File owner
3. Group
4. Local login
Answer
You can assign permissions to the individual user that owns the file, to the group
that owns the file, and to everyone else.
Option 1 is correct. You can set permissions for anyone that tries to access the
file. If you want everyone to be able to read a memo in a shared directory, you
would set the read permission in the third permission set.
Option 2 is correct. By default, the user who creates a file owns it, and usually has
full permissions to the file.
Option 3 is correct. By default, the user who creates a file is part of a primary
group that also owns the file You use group permissions to control who can
access a file and what they can do with it.
Option 4 is incorrect. You can't assign permissions at local login level. Instead you
use the group ownership of files to control access.
executing it directly
invoking the shell
using your environment
executing it directly
Direct execution is the simplest way to execute a script. You navigate to the directory in
which the script is stored and type its name at the shell prompt, preceded by ./.
Instead of navigating to the script's directory, you can specify its full file path.
invoking the shell
Invoking the shell to execute a script involves running an instance of the shell, and
specifying the path to the script as well as the script name. This allows you to specify shell
options.
using your environment
Using your environment to execute a script requires that the script is located in a directory
that is found in the $PATH environmental variable. This allows you to execute the script by
simply typing its name, regardless of your current working directory.
To run the add_pkg_files script using direct execution, you type ./add_pkg_files at
the command prompt.
This causes the shell to spawn a child shell process, also called a subshell. The script
executes in the subshell, which returns you to the main shell once the script has finished
running.
$ ./add_pkg_files
You can run the script in the background, which allows you to use the main shell while the
subshell executes the script.
$ ./add_pkg_files &
You can run the script by invoking the shell. To do this in the Korn shell, for example, you
type /bin/ksh add_pkg_files. As with direct execution, the script runs in a subshell.
$ /bin/ksh add_pkg_files
If you have added your script directory to the $PATH statement in your .profile file, you
don't need to use ./ or the shell to run the script.
In this case, you only need to type the name of the script. This causes the script to run in
a subshell.
$ add_pkg_files
If you don't want to run the script in a subshell, you can precede it with the keyword
source. This causes the current main shell to run each command in the script in
sequence.
You can't do anything else with the main shell while the script is executing.
$ source add_pkg_files
Question
Options:
Answer
You can use shell commands to run a script at a specified future time, at regular
specified intervals, or whenever system load levels are low.
In this example, you specify that the script must run at 6:00 pm on October 12.
You can also use more informal time specifications like 1pm tomorrow or even
teatime, which means 4:00 pm.
You need to make an entry in the /etc/crontab file, specifying the name of the script and
the interval at which to run it.
Then you run the cron command, which starts a process that checks the /etc/crontabs
file each minute and executes any jobs scheduled for the current minute.
$ cron
You use the batch command to run a script as soon as system load levels are low.
When you run a script with the batch command, the shell queues it as a pending job
until the system load average falls below 1.5.
$batch add_pkg_files
Question
Options:
1. at
2. batch
3. cron
Targets:
Answer
You use the at command to run commands at a schedule time, the cron command
to run commands at a regular interval, and the batch command to run scripts when
the system is relatively idle.
The at command allows you to specify the time and date at which a script will run.
You can use a variety of formats, including keywords such as "noon" and "today".
The batch command is useful if you want to run scripts at times when they won't
interfere with system performance. It runs commands or scripts when system load
levels drop below a specific amount. The amount will vary according to your
system.
The cron command allows you to specify the interval at which a command will
repeat. It can be set to recognize minutes, hours, days of the week, days of the
month, or specific months.
Question
Options:
1. addfile.sh
2. ./addfile.sh
3. /bin/bash addfile.sh
4. source addfile.sh
Answer
The source command executes the commands in a target file as if they were
entered into the current shell.
Option 2 is incorrect. The ./ syntax only allows you to execute the addfile.sh file
in a subshell. It is used when the location of the file is not listed in the $PATH
variable.
Option 3 is incorrect. This method launches a subshell in which the script is
executed. It's useful if you need to set options for the subshell.
Option 4 is correct. The source command reads the contents of the referenced file
directly into the current shell. It's useful if you want to apply changes to your
.profile file during your current session.
Summary
To write a shell script, you begin by invoking a shell. Then you add lines of commands
that perform the required tasks. You save the file and make it executable by assigning
execute permissions to it.
You can run a shell script by invoking it directly or by invoking it through the shell. Both
these processes cause it to run in a subshell. Alternatively, you can run a script in the
main shell by using the source keyword. If you include your script directory in the $PATH
statement of the .profile file, you can run a script simply by typing its name at the
command prompt.
This exercise is designed for practice use and does not contain new learning content. If your
computer doesn't have an application or applications required to carry out the exercise
tasks, or if you would prefer to perform the exercise at another time, you can proceed to the
next topic.
Scenario
You currently use a text file as a signature file for your e-mail messages. It contains your
name, job description, and contact details. You decide to liven it up a little by writing a script
that customizes it with a different message every day.
Exercise
Your assignment for this exercise is to write a script that customizes your e-mail signature
file – called mysig – with a random message.
You generate the messages using the fortune command, which outputs a random snippet
of wisdom each time you run it. The fortune program is located in /usr/games. You don't
need to supply any options or arguments to the fortune command.
You need to incorporate the fortune message in your signature file. You need to use the
cp command to make a temporary copy of mysig called dailysig. This file prevents a buildup
of previous fortune messages by storing a single message for each day only. Finally, you
append the output from fortune to dailysig by redirecting to dailysig.
The shell you're using for this script is the Korn shell, which is located at /bin/ksh.
Task list
Number Instructions
1 Write a script for the Korn shell that uses the fortune and cp
commands – as well as output redirection – to customize your e-mail
signature file with a random message.
Review your solution and, when you are satisfied with it, open the solution page.
Table of Contents
| Top of page |
| Scenario |
| Exercise |
| Print | Contents | Close |
After completing this topic, you should be able to manage the execution of UNIX shell
commands.
In UNIX, shell programs provide an interface between users and the kernel – the core of the
operating system.
You interact with the kernel by entering commands at the shell prompt.
It's possible to enter several commands at once. In this case, you separate each command
using a semicolon (;).
The code shown here redirects the output of the ls command to a file called listing, and
then uses the cat command to display the contents of the file. Because the two commands
are separated by a semicolon, they run consecutively.
If you end a line of code with a backslash (\), pressing Enter will move you to a new line and
allow you to continue entering commands. You can continue to do this over multiple lines.
When you run the commands, the shell will execute your code as if it were one long line,
ignoring the backslash characters.
Question
Most shell commands behave as filters – they can take data as input, process it, and
provide output. Because of this behavior, you can chain commands together, piping
the output of one command into the input of another command.
See if you can pipe the output of the ls -l command to the less command.
$ MISSING CODE
Answer
You type ls -l | less to pipe the output of the ls -l command to the less
command.
Sometimes you may want to run commands in the background, especially if they're likely to
take a long time to complete. Background commands use idle CPU time only, allowing you
to get on with other things.
You can background any command by suffixing it with an ampersand character (&). The
code shown here will run a script called myscript.sh in the background.
$ ./myscript.sh &
[1] 34030
$
Question
Options:
Answer
You use the backslash to make long commands more readable, the semi-colon to
enter consecutive commands on the same line, and the ampersand to run
commands in the background.
Option 1 is correct. Provided you precede it with a backslash, you can use the Enter
key to continue long commands on a new line. The shell will run your command as
one long line, ignoring the backslashes.
Option 2 is incorrect. The forwardslash does not prevent special characters from
being intepreted. It is used in the syntax for describing the filesystem.
Option 3 is correct. The ampersand allows you to run commands in the background
on idle processor cycles, leaving you free to continue working in your current
session.
Option 4 is correct. You can use the semi-colon to combine commands that execute
consecutively on a single command-line. This is useful when the first command will
take a long time to complete.
2. Conditional execution
Conditional execution is a programming strategy that allows you to specify different actions
based on the outcome of an initial operation.
For example, you could create a shell script that checks the size of a log file and then backs
it up only if it is larger than a certain size.
Regardless of their type or function, most UNIX commands return an exit status number to
the shell when they've completed executing. The exit status indicates whether a command
executed successfully.
Although you never see exit statuses at the shell prompt, you can use them to include
conditional decisions in your shell scripts.
Most shells provide at least the following two pairs of operators for performing conditional
execution based on exit status values:
For example, in the syntax shown here, the second command executes only if the first
command fails. And if the second command fails, the third command will not run.
Question
Options:
1. A non-zero exit status indicates that a command did not execute successfully
2. An exit status of 0 indicates that a command completed successfully
3. Conditional execution allows you to choose an action based on the exit status of a
command
4. Conditional execution allows you to choose what code to run based on the output of
a command
Answer
A non-zero exit status indicates that a command did not execute successfully, while
an exit status of 0 indicates that it did complete successfully. Conditional execution
allows you to choose an action based on the exit status of a command.
Option 1 is correct. Any non-zero exit status is a failed command. The range of
possible non-zero values for the exit status depends on the command that you
execute, and the shell you use.
Option 2 is correct. If the last command to execute completed successfully, the value
of the $? Variable, in which the exit status is stored, will be 0.
Option 3 is correct. Depending on the conditional operator you use, code will execute
only if the previous command executes successfully, or only if the previous
command fails.
Option 4 is incorrect. You can use the test command to return an exit status after
testing for a particular value, but conditional execution only operates on the exit
status of commands, not their results.
Question
You are creating a script that writes to the filesystem, and you don't want to alter any
existing files.
Options:
Answer
The || operator ensures that the hosts file is not overwritten if it already exists. You
use the && operator to ensure that a new hosts file is only created if the command to
rename the original succeeds.
Option 1 is incorrect. The ls hosts command will have an exit status of 0 if the host
file exists, and the && operator will cause the hosts file access and modification times
to be updated.
Option 2 is correct. The || operator will execute if the hosts file doesn't exist and the
ls command returns a non-zero exit status. This will cause the touch command to
execute and create a new hosts file.
Option 3 is incorrect. The ; operator merely allows consecutive execution of
commands. The execution of the command to overwrite the hosts file is not
dependant on the exit status of the command listing the host file.
Option 4 is correct. It is safe to overwrite the hosts file if the command that renames
the original file, as well as the command that prints the contents of the renamed file,
complete successfully.
Question
Let's say that you want to create a command that will check for the existence of a log
file. If the file exists, the log file should be copied to a backup directory. If the copy
operation fails, the system should send an e-mail to the root user.
See if you can type an operator in the first blank space that will back up the log file if
it exists.
Answer
Question
See if you can complete the code shown here to send an e-mail to the root user if the
copy operation fails.
$ ls -l /var/log/mylog && cp
/var/log/mylog /backup/mylog
> MISSING CODE mail -s "Error" root
< errmsg.txt
Answer
You use the || operator to execute the mail root@localhost "copy
operation failed" command if the copy operation returns a non-zero exit status.
3. Grouping commands
Conditional execution operations are based on the exit status value of a single command –
you can't have more than one command on either side of a conditional operator.
This behavior is sufficient for simple tasks, but often you'll need to base conditional
execution on the result of a series of commands. You achieve this using grouping.
You can enclose a series of commands in brackets to create a code block. The shell will
treat a code block as if it were a single command ( it will return one exit status value for the
entire block, rather than for each individual command that the block contains.
This allows you to create far more complex commands and shell scripts.
You can group commands using the following two types of brackets:
braces ({})
round brackets (())
braces ({})
You can use braces ({}) to create blocks of code that will execute in your current shell.
round brackets (())
You can use round brackets (()) to create blocks of code that will execute in a subshell ( a
new shell program spawned from the shell that you're running.
The advantage of running commands in a subshell ( by grouping them using round brackets
( is that subshells have their own environment variables. As a result, any radical
environment changes that take place during the running of a code block don't affect the shell
you started with.
Question
Options:
Answer
Option 2 is incorrect. The variable that stores the current working directory for the
shell in which the ls command is executed is not affected by the command to move
to the /etc directory in the subshell.
Option 4 is incorrect. You can include comments in shell scripts by preceding each
line of comments with the # symbol.
Question
Identify the operators that allow you to group commands in code blocks.
Options:
1. []
2. {}
3. ()
4. ;
Answer
You use the parentheses () and the braces {} operators to group commands in
code blocks that return a single exit status.
Option 1 is incorrect. The [] symbols are used by the shell in pattern matching
operations to specify character ranges.
Option 2 is correct. The {} symbols group commands into a single code block for
execution in the current shell.
Option 3 is correct. The () symbols group commands into a code block that is
executed in a subshell. This is useful if you want to avoid altering environment
variables in your current shell, for instance.
Option 4 is incorrect. The ; symbol is used to enter multiple commands on the same
line. However, the commands are executed in sequence, not as a code block.
Question
Suppose that you're in /home/gregl and you need to execute the code shown here. It
looks for a specific log file, and if the file exists a message displays.
What do you think your current working directory will be when the code has finished
executing?
$ { cd /var/log ; ls mylog } /
&& echo "Found the log file"
Options:
1. /var/log
2. /home/gregl
3. /usr/home
Answer
After the command runs, your current working directory will still be /home/gregl.
Option 1 is incorrect. The command that changes the directory to /var/logs runs in a
subshell. To run the code in the current shell, thereby changing the current directory,
you need to remove the braces, as shown here:
cd /var/log ; ls mylog && echo "Found the log file"
The command that changes the current working directory to /var/log is part of a
code block enclosed in brackets (()), so it will execute in a subshell. As a result, it
has no effect on the current shell you're in.
Option 3 is incorrect. At no point does any code change the directory to /usr/home
– either in the current shell or the subshell – so this can't be the current directory.
Summary
The shell's command line provides an interface between users and the UNIX kernel. You
can separate commands using a semicolon (;), continue them on new lines using a back
slash (\), and plug them into each other using a pipe (|).
You can use conditional operators to specify that commands execute based on the exit
status of other commands. You use two ampersand characters (&&) to specify that a
command run only if a previous command was successful, and two pipe characters (||) to
specify that a command run only if a previous command failed.
You can group commands into code blocks to specify that the shell return one exit status for
the entire block. You use braces ({}) to specify that code execute in the current shell, and
round brackets (()) to specify that code execute in a subshell.
Table of Contents
| Top of page |
| Learning objective |
| Summary |
| Print | Contents | Close |
Redirection in UNIX
Learning objective
After completing this topic, you should be able to redirect standard input and output, and
use file descriptors.
Most UNIX commands behave as filters – they can take input, process it in some way, and
provide output. A shell usually sends this output to the screen, but you can redirect it to
provide advanced functionality.
A common example of redirection is the use of the pipe operator (|) to plug the output of one
command directly into the input of another command. However, redirection most often
involves writing to and reading from files.
You can perform redirection with files by preceding a filename with one of the following
operators:
If you use a lot of file redirection in your scripts, it's quite easy to overwrite existing files
accidentally. To prevent this, most shells implement the noclobber environment variable.
When set, noclobber prevents file overwriting through redirection – but not through the
execution of commands such as cp or rm. You enable noclobber by typing set -o
noclobber, as shown here.
If you want to overwrite output files protected by noclobber, you can use the >| operator
to do this.
$ set -o noclobber
$
Question
Let's say that you want to send e-mail to a user called vincep. You've used the vi
editor to type the body of the e-mail, and you've saved it to a file called email.txt.
See if you can complete the code shown here to redirect input from the email.txt file
rather than from standard input.
$ mail –s "Guess what"
vincep@localhost MISSING CODE
email.txt
Answer
You use the < operator to redirect input from a file rather than from standard input.
You send the mail, and the shell reads the e-mail body from the text file you've specified.
Question
Let's say you're using the man command to find out about the exec command. You'd
like to dump the information to a file so that you can print it later.
See if you can complete the command shown here to dump the output of the
command to a file called exec.txt.
Answer
You use the > operator to redirect standard output to a file rather than to the screen.
The shell dumps the output of the man exec command to a text file. Because you've
redirected the output, you don't see it echoed to the screen.
Question
Options:
1. >
2. <
3. >>
Targets:
A. Used to append output from a command to a file
B. Used to redirect output from a command to a file
C. Used to specify a file as input for a command
Answer
You use the > operator to redirect the output from a command to a fle, the < operator
to use a file as input for a command, and the >> operator to append the output from a
command to an existing file.
You can use redirection to document your system by redirecting the output from
commands to a file, as in this example, which records the network interfaces
configured for a system: ifconfig –a > network.txt
Several UNIX commands accept files as input. For instance, you can redirect a text
file called report.txt to the mail utility and send it to Fred with the following command:
mail Fred < report.txt
2. File descriptors
File descriptors are files – usually with numeric filenames ( on disk that point to input/output
(I/O) objects, such as the keyboard, the screen, or other files. You can think of them as
placeholders.
By using file descriptors, processes don't have to interact mechanically with the I/O objects
that the descriptors represent – the kernel handles these mechanics.
Any running process can expect at least the three following default file descriptors to be
available:
0
1
2
0
Descriptor 0 is also known as standard input. It's normally mapped to the keyboard.
Processes write command output to this descriptor, and the output then appears on your
screen.
2
Descriptor 2 is also known as standard error. It's normally mapped to the screen, but when
UNIX was invented it was mapped to a dedicated printer.
Processes write error messages to this descriptor, and the messages then appear on your
screen.
Processes can also create their own descriptors, usually for interacting with files.
You can redirect file descriptors in a similar way to performing ordinary redirection, using the
following syntax – where n is a file descriptor:
<&n
>&n
n< filename
n> filename
n>> filename
n>| filename
<&n
You use <&n to redirect standard input from another descriptor. For example, you could
type <&2 to feed standard error into standard input.
>&n
You use >&n to redirect standard output to another descriptor. For example, you could type
>&2 to send standard output into standard error.
n< filename
You use n< with a filename to redirect a descriptor from the specified file.
n> filename
You use n> with a filename to redirect a descriptor to the specified file. As with ordinary
redirection, this will overwrite an existing file. To change this default behavior and not have
the file overwritten, set the noclobber variable, using the set noclobber command.
n>> filename
You use n>> with a filename to redirect a descriptor to the specified file. This will redirect to
a file but, as with ordinary redirection, this will append to an existing file.
n>| filename
You use n>| with a filename to redirect a descriptor to the specified file, overriding the
noclobber variable if this has been set.
Question
Options:
1. You use 2>| to ensure that existing files aren't overwritten by redirected standard
error
2. You use 2> to redirect standard error to a file
3. You use >&2 to redirect standard output to standard error
4. Use 1> to redirect standard output to a file
Answer
You use >&2 to redirect standard output to standard error, you redirect standard error
with 2>, and you redirect standard output to a file with 1>.
Option 1 is incorrect. In fact, this option allows you to overwrite existing files with
standard error.
Option 4 is correct. Instead of the output going to the screen it can be redirected to a
file using 1>
Question
Let's say that you want to use the find command to locate reports, and you want to
redirect both standard output and standard error to text files.
See if you can complete the first part of the code shown here to redirect standard
output to a text file called output.txt.
Answer
You type > to redirect standard output to a file.
Question
See if you can complete the code shown here to redirect standard error to a file
called error.txt.
Answer
Question
Options:
1. 0
2. 1
3. 2
Targets:
A. Standard error
B. Standard input
C. Standard output
Answer
The file descriptor for standard input has a value of 0, the descriptor for standard
output has a value of 1, and the descriptor for standard error has a value of 2.
Standard input is associated with file descriptor 0, and is usually mapped to the
keyboard. Typing commands with a keyboard is the most common way of interacting
with the shell.
Standard output is usually mapped to the screen. Most programs use standard
output to print usage messages or the results of commands.
Standard error is associated with file descriptor 2, and is usually mapped to the
screen – but it is sometimes useful to map it to a printer, to record errors.
3. Scripting with redirection
Sometimes, for convenience, you may want to pass multiple lines of input to a command.
You do this using a here document.
Here documents make it possible to give batch input to a command – to pass the command
a list of things you want it to do.
You can use here documents using either of the following two syntax forms:
The trailing dash in the syntax causes all leading tabs in the subsequent input to be
stripped.
Let's say that you want to automate a telnet session to e-mail yourself some data off a
remote machine. You begin by setting up the telnet command with a here document, using
the word END as a terminator.
You press Enter, and the shell waits for further input.
The first thing that a telnet session needs is the address of the target machine ( 190.100.2.3
in this case. So you enter the full command to open a session to that machine and then
press Enter.
In this case, your username is "gregl" and your password is simply "password".
The remote system should have logged you on by this point in the script, so you enter the
commands you need to fetch the file you want. You also enter the quit command to end
the telnet session.
To finish and execute the here document, you enter the terminator that you specified to
begin with ( in this case, the word END – and you press Enter.
The shell executes the telnet command and feeds it the here document line by line.
Question
Suppose that you're setting up a here document to run an ssh command, using the
word "END" as a terminator.
See if you can complete the code shown here to ensure that any leading tabs in the
here document will be stripped out.
You use the <<- operator to ensure that leading tabs are stripped out of here
documents.
Question
Options:
1. The <<- operator allows you to strip leading tabs from a here document
2. Here documents can be used to automate login processes
3. The first line of a here document is always a command.
4. The terminator in here documents is END
Answer
You use the <<- operator to strip leading tabs from lines in a here documents. Here
documents are often used to automate interactive logins and the first line of a here
document is always a command.
Option 1 is correct. Automatically stripping tabs from your here document allows you
to improve its legibility.
Option 2 is correct. Most interactive command line programs, such as telnet or ftp,
can be automated using here documents
Option 3 is correct. Here documents start with a command interpreter that loads a
command to read and execute the rest of the shell script.
Summary
Most UNIX commands behave as filters, accepting input and providing processed output.
You can redirect command output to a file, and you can redirect command input from a file.
File descriptors are numeric placeholders for input/output (I/O) objects. The default file
descriptors present in any shell environment include standard input (0), standard output (1),
and standard error (2). You can redirect file descriptors.
Here documents provide a convenient way of sending batch input to a command. They are
most commonly used for setting up scripts for interactive commands such as telnet.
Table of Contents
| Top of page |
| Learning objective |
| Summary |
| Print | Contents | Close |
After completing this topic, you should be able to use substitution and quoting to expand
and match files and commands.
You can use filename substitution when entering commands to specify groups of files that
you want to work with.
Filename substitution involves using wildcard characters either on their own, or with a
pattern fragment. The shell resolves a wildcard to a list of files, which it then passes to the
command you're trying to execute.
asterisk (*)
question mark (?)
asterisk (*)
You use the * wildcard to match a string of any length.
For example, if you specify w*, the shell will return a list of every file in your current working
directory with a name beginning with "w".
question mark (?)
You use the ? wildcard to match a single character.
For example, if you specify file0?, the shell might return a list containing file01 through
file09 – if these files existed.
You can specify limited lists of characters for filename substitution using square brackets
([]), using one of the forms shown here:
[abc]
[a-d]
[a-ce-g]
[abc]
You use the form [abc] to perform substitution with any character that appears between the
square brackets.
For example, if you specify file0[147], the shell will return a list containing file01, file04,
and file07 – if these files exist.
[a-d]
You use the form [a-d] to perform substitution with any character that appears in the range
between the square brackets.
For example, if you specify data_[a-c], the shell will return a list containing data_a,
data_b, and data_c – if these files exist.
[a-ce-g]
You use the form [a-ce-g] to perform substitution with any character that appears in any of
the ranges between the square brackets. You can include any number of ranges in this
way.
For example, if you specify log[1-36-8], the shell will return a list containing log1, log2,
log3, log6, log7, and log8 – if these files exist.
Question
See if you can complete the code shown here to retrieve a list of all the files in the
current directory that begin with a lowercase letter or a digit.
$ ls MISSING CODE
Answer
You type [a-z0-9]* or [0-9a-z]* to retrieve a list of all files that begin with a
lowercase letter or a digit.
In the filename substitution shown here, the square brackets hold two ranges – a to z and 0
to 9. The brackets are followed by an asterisk (*), which allows any type of string to follow
the specified ranges.
$ ls [a-z0-9]*
120415_log report1.txt report3.txt staffdata
120603_log report2.txt searchdata
$
You can perform reverse matching using the exclamation mark (!) character. For example,
the code shown here lists all files in the current directory that don't end in a digit.
$ ls –l *[!0-9]
120415_log report1.txt report3.txt staffdata
120603_log report2.txt searchdata
$
Occasionally, you may need to refer to a user's home directory in a script. You can refer to a
user's home directory without knowing their username using the tilde (~) character.
This enables you to write scripts with path references that any user can run.
$ echo ~
/home/vincep
$
You can use the tilde (~) character in substitution in the following forms:
~
~user
~-
~+
~
The shell resolves a lone tilde to the value of the current user's $HOME environment
variable.
~user
The shell resolves ~user to the path of a specific user's home directory.
~-
The shell resolves ~- to the path of the previous directory you were working in.
~+
The shell resolves ~+ to the path of the current directory.
Question
Options:
1. cd ~
2. cd ~$HOME
3. cd ~-
4. cd ~Fred
Answer
Option 1 is correct. In this case, the command changes the directory to the home
directory of the current user.
Option 2 is incorrect. The tilde substitutes for the $HOME variable, so this command
wouldn't work.
Option 3 is correct. Using the tilde command in this context changes the current
directory to its previous value.
Option 4 is correct. In this circumstance, the tilde command changes the current
directory to the home directory of the user Fred.
Question
Options:
1. [abc]
2. [a-e]
3. *
4. ?
Targets:
Answer
To substitute from a list of characters into a filename, you use the [abc] format. You
can also specify a range using the [a-e] format. The * substitutes for any character
zero or more times, while the ? matches any single character.
The command ls file[abc] would use filename substitution to match and list
files named filea, fileb, and filec if they exist.
The command ls text.[a-e] matches and lists every file named text with a suffix
consisting of a period followed by a character in the range of letters from a to e. In
other words, text.a, text.b, text.c, text.d, text.e will be listed.
The command ls file* will match and list any file with a filename that starts with
file, including file123.
The command ls file? will match and list any file with a filename that starts with
file and has one additional character, such as file1 or filea if they exist.
2. Command substitution
Command substitution allows you to use the output of one command as an argument for
another command.
$(command)
$(< filename)
$(command)
When you use $(command) as an argument for another command, the shell executes the
bracketed command first. It then passes this command's entire output to the unbracketed
command.
$(< filename)
When you use $(< filename) as an argument for a command, the shell reads the specified
file and passes its entire contents to the command.
Question
See if you can complete the command shown here to display a list of files that belong
to you, without knowing what your username is.
Answer
The $(whoami) command returns your current username, which the grep
command uses to filter the output of the ls –l command.
The code shown here shows a list of the files in the current directory that you own.
$ ls –l | grep $(whoami)
-rw-r--r-- 1 vincep vincep 9880 Feb 11 15:40 120415_log
-rw-r--r-- 1 vincep vincep 9372 Feb 11 15:41 120603_log
-rw-r--r-- 1 vincep vincep 73234 Jan 8 09:22 report1.txt
-rw-r--r-- 1 vincep vincep 247416 Jan 8 09:23 report2.txt
-rw-r--r-- 1 vincep vincep 451008 Jan 8 09:24 report3.txt
When you include a special character such as a wildcard or a pipe as part of an argument
for a command, the command does not interpret that character. Instead, the shell resolves it
and passes the output to the command as an argument.
If you intend a command to use a special character as an argument, you must prevent the
shell from interpreting it by quoting the character.
The code shown here displays an asterisk on the screen. Without the quotation marks, the
shell would have resolved the asterisk to a list of all the files in the current directory.
$ echo '*'
*
$
Instead of quote marks, you can use backslash escaping to perform quoting. To do this, you
place a backslash (\) immediately before a special character. This instructs the shell to pass
the character literally to the command instead of trying to interpret it.
Backslash escaping is also useful for situations in which you need to pass an actual quote
mark as an argument.
The code shown here uses backslash escaping to print out the message, "'*' is a special
character."
$ echo \'\*\' is a special character
'*' is a special character
$
Question
Which commands use a suitable alternative method for escaping special characters?
Options:
1. echo \$USER
2. echo $($)USER
3. echo $(< $)USER
Answer
Option 1 is correct. The backslash prevents the $ from being interpreted as a special
character, so $USER will be printed onscreen, rather than the value of the $USER
variable.
Option 2 is incorrect. The $(command) syntax allows you to pass the output from a
command as a parameter to another command.
Option 3 is incorrect. The $(< filename) syntax allows you to pass the contents
of a file as a parameter to a command.
Let's say that you're a system administrator, and you're preparing a startup script that will
run each time a user logs in. You start the vi editor, go into insert mode, and begin typing.
You begin the script by clearing the screen and you use command substitution to greet the
user using the whoami command.
#!/usr/local/bin/ksh
# Logon welcome script
clear
echo Welcome $(whoami)
You add a line that uses command substitution to display the current date.
#!/usr/local/bin/ksh
# Logon welcome script
clear
echo Welcome $(whoami)
echo The current date is $(date "+%d-%M-%Y")
Question
You want to display a list of all the reports in the user's home directory, but you don't
know what the user's username is.
Complete the code shown here to navigate to the user's home directory.
#!/usr/local/bin/ksh
# Logon welcome script
clear
echo Welcome $(whoami)
echo The current date is $(date "+%d-%M-%Y")
MISSING CODE
Answer
You type cd ~ to navigate to a user's home directory without knowing the user's
username.
Question
You use the ls command to list the contents of the user's home directory.
See if you can complete the code shown here to use the grep command to filter the
output of the ls command to show only those files whose names begin with "report".
#!/usr/local/bin/ksh
# Logon welcome script
clear
echo Welcome $(whoami)
echo The current date is $(date "+%d-%M-%Y")
cd ~
ls -l | grep "MISSING CODE"
Answer
You type report* to have the command return all files that begin with "report".
Welcome vincep
The date is 26-08-2003
You have the following reports:
-rw-r--r--1 vincep vincep 73235 Jan 8 09:22 report1.txt
-rw-r--r--1 vincep vincep 247416 Jan 8 09:23 report2.txt
Question
Options:
1. Command substitution allows you to use the output from a command as standard
input for a second command
2. Command substitution allows you to use the output from a command as part of the
argument for another command
3. Wild cards are used to perform command substitution
4. Wild cards are used to perform filename substitution
Answer
You use command substitution to include the output from a command in an argument
to another command and you perform filename substitution with wildcards.
Option 3 is incorrect. The shell only performs filename substitution with wildcards.
The command echo It is $(dat*) will yield a "command not found" error
message.
Summary
Filename substitution involves specifying groups of files to use as arguments for commands.
Most shells let you use wildcards like asterisk (*) or a question mark (?) in filename
substitution. The shell resolves a wildcard into a list of files that match specified search
criteria.
Similarly, you can perform command substitution to use the output of one command as an
argument for another command. The shell resolves the argument command first and passes
its output to the main command.
You can use any combination of filename substitution and command substitution when you
write shell scripts.
Table of Contents
| Top of page |
| Learning objective |
| Summary |
This article serves as a quick reference for the most commonly used shell scripting syntax.
Introduction
There are hundreds of UNIX commands that you can execute at the shell prompt.
Shells have their own built-in syntax that helps you to work more effectively with
existing commands by allowing you to perform functions like plugging commands
into each other and controlling the flow of execution.
command1 || command2
In the above syntax, the second command executes only if the first command fails.
&&
In the above syntax, the second command executes only if the first command
executes successfully.
{}
You can enclose multiple statements in braces ({}) to create a code block. The shell
returns one exit status value for the entire group, rather than for each command in
the block.
()
You can enclose multiple statements in round brackets to create a code block. This
code block functions in the same way as a code block enclosed in braces, but runs
in a subshell.
>
You use this operator to redirect command output to a file. If the specified file
doesn't exist, the shell creates the file. If the file does exist, the shell overwrites it
with the command output unless the noclobber environment variable is set.
>|
You use this operator to redirect command output to a file. If the specified file
doesn't exist, the shell creates the file. If the file does exist, the shell overwrites it
with the command output even if the noclobber environment variable is set.
>>
You use this operator to redirect command output to a file. If the file doesn't exist,
the shell creates the file. If it does exist, the shell appends the new data to the end
of it.
<
<&n
You use this operator to redirect standard input from file descriptor n.
>&n
n< filename
You use this operator with a filename to redirect descriptor n from the specified file.
n> filename
You use this operator with a filename to redirect descriptor n to the specified file.
Unlike ordinary redirection, this will not overwrite an existing file.
n>| filename
You use this operator with a filename to redirect descriptor n to the specified file,
overriding the noclobber environment variable if it is set.
n>> filename
You use this operator with a filename to redirect a descriptor to the specified file.
This will redirect to a file but, unlike ordinary redirection, this will append to an
existing file.
Filename substitution
You use square brackets to match only characters that appear inside the specified
set. For increased convenience, you can specify multiple ranges.
!pattern
You use the ! operator with a pattern to perform a reverse match. The shell returns
only filenames that don't match the pattern.
Command substitution
$(command)
You use this form of command substitution to resolve a command and pass its
output to another command as an argument.
$(< filename)
You use this form of command substitution to pass the entire contents of a file to a
command as an argument.
Tilde substitution
~
You use the ~ operator to instruct the shell to return the value of the $HOME
variable.
~username
You use the ~ operator with a username to instruct the shell to return the full path of
a specific user's home directory.
~+
You use the ~+ operator to instruct the shell to return the full path of the current
working directory.
~-
You use the ~- operator to instruct the shell to return the full path of the previous
working directory you used.
Miscellaneous syntax
If you enter several commands on the same line, you need to separate the
commands with semicolons. The shell will execute each command successively
once you press Enter.
You use a backslash to allow you to press Enter and continue typing commands on
a new line. The shell will only begin executing your commands when you press
Enter on a line that doesn't end in a backslash. Using a backlash in this way is
known as backslash escaping.
&
You add a single ampersand at the end of a command to run that command as a
background process. This is useful for tasks that are likely to take a long time to
complete.
Summary
Shell programs can execute a wide range of UNIX commands, but they also have
built-in functions to help you use shells more effectively.
Most shells support standard operators for conditional execution, input/output (I/O)
redirection, file descriptor redirection, and command grouping. They also allow you
to perform filename, tilde, and command substitution.
Table of Contents
| Top of page |
| Abstract |
| Introduction |
| Filename substitution |
| Command substitution |
| Tilde substitution |
| Miscellaneous syntax |
| Summary |
| Print | Contents | Close |
Exercise overview
In this exercise, you're required to complete a script that will perform a search for certain
types of files, output the search results to a file, and e-mail that file to the root user.
Let's say that you're the root user and that you're editing a script in the vi editor.
Step 1 of 1
You want the first two lines of the script to act as a distinct code block that returns
one exit value for both commands. The commands should execute in the current
shell.
Options:
You use braces ({}) to create a code block that will execute in the current shell.
Option 3 is correct. The syntax for a function – a sequence of code that is available
to the shell into which it is read – is:
function_name() {commands}
Option 4 is incorrect. The <> special characters are used to perform redirection.
They are also used as conditional operators (less-than and greater-than) within
conditional expressions.
Step 1 of 1
Suppose that you want to use the ls command to search for all files that begin with
the word "report" and end in a number between 0 and 9.
Result
You specify report[0-9] with the ls command to list all files that begin with the
word "report" and end in a number between 0 and 9.
Task 3: Using redirection and conditionals
Step 1 of 3
Suppose that you want to redirect the output of the ls command to a file named
localreports.
Result
You use the greater than symbol (>) to redirect output to a file.
Step 2 of 3
In the second line of the script, you want to redirect the contents of a mail from the
localreports file.
Result
You use the less than symbol (<) to redirect input from a file.
Step 3 of 3
You want the last line of the script to execute only if the code block produces an
error.
Result
You use the double pipe symbol (||) to execute a command only if the preceding
command ( or, in this case, a command block ( fails.
You've now successfully completed a script that will perform a search for specific files,
output the search results to a file named localreports, and e-mail the localreports file to the
root user. It will also display an error message if the code fails.
Table of Contents
| Top of page |
| Learning objective |
| Exercise overview |
| Print | Contents | Close |
After completing this topic, you should be able to use variables in UNIX programming.
1. Variables and data types
Local variables are available only in the function or subshell in which they were created or
declared.
Global variables are available throughout the shell, including all its subshells and functions.
You can turn local variables into global variables by exporting them using the export
command.
In this example, the code exports the local variable DEST from a subshell, thus making it a
global variable.
$ export DEST
$
Like most programming languages, UNIX shells allow you to define variables using data
types.
Data types define the types of data contained in variables. For example, you can define a
variable as an integer or as a character string.
This can change the way the shell uses the variables. For example, the shell performs faster
arithmetic with integer variables.
Each of the following popular UNIX shells supports variables and data types in different
ways:
Bourne shell
Bash shell
TC shell
Korn shell
Bourne shell
The Bourne shell supports string variables only. All variables in the Bourne shell are global.
Bash shell
The Bash shell supports string and integer variables. You can only declare local variables in
the Bash shell.
TC shell
The TC shell supports string, integer, float, and array variables. TC shell variables can be
either local or global.
Korn shell
Like the TC shell, the Korn shell supports string, integer, float, and array variables. Korn
shell variables can be either local or global.
Question
Options:
1. Bash shell
2. Bourne Shell
3. Korn Shell
Targets:
Answer
The Bourne shell only supports string variables, the Bash shell supports string and
integer variables, while the Korn shell supports string, integer, float, and array
variables
The Bash shell is based on the Bourne shell. Although you can export variables to
make them global, they can only be declared locally.
The Bourne shell is the most universally deployed shell and, although it has a limited
feature set, it's useful for writing platform-independent shell scripts.
The Korn shell provides good support for variable typing and other useability
features. It's more likely to be deployed on proprietary UNIX systems than on open
source systems.
2. Using variables
The shell defines variables automatically when you assign a value to them for the first time.
You can assign a value to a variable using the equal sign (=).
In this example, the code assigns the value 4070 to a variable called distance.
$ distance=4070
$
You can use the typeset command to assign values to variables. Using typeset allows
you to specify command-line options for the assignment operation.
$ typeset distance=4070
$
If you don't specify a value for a variable in an assignment statement, the shell assigns a
null value to the variable.
$ distance=
$
To refer to the value of a variable – rather than to the variable itself – you precede the
variable name with a dollar sign ($).
To view the value of a variable, for example, you use the print command and precede the
variable name with a $ sign.
In this example, the code allows you to view the value of the distance variable.
$ print $distance
4070
$
You can assign the value of one variable to another variable, also using the dollar sign.
In this example, the code assigns the value of the distance variable to another variable
called temp.
$ temp=$distance
$
$ unset temp
$ print $temp
$
Question
$ t=17h45
$ departure=$t
$ print $departure
Options:
1. 17h45
2. t
3. departure
Answer
Option 1 is correct. The value of the $departure variable is set to the value of the $t
variable. This variable is assigned a value of 17h45 in the first line of code.
Option 3 is incorrect. The command print $departure prints the value assigned to the
departure variable – in this case the value of the $t variable set in the first line of
code. It does not interpret $departure as the literal string departure.
3. Quoting
If you want to prevent the shell from processing spaces and special characters such as &
and *, you can enclose them in quotes.
For example, this line of code displays a heading that includes special characters.
Because the character string is enclosed in quotes, the shell displays it without processing
the * and & characters.
$ echo '***** Spain & Portugal *****'
***** Spain & Portugal *****
$
If you omit the quotes, the shell processes the special characters, returning a list of files and
an error message.
You can use the following kinds of quotes in UNIX shell scripts:
Single quotes also prevent variable and command substitution because they hide the
meaning of the $ character.
This means that command and variable substitution still take place in strings enclosed by
double quotes. Therefore, you can use double quotes to combine variable values with a text
string.
back quotes (``)
Back quotes (``) are a method of command substitution that the Bourne shell uses. The
Korn shell retains them for compatibility with this shell.
When you enclose a command in back quotes, the shell executes them and returns the
value of their output. You can assign this value to a variable.
Say you want to define a variable called pricetag and to specify that this variable must
contain the string $50. You need to do this using single quotes to prevent the shell from
treating the characters after the $ sign as a variable name.
$ pricetag='$50'
$ print $pricetag
$50
$
Let's say that you want the shell to read the value of the item and pricetag variables and
to include these values in the value of another variable called priceline.
You need to use double quotes so that you can combine the variable values with text.
$ pricetag='$50'
$ item='trombone'
$ priceline="The cost of a $item is $pricetag."
$ print $priceline
The cost of a trombone is $50.
$
Say you need to declare a variable that contains a full directory listing of the scripts
directory. Because the listing needs to be up to date, you need to use the ls command to
obtain it.
Using back quotes, you can assign the output of the ls command to the drlist variable.
$ drlist=`ls -a scripts`
$ print $drlist
. .. add_pkg_files$
Question
Options:
1. Var=&$100
2. Var=\$100
3. Var="$100"
4. Var='$100'
Answer
The backslash and single quotes prevent the $ symbol from being interpreted as a
special character.
Option 1 is incorrect. Placed at the end of a command, the ampersand causes the
command to run as a background job, but it does not escape special characters.
Option 2 is correct. The backslash escapes all special characters, including itself. So
the expression \\ will be interpreted by the shell as \.
Option 3 is incorrect. Double quotes prevent most special characters from being
interpreted, except $, \, and `.
Option 4 is correct. Single quotes prevent all special characters from being
interpreted, but they can't be nested.
Question
$ airline='PA'
$ flightno="$airline 771"
$ print $flightno
Options:
1. airline 771
2. flightno 771
3. PA 771
4. PA771
Answer
The output of the code is PA 771 because the double quotes allow variable
substitution and the single quotes include the space character.
Option 2 is incorrect. The print command prints the flightno variable, not the flightno
string.
Option 3 is correct. The first command assigns the value PA to the airline variable.
The second command substitutes the variable at the start of a string that consists of
a space and the number 771, and assigns the resulting string to the flightno variable,
which is then printed.
Option 4 is incorrect. Double quotes allow you to create variables that include spaces
because everything within the double quotes is interpreted as a single word by the
shell.
Summary
Variables are named spaces for storing data. The value of a variable can be one of several
data types, such as an integer or a character string. Some shell environments support more
data types than others.
You can set the value of a variable using an assignment statement or the typeset
command. You can unset them using the unset command. To access the value of a
variable, you precede the variable name with a dollar sign ($).
You can use quotes when assigning a value to a variable. Single quotes and double quotes
allow you to include special characters in strings. Back quotes allow you to assign the output
of a command to a variable.
Table of Contents
| Top of page |
| Learning objective |
| 3. Quoting |
| Summary |
| Print | Contents | Close |
After completing this topic, you should be able to describe and use script variables and
parameters.
The integer attribute, for example, defines a variable as an integer, and the left justify
attribute formats a string variable so that it's left-justified.
integer (-i)
float (-E, -F)
lowercase and uppercase (-u, -l)
right and left justify (-R, -L)
right and left justify with zeroes (-RZ, -LZ)
autoexport (-x)
integer (-i)
The integer attribute (-i) defines a variable as having the integer data type. This allows the
shell to process integer values faster, but it prevents you from assigning any non-integer
value to the variable.
float (-E, -F)
The float attributes (-E and -F) define a variable as a floating-point decimal number.
When you use the -E attribute, you can specify the number of digits in the number,
regardless of the position of the decimal point. For example, 3.1415 and 314.15 both have
five digits.
When you use the -F attribute, you can specify the number of digits after the decimal point.
For example, 196.543 and 2.160 both have three digits.
lowercase and uppercase (-u, -l)
The lowercase attribute (-l) converts any alphabetical characters in a variable's value into
lowercase characters.
right and left justify (-R, -L)
The right justify (-R) and left justify (-L) attributes allow you to format data. You need to
specify a field width when you set either of these attributes.
Variables with the -R attribute will display at the right end of the specified field width,
whereas variables with the -L attribute display at the left end of the specified field width.
right and left justify with zeroes (-RZ, -LZ)
The right justify with zeroes (-RZ or -Z) attribute right-justifies a variable to a specified field
width and fills the remainder of the field with leading zeroes.
The left justify with zeroes (-LZ) attribute left-justifies a variable to a specified field width and
strips all leading zeroes.
autoexport (-x)
The autoexport (-x) attribute causes the shell to export a local variable whenever its value is
set. This is useful if you want to ensure that the new value is available throughout the shell.
Paradoxically, you use a minus sign (-) to set an attribute and a plus sign (+) to unset it.
In this example, the code sets the integer attribute (-i) on the dist variable.
$ typeset -i dist
$
In this example, the code unsets the integer attribute on the dist variable.
$ typeset +i dist
$
You can set multiple attributes at the same time. In this example, the code sets the
uppercase attribute (-u) and the autoexport attribute (-x) on the name variable.
Question
Options:
Answer
You can set attributes on a variable either before, during, or after assignment.
However, you can't set attributes on a variable that has been unset.
To assign an attribute and a value to a variable at the same time, you use the typeset
command and specify the attribute before the value.
In this example, the code sets the integer attribute (-i) on the dist variable and assigns it
the value 489.
$ typeset -i dist=489
$
For example, if the value of name is Novak and you set the uppercase attribute (-u) on
name, its value becomes NOVAK.
$ name=Novak
$ typeset -u name
$ print $name
NOVAK
$
You can use the readonly attribute (-r) to prevent any further changes to the value or
attributes of a variable.
Unlike other attributes, you cannot unset the readonly attribute on a variable once you've set
it.
You can use the typeset command to display a list of all shell variables that currently have
a particular attribute.
For example, this command displays a list of all variables with the integer attribute (-i).
$ typeset -i
MAILCHECK=600
OPTIND=1
PPID=55394
RANDOM=22348
SECONDS=700
TMOUT=0
dist=489
$
Question
Options:
1. typeset-i
2. typeset -l
3. typeset -L
4. typeset -x
Targets:
Answer
The –i attribute defines a variable as an integer, the –x attribute exports it, the –l
attribute ensures all its characters are lowercase, and the –L attribute ensures that
the variable is left justified.
The –i attribute is useful when you are using variables in calculations. It enables you
to make sure that numbers are assigned to the variables, rather than letters.
The –l attribute is useful if you need to format the case of a variable consistently,
irrespective of its value.
The –L attribute strips any leading spaces from a variable, which is useful when you
want to avoid errors in shell scripts in which variables are set interactively by the
user.
If you want to make a variable available to subshells, you must export it using the –x
attribute. However if you want the variable to be available between logins, or to other
users, you will need to add it to a shell configuration file.
Parameters are reserved variables that the shell uses. They allow you to control the
behavior of the shell or to obtain information about current processes.
Environment variables are parameters that control the shell's behavior. They are global
variables declared in each user's .profile file.
User-defined variables can't use the same names as the environment variables because the
system has reserved these names. However, users can change the value of environment
variables.
The shell has some special parameters that allow you to interact with current processes,
including
?
-
!
$
PPID
ERRNO
?
The ? parameter contains the exit status of the last command that the shell has executed. If
the command executed successfully, the exit status is 0. If it didn't execute successfully, the
exit status is non-zero.
-
The - parameter contains the shell options that are set on the current shell process.
!
The ! parameter contains the process ID (PID) of the last background command or co-
process that the shell has executed.
$
The $ parameter contains the process ID (PID) of the current shell process.
PPID
The PPID parameter contains the process ID (PID) of the current shell's parent shell
process.
ERRNO
The ERRNO parameter contains the error number returned by the most recent failed system
call.
Question
What do you think you can do to the values of special parameters such as ? and $?
Options:
1. Reassign them
2. Set them
3. Unset them
4. View them
Answer
You can view the values of special parameters, but you can't reassign them, set
them, or unset them.
Positional parameters contain arguments that have been passed to a command, script, or
function. They allow you to use these arguments in subsequent lines of a script.
Positional parameters are named according to the positions of the arguments they contain.
Parameter 0 contains the name of the command or function to which the arguments apply,
parameter 1 contains the first argument, parameter 2 contains the second argument, and so
on.
In this command, for example, the value of parameter 0 is ls, the value of parameter 1 is
-a, and the value of parameter 2 is the specified directory path.
$ ls -a /usr/home
This example of a function searches the current directory for a specified file and displays the
contents of the file one screen at a time.
When you call the function and specify a filename as an argument, the filename is stored in
the positional parameter $1. This allows the commands in the function to work with the
filename.
disf() {
> ls -al | grep $1
> cat $1 | more
> }
$
You can call positional parameters only up to $9. If you want to use ten or more positional
parameters, you need to enclose the identifying numbers in the parameter names in braces
({}).
Question
Options:
1. Although commands can take any number of arguments, you can only view nine
positional parameters
2. Positional parameter 2 contains the second argument to be passed to a command
3. Positional parameter 0 always contains the value of a command or name of a
function
4. You can use positional parameters within a function to manipulate its arguments
Answer
Option 1 is incorrect. You can view positional parameter 10 or greater, but you must
enclose it in braces, as in ${10}.
Option 2 is correct. For the command cat file1 file2, the value of $2 is file2.
Option 3 is correct. For the command cat file1 file2, the value of $0 is cat.
Summary
You can set attributes that apply to a variable. Attributes can modify the way the shell
processes a variable – for example by defining it as an integer or floating-point decimal
number. They can also modify the way the shell displays a variable value – for example by
specifying that it display as uppercase or left-justified characters.
Parameters are predefined system variables. They include environment variables, which
define characteristics of the shell environment, special parameters, which allow you to
obtain data about current processes, and positional parameters, which allow you to work
with arguments.
Table of Contents
| Top of page |
| Learning objective |
| Summary |
| Print | Contents | Close |
After completing this topic, you should be able to explain variable expansion and use array
variables.
1. Variable expansion
Variable expansion allows you to access the values of variables. The simplest form of
variable expansion involves referring to the value of a variable directly. To do this, you place
a dollar sign ($) in front of a variable name.
Other forms of expansion allow you, for example, to refer to portions of a variable's value or
to the length of a variable's value, or to specify default or alternative variable values.
You can use braces ({}) to delimit the value of a variable from any characters following the
variable name. This allows you to use the variable name in assignment statements that
change the value of the variable.
In this example, the variable en contains the name "EasyNomad." Using braces, you can
change its value to "EasyNomadTravel" by invoking its original value.
$ en=EasyNomad
$ en=${en}Travel
$ print $en
EasyNomadTravel
$
If you precede a variable name with a hash sign (#), you can determine the length of its
value – the number of characters it contains.
In this example, the output of the command specifies that the value of the en variable
contains 15 characters.
$ print ${#en}
15
$
The following variable expansion formats allow you to work with default and alternative
values:
${variable:-word}
${variable:=word}
${variable:+word}
${variable:?word}
${variable:?}
${variable:-word}
The ${variable:-word} syntax returns the value of the variable if it's set and non-null. If it's not
set or if it's set to null, the expression returns the text string word.
For example, ${name:-Name not found} returns the value of the name variable. If name
is null or not set, it displays the message "Name not found".
${variable:=word}
The ${variable:=word} syntax returns the value of the variable if it's set and non-null. If it's
not set or if it's set to null, the expression assigns the text string word to the variable and
then returns the variable's new value.
For example, ${name:=anonymous} returns the value of the name variable. If name is null
or not set, it assigns the value "anonymous" to name and returns the value "anonymous".
${variable:+word}
The ${variable:+word} syntax returns the text string word if the variable is set and non-null. If
it's not set or if it's set to null, the expression returns nothing.
For example, ${name:+Name already exists} returns the string "Name already
exists" if the value of the name variable is set and non-null. If name is null or not set, nothing
is displayed.
${variable:?word}
The ${variable:?word} syntax returns the value of the variable if it's set and non-null. If it's
not set or if it's set to null, the expression returns the text string word and the shell
terminates.
You can use this format to ensure that essential variables are set before a script executes
further.
${variable:?}
The ${variable:?} syntax returns the value of the variable if it's set and non-null. Otherwise,
the expression returns the default error message "parameter null or not set" and the shell
terminates.
The following variable expansion formats allow you to extract substrings from variable
values:
${variable:start}
${variable:start:length}
${variable#pattern}
${variable##pattern}
${variable%pattern}
${variable%%pattern}
${variable:start}
The ${variable:start} syntax returns part of the value of a variable, from the character
position specified by the integer start to the end of the value.
For example, if the value of the en variable is EasyNomad, the expression ${en:4} returns
"Nomad".
${variable:start:length}
The ${variable:start:length} syntax returns part of the value of a variable. The selected
substring begins at the character position specified by the integer start and contains as
many characters as specified by length.
For example, if the value of the en variable is EasyNomad, the expression ${en:4:3}
returns "Nom".
${variable#pattern}
The ${variable#pattern} syntax finds the shortest substring that matches pattern, beginning at
the left of the variable's value. It then removes this substring and returns the remainder of
the value.
For example, say the value of the tour variable is "landing ground on island". The
expression ${tour#land*} finds the first and shortest instance of land* and removes it. It
therefore returns "ing ground on island".
${variable##pattern}
The ${variable##pattern} syntax finds the longest substring that matches pattern, beginning
at the left of the variable's value. It then removes this substring and returns the remainder of
the value.
For example, if the value of the tour variable is "landing ground on island", the expression
${tour##land*} finds the first and longest instance of land*, which is the entire string. It
removes this and returns nothing.
${variable%pattern}
The ${variable%pattern} syntax finds the shortest substring that matches pattern, beginning
at the right of the variable's value. It then removes this substring and returns the remainder
of the value.
For example, if the value of the tour variable is " landing ground on island", the expression
${tour%*land} finds the last and shortest instance of land* and removes it. It therefore
returns "landing ground on is".
${variable%%pattern}
The ${variable%%pattern} syntax finds the longest substring that matches pattern, beginning
at the right of the variable's value. It then removes this substring and returns the remainder
of the value.
For example, if the value of the tour variable is "landing ground on island", the expression
${tour%%*land} finds the last and longest instance of land*, which is the entire string.
Removing this, it returns nothing.
The following variable expansion formats allow you to find and replace substrings in variable
values:
${variable/pattern1/pattern2}
${variable//pattern1/pattern2}
${variable#pattern1/pattern2}
${variable%pattern1/pattern2}
${variable/pattern1/pattern2}
${variable/pattern1/pattern2} replaces the first instance of pattern1 in a variable's value with
pattern2 and returns the result.
For example, say the variable heading contains the string "Last-minute flights and cut-
price flights". In this case, the expression ${heading/flight/tour} returns the string
"Last-minute tours and cut-price flights".
${variable//pattern1/pattern2}
The ${variable//pattern1/pattern2} syntax replaces all instances of pattern1 in a variable's
value with pattern2 and returns the result.
For example, if the variable heading contains the string "Last-minute flights and cut-price
flights", the expression ${heading//flight/tour} returns the string "Last-minute tours
and cut-price tours".
${variable#pattern1/pattern2}
The ${variable#pattern1/pattern2} syntax replaces the first instance of pattern1 in a variable's
value with pattern2 if the value begins with pattern1.
For example, say the variable col3 contains the string "Cost of flight." In this case, the
expression ${col3#Cost/Price} returns the string "Price of flight".
${variable%pattern1/pattern2}
The ${variable#pattern1/pattern2} syntax replaces the last instance of pattern1 in a variable's
value with pattern2 if the value ends with pattern1.
For example, if the variable col3 contains the string "Cost of flight", the expression $
{col3%flight/cruise} returns the string "Cost of cruise".
Question
Select the statements that correctly describe variable expansion of the file variable.
Options:
Answer
Variable expansion allows you to reuse the current value of a variable when
assigning a new value, return the number of characters in a variable, and return just
part of a variable.
Option 1 is correct. You can expand the current value of a variable and include it as
part of a new string, or perform an arithmetic operation on it if it is an integer to
create a new value.
Option 2 is correct. When using braces to expand a variable, preceding the variable
name with a # symbol returns the number of characters in a variable, not the string or
integer itself.
Option 3 is correct. The number in this syntax identifies where the returned string
starts. In this case the variable has been assigned a five-character string, so only the
last character is returned.
Option 4 is incorrect. This only happens if the file variable has either not been
assigned, or has been assigned a null value.
Question
Options:
Answer
Using variable expansion formats, you can display a custom error message if a
variable is not set, extract a substring from the value of a variable, and replace a
substring in the value of a variable with another substring.
Option 1 is correct. If the file variable is not set, or has a null value, this command
will output the message "File not found":
echo ${file:-File not found}
Option 2 is correct. If the file variable has been assigned the value file1, you
can return just the 1 using the command:
echo ${file:4}
Option 3 is correct. If the file variable has been assigned the value file1, you
can replace the 1 with a 2 using the command:
echo ${file/1/2}
Option 4 is incorrect. Variable expansion isn't used for this. You simply assign the
variable as shown:
newvar=$file
2. Arrays
Arrays are variables that can contain multiple values, stored as a sequence of elements.
The name of an array variable denotes the entire array. You refer to an individual element in
an array using a subscript that identifies its position in the sequence of elements in the
array. This subscript takes the form of a number enclosed in square brackets.
Numbering begins at zero. Therefore, the subscript of the first element is [0], the subscript of
the second element is [1], and so on.
Question
Options:
Answer
If a variable already has an existing value when you convert it into an array, that value
becomes the value of the array's first element.
In this example, the code assigns the value "January" to the mth variable. It then assigns
the value "February" to the mth[1] element, thereby making it an array. The value of
mth[0] is still "January."
$ mth=January
$ mth[1]=February
$ print ${mth[0]}
January
$
If you use arithmetic expressions or variables in the subscript of an array element, the shell
evaluates the expressions or variables and then references the array.
In this example, the code assigns a value to element 4 + 9 of the items array, which lists
souvenirs for sale. The next line of code displays the element whose subscript is contained
in the variable count. Assuming that the value of count is 3 and the fourth element in the
array is "wooden giraffe", the resulting output is "wooden giraffe."
$ items[4+9]="safari hat"
$ print ${items[$count]}
wooden giraffe
$
The following variable expansion formats allow you to work with array variables:
${array[*]}
${!array[*]}
${!array[*]:n}
${!array[*]:n:x}
${#array[*]}
${#array[n]}
${array[*]}
${array[*]} lists all elements of an array.
${!array[*]}
${!array[*]} lists all initialized subscript values of an array.
${!array[*]:n}
The ${!array[*]:n} syntax lists all array elements, beginning with the subscript n.
For example, ${!items[*]:5} lists all elements of the items array from the sixth
element onwards.
${!array[*]:n:x}
The ${!array[*]:n:x} syntax lists x array elements, beginning with the subscript n.
For example, ${!items[*]:5:3} lists the sixth, seventh, and eighth elements of the
items array.
${#array[*]}
The ${#array[*]} syntax returns the number of elements in an array.
${#array[n]}
The ${#array[n]} syntax returns the length of the array element specified by the subscript [n].
Let's say that you want to write a function that accepts an element number as an argument
and displays a specified element, as well as the number and length of this element, in the
items array. The items array is an existing array that lists souvenirs for sale.
$ get_item() {
>
You begin by printing the subscript specified in the argument. You do this using a positional
parameter.
$ get_item() {
> print "Item no. $1"
Then you print the length of the specified element, using the appropriate array variable
expansion format.
$ get_item() {
> print "Item no. $1"
> print "Length ${#items[$1]}"
$ get_item() {
> print "Item no. $1"
> print "Length ${#items[$1]}"
> print ${items[$1]}
> }
$
When you call the get_item function for element 2, the data for the third element of the
items array displays.
$ get_item() {
> print "Item no. "$1+1
> print "Length "${#items[$1]}
> print ${items[$1]}
> }
$ get_item 2
Item no. 2
Length 14
leopard poster
$
Question
In this question, the variable var is an array.
Match the expansion statements for the var variable to their corresponding
functions.
Options:
1. ${var[*]}
2. ${!var[*]}
3. ${#var[*]}
Targets:
Answer
The statement ${var[*]} returns all the elements in the array, ${!var[*]}
returns every initialised subscript, and ${#var[*]} returns the number of elements
in an array.
The value returned when you use the # metacharacter will be equivalent to the value
of the last subscript to be initialized for the array.
Summary
The process of variable expansion allows you to work with the values of variables. Using
variable expansion formats, you can extract or replace part of a variable's value. You can
also provide default and alternative values.
You can store multiple values in a variable by turning it into an array. You assign and access
the elements of an array using a subscript in square brackets.
Table of Contents
| Top of page |
| Learning objective |
| 2. Arrays |
| Summary |
| Print | Contents | Close |
After completing this topic, you should be able to use arithmetic operators in shell scripts.
You can incorporate arithmetical and logical expressions in UNIX command-line statements
and shell scripts. The shell evaluates these expressions and substitutes the results for the
expressions themselves.
To declare an integer variable, you can simply assign an integer value to it. For example,
the code itemID=5 declares the integer variable itemID.
However, you can also declare an integer variable explicitly, which improves performance
and allows type checking. To do this, you use the integer command.
$ integer itemID=5
Alternatively, you can declare variables explicitly using the typeset command with the -i
option to set the integer attribute on them.
$ typeset -i itemID=5
Integer variables in the UNIX shell don't need to use the base-10 decimal system, although
this is the default.
To specify a base number other than 10 for an integer variable, you use a hash symbol (#)
between the base and the value. In this example, the code assigns the binary value 1101 to
the nibble variable.
$ nibble=2#1101
You can specify a base number after the -i option of the typeset command. In this
example, the code assigns the hexadecimal value of B1F7 to the position variable.
Note
In early versions of UNIX, the expr command was used to evaluate expressions. This
example shows the variable i being incremented by 1.
$ i = `expr $i + 1`
The let command doesn't require that you use a dollar sign ($) when referring to a
variable's value. This bypasses the process of variable expansion, which makes it up to 60
times faster than the expr command.
$ let i=i+1
As an alternative to using the let command to evaluate expressions, you can use double
brackets as shown here. The shell treats the code in the brackets as an arithmetical
expression and evaluates it.
$ ((i=i + 1))
When the shell evaluates an arithmetical expression, it follows the conventional order of
precedence for arithmetical operators.
Multiplication (*) and division (/) operators take precedence over addition (+) and subtraction
(-) operators, unless you use brackets to override this precedence.
Question
What do you think is the result value of the following expression?
12 * (48 / 6 + 2)
Options:
1. 36
2. 72
3. 120
4. 144
Answer
The UNIX shell supports a modulo operator (%), which returns the remainder that results
from the division of one expression by another.
$ ((rem = 18 % 8))
$ print $rem
4
$
You can assign negative values to variables by placing a minus sign (-) immediately in front
of the value.
However, you need to use the print - command to display negative values, because
otherwise the print command interprets the minus sign in a variable value as an
argument.
Question
Options:
Answer
You can use the typeset option to explicitly declare integer variables that aren't base
10, such as variables containing binary numbers, which are base 2.
Option 1 is correct. Integer variables can use any base between 2 and 36, although
the default is 10. You can declare hexadecimal integers like this:
Variable=16#b6
Option 2 is correct. Binary numbers are base 2, and you can use variables with base
2 integers as values.
Option 3 is correct. To explicitly declare an integer variable, you use the typeset –
i option.
The UNIX shell supports logical operators that perform Boolean logic operations on
expressions.
If both are true or evaluate to zero, the entire expression evaluates to zero.
If either or both are false or evaluate to one, the entire expression evaluates to one.
logical OR (||)
The || operator compares two expressions logically and evaluates the entire expression to
zero if either of them are true.
The entire expression evaluates to one only if both expressions are false.
logical negation (!)
The ! operator evaluates an expression and then reverses it logically.
If an expression evaluates to zero, the ! operator changes it to one, and vice versa.
Whereas logical operators compare and manipulate entire expressions, bitwise operators
perform Boolean logic on individual bits.
bitwise OR (|)
The | operator compares two expressions bit by bit. If either of the bits is 1, it returns a 1. It
returns a 0 only if both bits are 0.
The UNIX shell supports bit shift operators. These act on binary number values, shifting
their bits either to the left or to the right by a specified number of places.
For example, ((2#1001 >> 1)) evaluates to 2#100. Bits at the right end of the original
values are lost.
Question
Which operator performs bitwise reverses on the value of each bit in a binary
expression?
Options:
1. ~
2. &
3. ^
4. |
Answer
The ~ operator performs bitwise negation on the value of each bit in a binary
expression.
Option 1 is correct. If you perform bitwise negation on the base2 integer 10001, the
result will be 1110.
Option 2 is incorrect. You use the & symbol to perform a bitwise AND on two binary
expressions.
Option 3 is incorrect. You use the ^ symbol to perform a bitwise XOR on two binary
expressions.
Option 4 is incorrect. You use the | symbol to perform a bitwise OR on two binary
expressions.
Question
1. 0100
2. 11
3. 0
4. 100
Answer
The output of this code is 2#1011. The first statement sets x to 2#0100 and the
second statement performs bitwise negation on this value.
Option 1 is incorrect. This is not valid output for a binary integer. If it were, the
leading zero would be stripped.
Option 2 is correct. The first statement compares the two binary numbers, bit by bit.
Where both bits are set, the returned bit is also set. Where one bit isn't, the returned
bit is also not set. The second statement performs bitwise negation, unsetting set bits
and setting unset bits.
Option 3 is incorrect. This would be the result if the first statement were a bitwise
OR, yielding a result of 1111.
Option 4 is incorrect. This would be the result if the first statement were a bitwise
XOR, yielding a result of 1011.
3. Additional operators
The UNIX shell supports a range of comparative operators that compare the values of two
expressions. These are useful when you need to set conditions for conditional statements.
The UNIX shell simplifies this task with increment and decrement operators.
The increment operator (+=) sets a variable to a new value that's equal to its previous value
plus any number you specify after the operator.
Question
Options:
1. 419
2. 2
3. 3
Answer
Option 3 is correct. The $? variable is assigned the exit status of the previous
command. In this case, the previous command is the conditional expression that
evaluates as false, so the value that is assigned to $x is 1. The next expression
increments the value of $x by 2, and the print statement prints the new value of $x –
3 – to the screen.
The decrement operator (-=) sets a variable to a new value that's equal to its previous value
minus any number you specify after the operator.
Question
Options:
1. $x >= $y
2. $x < $y+=4
3. $x == $y
4. $x != $y
Answer
$x < $y+=4, $x >= $y, and $x != $y all use comparative and increment operators
correctly to construct valid statements.
Option 1 is correct. The >= operator means that the value on the left is equal to or
greater than the value on the right of the operator.
Option 2 is correct. The +=4 operator increments the y variable by 4, making it larger
than the x variable, so the statement is correct.
Summary
You can evaluate arithmetical expressions using the let command or the ((…)) construct.
The basic arithmetical operators include addition, subtraction, multiplication, division, and
modulo operators. These follow the conventional precedence order – with multiplication and
division taking precedence over addition and subtraction – unless parentheses override this
convention.
The UNIX shell supports logical and bitwise operators that perform Boolean logic on entire
expressions or on individual bits of binary numbers.
The UNIX shell also supports comparative operators that compare the values of two
expressions.
Table of Contents
| Top of page |
| Learning objective |
| Summary |
| Print | Contents | Close |
After completing this learning object, you should be able to use variables and arithmetic in
a shell script.
Exercise overview
In this exercise, you're required to write a piece of code that performs calculations using
variables.
Let's say that you're working for the EasyNomad travel company and you're required to write
a script that calculates the amount of commission that agents earn on each package tour
they sell.
For the commission script, you need a variable to store the commission percentage that
agents receive and another variable to store the amount received.
Step 1 of 2
Type the command that declares variables explicitly and the attribute that specifies a
variable to be an integer.
#!/usr/local/bin/ksh
MISSING CODE
Result
Step 2 of 2
Complete the line of code that creates a variable called commrate and assigns the
value of 10 to it.
#!/usr/local/bin/ksh
typeset -i MISSING CODE
Result
#!/usr/local/bin/ksh
typeset -i commrate=10
typeset -i amount
The script you are writing needs to accept the value of a tour package that an agent sells as
its first argument.
Step 1 of 3
#!/usr/local/bin/ksh
typeset -i commrate=10
typeset -i amount
MISSING CODE
Result
Step 2 of 3
Type the code that assigns the value of the script's first argument to the amount
variable.
#!/usr/local/bin/ksh
typeset -i commrate=10
typeset -i amount
let MISSING CODE
Result
You type amount=$1 to assign the value of the script's first argument to the amount
variable.
Step 3 of 3
Type the arithmetical expression that applies the commission percentage to the value
of the amount variable.
#!/usr/local/bin/ksh
typeset -i commrate=10
typeset -i amount
let amount=$1
let amount= MISSING CODE
Result
You add a print statement to output the value of the amount variable. The script is now
complete.
#!/usr/local/bin/ksh
typeset -i commrate=10
typeset -i amount
let amount=$1
let amount=amount*commrate/100
print amount
Table of Contents
| Top of page |
| Learning objective |
| Exercise overview |
| Print | Contents |
This article discusses the history of UNIX text and format processing tools.
Plain text
Although computer processors act on binary data, binary output is not legible to human
beings. Output needs to be in plain text for processed data to be accessible to both the
computer and its human operator.
One of the original purposes of UNIX was to develop, maintain, and track documents and
memos in the AT&T/Bell Telephone Company in the late 1960s.
The text editors at this time were referred to as line editors because they required all
changes to a file to be made on a line-by-line basis. Each command to a line editor specified
a line number, and the changes to be applied to that line. The original standard UNIX line-
mode text editor – named "ed" – was first compiled in assembly language. It was later
rewritten in C, although it retained the same functionality.
Text formatting
To improve the readability of text created in a text editor on UNIX systems – and to print this
text, you need a separate program to format it. Text formatters allow you to specify features
such as fonts, spacing, paragraphs, margins, and footnotes when displaying or printing
documents. The earliest UNIX text formatter was runoff, which was named after print runs in
the publishing industry. You can run this formatter using the command roff. Since runoff,
next runoff (nroff) and typesetting runoff (troff) have been developed to provide additional
features and greater flexibility.
Other text formatters such as SCRIPT for IBM mainframes, Scribe, Digital Standard Runoff
(DSR), and TeX have also been developed.
TeX is a text processor used mainly in scientific and mathematical environments to produce
papers, books, and theses. It is especially suited to material with a high mathematical
content. TeX provides fonts and font sizes, page layout, mathematical equations,
mathematical characters, diacriticals, and alternative character sets. For example, it
provides Greek symbols, as well as support for indexing, footnotes, line drawing, diagrams
and boxes, multicolumn printing, and tables.
LaTeX is a set of macros based on TeX macros that facilitates the structuring of large
documents. It provides automated section and chapter, cross-referencing, and bibliography
macros.
Emacs
The Emacs text editor was written mostly in the programming language Lisp. At the core of
Emacs, there is a full-featured Lisp interpreter written in C. This built-in programming
language allows you to customize and extend Emacs.
You can start up a UNIX shell from inside Emacs. In this case, you receive a UNIX prompt
inside the current Emacs window. You can then run commands – including compile
commands – as you would outside of Emacs and save the shell output to a file.
In addition to the runoff text formatter, Emacs, and TeX, there are a number of text tools that
system administrators commonly use in shell programming. The most commonly used tools
are
sed
The sed utility is a stream editor that allows you to search for and replace text strings in a
file. This allows you to delete lines of text by "replacing" them with nothing, which is
especially useful in the case of lengthy system logs and e-mails.
You can execute sed commands at the command line or you can run a command file to
replace characters and strings in a specified data file.
In the following example, the sed command deletes the first two lines of the easynomad.txt
file:
awk
Named after its originators, Aho, Weinberger, and Kernighan, awk is a complete pattern-
scanning and processing language. It allows you to extract parts of a large body of text, to
format text, or to extract information for other programs. However, it is most commonly used
as a UNIX command-line filter to reformat the output of other commands, such as PERL
commands.
You can run awk commands from the command line as a simple text-processing tool, or you
can create a command file containing awk statements.
In the following example, the awk command will remove the second field from each line in a
file called profile.
PERL
PERL (Practical Extraction and Report Language) is a portable text, process, and file-
manipulation tool that bridges the gap between shell and C programming. Its syntax
resembles the syntax you use in shell programs and in C programming. PERL also includes
awk, sed, and grep as built-in functions.
PERL was developed originally as a data reduction language capable of processing large
quantities of data efficiently.
Summary
The three most commonly used text-processing tools are awk, sed, and PERL. Awk allows
you to perform pattern-scanning and processing, sed allows you to perform search and
replace operations, and PERL is a text, process, and file-manipulation tool.
Table of Contents
| Top of page |
| Abstract |
| Plain text |
| Emacs |
| Summary |
| Print | Contents | Close |
After completing this topic, you should be able to explain how to use the sed line editor.
1. Introducing sed
Unlike text editors that allow you to alter a file by moving the cursor on the screen, sed is a
noninteractive line editor. It processes one line of standard input at a time and returns the
results as standard output.
In most cases, it processes input via a keyboard and returns output to the screen. It doesn't
alter original files.
In its simplest operation, sed works as a filter that searches for a specified pattern and takes
a specified action if it matches the pattern.
You often use sed to perform search and replace operations on the content of specified text
files. However, you can also use it to append, display, delete, or insert text.
You can perform many of these functions using the vi text editor. However, sed provides a
better solution than the vi text editor for working with multiple text files.
Sed applies each editing command to the first line in a file and then moves on to subsequent
lines one at a time.
Question
Because sed works with open format files – or plain text files – it is portable and
supports a wide range of text applications.
Options:
Answer
You can use sed to convert line spacing in a file and to convert DOS newlines to
UNIX newlines. You can also use sed to delete leading or trailing spaces and
consecutive blank lines, and to perform substitutions on all or specific lines.
You can create a script containing only editing commands, which you can then pass as a
file for execution by sed using the -f option.
For example, the code sed -f sample.sed memo.txt causes sed to execute the
commands in a file called sample.sed on a file called memo.txt.
creating a sed executable file
An executable sed script contains the line #!/usr/bin/sed -f, which allows you to run it
through the shell using the ./ command.
However, you need to use the chmod command to give the file execute permissions before
you can run it.
Question
You have stored sed editing commands in a text file named edit.sed.
Which sed command do you execute at the command line to edit the file
reports.txt?
Options:
Answer
You use sed –f edit.sed reports.txt to edit the reports.txt file using the sed
commands in edit.sed.
Option 1 is incorrect. It is possible to pipe input into sed, but to do so, you would use
the following command:
cat reports.txt | sed –f edit.sed
Option 2 is correct. You use the –f option to specify the file containing the sed
commands, and then you provide one or more files for sed to operate on.
Option 3 is incorrect. The file containing the sed commands is provided after the –f
option, and not after the file on which sed is to operate on.
Option 4 is incorrect. This command would work if edits.sed were an executable file,
but in this example it just contains sed commands.
To specify a sed command in a script or on the command line, you follow the command with
a string that includes the following components:
editing command
regular expression and pattern string
replacement text
For instance, the code sed 's/^night/day/' substitutes the word "night" with the word
"day" when the word night occurs at the beginning of a line.
Note
The sed utility supports most of the regular expressions that other UNIX text utilities (
such as awk, gawk, and grep ( and UNIX shells support.
The most commonly used regular expressions with the sed command are
^
$
*
?
[ ]
^
The ^ expression attempts to match a pattern string with the beginning of a line in a
specified file. If a ^ replaces a line number, for example, it denotes the first line in a file.
For instance, the command sed 's/^Thu /Thursday/' easynomad will substitute Thu
with Thursday only when Thu occurs at the beginning of a line.
$
The $ expression attempts to match a pattern string with the end of a line in a specified file.
For instance, the command sed 's/ $//' easynomad will replace any space character
( indicated by a forward slash ( that occurs at the end of a line.
*
The * expression attempts to match any occurrences of a specified character with
characters in the content of a specified file.
For instance, the command sed 's/[ ]*//' easynomad will delete any spaces in the
file.
?
The ? expression attempts to match one instance of a specified character with the content
of a specified file.
For instance, the command sed 's/?D/4/' easynomad will substitute the first instance
of the letter D with the number 4.
[ ]
The [ ] expression attempts to match the content of the square brackets with a specified
pattern string.
Question
Options:
Targets:
Answer
The command sed 's/ *$//' test1 deletes all trailing spaces from each line,
sed 's/[ ][^ ]*$//' test2 deletes the last word on each line, and sed
's/^$/ blank line/' test3 finds all blank lines by identifying each line end
that follows immediately after a line beginning.
The -s option to sed substitutes anything matched by the first expression with the
contents of the second expression. In this case, the metacharacter * matches the
preceding character – a space – zero or more times. The $ matches the end of a
line. Because nothing is specified in the second expression, the effect is to delete all
trailing spaces from each line.
The -s option to sed substitutes anything matched by the first expression with the
contents of the second expression. The expression [.][^ ]*$ matches any string
that starts with a space, but which contains anything other than a space thereafter,
and is at the end of a line. The effect of this is to match every word at the end of a
line.
When it is not preceding a class or range, the circumflex (^) matches the start of a
line or string. In this example, ^$ matches the start and end of an empty line. To
insert a paragraph separator such as --- you would adapt the expression as follows:
sed 's/^$/---/' test3
In addition to the substitute command (s), you can use the insert (i), append (a), and delete
(d) commands with the sed command.
For instance, the command sed '$d' text.txt deletes the last line in a file named
text.txt.
You can preface sed commands with line numbers to specify which lines in the input file are
to be modified by the sed command.
For instance, 2 s/tourist/tourists/ looks for the string "tourist" in line 2 of a specified
file and replaces it with the string "tourists."
In the example sed 9 a\Copyright EasyNomad 2003, the sed command appends the
string "Copyright EasyNomad 2003" after line 9 in a specified file.
If you wanted the string inserted before line 9, you would specify the command sed 9
i\Copyright EasyNomad 2003.
You can use two advanced sed commands – branching (-b) and testing (-t) – to rearrange
the order in which portions of a script execute.
These commands transfer control to a line containing a specified label. If the label does not
appear in the script, control passes to the end of the script.
Branching passes control to the end of the script automatically, but testing requires an s
(substitution) command to change the current line before control is passed.
Note
The UNIX man pages provide a full listing of the sed options, with examples of their
usage.
Question
Options:
1. You can specify the line on which a sed command will operate
2. You use the a command to append text in sed
3. You use the i command to insert text in sed
4. You use the –n option to insert new lines
Answer
The a command appends text to the specified location, while the i command inserts
it. You preface sed commands with the specific line on which you want them to
operate.
Option 1 is correct. You can precede your sed command with the number of the line
on which it is to operate.
Option 2 is correct. You use the a command to append text in sed. For example, this
command adds the text "circa 1848" to the end of line 10:
sed 2 a\circa 1848
Option 3 is correct. You use the i command to insert text in sed. For example, this
command adds the text "circa 1848" to the start of line 10:
sed 2 i\circa 1848
Option 4 is incorrect. You use the –n option to suppress printing of all output except
that explicitly matched by the sed command.
Question
Options:
1. ^
2. $
3. *
4. [ ]
Targets:
Answer
You match the start of a line with ^ and the end of a line with $. You use [ ] to
match a class of characters and * to match the preceding character, zero or more
times.
The ^ command matches the beginning of a line. In this example, the sed command
will delete any new line that starts with a space:
sed '/^ /d'
You match the end of a line with the $ command. For example, this command
changes DOS newlines to the UNIX format:
sed 's/.$//'
The * command matches the preceding character, zero or more times. The following
sed command will replace the character 2 with 1:
sed 's/2*/1/g'
The [ ] command matches a class of characters. For example, the following sed
command will delete any line that starts with a number:
sed '/^[1-9]/d'
Let's say that you have a simple phone list that you need to extract data from using sed
commands.
Suppose that you begin by replacing instances of 100- with (100) so that you can distinguish
the area codes from the rest of phone numbers more easily.
The output shows that only the first instances of "100-" in each line were changed.
To change all instances of "100-" in each row, you use the g option.
For instance, both Greg and Tanya's cell numbers are now prefixed with "(100)".
Note
Even though you've used s (substitution) options on the phone list file, its original
content doesn't change. The command to change "100-" to "(100)" , for instance,
changes only the data output and not the original text.
Let's say that you want to replace the string "2222" with "5555" and to instruct sed to print
only the lines affected by the change.
Using the p option prints Tanya and Sam's records because these are the only records that
contained the string "2222".
The example here shows how you can use the d (delete) option to display a specific number
of lines – in this case, the first three lines ( in a file.
You can use the q (quit) option to display lines up to and including a specified line – line 5,
in this example.
Let's say that you want to print all records in the phone list except Sam's record.
To do this, you use the d (delete) option with the specification that all lines beginning with
the word "Sam" be excluded from the output.
In this example, the code replaces a 3-digit string with a 2-digit string and saves the output
in a new text file named newphone.txt.
If you want to execute a number of sed commands on the same text file, it's advisable to
write a script that you can then run against the file.
cat sedscript
s/555/55/g
s/100-/(100) /g
s/101-/(101) /g
s/easynomad/EasyNomad/g
When you run a sed script against the phone list text file, it outputs the table with all the
substitutions in the script implemented.
Summary
Sed is a noninteractive line editor, which processes one line of standard input at a time and
returns the results as standard output.
To specify a sed command in a script or on the command line, you follow the command with
regular expressions, the pattern string that you want the command to locate, and
replacement text for this string.
Sed allows you to perform a range of text edits on plain text files, either as single or multiple
command-line instructions or by passing multiple instructions in a script.
Table of Contents
| Top of page |
| Learning objective |
| Summary |
| Print | Contents | Close |
After completing this topic, you should be able to use sed in a UNIX shell script to edit a
file.
Exercise overview
In this exercise, you're required to create a new configuration file and perform a sed
substitution.
You decide to use sed to search and replace host addresses and to save the output as a
new file called "en_hosts".
Step 1 of 2
Let's say that you want to change all instances of "190" to "192".
See if you can type the sed command that will substitute the 190 string.
Result
You enter 's/190/192/g' to specify the global substitution that will change all instances
of "190" to "192."
Step 2 of 2
Let's say that you want to create a new configuration file containing the changed
output.
Choose the code that you think will specify hosts as the input file and en_hosts as
the output file.
Options:
Result
The code that specifies hosts as the input file and en_hosts as the output file is
hosts > en_hosts
Option 1 is incorrect. This option uses en_hosts as the input file and hosts as the
output file.
Option 2 is correct. The data on the left of the > redirector is written to the file on the
right of the redirector.
Option 3 is incorrect. This command would redirect the hosts file as an argument to
the command on the left of the redirector. Since en_hosts is a file, this command will
generate an error.
Step 1 of 2
Let's say that you need to change all instances of "gleeson" to "gleeson_assoc" in
the en_hosts configuration file.
Choose the code that you think will substitute all the text strings that match
"gleeson."
Options:
1. 's/^gleeson/gleeson_assoc/g'
2. 's/gleeson_assoc/gleeson/g'
3. 's/gleeson/gleeson_assoc/p'
4. 's/gleeson/gleeson_assoc/g'
Result
You enter 's/gleeson/gleeson_assoc/g' to specify the substitution that will change all
instances of "gleeson" to "gleeson_assoc" in the en_hosts configuration file.
Option 1 is incorrect. This would only replace the string "gleeson" if it appeared at the
start of a line.
Option 2 is incorrect. This would replace gleeson_assoc with gleeson, instead of vice
versa.
Option 3 is incorrect. The g flag is required to ensure the each instance is replaced,
not just the first. The p flag sends the results of each replacement to standard output.
Option 4 is correct. You use the s command with the g flag to perform global
substitutions. The first term is the search term that is replaced with the second term.
Step 2 of 2
See if you can complete the sed command to output the new configuration
information to screen.
Result
You enter en_hosts to specify the filename of the new configuration file.
Table of Contents
| Top of page |
| Learning objective |
| Exercise overview |
| Print | Contents | Close |
After completing this topic, you should be able to explain how to use awk.
1. Introducing awk
Like the sed line editor, awk scans and processes patterns in a text file. However, its
extensive programming syntax classifies it as a programming language rather than as a
simple text utility.
Awk allows you to perform a variety of data processing tasks, including data extraction,
analysis, and reporting.
Note
Awk derives its name from the surnames of its developers, Aho, Weinberger, and
Kernighan.
Question
Which of the following operations do you think awk allows you to perform on a text
file?
Options:
Answer
Awk allows you to view text files as tables of records and fields, use variables to
manipulate the text as if it were data in a database, and generate formatted reports.
You can specify this data as one or more files or you can use standard input from the
keyboard after executing the awk command.
You can also provide standard input by piping another program or UNIX command into awk.
the awk program or script
The awk script specifies how the input data is to be processed.
You can specify the awk script as a file passed with the awk command or you can specify it
on the command line.
In addition to the input data and the awk script, you can specify options with the awk
command using the following syntax:
-f program file
program source
-F character
-v var=value
-f program file
You can use the -f option with the name of a program file to specify the file containing the
awk program code to execute. This is an alternative to entering the required source code on
the command line.
program source
You can enter the program source – the code that the awk program needs to execute
against – on the command line. This is an alternative to using the -f option to specify the
name of a program file.
When you enter the program source on the command line, you need to enclose the code in
single quotes to prevent it from being interpreted as shell code.
-F character
You use the -F character option to set the field separator in the input data to a character
other than the default white space.
For instance, to set the field separator that the input data uses to a colon, you enter -F:.
-v var=value
You use the -v var=value option to initialize a variable on the command line. This sets the
specified variable prior to execution.
Question
Options:
Answer
Awk program code can be entered on the command line or you can use the –f
option to read it from a file. You can initialize variables used by awk from the
command line.
Option 1 is incorrect. You can specify multiple files for an awk program to use as
input.
Option 2 is correct. You can specify the patterns to be matched and the actions to be
taken as a single awk command line statement.
Option 3 is correct. You can use the –v option on the awk command line to name a
variable used in your awk code and assign it an initial value.
Option 4 is correct. If you have a complex awk program that you want to reuse, you
can store the program instructions in a text file to read in from the awk command
line.
Awk works by matching a specified pattern with data in an input file and taking a specified
action on all lines that contain pattern matches. To specify an awk command, you use the
following basic syntax.
awk [-v var=value] [-F field separator] [--] 'pattern {action}' var=value datafile(s)
You enclose the actions that awk must perform in braces ({}) so that awk can differentiate
them from their associated patterns.
A string of characters placed between forward slashes (//) is called a regular expression.
When awk encounters any occurrence of that pattern within a line, it will select the line.
To select records on the basis of data in a particular field, you can use a matching operator
such as the equals (=) sign. For instance, the code fragment deposit = $5 instructs awk
to use the value in column 5 for calculations involving the deposit.
In addition to regular expressions in an awk command, you can search for patterns using
relational expressions such as greater than (>), less than (<), and less than or equal to (<=).
And you can address both regular and relational expressions in compound forms using the
logical OR (||) and logical AND (&&) operators.
The BEGIN statement allows you to specify actions – such as resetting variables or creating
column headings for output – that need to occur before awk starts processing input lines.
The END statement allows you to specify actions – such as printing an ending text line or
calculating cumulative variables such as line counts or column totals – that need to occur
after awk has processed the last input line.
The BEGIN statement is called once only before awk reads any data lines.
The END statement is called after awk has read and processed all lines.
If the source program for awk includes multiple files, END is called after awk has finished
processing the last file.
You don't need to specify awk variables before you use them. Rather, you can specify awk
variables only when they're needed.
bash-2.05a$
bash: / : is a directory
bash-2.05a
for i in terry larry mary
do
mail $i << END
$(date)
Have a good holiday $i!
END
quit
exit
clear
{
en='EasyNomad'
print en;
}
Awk provides reserved program variables. You can change some of these variables – such
as the default field separator. Awk updates other reserved program variables – such as
current record number – automatically.
NR or $0
NF or the $1-$n syntax
FS and OFS
RS and ORS
NR or $0
The NR variable denotes the number of the current record, whereas the $0 expression
represents the number of the current record as a single variable.
NF or the $1-$n syntax
Both the NF variable and the $1-$n syntax denote the number of fields in the current record.
FS and OFS
The FS and OFS variables represent the input field separator and the output field separator
respectively.
RS and ORS
The RS and ORS variables represent the input record separator and the output record
separator respectively.
By default, both separators are newline characters, which means that awk interprets each
line in an input file as a separate record. It also appends a newline character to the end of
each record that it sends to standard output.
Awk provides a number of functions, which you can use in awk scripts to return values
needed by the script.
These are divided into arithmetic functions – such as cos, exp, int, log, sin, and
rand – and string functions – such as index, length, and match.
Awk allows you to define your own functions for use in multiple scripts by using the
function command to specify a list of statements.
For instance, you can write a function that returns values sorted in a specified order.
Awk allows you store sets of values in associative arrays. These arrays are known as
associative arrays because you can reference the values they include using a string or an
index number.
array[index] = value
You can use a special for structure with an awk array to loop through the array elements,
using this syntax.
for(elem)action
Question
Options:
1. The first argument to the awk command
2. The first field in a line of input
3. An entire line of input
4. A null variable
Answer
The $0 expression is an awk field variable that matches an entire line of input.
Option 2 is incorrect. The first field in a line of input is referred to with the $1 field
descriptor.
Option 3 is correct. The $0 field descriptor matches every field in a line of input. The
default field delimiter is a space, but you can use the –F option to define the field
delimiter.
Option 4 is incorrect. In awk, all variables have a null value until a value is assigned
to them.
Question
Options:
1. If multiple files are called, the END statement is called after each is processed
2. You must declare any variables used in your awk program on the command line
3. You use the BEGIN statement to specify actions that take place before processing
any input
4. You use the END statement to specify actions that are performed after input is
processed
Answer
The BEGIN statement specifies actions to be taken before processing all input, while
the END statement specifies actions taken after processing all input.
Option 1 is incorrect. Irrespective of the number of files on which the awk code
operates, the END statement is called only once, after all the input has been
processed.
Option 2 is incorrect. As is generally true of shell scripting, awk variables can be
initialized as you need them.
Option 3 is correct. You can use the BEGIN statement to perform initial operations,
such as creating columns or assigning values to variable, before processing any
input.
Option 4 is correct. Once all input has been processed, you can use actions defined
in the END statement to manipulate the processed data.
One of the main uses of awk is to find out about the content of a text file.
For instance, you may want to know whether a configuration file contains comments –
indicated by a hash sign (#) – and, if so, where they are located in the file.
The code awk '/#/ {print "Comment found "}' /etc/hosts will print "Comment
found" for each line that contains at least one hash sign anywhere in the line.
Let's say that you want to print a text file to a screen that is limited to a width of 80
characters per line. You don't want the lines to wrap.
Before entering complex formatting instructions, you want to check whether there are any
lines in the file that exceed 80 characters.
To do this, you use the code awk 'length > 80' hosts.
You can use awk to reformat text files awk for improved readability or for export into other
applications.
For instance, the code awk '{print ; print ""}' nomad.txt > nomad1.txt
changes the single line spacing in the file nomad.txt to double-line spacing in a file named
nomad1.txt.
In this example, the single quotes allow the use of double quotes in the awk expression.
You use single quotes in this way to hide special characters from the shell.
Awk can process text that has been broken into multiple logical fields. And you can
reference individual fields in an awk script.
The code awk -F":" '{ print $1 }' /etc/passwd, for example, prints a list of all
user accounts on the local system.
In the code, the -F option changes the field separators between multiple fields in the
passwd file from the default spaces to colons. When awk processes the print $1
command, it will print out the first field that appears on each line in the input file.
Question
Awk allows you to perform integer and floating point mathematical operations on
input files.
What do you think the output of the following script will be?
Options:
Answer
The code will calculate the number of blank lines in a file and return the result to the
screen.
Option 1 is correct. Each time a blank line is detected, the awk program initialises the
x variable with a value of 0, and increments it by one. The print command sends the
value of x to standard output, accompanied by the text strings enclosed in double
quotes.
Option 3 is incorrect. The print command only prints the value of the x variable and
some helpful text. The pattern that is matched – in this case blank lines – is not
automatically printed.
You can use awk with other UNIX commands, such as cat and grep.
In the example cat text1 | awk '$1 < $2 {print $0, $1/$2}' > text2, the
content of the text1 file is piped to awk for pattern matching. The output of the command will
be stored in a new file named text2.
Summary
Awk allows you to perform a variety of data processing tasks, including data extraction,
analysis, and reporting.
Awk works by matching a specified pattern with data in an input file and taking a specified
action when it locates this pattern.
One of the main uses of awk is to find out about or change the content of a text file. For
instance, you can use it to determine whether a file includes comments or to reformat a file
for improved readability.
Table of Contents
| Top of page |
| Learning objective |
| Summary |
| Print | Contents | Close |
After completing this topic, you should be able to explain the principles of logical flow in a
shell program.
The shell is an interface between the user and a UNIX operating system.
You can create shell scripts – consisting of ASCII text files – that the shell interprets and the
operating system executes.
The shell supports a set of functions implemented as keywords that provide a simple
programming language. You can combine these functions with UNIX command-line
programs.
automating tasks
prototyping
system administration
automating tasks
A shell program – or script – allows you to specify operations in a certain sequence.
You can use shell scripting to automate tasks that need to be performed in sequence, such
as transferring files using the File Transfer Protocol (FTP) and then e-mailing users once
the files have been downloaded.
prototyping
You can use shell programming to prototype complex applications so as to provide proof-of-
concept implementations and to test logical functions.
Once you've prototyped an application, you can rewrite it in languages such as C++ and
Java to improve its performance and extend its functionality.
system administration
Shell scripts run at system bootup to restore system configurations and to set up services.
They manage the initialization and shutting down of essential system services during
system startups and shutdowns, as well as the transition from one run-level to another.
System administrators can write shell scripts to analyze the behavior of a system and to
modify it, as well as to automate repetitive tasks.
Question
In certain cases, you should use compiled languages such as C, C++, or Java, rather
than shell scripting.
For which of the following purposes do you think a compiled language is more
appropriate than shell scripting?
Options:
Answer
You should use more powerful scripting languages than shell scripting for improving
system performance, designing proprietary applications, and manipulating graphics
or GUIs.
Question
Options:
1. Automation
2. Prototyping
3. System Administration
Targets:
Answer
Repetitive tasks can be automated by shell scripts. For instance, if every project you
work on has a similar directory structure, you can automate the process of
generating the directories for a particular project.
Because shell programs are relatively simple to write, it's often useful to test
procedures in the shell before incorporating them into programs written in more
sophisticated languages.
Services and system configurations on UNIX systems are usually initialized by shell
scripts, so understanding shell programming techniques is essential to optimizing
your system.
2. Programming fundamentals
Programming effectively requires that you first determine the solution that a program must
provide. You then plan the components that the program requires to do this and determine
how the components will fit together to execute the required tasks.
General programming principles that make it easier to write and interpret programs include
writing pseudocode
debugging during code construction
indenting code for readability
writing pseudocode
To identify the steps that a program needs to include, you should use pseudocode.
Pseudocode consists of an informal written plan that describes what each item of code must
do, together with the programming language components required to achieve the specified
tasks.
debugging during code construction
You should debug a program as you work on it – rather than after completing it – to verify
that each logical section of code works correctly. Ensuring that each section of code works
correctly before you continue programming simplifies the debugging process because it
allows you to isolate incorrect code more easily.
indenting code for readability
You should enter spaces between commands and indent code to make it more readable.
Core tools available in most programming languages – including shell scripting – include
conditional statements
functions
loops
variables
conditional statements
You use conditional statements to control the flow of statement execution in a program.
You use if then else statements to check a condition and then to perform an action
based on whether the condition evaluates to true or false.
If the condition specified in the if statement in this example evaluates to true, the then
statement executes. If the if statement evaluates to false, the else statement executes to
repeat the request that the user enter a valid number.
functions
Functions – also known as procedures or subroutines – consist of code that implements a
program that you can embed and reuse in a shell script. They allow programmers to create
modular code that is easier to maintain and extend than code implemented as a single
routine.
In the example shown here, the check_value function ensures that a variable set from
standard input corresponds to an appropriate range of values.
In this statement, the loop will print "Hello World" the number of times that the user
specifies. If the user enters a value of 5, for example, the code will print "Hello World" while
the count is smaller than or equal to 5.
Modern shells – such as the Korn and Bash shells – provide for the use of variables with
both a local and global scope in shell scripts. You can use the local keyword in a function
to define a variable that is available only to that function. This is useful for maintaining
modularity in complex programs.
When naming a variable, you should ensure that the name isn't used by a function. You
should choose a descriptive name. And a useful convention for shell scripts is to capitalize
variable names.
Question
Options:
Answer
Option 1 is incorrect. To reduce errors and save time and effort, you should debug
code as you write it.
Option 3 is correct. Creating modular code through the use of functions makes your
code easier to maintain and reuse, and thereby improving efficiency.
Question
Why would you write pseudocode?
Options:
Answer
Pseudocode helps you identify the logical steps needed in your program.
Option 2 is correct. Although pseudocode isn't needed for simple tasks, it is difficult
to accomplish complicated programming tasks without first breaking them down into
steps.
Summary
The shell allows users to interact directly with a UNIX operating system. You can use shell
scripts to instruct the shell to perform operations in a specific sequence. Shell programming
is useful for automating tasks, creating prototype programs, and simplifying system
administration.
The core tools for programming include variables, conditional statements, loops, and
functions. General programming principles that you should follow include writing
pseudocode, debugging a program as you construct it, and indenting code for readability.
Table of Contents
| Top of page |
| Learning objective |
| Summary |
| Print | Contents | Close |
After completing this topic, you should be able to use conditional expressions to provide
flow control in shell programs.
1. Testing conditions
Programmers frequently need to make the execution of their code dependent on certain
circumstances being met.
For instance, if your code needs to append data to a file, it is a good idea to check that the
file exists before executing code that tries to append data to the file.
The shell provides a number of tools for testing conditions that simplify the process of
building conditional execution into shell scripts.
Question
Options:
Answer
You can use conditional expressions to evaluate and compare integers, evaluate and
compare strings, and test file attributes.
You can test conditional expressions using three different forms of syntax:
[ expression ]
test expression
[[ expression ]]
In the syntax, expression is the condition that you are testing. You must include a space
between the expression and the brackets.
When testing an expression, the command will return an exit status of 0 if the conditional
expression is true and an exit status of 1 if it is false.
For instance, when you compare two variables using the test and [ … ] commands and
one of the variables is set to null or is unset, it will return a syntax error.
Furthermore, because wildcard expansion and word splitting aren't automatically applied to
expressions in the [[ … ]] command, you're less likely to need to resort to quoting.
For example, you can test whether x is less than or equal to y using the following
statements:
[[ $x le $y]]
[ $x le $y ]
test $x le $y
For example, you can combine two expressions using the syntax shown here.
Suppose that you want to ensure that two conditions are met before printing a message. In
this case, a student's class mark has to be higher than the average mark of the class and
higher than 80.
You can use the [[ … ]] syntax together with string operators to test variables.
The following string operators are useful for checking that variables used in a program have
been assigned appropriate values:
-n – true if length of string is not zero
-o – true if option is set
-z – true if length of string is zero
The following code fragment executes the set_var function until the $VAR variable is
assigned a value that is not null.
set_var (){
echo "Enter a variable"
read VAR
}
while [[ -z $VAR ]]
do
set_var
done
Question
Identify the statements regarding the test command that are true.
Options:
1. Spaces on either side of the [ symbol are not interpreted by the shell
2. The –n option to the test command returns a value of 1 if the variable being tested
has a null value
3. You can use conditional operators within a test command
4. The [[ condition ]] syntax is backwardly compatible with the Bourne shell
Answer
If test –n returns a 1 when testing a variable, then that variable has a null value.
Conditional expressions can form part of conditions evaluated by the test command.
Option 3 is correct. You can use the && operator to require two conditions to be met,
and you use the || operator to require only one of two conditions to be met.
You can compare one string with another string using the syntax [[ string1 = string2 ]].
For example, this code tests whether the string "Belize" has been assigned to the
$DESTINATION variable and if it has, prints the message "flights available."
=
!=
<
>
=
You use the [[ string = pattern ]] syntax shown here to test if a string matches a pattern
constructed using a combination of literals.
DOTW=$(date +%A)
[[ $DOTW = S* ]] &&
echo "Weekend!"
In this case, the format options to the date command assign values such as "Saturday" and
"Wednesday" to the DOTW variable. The DOTW variable is then tested to see whether it starts
with an "S", and if it does, the string "Weekend!" is echoed.
!=
You use the [[string! = pattern ]] syntax to test if a string doesn't match a pattern. The syntax
returns a true value if the string fails to match the specified pattern.
DOTW=$(date +%A)
[[ $DOTW != S* ]] &&
echo "Workday!"
In this example, the DOTW variable is tested to see whether it does not start with an "S", and
if it doesn't, the string "Workday!" is echoed.
<
You use the [[ string1 < string2 ]] syntax to test if the characters in string1 comes before
characters in string2 in the dictionary.
The string operators test the lexigraphical order of two strings. A string that appears before
a second string in the dictionary is considered "smaller" than the second string.
In this code, the string "abc" is smaller than "bcd" and so the code echoes the string
"Smaller."
>
You use the [[ string1 > string2 ]] syntax to test if the characters in string1 comes after
characters in string 2 in the dictionary.
In this example, the string "bcd" is greater than the string "abc", so the code echoes the
string "Bigger."
You use the following options ( in the syntax [[ option filename ]] ( to test files:
To test whether a file named "new" contains data so that you don't overwrite it if it does, for
example, you use this statement.
Additional file options that the Korn shell provides for testing file attributes include
-G
-L
-O
-S
-G
The [[ -G filename ]] command returns an exit status of 0 if the specified file exists and the
group ID of the file matches the effective group ID for the current process.
-L
The [[ -L filename ]] syntax returns an exit status of 0 if the specified file exists and is a
symbolic link. A symbolic link is a directory or file name that you use to change to another
directory without typing its full pathname. Symbolic links can bridge file systems and link to
directories.
-O
The [[ -O filename ]] syntax returns an exit status of 0if the user ID for the specified file
matches the effective user ID for the current process.
-S
The [[ -S filename ]] syntax returns an exit status of 0 if the specified file exists and is a
socket. Sockets are file descriptors and obey file descriptor semantics.
In addition to using options to test file attributes, you can compare files using the following
syntax:
Question
Suppose that you're writing a program that backs up files. You want to back up a file
named "currentfile" and to ensure that the program only overwrites files by this name
that are older than the backup version to ensure that the information it contains
remains up to date.
What statement do you think you need to use to ensure that only older files are
overwritten?
[[ MISSING CODE ]] && print "Overwriting older file"
Options:
Answer
To compare expressions involving integers, you use the following integer operators:
-eq
-ne
-le
-ge
-gt
-eq
You use the -eq option with the syntax [[ expr1 -eq expr2 ]] to test whether expression1 is
equal to expression2.
To return an appropriate message to users who enter an age of 64, for example, you use
this code.
[[ $age –ne 64 ]] && print " Your age is not equal to 64"
You use this code to return an appropriate message to users who enter an age other than
64, for example.
-le
You use the -le option with the syntax [[ expr1 -le expr2 ]] to test whether expression1 is
less than or equal to expression2.
To return an appropriate message to users who enter an age at or below 64, for example,
you use this code.
You use this code to return an appropriate message to users who enter an age equal to or
above 64, for example.
-gt
You use the -gt option with the syntax [[ expr1 -gt expr2 ]] to test whether expression1 is
greater than expression2.
To return an appropriate message to users who enter an age greater than 64, for example,
you use this code.
Question
See if you can identify the command that you think will do this.
#!/usr/local/bin/ksh
if MISSING CODE
then
Answer
You use the statement [[ -a travel ]] to check if the file named "travel" exists.
Question
Suppose that if the file named "travel" exists, you want to test that it contains data.
Answer
Footnote
The Bash shell supports the use of single and double square brackets ([] and [[]]) for
conditional testing. Double square brackets are a recently added feature, and are
slightly more versatile.
-eq – equal to
Question
Which operator will return a 0 if you compare two strings with different values?
Options:
1. !=
2. =
3. >
4. <
Answer
The != string operator will cause a test condition to return a value of 0 if the strings
being compared are not the same.
Option 1 is correct. If variables X and Y have different values, the test shown here
will evaluate as true, as represented by a return value of 0:
[[ $X != $Y ]]
Option 2 is incorrect. This operator will return a value of 1 if the two strings have
different values.
Option 3 is incorrect. This operator will return a value of 0 if the string on the left of
the operator has a greater value than the string on the right of the operator.
Option 4 is incorrect. This operator will return a value of 0 if the string on the right of
the operator has a greater value than the string on the left of the operator.
Question
Options:
1. [[ -a filename ]]
2. [[ -s filename ]]
3. [[ -x filename ]]
Targets:
Answer
The –a option tests a file's existence, the –s option tests whether the file contains
data, and the –x option tests that the file is executable.
A test using the –a option on a filename will evaluate as true if the file exists.
A test using the –s option will evaluate as true if the file is not empty.
A test using the –x option will evaluate as true if an executable permission has been
set on the file.
Summary
The Korn shell allows you to test strings, files, and expressions for specific conditions. You
can use the [ … ], test, and the [[ … ]] commands to test conditional statements.
The [[ … ]] command is preferable because it results in fewer syntax errors. You can use
compound expressions to test conditions that require the comparison of files, strings, or
expressions by using the double ampersand (&&) or the double pipe (||) operators.
You can compare strings, check file attributes and do arithmetic tests with the Korn shell.
This allows you to manipulate files, integers, and strings.
Table of Contents
| Top of page |
| Learning objective |
| Summary |
| Print | Contents | Close |
After completing this topic, you should be able to use flow control commands in shell
programs.
1. Flow control
You use conditional statements to control the flow of code execution in UNIX shell scripts.
Conditional statements include statements that use the if then command, the if then
else command, the if then elif command, and the case command.
You use the if then command to execute subsequent commands when a given condition
is true.
if condition
then
commands
fi
This code, for example, checks that the password that a user enters is "password01". If it is,
the conditional statement evaluates to true and the subsequent code executes to display the
message "You can proceed".
#!/usr/local/bin/ksh
echo "Enter password"
read password
if [[ $password = password01 ]]
then
print "You can proceed"
fi
The if then else command allows you to specify that one set of commands must
execute if the condition is true and that another set of commands must execute if the
condition is false. You use the if then else command with this syntax.
if condition
then
commands
else
commands
fi
In this example, the statements after the then keyword will execute if the condition for the
if command evaluates to true.
If the condition is false – because a user enters a password other than "password01" – the
statements after the else keyword will execute to display the message "Goodbye" and to
exit.
#!/usr/local/bin/ksh
echo "Enter password"
read password
if [[ $password = password01 ]]
then
print "You can proceed"
else
print "Goodbye"
exit
fi
The if then elif command allows you to specify more than one condition. It executes a
set of commands if the first condition is true and another set of commands if a subsequent
condition is true.
If no condition evaluates to true, the statements that follow the else keyword execute.
if condition1
then
commands
elif condition2
then
commands
elif condition
then
commands
else
commands
fi
In the example, the elif command allows you to ask the user to re-enter a password if they
typed it incorrectly. The program will then proceed once the correct password is typed.
#!/usr/local/bin/ksh
echo "Enter password"
read password
if [[ $password = password01 ]]
then
print "You can proceed"
elif [[! $password = password01 ]]
then
print "Please retype password"
read password
if [[ $password = password01 ]]
then
print "You can proceed"
else
print "Goodbye"
exit
fi
print "Welcome $USER"
The case command can compare a single value to a number of other values. If the single
value matches one or more specified patterns, the commands associated with the patterns
will execute.
case value in
pattern1) command
command;;
pattern2) command
command;;
patternn) command
command;;
esac
This Korn shell script illustrates the use of a simple case statement.
The following code prompts the user to supply a value for a variable called password. If the
user input for the variable conforms to one of three options, the code performs the specified
action. In this case, it prints a suitable message to standard output.
Question
Suppose that you are writing a menu program that will load a certain program when
the user enters a specific number.
See if you can complete the code that allows the user to type a number to execute a
program.
#!/usr/local/bin/ksh
echo "Enter menu number"
read menu
MISSING CODE
1 ) ./travelpackage ;;
2 ) ./flights ;;
3 ) ./bookings ;;
esac
Answer
You type case $menu in to complete the case command that allows the user to
type a number to execute a program.
Question
Suppose that you want to add another menu item to run a new program named
"customers".
See if you can type the code that adds this program as number 4 to the menu
program.
#!/usr/local/bin/ksh
echo "Enter menu number"
read menu
1 ) ./travelpackage ;;
2 ) ./flights ;;
3 ) ./bookings ;;
MISSING CODE
esac
Answer
You type 4 ) ./customers to create another menu item that will execute the
program named "customers".
Question
Options:
1. if then fi
2. if then else fi
3. if then elif fi
4. case esac
Answer
To specify the execution of code based on whether a single condition is true or false,
you use an if then else fi statement.
Option 2 is correct. This statement executes code after then if the specified
condition is met, and after else if it isn't.
Option 3 is incorrect. This statement is useful if you need to use additional conditions
after the elif syntax.
Option 4 is incorrect. This statement is used to execute code based on whether one
or more patterns match a particular value.
Loops are recursive code that executes repeatedly until a specific condition is met. This
process is called iterative execution. Loops can be infinite, in which case they continue to
execute indefinitely, so you need to be careful when specifying conditions.
You use loops to perform various tasks in a sequence and to automate system
administration.
You use the for command with this syntax to execute commands a specific number of
times.
A word is a piece of a line that the shell has broken up, such as "a", "apple", and "airplane."
The commands execute after the do statement and the done statement ends the loop.
For example, you can use this program to display a loop number and a loop value for each
iteration of the loop.
#!/usr/local/bin/ksh
integer loopnum=1
for x in a b c
do
print "Loop $loopnum: x=$x"
(( loopnum+=1 ))
done
This is the output of the script.
Loop 1: x=a
Loop 2: x=b
Loop 3: x=c
You can assign parameters to a loop dynamically, as in this code, which you can use to
make one or more files executable.
#!/usr/local/bin/ksh
for FILE
do
chmod 700 $FILE
print "$FILE is executable"
done
If you save a file named "exe" and you have a script named "sample" then you can make
the sample file executable by typing the command
./mkexe sample
To make one or more files that you own executable, you use the syntax shown here.
A loop that uses the while command continues to execute until a specified condition is
true.
while condition1
do
commands
done
If the exit status of the conditional statement is zero the commands between do and done
are executed.
For example, to create directories using the while command you use the code shown here.
#!/usr/local/bin/ksh
echo "What is the name of the directory"
read directory
echo "How many directories do you want to make?"
read dirnumber
count = 1
while [[ $count -le $dirnumber ]]
do
mkdir $directory$count
(( count+=1 ))
done
A loop that uses the until command continues to execute while a specified condition
continues to evaluate to false. In other words, the commands continue to execute until
condition1 returns a zero exit status.
until condition1
do
commands
done
Question
Suppose that you want to debug a Korn shell script that another person wrote.
Tracking the value of variables is an important part of debugging a script. Syntax
errors are usually fairly easy to detect, but logical errors may require monitoring how
a variable changes during the course of a script's execution.
#!/usr/local/bin/ksh
integer y=3
while (( y < 5 ))
do
print "The value of y is $y"
(( y = y + 1 ))
done
Options:
1. The value of y is 2
The value of y is 3
The value of y is 4
2. The value of y is 3
The value of y is 4
3. The value of y is 3
The value of y is 4
The value of y is 5
4. The value of y is 4
Answer
The initial value of y is 3 and the loop executes until the value of y is less than 5. The
output is then as follows:
The value of y is 3
The value of y is 4
Option 1 is incorrect. Although y having a value of 2 meets the conditions of the test
on which the loops is performed, the first line of code assigns a value of 3 to y.
Option 2 is correct. The first iteration of the loop prints the current value of y, and
then increments it by 1. The second iteration prints the new value, 4, before
incrementing it.
Option 3 is incorrect. Only two iterations of the loop take place, because after the
second iteration the condition on which the loop takes place is no longer true.
Option 4 is incorrect. The statement that prints the value of y is executed before y is
incremented, so the first iteration of the loop prints the initial value of y, which is 3.
Question
Options:
1. The shell checks the logic of loops to prevent code being executed an infinite
number of times
2. Loops execute code while a specified condition is met, or a specified number of
times
3. You must specify the number of parameters that you intend to pass to a for loop
4. You use the test command with the while loop
Answer
Loops execute code for each argument supplied, or until a condition is met. If the
condition isn't met, the loop will be infinite. You use the test command with the
while loop.
Option 1 is incorrect. Loops can be infinite, so you should take care when specifying
conditions on which they take place.
Option 2 is correct. You supply parameters to the for loop on which code is
executed, and you specify a condtion for which while loops take place.
Option 3 is incorrect. You can dynamically assign parameters to the for loop. For
example, you can use command-line arguments to assign them.
Option 4 is correct. You can use the test command to specify a condition, which
while it holds true, will cause a while loop to execute.
Summary
To control the flow of execution in UNIX shell scripts, you use conditional statements. These
include statements that use the if then command, the if then else command, the if
then elif command, and the case command.
A loop is a set of commands that executes – or iterates – repeatedly, either while or until a
specific condition is met or for a specified number of iterations.
Table of Contents
| Top of page |
| Learning objective |
| Summary |
| Print | Contents | Close |
After completing this topic, you should be able to use conditional expressions and flow
control in a shell program.
1. Exercise overview
In this exercise, you're required to use conditional expressions and flow control in a shell
program.
writing a function
writing the main script
2. Task 1: Writing a function
Suppose that you want to create a script that will notify a user by e-mail if specific files have
been modified during week days.
First, you need to write a Korn shell script that exits if the date is not a week day. To do this,
the script must check the date.
If it is a week day, the script must check to see if any files have been modified in a specified
directory. If files have been modified, it must e-mail the user. If not files have been modified,
the script must terminate.
Question
Let's say that you want to write a function called check_date that determines whether
the current date falls on a weekend.
See if you can type the code indicating the function name and the start of the
function.
#!/usr/local/bin/ksh
#=====================
# Function" Check if today is a weekday
#======================
MISSING CODE
today=$(date '+%A')
#======================
check_date
Answer
Question
You want to ensure that only files modified on weekdays are mailed to the user.
See if you can supply the operator that will exit the shell script on Saturdays and
Sundays.
#!/usr/local/bin/ksh
#==================
#Function "Check if today is a weekday
#======================
function
today=$(date '+%A')
if [[$today == Saturday MISSING CODE
$today == Sunday ]]
then
return=false
fi
}
#=======================
check_date
Answer
The operator that will exit the shell script on Saturdays and Sundays is ||.
Question
See if you can complete the code that allows you to provide an alternative condition
for the if statement.
#!/usr/local/bin/ksh
#=====================
# Function" Check if today is a weekday
#======================
function check_date {
today=$(date '+%A')
if [[ $today == Saturday || $today == Sunday ]]
then
return=false
MISSING CODE
fi
}
#=======================
check_date
Answer
Question
The condition you include needs to evaluate to true on weekdays so that the program
continues to run on these days.
#!/usr/local/bin/ksh
#=====================
# Function" Check if today is a weekday
#======================
function check_date {
today=$(date '+%A')
if [[ $today == Saturday || $today == Sunday ]]
then
return=false
else
print "Today is a weekday"
MISSING CODE
}
#=======================
check_date
Answer
After writing the check_date function to check if the current day is a weekday, you also
write this modify function to check if any files have been modified in the last 24 hours.
#================
#Function: checks what files are modified today
#================
function modify {
find /home/vincep –ctime 1 > logfile
if [[ ! –s logfile ]]
then
print "No files were modified"
modified=false
else
print "Files were modified"
modified=true
fi
}
Question
You now want to complete the code so that the program checks to see if any files
have been modified in a directory and then outputs any files that have been modified.
See if you can complete the code that allows you to call the modify function.
return=false
check_date
if [[ $return == true ]]
then
print "Finding out if files were modified today"
MISSING CODE
Answer
Question
You want to print the list of any files that have been modified using the cat
command.
Choose the flow control statements that will print out the modified files.
return=false
check_date
if [[ $return == true ]]
then
print "Finding out if files were modified today"
modify
MISSING CODE
fi
Options:
1. if [[ ! $modified == true ]]
then
print "The modified files:"
cat logfile
fi
2. if [[ $modified == true ]]
then
print "The modified files:"
cat logfile
fi
3. while [[ $modified == false ]]
do
print "The modified files:"
cat logfile
done
4. while [[ $modified == true ]]
do
print "The modified files:"
cat logfile
done
Answer
Option 1 is incorrect. The use of the ! operator on the first line means that the
condition evaluates as false if files are modified, whereas you need it to evaluate as
true.
Option 2 is correct. The first line tests that files have been modified. If this test
evaluates as true, the rest of the if fi statement is executed, printing a list of the
modified statement accompanied by a message.
Option 3 is incorrect. The loop executes if files are not modified, in which case there
will be nothing to print. Also, if the condition is true, this loop will be infinite.
Option 4 is incorrect. To avoid infinite looping if the condition is met, you need to
include the following command at the end of the code that is executed by the loop:
modified=false
You have now created a function that tests whether the current day is a weekday. The shell
script then checks if the files in a directory have been modified and outputs these files with
the cat command. The shell script exits if it is a weekend or if no files have been modified.
return=false
check_date
if [[ $return == true ]]
then
print "Finding out if files were modified today"
modify
if [[ $modified == true ]]
then
print "The modified files:"
cat logfile
fi
fi
Table of Contents
| Top of page |
| Learning objective |
| Print | Contents | Close |
After completing this topic, you should be able to use functions to improve the efficiency of
program code.
1. Introducing functions
A function is a set of commands grouped together and called by the same name that
accepts a number of arguments as input and can return a value.
When functions are self-contained, they can be stored in memory and executed at the
command line.
For example, this self-contained function will create a directory and then access that
directory.
function md {
mkdir $1 && 'cd' $1
}
When functions are contained within shell scripts, they need to be called in the main body of
the shell script.
For example, you can create a function to print an error message whenever specific errors
occur.
function name {
commands
}
A simple function named quit is called from the main shell program to quit after printing
"Hello world".
#!/usr/local/bin/ksh
function quit {
exit
}
print "Hello world"
quit
This code uses a get_time function to return the current time in the format that shell
scripts require.
#!/usr/local/bin/ksh
function get_time {
TIME=$(date +%m/%d/%y-%H:%M:%S)
printf "$TIME\n"
}
get_time
You can declare a local variable in a function using the typeset command. All the other
function variables are inherited and shared by the Korn shell script.
Because functions can have local variables, recursion is possible. This reduces the need for
extra code.
#!/usr/local/bin/ksh
x=1
function myfunc {
x=3
typeset y=4
print "In local function x is $x"
print "In local function y is $y"
}
myfunc
print "Outside local function x is $x"
print "Outside local function y is $y"
Note
You can declare functions using the Portable Operating System Interface (POSIX)
format shown here to maintain compatibility with the Bash shell. Functions that have
this format cannot contain local variables.
function_name() {
commands
}
You can use the return command to pass an exit value from a function to the invoking
Korn or Bash shell script.
return variable
You can use this script to ensure that the user's input is an integer value and then to return
this value for further calculation.
This allows you to pass variables to a function and to obtain a return value from the function.
#!/usr/local/bin/ksh
function get_number {
print "Input a number between 1 and 9"
read num
case $num in
+([0-9])) print "Input is numeric" ;;
*) print "Non-numeric input, try again" ;;
esac
}
get_number
print "The number is $num"
integer answer=$num*5
print "Five times number is $answer"
Question
Options:
Answer
Functions improve the performance of shell programs because they are loaded into
memory.
Option 1 is incorrect. While functions do allow you to reuse code, it is the fact that the
function's code is copied into memory that improves performance.
Option 2 is correct. When a function is first declared it is copied into memory, from
where it is executed whenever a script calls it. This is more efficient than reading the
code from disk.
Option 4 is incorrect. Functions can use local or global variables, and there is no
significant performance gain associated with either.
Question
Identify the true statements concerning functions.
Options:
Answer
Functions can be defined within a shell script or read directly into memory. Usually,
functions accept one or more arguments.
Option 1 is incorrect. You can use the return command to specify a return value for
a function.
Option 2 is correct. Functions can be defined at any point in a shell script, as long as
it is before they are first called.
Option 4 is correct. You can store a function in an ordinary text file and use the .
command to read it into memory.
2. Uses of functions
self-contained
used within shell scripts
self-contained
You use self-contained functions to execute a number of commands from the shell.
You need to read these functions into memory so that you can execute them from the
command line.
In the following example, a self-contained function – named get_time – outputs the time in
a specific format.
function get_time {
TIME=$(date +%m/%d/%y-%H:%M:%S)
printf "$TIME\n"
}
Functions can accept arguments and then return values after processing the data. Because
functions have local variables, they allow recursion.
You normally use functions for commands that are invoked often. For example, you can use
a function to verify that a user inputs the correct data in response to a prompt, as illustrated
in the following code:
#!/usr/local/bin/ksh
function get_number {
print "Input a number between 1 and 9"
read num
case $num in
+([0-9])) print "Input is numeric" ;;
*) print "Non-numeric input, try again" ;;
esac
}
get_number
print "The number is $num"
integer answer=$num*5
print "Five times number is $answer"
Once you read a function in, you can invoke it from a shell script using the syntax
function shell_script
function get_number {
print "Enter a number from 1-9"
read num
case $num in
([0-9]) ) print "Thanks"
* ) print "Try again!"
esac
}
typeset –fx get_number
You can autoload functions to improve performance. In this case, the shell reads in the
functions when they are first invoked and keeps them in memory, rather than reading them
in each time they are invoked.
To autoload a function, you first need to set the FPATH variable with the name of the
directory that stores it.
The shell searches each directory in the FPATH variable from left to right for a filename that
matches the name of the function. When it finds the appropriate file, it reads and executes it
in the current environment.
You can define these functions, although you don't call them specifically in shell scripts.
Rather, a script calls a discipline function when it accesses the variable associated with the
function.
For example, the shell calls the discipline functions time.get, time.set, and
time.unset when the variable time is accessed, set, or unset respectively.
Question
Suppose that you have written a shell script to ensure that users enter an integer
value for use in a calculation.
What do you think the output of this script – excluding interactive prompts – will be if
a user first enters a text value and then the value 6?
#!/usr/local/bin/ksh
function get_number {
print "Enter a number from 1-9"
read num
case $num in
([1-9]) ) print "Thanks" ;;
* ) print "Try again!" ;;
esac
}
while [ $num != [1-9] ]
do
get_number
done
((square=num*num))
print "$num squared is: $square"
Options:
1. Thanks
6 squared is 36
2. Try again!
3. Try again!
Thanks
6 squared is 36
4. Try again!
6 squared is 36
Answer
Option 1 is incorrect. The user first enters text, which matches the "catch-all" pattern
in the case statement:
*) print "Try again!"
Option 2 is incorrect. The while loop ensures that the get_number function is
executed repeatedly until the user supplies a value between 1 and 9.
Option 3 is correct. Once the user has supplied a valid value for the num variable, the
while loop exits because its condition is met, and the code that prints the square of
the num variable is executed.
Summary
You can read a function in and then invoke it from the shell. After autoloading a function, the
shell reads it in when it is invoked rather than each time a Korn shell script invokes it. To
allow the autoloading of functions, you need to set the FPATH variable with the pathnames
to files for autoloading functions.
Table of Contents
| Top of page |
| Learning objective |
| Summary |
| Print | Contents | Close |
After completing this topic, you should be able to use the UNIX I/O system in shell
programs.
Each program you run from the shell opens files that provide the primary means of
communication between it and other processes or programs.
standard input
standard output
standard error
standard input
The standard input file provides a way to send data to a program. By default, the standard
input is read from the terminal keyboard.
standard output
The standard output file provides a means for the program to output data. By default, the
standard output goes to the terminal display screen.
standard error
The standard error file is where the program reports any errors encountered during
execution. By default, the standard error goes to the terminal display.
You need to use file descriptors with some redirectors – such as the exec command, which
opens specified files.
The Korn shell automatically assigns the most common file descriptors, which are
If you want input to go to standard error, you use the following syntax:
input 2>standard_error_output
You can control program input/output (I/O) – including the source that a program uses for
input and where it sends output and standard error – using redirection.
You can instruct the shell to use the content of a file as input. For instance, you can include
the instruction <text1 in a shell command to instruct the shell to read input from a file
called text1 instead of from the terminal keyboard.
In the example more < /etc/passwd, the shell sends the content of the password file to
the more command for display on the screen.
You use a "here" document to redirect multiple lines from standard input to a command. To
do this, you redirect the standard input using a marker string ( most commonly, EOF (end-of-
file).
The shell reads the input to the command until it encounters a line that contains only the
marker string. All the lines it has read – except for the marker string line – are then fed to the
command as its standard input.
In the example cat > script4 << EOF, the shell reads standard input for cat until it
reaches a line containing only the string EOF. It then creates a file called script#4.
command<<word
For instance, in a script that sends e-mail to a list of users named Terry, Larry, and Mary,
the here document feeds instructions to the mail command, line by line, until it encounters
the specified word – in this case, END.
Question
What command do you think transfers the user's response to memory so that the
input can be used for processing?
Options:
1. echo
2. print
3. quote
4. read
Answer
The read command transfers the user's response to memory so that it can be used
for processing.
You can modify the behavior of the read command using the following options:
-p
-r
-s
-un
-p
The -p option instructs the shell to read an input line from a coprocess.
-r
The -r option forces the shell not to interpret the backslash (\) as a line continuation
character.
-s
The -s option causes the shell to save a copy of an input line in the command history file.
-un
The -un option causes the shell to read a specified input line from the specified file
descriptor.
The print command displays the content of a specified string – the prompt you want the
user to respond to – on screen.
-
-n
-p
-r
-R
-s
-un
-
The - option causes the shell to interpret all strings following the hyphen as an argument,
even when the string itself begins with a hyphen.
-n
The -n option prevents the shell from adding an ending newline to the output.
-p
The -p option causes the shell to redirect arguments to a coprocess.
-r
The -r option causes the shell to ignore the backslash (\) escape conventions.
-R
The -R option causes the shell to ignore the backslash (\) escape conventions, as well as
preventing it from interpreting arguments preceded by a hyphen as options.
-s
The -s option causes the shell to redirect specified arguments to the history file.
-un
The -un option causes the shell to redirect arguments to the specified file descriptor.
You can direct the flow of output from a command to a file instead of to the default terminal
screen.
For instance, the code ls /tmp > ~/ls.out creates a file containing the output – in this
case a listing of the /tmp directory – of the ls command into a new file called ls.out.
Note
If the file named ls.out already exists, the old version will be overwritten. To prevent
this, you can append content to the end of a file using the >> command.
Instead of displaying errors on screen, you can redirect a program's standard error to a file
or command.
For instance, in the Korn shell you could sort the /etc/passwd file, place the results in a file
called prog_errors, and trap any errors in a file called log_errors using the command
Question
Options:
1. >>
2. >
3. >|
Targets:
Answer
The > redirects standard output to a file. If the file already exists and you want to
overwrite it despite the noclobber option being set, you use the >| operator. You
append standard output to a file with the >> operator.
You can append the output from the ls command to a file named dirlist.txt
with this command:
ls >> dirlist.txt
You can redirect the output from the ls command to a file named dirlist.txt
with this command:
ls > dirlist.txt
Even if the noclobber option is set, you can redirect standard ouptut to overwrite
an existing file called dirlist.txt with this command:
ls >| dirlist.txt
Question
Which option for the read command do you use if you don't want the backslash (\) to
be interpreted as a special character when supplied as input?
Options:
1. -p
2. -r
3. -s
4. -u
Answer
You use the –r option to prevent the backslash (\) being interpreted as a special
character in input supplied by the read command.
Option 1 is incorrect. The read –p option is used to accept input from a co-process
spawned using the |& operator.
Option 2 is correct. If the option is not inserted, the backslash causes special
characters to be treated as literals.
Option 3 is incorrect. The read –s option will cause input to be saved in the
command history file.
Option 4 is incorrect. The read –u option is used to specify the file descriptor used
to provide input. The default is standard input, as in read –u 0.
During planning, you decide to divide the program into two logical sections. The first section
will gather user data ( such as a user's username and password ( and the second section
will use this data to connect to a server and transfer local files to the server.
A user may need to transfer a file to an anonymous FTP site – which doesn't require the
user's username or password – or to a secure site – which does require authentication.
So your code first needs to determine how the user intends to transfer a file.
Using an echo command, you start interaction with the user by printing the question Do
you want to perform an anonymous ftp [y/n]: to the screen.
You then use the read command to create a new variable called ANON to place the user's
reply – yes or no – in memory.
To ensure that the value that a user enters is either y or n, you create an echo … read
loop inside a while … do statement.
Each time a user enters a value other than y or n, the commands in the loop execute to
prompt the user for a correct response. The loop exits only when the user enters a correct
value, which the program then reads into memory as the value of the ANON variable.
To cater for the two possible values of the ANON variable – yes or no – you need to use an
if … else statement to create two conditions.
If the value of ANON is yes, the statement will create two new variables – FTPLOGIN and
PASSWD. The value of FTPLOGIN will be "anonymous" and PASSWD won't contain a value.
if [[ $ANON != y ]]
then
FTPLOGIN=anonymous
PASSWD=" "
else
echo "Enter the username to use: "
read FTPLOGIN
echo "Enter the password to use: "
read PASSWD
fi
Question
If the FTP session that a user requests is not anonymous, the user needs to provide
authentication details to log in to the remote server.
You decide to use two echo … read statements to allow the user to supply these
values.
Options:
1. ANON
2. FTPLOGIN
3. PASSWD
4. A new variable to store the user's login details
Answer
You need to specify the FTPLOGIN and PASSWD variables to retrieve the user's
username and password.
Once the program has obtained user authentication details, it needs to prompt the user to
supply the address of the server to which a file is to be transferred and the name and
location of this file.
To make the program do this, you use two print … read statements, in which you create
two new variables – FTPSERVER and FILE – to contain the new information.
if [[ $ANON != y ]]
then
FTPLOGIN=anonymous
PASSWD=" "
print "Enter the address of the FTP server: "
read FTPSERVER
To allow the program to establish an FTP session with a server, you need to specify the ftp
commands that the user's local computer must use to make the actual file transfer.
To do this, you write a here document, which terminates itself and the FTP program when it
reaches the END marker.
(
ftp -n <<- END
open $FTPSERVER
quote USER $FTPLOGIN
quote PASS $PASSWD
binary
cd /home/gregl
put $FILE
END
)
Let's say that you want to test your program by anonymously sending a file called case.sh to
a server at the IP address 190.100.2.2.
You follow the screen prompts and the file is transferred without error.
$ ./ftp.sh
Do you want to perform an anonymous ftp [y/n]:
y
Enter the address of the FTP server:
190.100.2.2
Enter the file to transfer:
case.sh
$
Question
You are writing a script to automate the process of adding a user account to the
system.
Identify the command that completes the code that obtains an account name from
standard input.
print "Enter the account name: "
Missing_code NAME
Options:
1. echo
2. read
3. typeset -x
4. <<
Answer
You use the read command to obtain the account name from standard input.
Option 1 is incorrect. The echo command will merely print the string NAME to the
screen.
Option 2 is correct. The read command will provide a prompt, and input at the
prompt will be assigned to the NAME variable.
Summary
Programs that run from the shell open files that provide the primary means of
communication between it and other processes or programs.
These files are standard input, standard output, and standard error. You need to use file
descriptors with some redirectors. You can direct the flow of output and errors from a
command to a file instead of to the default terminal screen.
You can write programs that use input/output (I/O) control. For example, you can create a
program that automates file transfer across a network for users. This program gathers data
and then uses this data to connect to a server and then transfer local files to the server.
Table of Contents
| Top of page |
| Learning objective |
| Summary |
Copyright © 2003 SkillSoft. All rights reserved.
SkillSoft and the SkillSoft logo are trademarks or registered trademarks
of SkillSoft in the United States and certain other countries.
All other logos or trademarks are the property of their respective owners.
| Print | Contents | Close |
After completing this topic, you should be able to create user menus with the select
command and use them in shell programs.
The select command allows you to create a simple menu that users can use to interact
with a program. The menu contains numbered items and displays a prompt message.
Note
The select command is available only in the Korn shell and version 1.14 and later
of the Bash shell.
You specify a select command using the syntax shown here. In the syntax, you need to
supply a list of menu item names, followed by a do … done statement loop. The command
will associate the names you specify with the commands contained in the loop.
Question
See if you can place the steps that the select command performs when a user
selects a menu item in the correct sequence.
Options
Option Description
A Executes the statements in the body of the command
Options
Option Description
B Prompts the user for a number
C Breaks on encountering a break, exit, return, or EOF marker
Answer
Correct ranking
Option Description
B Prompts the user for a number
D Stores the selected option in a built-in variable named REPLY
A Executes the statements in the body of the command
C Breaks on encountering a break, exit, return, or EOF marker
2. Creating menus
Let's say that you want to write a shell script that allows users to view the files in a directory,
run a text editor, or exit the script.
Whenever a shell displays a menu prompt, it uses a default prompt character stored in a
shell variable named PS3.
You decide to change the default menu prompt ( which is #?, meaning "which number?" ( to
a new prompt that you specify.
To do this, you set the PS3 variable to the text you want to appear as the prompt.
#!/usr/local/bin/ksh
PS3="Select an option [1-3]> "
You name the menu items "Show files", "Run editor", and "Exit", in the order in which they
are to appear on screen.
When the user selects an option, its corresponding value is recorded in the user-defined
OPTION variable and stored in the built-in REPLY variable.
#!/usr/local/bin/ksh
PS3="Select an option [1-3]> "
select OPTION in "Show files" "Run editor" "Exit"
The do … done statement in the select command needs to specify actions that
correspond to a user's selection of each of the three specified menu items.
In this case, you first specify that if the user chooses the third menu option ( the Exit option
for quitting the menu ( the select command will execute the exit command. You do this
using an if ... else statement.
#!/usr/local/bin/ksh
PS3="Select an option [1-3]> "
select OPTION in "Show files" "Run editor" "Exit"
do
if [[ $REPLY = 3 ]]
then
exit
Question
Let's say that you want to test the menu at this point before adding functionality to the
first two menu items.
What command do you think you can use to insert dummy text for these options?
Options:
1. echo
2. read
3. select
4. quit
Answer
You can use echo statements to print dummy option text to test the functionality of
the menu.
To test the code at this point – without adding functionality for options 1 and 2 ( you add an
echo statement that is printed if a user selects any item other than the third menu item.
#!/usr/local/bin/ksh
PS3="Select an option [1-3]> "
select OPTION in "Show files" "Run editor" "Exit"
do
if [[ $REPLY = 3 ]]
then
exit
else
echo "You chose to $OPTION"
fi
done
When you execute the select command, it generates a menu with labeled options and the
changed prompt text you've specified.
$ ./select.sh
1) Show files
2) Run editor
3) Exit
Select an option [1-3]>
This returns the message "You chose to Show files" and returns to the menu.
$ ./select.sh
Show files
Run editor
Exit
Select an option [1-3]>
You chose to Show files
Select an option [1-3]>
The example here shows a dummy menu generated by a select command that uses the
case command to structure the menus, instead of an if … else statement.
When you use the case command, you need to number the menu items manually with
echo statements and you need to terminate each command line with a double semicolon
(;;).
#!/usr/local/bin/ksh
echo "1. Menu item one"
echo "2. Menu item two"
echo "3. Menu item three"
echo "Choose an option [1-3]"
read MENUITEM
case $MENUITEM in
1 ) echo "Do action 1" ;;
2 ) echo "Do action 2" ;;
3 ) echo "Do action 3" ;;
esac
The output of the dummy menu using the case command is shown here.
$ ./case.sh
1. Menu item one
2. Menu item two
3. Menu item three
Choose an option [1-3]
Question
Options:
1. The prompt used by the select command is determined by the PS3 built-in
variable
2. The select command can be combined with the case command
3. The select command is available in the Korn and C shells
4. The select command uses the REPLY variable to store user input
Answer
The PS3 variable determines the prompt used by the select command, and
select command input is stored in the REPLY variable. The select and case
commands are often combined.
Option 1 is correct. The shell's interactive prompt is determined by the PS3 variable.
The Korn shell default is #?
Option 2 is correct. The case command can be used instead of if then else
statements to execute code based on the user's menu selection.
Option 3 is incorrect. The select command is supported by the Korn shell and
recent versions of the Bash shell, but not the C shell.
Option 4 is correct. The select command presents the user with a numbered menu
and a prompt. The value entered by the user is stored in the REPLY variable.
Summary
You can use the select command in scripts to create basic menus. The select
command is available in the Korn shell and in version 1.14 and later of the Bash shell.
The case command provides similar functionality, but is more globally applicable. Both the
case and select commands allow you to specify behavior based on the value of a
variable.
Table of Contents
| Top of page |
| Learning objective |
| Summary |
| Print | Contents | Close |
After completing this topic, you should be able to write an interactive program that makes
use of standard input and output.
Exercise overview
In this exercise, you're required to create a shell script that generates a simple menu.
Let's say that you want to create a program that allows users to view files in the current
directory, edit a specified file, or quit the program using a simple menu.
$ ./editor.sh
View files
Launch Editor
Quit
Choose a menu item [1-3]>
Step 1 of 2
Let's say that you want to specify the menu prompt "Choose a menu item [1-3]>."
See if you can type the code that will change the prompt.
#!/usr/local/bin/ksh#
MISSING CODE "Choose a menu item [1-3]>"
Result
Step 2 of 2
See if you can type the command that will generate a menu.
#!/usr/local/bin/ksh#
PS3="Choose a menu item [1-3]>"
MISSING CODE ITEM in "View files" "Launch Editor" "Quit"
Result
Let's say that you now want to specify the commands that the select command must call
when a user selects one of the menu items.
#!/usr/local/bin/ksh#
PS3="Choose a menu item [1-3]>"
select ITEM in "View files" "Launch Editor" "Quit"
do
Step 1 of 4
#!/usr/local/bin/ksh#
PS3="Choose a menu item [1-3]>"
select ITEM in "View files" "Launch Editor" "Quit"
do
MISSING CODE
Options:
1. case REPLY in
2. case $REPLY in
3. case ITEM in
4. case $ITEM in
Result
Option 2 is correct. The case command executes code based upon the value of a
variable – in this case the REPLY variable returned by the select command.
Option 3 is incorrect. The ITEM variable needs to be expanded using the $ITEM
syntax. Furthermore, it is more efficient to use the REPLY variable.
Step 2 of 4
Choose the code that you think will retrieve a listing of the current directory if a user
selects the first menu item, and will allow to then choose other menu options.
#!/usr/local/bin/ksh#
PS3="Choose a menu item [1-3]>"
select ITEM in "View files" "Launch Editor" "Quit"
do
case $REPLY in
MISSING CODE
Options:
1. 1)
echo "The current directory is $(pwd)"
ls ;;
2. 1)
echo "The current directory is (pwd)"
ls ;;
3. 1)
echo "The current directory is $(pwd)"
ls
break ;;
Result
The code that will retrieve a listing of the current directory is shown here:
1)
echo "The current directory is $(pwd)"
ls ;;
Option 1 is correct. The pwd command provides the name of the current directory as
output. Command substitution is used to insert the output into the string that is sent
to standard output by the echo command. The ls command then lists the contents
of the current directory.
Option 2 is incorrect. To perform command substitution, the brackets that contain the
command must be preceded by the $ special character.
Option 3 is incorrect. The break command will exit the case loop, causing the
program to end once the user has viewed the contents of the current directory.
Step 3 of 4
Choose the code that you think will specify that the second menu item is to load a
specified file in the vi editor.
#!/usr/local/bin/ksh#
PS3="Choose a menu item [1-3]>"
select ITEM in "View files" "Launch Editor" "Quit"
do
case $REPLY in
1)
echo "The current directory is $(pwd)
ls ;;
MISSING CODE
Options:
1. 2)
echo "Name the file to edit"
read $FILE
vi FILE ;;
2. 2)
echo "Name the file to edit"
read FILE
vi $FILE ;;
Result
The code that will load a specified file in the vi editor is shown here:
2)
echo "Name the file to edit"
read FILE
vi $FILE ;;
Option 1 is incorrect. This command will open the vi editor with a file named FILE in
the edit buffer. To achieve the desired result of expanding the FILE variable set
using the read command, you need to perform variable expansion using the $FILE
syntax.
Step 4 of 4
Choose the code that you think will exit the menu if a user selects the third menu
item.
#!/usr/local/bin/ksh#
PS3="Choose a menu item [1-3]>"
select ITEM in "View files" "Launch Editor" "Quit"
do
case $REPLY in
1)
echo "The current directory is $(pwd)
ls ;;
2)
echo "Name the file to edit"
read FILE
vi $FILE ;;
MISSING CODE
Options:
1. 3)
;;
2. 3
break ;
3. 3)
break ;;
Result
The code that will exit the menu if a user selects the third menu item is
3)
break ;;
Option 1 is incorrect. This code will simply result in nothing being executed when the
third option is selected – but the loop will not be exited and the user will be prompted
to choose a menu item again.
Option 2 is incorrect. Each case statement must end with two semicolons, not one.
This code will result in a syntax error, crashing the program.
Option 3 is correct. The break command is used to exit a case, while, or for
loop.
When you have specified the functionality for each menu item, you end the case and do
commands and print a statement to screen.
#!/usr/local/bin/ksh
PS3="Choose a menu item [1-3]>"
select ITEM in "View files"\
"Launch Editor"\
"Quit"
do
case $REPLY in
1)
echo "The current directory is $(pwd)
ls ;;
2)
echo "Name the file to edit>"
read FILE
vi $FILE ;;
3)
break ;;
esac
done
Table of Contents
| Top of page |
| Learning objective |
| Exercise overview |
| Print | Contents |
This article describes recommended practices for writing human-readable shell scripts,
including the use of comments, white space, and indentation.
Introduction
In addition to ensuring that your shell scripts contain accurate code, it's a good idea to lay
them out so that human readers can read them easily.
You can use blank lines and indentation to indicate the structure of the script, and you can
give meaningful names to variables and functions so that the reader can understand their
purposes. These techniques make it easier for you to debug the script after you've written it,
and they allow other shell programmers to read and understand your script.
Variable names
If you name variables and functions after the tasks they perform, it becomes much easier for
readers to figure out what's going on in a script. This may require that names be quite long,
but they will be more transparent.
Note that dots (.) and underscores (_) don't split a name, so they're a good way to build
names that contain multiple words. If, for example, you need a variable to contain the name
of a customer, you could call it customer_name. A function that reads customer data could
be read.customer.data. This allows you to create similar variables – such as
package_name and read.flight.data – without causing any confusion.
Comments
The shell disregards any line in a script that begins with a hash (#). This allows you to insert
comments in a script. Although comments aren't essential to the functioning of a script, it's a
good idea to insert them liberally so that other users can work out what each part of your
script is for.
At the beginning of the read.customer.data function, for example, you should insert a
comment like the following:
It's also a good idea to insert a comment at the beginning of a script. This comment needs
to explain the general purpose and structure of the script. For example, an appropriate
comment might be
# This script obtains data about one or more new bookings using
input functions and then inserts the data as a new entry in the
bookings table.
When you create a variable, you can add a comment that explains its function. This is
especially useful if the variable is an important global variable or if it performs a complex
function in a script. For example, an appropriate comment for explaining a specific variable
is
White space
Except for spaces in individual command lines, the UNIX shell ignores white space in
scripts. You can insert empty lines and spaces before lines of code without affecting the
running of a script.
This is useful if you want to make a script more readable by separating functions, both from
each other and from the main script. You can split the main script at any point you choose –
for example when one task has been accomplished and another begins. There's no such
thing as a script with too much white space – empty lines can only make a script more
readable and reduce eye strain for the reader.
Indentation
Because the shell doesn't respond to initial spaces at the beginning of a line, you can indent
lines by a number of spaces. You should do this for any lines enclosed in a loop or an if
statement, so that the beginning and the end of the statement are obvious to the reader.
if [$bool1 = n]
then
rmdir /shared/$tour
echo "Rerun add_tour to start again."
exit
else
touch /shared/$tour/status$tour
chmod 666 /shared/$tour/status$tour
fi
Summary
When writing a script, you should ensure that it's readable to human readers. To help
human readers understand the script, you should use variable and function names that
reflect the purposes of the corresponding variables and functions. You can also make the
structure of the script easier to understand by using blank lines and indentation. You should
insert explanatory comments wherever necessary so that readers know what each function,
variable, or script is for.
Table of Contents
| Top of page |
| Abstract |
| Introduction |
| Variable names |
| Comments |
| White space |
| Indentation |
| Summary |
After completing this topic, you should be able to explain how to debug shell scripts.
1. Troubleshooting principles
Once you've written a shell script – unless it's a very simple one – it's likely to contain a few
bugs. The more complex a script is, the more time you'll have to spend debugging it.
For this reason, it's very useful to structure the script in a way that makes it easy for you to
make sense of it.
You can reduce the effort of debugging by carefully examining your script at the design
stage.
Question
What do you think you need to keep track of when you analyze a script at the design
stage?
Options:
1. Arithmetic substitution
2. Function calls and arguments
3. Program flow
4. Variable values
Answer
You need to analyze the logical flow of your script. For example, you need to be
aware of the functions that should be called at each stage and of which arguments
these functions should use. You also need to keep track of variables you've declared,
including where they are used and what values they're likely to have.
Once you've sketched the program flow for a script and have eliminated any logical
problems at the design stage, you write the script code.
Then you try running the script. If it requires user input, you should test a wide range of input
values ( including completely inappropriate values ( to see how the script responds.
When testing a script, you may encounter the following types of errors:
logical errors
resource errors
syntax errors
logical errors
Logical errors occur as a result of errors in program flow. For example, a script may fail to
terminate because it's caught in a logical loop, or it may be passing a string to a function
that performs integer arithmetic.
resource errors
Reference errors occur when scripts require resources that they can't access. For example,
a script may refer to a file that doesn't exist or it may need to modify a file to which it doesn't
have write permissions.
$ user_backup
cp: file: No such file or directory
$
syntax errors
Syntax errors occur as a result of lexical errors ( such as misspelled variable names and
parentheses or quotes that haven't been closed ( in scripts.
$ user_backup
user_backup[3]: syntax error: 'while' unmatched
$
If one of your test runs of a script produces an error, you should try to reproduce the error by
running the script again with similar input.
If you can't get the error to repeat, you can't make any further progress towards solving the
problem. However, if it repeats in a predictable way, this indicates a fixable bug.
Once you've established what the effects of a bug are, you can begin looking for its causes.
You first need to find the place in the script at which things begin to go wrong.
If the script gets stuck in a logical loop, for example, you need to find out why the exit
conditions for the loop aren't being met. Perhaps it's because a variable value is wrongly
assigned at some other point in the script.
Once you've located the cause of a bug, you can begin finding a way to fix the bug. This
may be as simple as correcting a spelling mistake, or it may involve rewriting sections of
code.
Question
Options:
Targets:
A. Logical error
B. Resource error
C. Syntax error
Answer
Syntax errors are generated by mistyping variable or command names, and not
using the correct format for commands.
Logical errors result from design flaws, such as passing improper arguments to
functions or failing to terminate loops.
Resource errors result from a script not being able to access external commands or
filesystem locations – either because they don't exist or because the requisite
permissions aren't in place.
Question
Which of the following types of error can be minimized at the design stage of writing
a program?
Options:
1. Logical errors
2. Resource errors
3. Syntax errors
Answer
Logical errors can usually be minimised by planning your programs carefully at the
design stage.
Option 1 is correct. Logical errors are usually the result of design flaws. Preparing
your program in pseudocode helps to eliminate this type of error.
Option 3 is incorrect. Syntax errors are only encountered once you start to write a
program, so they aren't encountered at the design stage.
2. Tracking variables
Finding the place in a script that's causing a bug is the most difficult part of debugging. To
do this, you need to track changes in variable values and the direction of program flow.
The easiest and simplest way to track variables is by introducing large numbers of echo
statements into a script.
You use the echo statements to output the name and value of a variable at different stages
in the script.
For example, this echo statement at the beginning of a function displays the values of any
arguments passed to the function.
read.customer.info () {
> echo $*
> new_entry[0]=$1
> new_entry[1]=$2
> new_entry[2]=$3
> new_entry[3]=$4
> }
$
This example shows an echo statement in a loop. Each time the loop executes, the echo
statement displays the current value of two variables.
while (($count<$total))
do
name=new_entry[0]
touch "$name _data"
count+=1
echo "iteration $count, name is $name"
done
Detailed output from echo statements is necessary for debugging only. Once you've
finished debugging a script, you can disable the echo statements by commenting them out.
This allows you to re-enable them if you need to debug the script again in the future.
while $count<$total
do
name=new_entry[0]
touch "$name _data"
count+=1
# echo "iteration $count, name is $name"
done
You can use traps to track a particular variable or set of variables. Traps are statements that
execute if the script receives a particular signal.
Question
What kind of statements do you think you need to execute in traps when you're
debugging a script?
Options:
Answer
When you debug a script using traps, you need to use the traps to run statements
that echo variable values.
DEBUG
EXIT
ERR
DEBUG
The DEBUG trap causes a statement to execute each time a line of code in the script
executes. This allows you to display the value of a variable every time the script executes a
statement.
EXIT
The EXIT trap causes a statement to execute when a script exits. This is especially useful if
you're dealing with an error that causes a script to exit prematurely because it allows you to
display the values of important variables immediately before the script crashes.
ERR
The ERR trap causes a statement to execute whenever a statement in the script causes an
error – in other words, if its exit status is non-zero. This allows you to view variable values at
the point at which errors occur, but it doesn't allow you to trace them through the statements
that lead up to that point in a script.
In this example, you use the DEBUG trap to run the showvars function after each script
statement executes.
A trap is active from the position of the trap statement onwards. If you want a trap to remain
active throughout a script, you need to enter the trap statement at the beginning of the
script.
To deactivate a trap, you use another trap statement without specifying a command.
$ trap - DEBUG
In this example, you use the EXIT trap to run the exitmsg function if the script receives an
EXIT signal. This function might contain commands that display the values of all variables
and the line number of the last executed statement, for instance.
Question
Options:
1. You can include echo commands in your code without affecting control flow
2. You can use the echo command to rename variables
3. You use the echo command to print a variable without modifying its value
4. You use the # symbol to turn echo commands into comments
Answer
The echo command does not modify the values of variables and doesn't affect
control flow within scripts. You use the # symbol to turn echo commands into
comments.
Option 1 is correct. The echo command simply prints any arguments to standard
output. It doesn't affect control flow, so is useful for debugging programs.
Option 2 is incorrect. The echo command will print the value of any variable passed
to it as an argument, but it can't be used to rename variables.
Option 3 is correct. The echo command can't modify the value of variables passed
to it in arguments, so it is useful for monitoring variable values in order to debug
programs.
Question
You have written a shell program that creates several temporary files. You want to
use the trap command to remove these files if the program exits unexpectedly.
Options:
1. DEBUG
2. ERR
3. EXIT
Answer
You use the EXIT signal if you need to execute code when a program exits.
Option 1 is incorrect. The DEBUG signal is used to create a trap that executes code
after every command in the scripts is executed.
Option 2 is incorrect. The ERR signal is used to create a trap that executes code after
every error command, whether or not the error causes the program to exit.
Option 3 is correct. The EXIT command is used to create a trap that executes code
– in this case code that removes temporary files – when a program terminates. The
specified code will be executed even if the program terminates unexpectedly.
To assist in debugging a script, you can set debugging options when you run it.
noexec
verbose
xtrace
noexec
The noexec option causes the shell to read a script without executing any commands. This
is useful when you want to check for syntax errors or when you suspect that a bug may
hang the system.
verbose
The verbose option echoes all input that a script receives to the standard error file. This
can help you to locate a bug by allowing you to identify which input the script received
immediately before an error occurred.
xtrace
The xtrace option echoes each statement in a script as it's executed, and it echoes each
stage of variable, arithmetic, or command substitution. This is a powerful way of detecting
bugs that occur in the substitution stages of a statement.
This example of xtrace output shows how stages of execution are displayed. The xtrace
prompt – in this case a plus sign (+) – indicates how many levels of substitution are
occurring for a particular line. The example shows one level of variable substitution in which
the number 3 is substituted for the variable i.
Note
You can customize the xtrace prompt using the PS4 environment variable.
You set debugging options using the set command with the -o option. The code shown
here sets the verbose option.
$ set -o verbose
You can run the set command without the -o option. You do this by using the first letter of
the debugging option. This code, for example, sets the xtrace option.
$ set -x
To unset a debugging option, you use the set +o command. The code shown here unsets
the verbose option.
$ set +o verbose
Note
You can't unset the noexec option because it prevents the set +o command from
executing.
Question
Which debugging option do you think allows you to view the input that a script
receives?
Options:
1. noexec
2. verbose
3. xtrace
Answer
The verbose debugging option allows you to view the input that a script receives.
The most powerful tool for debugging a script is a debugging script. A debugging script
appends itself to the target script and enables various debugging tools.
You can write your own debugging script, but some shells provide an existing debugger. For
example, the Bash shell provides the Bash debugger.
To run the Bash debugger on a script, you run a session of the Bash shell and specify the
--debugger option.
Question
Options:
1. noexec
2. verbose
3. xtrace
Answer
Option 1 is incorrect. The noexec option reads commands without executing them.
It's useful for discovering syntax errors.
Question
Options:
Answer
A debugging script appends itself to the target script. You can use it to enable
debugging options and display data when a script exits unexpectedly.
Option 1 is correct. When you execute a debugging script you supply the target script
as a parameter. The debugging code is appended to the code of the target script.
Option 2 is incorrect. A debugging script may help you locate the source of a logical
error, but it won't correct it.
Option 3 is correct. A debugging script can be used to set traps that print useful data
– such as the values of variables – when a program exits unexpectedly.
Option 4 is correct. You can use a debugging script to set shell options – such as the
xtrace option – that facilitate debugging.
If you're developing a script that requires input from users, you need to include statements
that check the input to make sure it makes sense.
For example, an input function that reads a time shouldn't accept hours lower than zero or
higher than 24.
Question
Options:
Answer
If user input doesn't fit into the range of input that makes sense, the error-handling
code in the script should provide a simple error message and exit the script.
This example shows part of a function that reads a date from user input. It checks the input
to ensure that it makes sense and displays an error message and exits if it doesn't fit into
the required range of values.
read.date () {
> echo "Enter month:"
> read month
> if ((month<0)) | ((month>12))
> echo "Invalid month!"
> exit
> fi
Summary
When you write complex shell scripts, they frequently contain bugs. You can avoid some
bugs by checking program logic thoroughly before you write a script, but you need to
remove other bugs after writing. To do this, you need to find the place in the script that
causes the bug and then formulate a solution that fixes it.
You can locate bugs by inserting echo statements that allow you to trace the values of
variables at different points in the script's execution. You can also use traps to display
variable values when certain conditions occur.
You can set debugging options when you run a script. These allow you to investigate how
the script runs. You can also use debugging scripts, which incorporate various debugging
tools.
When you write scripts that accept user input, you should include statements that test the
input to make sure it makes sense.
Table of Contents
| Top of page |
| Learning objective |
| Summary |
| Print | Contents |
This exercise is designed for practice use and does not contain new learning content. If your
computer doesn't have an application or applications required to carry out the exercise
tasks, or if you would prefer to perform the exercise at another time, you can proceed to the
next topic.
Scenario
You've written a script that creates a shared directory. It allows a user to specify the
directory's name and to undo the creation process if they've made a mistake. However,
you've run the script and discovered that it has bugs.
Exercise
Your assignment for this exercise is to find and eliminate all bugs in the script. In its current
state, the script contains two logical errors and three syntax errors.
Task list
Numbe Instructions
r
1 Remove the bugs from the following script:
#!/usr/local/bin/ksh
DirPath="/home/shared/"
Exist=0
clear
mkdir -m 777 $DirPath$DirName
echo "The $DirName directory has been created in
$DirPath. Are you happy with this?"
read -p "[y/n]" BOOL1
clear
if [ $BOOL1 = n ]
then
rmdir $DirPathDirName
echo "$DirName has been removed"
exit
else
touch $DirPath$DirName/status$DirName
chmod 666 $DirPath$DirName/status$DirName
if
Review your solution and, when you are satisfied with it, open the solution page.
Table of Contents
| Top of page |
| Scenario |
| Exercise |