Unit V

Shell:

Shell is an environment in which we can run our commands, programs, and shell scripts. There
are different flavors of a shell, just as there are different flavors of operating systems. Each flavor
of shell has its own set of recognized commands and functions.

Shell Prompt
The prompt, $, which is called the command prompt, is issued by the shell. While the prompt
is displayed, you can type a command.

Shell reads your input after you press Enter. It determines the command you want executed by
looking at the first word of your input. A word is an unbroken set of characters. Spaces and tabs
separate words.

Following is a simple example of the date command, which displays the current date and time −
$date
Thu Jun 25 08:30:19 MST 2009

You can customize your command prompt using the environment variable PS1 explained in the
Environment tutorial.

Shell Types
In Unix, there are two major types of shells −

 Bourne shell − If you are using a Bourne-type shell, the $ character is the default
prompt.

 C shell − If you are using a C-type shell, the % character is the default prompt.

The Bourne Shell has the following subcategories −

 Bourne shell (sh)

 Korn shell (ksh)
 Bourne Again shell (bash)
 POSIX shell (sh)
The different C-type shells follow −

 C shell (csh)
  TENEX/TOPS C shell (tcsh)

The original Unix shell was written in the mid-1970s by Stephen R. Bourne while he was at the
AT&T Bell Labs in New Jersey.

Bourne shell was the first shell to appear on Unix systems, thus it is referred to as "the shell".

Bourne shell is usually installed as /bin/sh on most versions of Unix. For this reason, it is the
shell of choice for writing scripts that can be used on different versions of Unix.

In this chapter, we are going to cover most of the Shell concepts that are based on the Borne
Shell.

Shell Scripts
The basic concept of a shell script is a list of commands, which are listed in the order of
execution. A good shell script will have comments, preceded by #sign, describing the steps.

There are conditional tests, such as value A is greater than value B, loops allowing us to go
through massive amounts of data, files to read and store data, and variables to read and store
data, and the script may include functions.

Example Script
Assume we create a test.sh script. Note all the scripts would have the .shextension. Before you
add anything else to your script, you need to alert the system that a shell script is being started.
This is done using the shebangconstruct. For example −

#!/bin/sh
This tells the system that the commands that follow are to be executed by the Bourne shell. It's
called a shebang because the # symbol is called a hash, and the ! symbol is called a bang.

To create a script containing these commands, you put the shebang line first and then add the
commands −
#!/bin/bash
pwd
ls

Shell Programming in bash:

Example 1. Hello World

#!/bin/sh
echo "Hello world"

Using Arguments

Example 2. Shell Script Arguments

#!/bin/bash

# example of using arguments to a script

echo "My first name is $1"
echo "My surname is $2"
echo "Total number of arguments is $#"

Conditions and Loops:

For Loops
for loops iterate through a set of values until the list is exhausted:
for.sh
#!/bin/sh
for i in 1 2 3 4 5
do
echo "Looping ... number $i"
done

for2.sh
#!/bin/sh
for i in hello 1 * 2 goodbye
do
echo "Looping ... i is set to $i"
done

While Loops
while.sh
#!/bin/sh
INPUT_STRING=hello
while [ "$INPUT_STRING" != "bye" ]
do
echo "Please type something in (bye to quit)"
read INPUT_STRING
echo "You typed: $INPUT_STRING"
done

while2.sh
#!/bin/sh
while :
do
echo "Please type something in (^C to quit)"
read INPUT_STRING
echo "You typed: $INPUT_STRING"
done

while3a.sh
#!/bin/sh
while read f
do
case $f in
hello) echo English ;;
howdy) echo American ;;
gday) echo Australian ;;
bonjour) echo French ;;
"guten tag") echo German ;;
*) echo Unknown Language: $f
;;
esac
done < myfile

If Condition:

#!/bin/sh

a=10

b=20
if [ $a == $b ]

then

echo "a is equal to b"

elif [ $a -gt $b ]

then

echo "a is greater than b"

elif [ $a -lt $b ]

then

echo "a is less than b"

else

echo "None of the condition met"

fi

Ex2:

1. #!/bin/bash
2. # Basic if statement
3.
4. if [ $1 -gt 100 ]
5. then
6. echo Hey that\'s a large number.
7. pwd
8. fi
9.
10. date

Case statement:

Ex1:
#!/bin/sh
x=0
echo Enter the number
read x
case $x in

1)
echo January
;;
2)
echo February
;;
3)
echo March
;;
4)
echo April
;;
5)
echo May
;;
6)
echo June
;;
7)
echo July
;;
8)
echo August
;;
9)
echo September
;;
10)
echo October
;;
11)
echo November
;;
12)
echo December
;;
*)
echo Not sure that about one
;;

esac

Ex2:

#!/bin/sh

FRUIT="kiwi"

case "$FRUIT" in

"apple") echo "Apple pie is quite tasty."

 ;;

"banana") echo "I like banana nut bread."

;;

"kiwi") echo "New Zealand is famous for kiwi."

;;

esac

To pass argument to shell script:

#!/bin/bash
# Call this script with at least 3 parameters, for example
# sh scriptname 1 2 3

echo "first parameter is $1"

echo "Second parameter is $2"

echo "Third parameter is $3"

exit 0

Output:
[root@localhost ~]# sh parameters.sh 47 9 34

first parameter is 47

Second parameter is 9

Third parameter is 34

[root@localhost ~]# sh parameters.sh 4 8 3

first parameter is 4

Second parameter is 8

Third parameter is 3

Shell Keywords:
Keywords are the words whose meaning has already been explained to the shell.the
keywords cannot be used as variable names because of it is a reserved words with
containing reserved meaning.

Echo read set unset

Readonly shift export if

Fi else while do

Done For until case

Esac break continue exit

Return trap wait eval

Exec ulimit umask

Grep Command:

Grep command in Unix/Linux is a powerful tool that searches for matching a regular

expression against text in a file, multiple files or a stream of input. It searches for the pattern of

text that you specify on the command line and prints output for you.

In addition, three variant programs egrep,fgrep and rgrep are available.

 egrep is the same as grep -E

 fgrep is the same as grep -F
 rgrep is the same as grep -r

Direct invocation as either egrep or fgrep is deprecated, but is provided to allow historical
applications that rely on them to run unmodified.
1. Search for a string “linux” in a file using grep command in unix

This is the basic usage of grep command. It searches for the given string in the specified file.

grep “linux” index.html

2. Insensitive case search with grep -i

The below grep command searches for the words like “LINUX”, “Linux”, “linux” case

insensitively.

grep -i “linux” index.html

3. Searching for a string in multiple files.

This command will search for "linux" string in multiple files at a time. It searches in all files with

file1.txt, file2.txt and along with different extensions too like file1.html, file2.php and so on.

grep “linux” file*.*

4. Specifying the search string as a regular expression pattern.

It is a very powerful feature and can use as a regular expression with much effectively. In the

below example, it searches for all the pattern that starts with “fast” and ends with “host” with

anything in-between. i.e To search “fast[anything in-between]host” in index.html file.

grep “fast.*host” index.html

5. Displaying the line numbers.

You can use this grep command to display the line number which contains the matched string in

a file using the -n option

grep -n “word*” file.txt

6. Highlighting the search using grep

If we use the –color option, our successful matches will be highlighted for us.

grep –color “linux” index.html

7. Print the line excluding the pattern using -v option

List all the lines of the file /etc/passwd that does not contain specific word “string”.

grep -v linux /etc/passwd

8. Display all the lines that starts with specified pattern using ^ symbol

Bash shell treats carrot symbol (^) as a special character which treat as the beginning of line.

Let’s display the lines which starts with “root” word in the file /etc/passwd.

grep ^root /etc/passwd

9. Display all the lines that ends with specified pattern using $ symbol.

List all the lines of /etc/passwd that ends with “bash” word.

grep bash$ /etc/passwd

10. Search the pattern recursively using -r option

The below command will search linux in the “/etc” directory recursively.

grep -r linux /etc/

11. Counting the lines when words match

This grep command can report the number of times the pattern matches for each file by using -c

(count) option.

grep -c 'test' /home/example/test.txt

Conclusion

By the time you complete reading this article, I am sure you will get basic idea about grep

command and how to use it on your Linux server with different search patterns. If you like this,

please post your comment below.

Awk Programming:

Awk is a scripting language used for manipulating data and generating reports.The awk

command programming language requires no compiling, and allows the user to use variables,

numeric functions, string functions, and logical operators.

Awk is a utility that enables a programmer to write tiny but effective programs in the form of
statements that define text patterns that are to be searched for in each line of a document and the
action that is to be taken when a match is found within a line. Awk is mostly used for pattern
scanning and processing. It searches one or more files to see if they contain lines that matches
with the specified patterns and then performs the associated actions.
Awk is abbreviated from the names of the developers – Aho, Weinberger, and Kernighan.
WHAT CAN WE DO WITH AWK ?
 1. AWK Operations:
(a) Scans a file line by line
(b) Splits each input line into fields
(c) Compares input line/fields to pattern
(d) Performs action(s) on matched lines

2. Useful For:
(a) Transform data files
(b) Produce formatted reports

3. Programming Constructs:
(a) Format output lines
(b) Arithmetic and string operations
(c) Conditionals and loops

Syntax:

awk options 'selection _criteria {action }' input-file > output-file

Options:
-f program-file : Reads the AWK program source from the file

program-file, instead of from the

first command line argument.

-F fs : Use fs for the input field separator

Example:

Consider the following text file as the input file for all cases below.
$cat > employee.txt

ajay manager account 45000

sunil clerk account 25000

varun manager sales 50000

amit manager account 47000

tarun peon sales 15000

deepak clerk sales 23000

sunil peon sales 13000

satvik director purchase 80000

1. Default behavior of Awk : By default Awk prints every line of data from the specified
file.
$ awk '{print}' employee.txt

Output:
ajay manager account 45000

sunil clerk account 25000

varun manager sales 50000

amit manager account 47000

tarun peon sales 15000

deepak clerk sales 23000

sunil peon sales 13000

satvik director purchase 80000

In the above example, no pattern is given. So the actions are applicable to all the lines. Action
print without any argument prints the whole line by default, so it prints all the lines of the file
without failure.
2. Print the lines which matches with the given pattern.
$ awk '/manager/ {print}' employee.txt

Output:
ajay manager account 45000

varun manager sales 50000

amit manager account 47000

In the above example, the awk command prints all the line which matches with the
‘manager’.

3. Spliting a Line Into Fields : For each record i.e line, the awk command splits the
record delimited by whitespace character by default and stores it in the $n variables. If
the line has 4 words, it will be stored in $1, $2, $3 and $4 respectively. Also, $0
represents the whole line.
$ awk '{print $1,$4}' employee.txt

Output:
ajay 45000

sunil 25000

varun 50000
amit 47000

tarun 15000

deepak 23000

sunil 13000

satvik 80000

Find the employees who has employee id greater than 200

$ awk '$1 >200' employee.txt

300 Sanjay Sysadmin Technology $7,000

400 Nisha Manager Marketing $9,500

500 Randy DBA Technology $6,000

Ex: We can create a favorite_food.txt file which lists an item number and the favorite foods of a
group of friends:

echo "1 carrot sandy

2 wasabi luke
3 sandwich brian
4 salad ryan
5 spaghetti jessica" > favorite_food.txt
If we want to find all foods from this file that begin with "sa", we might begin by trying
something like this:

awk '/sa/' favorite_food.txt

1 carrot sandy
2 wasabi luke
3 sandwich brian
4 salad ryan

Here, we are matching any instance of "sa" in the word. This does exclude things like "wasabi"
which has the pattern in the middle, or "sandy" which is not in the column we want. We are only
interested in words beginning with "sa" in the second column.
 We can tell awk to only match at the beginning of the second column by using this command:

awk '$2 ~ /^sa/' favorite_food.txt

3 sandwich brian
4 salad ryan
The "^" character tells awk to limit its searches to the beginning of the field. The "field_num ~"
part specifies that it should only pay attention to the second column.

We can just as easily search for things that do not match by including the "!" character before the
tilde (~). This command will return all lines that do not have a food that starts with "sa":

awk '$2 !~ /^sa/' favorite_food.txt

1 carrot sandy
2 wasabi luke
5 spaghetti jessica
If we decide later on that we are only interested in lines where the above is true and the item
number is less than 5, we could use a compound expression like this:

awk '$2 !~ /^sa/ && $1 < 5' favorite_food.txt

This introduces a few new things. The first is the ability to add additional requirements for the
line to match by using the && operator. Using this, you can combine an arbitrary number of
conditions for the line to match.

Boolean expression when working with multiple data types including strings and numbers:

While doing logic examples we will use following test data which file name is students.txt . The
numbers are exam1, exam2 and exam3

ismail 45 80 75
ali 80 90 100
elif 80 95 45

1ismail 45 80 75
2ali 80 90 100
3elif 80 95 45

Awk AND Logic

and logic is used to check if all provided Boolean values are True . We use && to specify and
logic. The basic usage is providing two Boolean values and we can also provide more than 2
 values. The best practice while using and logic is surrounding the operators and values with
parenthesis ( ... ) . Syntax of and operator is like below.

(CONDITION && CONDITION &&

1(CONDITION && CONDITION && ... )

In this example we will check two exam1 and exam2 grades if they are over 50we will print the
name of the student.

1$ awk '{ if( $2>50 && $3>50 ) print $1 }' students.txt

Awk OR Logic

OR logic is used to check given conditions and if one of the condition is true the
whole or logic will return true. We will use || signs to specify OR logic. Syntax of OR
logic is like below.

(CONDITION || CONDITION || .

1(CONDITION || CONDITION || ... )

In this example we will check all three exams degrees and find students those have
get a degree lower than 50 at least one time.

$ awk '{ if( $2<50 || $3<50 || $

1$ awk '{ if( $2<50 || $3<50 || $4<50 ) print $1 }' students.txt

Built In Variables In Awk

Awk’s built-in variables include the field variables—$1, $2, $3, and so on ($0 is the entire line) — that
break a line of text into individual words or pieces called fields.

NR: NR command keeps a current count of the number of input records. Remember that records are
usually lines. Awk command performs the pattern/action statements once for each record in a file.

NF: NF command keeps a count of the number of fields within the current input record.

FS: FS command contains the field separator character which is used to divide fields on the input line.
The default is “white space”, meaning space and tab characters. FS can be reassigned to another
character (typically in BEGIN) to change the field separator.

RS: RS command stores the current record separator character. Since, by default, an input line is the
input record, the default record separator character is a newline.

OFS: OFS command stores the output field separator, which separates the fields when Awk prints
them. The default is a blank space. Whenever print has several parameters separated with commas, it
will print the value of OFS in between each parameter.

ORS: ORS command stores the output record separator, which separates the output lines when Awk
prints them. The default is a newline character. Print automatically outputs the contents of ORS at the
end of whatever it is given to print.

Create temporary files or directories:

Use mktemp to create a temporary file or directory:

temp_file=$(mktemp)
Or for a direcotry:

temp_dir=$(mktemp -d)
At the end of the script you have to delete the temporary file/dir:

rm ${temp_file}
rm -R ${temp_dir}
You need to generate 10 random numbers. How would you achieve this?

Consider the following script:

Let’s execute it to see what we will get: