18cs56 Unix Programming Notes
18cs56 Unix Programming Notes
18cs56 Unix Programming Notes
* Visit https://vtuconnect.in for more info. For any queries or questions wrt our
platform contact us at: support@vtuconnect.in
VTU Connect Android App Download Now On Google Play Store
LESSON NOTES
Vision
Mission
To keep pace with advancements in knowledge and make the students competitive
and capable at the global level.
To create an environment for the students to acquire the right physical, intellectual,
emotional and moral foundations and shine as torch bearers of tomorrow's society.
To strive to attain ever-higher benchmarks of educational excellence.
1. Demonstrate understanding of the principles and working of the hardware and software
aspects of Embedded Systems.
2. Use professional Engineering practices, strategies and tactics for the development,
implementation and maintenance of software.
3. Provide effective and efficient real time solutions using acquired knowledge in
various domains.
UNIX PROGRAMMING
(Effective from the academic year 2018 -2019) SEMESTER – V
Subject Code 18CS56 CIE Marks 40
Number of Contact Hours/Week 3:0:0 SEE Marks 60
Total Number of Contact Hours 40 Exam Hours 3 Hrs
CREDITS –4
Course Objectives:This course (18CS56) will enable students to
Interpret the features of UNIX and basic commands.
Demonstrate different UNIX files and permissions
Implement shell programs.
Explain UNIX process, IPC and signals.
Module 1 Contact
Hours
Introduction: Unix Components/Architecture. Features of Unix. The UNIX Environment 08
and UNIX Structure, Posix and Single Unix specification. General features of Unix
commands/ command structure. Command arguments and options. Basic Unix commands
such as echo, printf, ls, who, date,passwd, cal, Combining commands. Meaning of Internal
and external commands. The type command: knowing the type of a command and locating
it. The root login. Becoming the super user: su command.
Unix files: Naming files. Basic file types/categories. Organization of files. Hidden files.
Standard directories. Parent child relationship. The home directory and the HOME variable.
Reaching required files- the PATH variable, manipulating the PATH, Relative and absolute
pathnames. Directory commands – pwd, cd, mkdir, rmdir commands. The dot (.) and double
dots (..) notations to represent present and parent directories and their usage in relative path
names. File related commands – cat, mv, rm, cp, wc and od commands.
Module 2
File attributes and permissions: The ls command with options. Changing file permissions: 08
the relative and absolute permissions changing methods. Recursively changing file
permissions. Directory permissions.
The shells interpretive cycle: Wild cards. Removing the special meanings of wild cards.
Three standard files and redirection. Connecting commands: Pipe. Basic and Extended
regular expressions. The grep, egrep. Typical examples involving different regular
expressions.
Shell programming: Ordinary and environment variables. The .profile. Read and readonly
commands. Command line arguments. exit and exit status of a command. Logical operators
for conditional execution. The test command and its shortcut. The if, while, for and case
control statements. The set and shift commands and handling positional parameters. The here
( << ) document and trap command. Simple shell program examples.
Module 3
UNIX File APIs: General File APIs, File and Record Locking, Directory File APIs, Device 08
File APIs, FIFO File APIs, Symbolic Link File APIs.
UNIX Processes and Process Control:
The Environment of a UNIX Process: Introduction, main function, Process Termination,
Command-Line Arguments, Environment List, Memory Layout of a C Program, Shared
Libraries, Memory Allocation, Environment Variables, setjmp and longjmp Functions,
getrlimit, setrlimit Functions, UNIX Kernel Support for Processes.
Process Control: Introduction, Process Identifiers, fork, vfork, exit, wait, waitpid, wait3,
wait4 Functions, Race Conditions, exec Functions
Module 4
Changing User IDs and Group IDs, Interpreter Files, system Function, Process Accounting, 08
User Identification, Process Times, I/O Redirection.
Overview of IPC Methods, Pipes, popen, pclose Functions, Coprocesses, FIFOs, System V
IPC, Message Queues, Semaphores.
Shared Memory, Client-Server Properties, Stream Pipes, Passing File Descriptors, An
Open Server-Version 1, Client-Server Connection Functions.
Module 5
Signals and Daemon Processes: Signals: The UNIX Kernel Support for Signals, signal, 08
Signal Mask, sigaction, The SIGCHLD Signal and the waitpid Function, The sigsetjmp and
siglongjmp Functions, Kill, Alarm, Interval Timers, POSIX.lb Timers. Daemon Processes:
Introduction, Daemon Characteristics, Coding Rules, Error Logging, Client-Server Model.
Course Outcomes: The student will be able to :
Explain Unix Architecture, File system and use of Basic Commands
Illustrate Shell Programming and to write Shell Scripts
Categorize, compare and make use of Unix System Calls
Build an application/service over a Unix system.
Question Paper Pattern:
The question paper will have ten questions.
Each full Question consisting of 20 marks
There will be 2 full questions (with a maximum of four sub questions) from each module.
Each full question will have sub questions covering all the topics under a module.
The students will have to answer 5 full questions, selecting one full question from each module.
Textbooks:
1. Sumitabha Das., Unix Concepts and Applications., 4thEdition., Tata McGraw Hill ( Chapter 1,2
,3,4,5,6,8,13,14)
2. W. Richard Stevens: Advanced Programming in the UNIX Environment, 2nd Edition,
Pearson Education, 2005 ( Chapter 3,7,8,10,13,15)
3. Unix System Programming Using C++ - Terrence Chan, PHI, 1999. ( Chapter 7,8,9,10)
Reference Books:
1. M.G. Venkatesh Murthy: UNIX & Shell Programming, Pearson Education.
2. Richard Blum , Christine Bresnahan : Linux Command Line and Shell Scripting
Bible, 2ndEdition, Wiley,2014.
Module - I
INTRODUCTION
WHAT IS AN OPERATING SYSTEM (OS)?
An operating system (OS) is an interface between hardware and user. It manages hardware and
software resource. It takes the form of a set of software routines that allow users and application
programs to access system resources (e.g. the CPU, memory, disks, modems, printers, network
cards etc.) in a safe, efficient and abstract way.
For example, an OS ensures safe access to a printer by allowing only one application program
to send data directly to the printer at any one time. An OS encourages efficient use of the CPU
by suspending programs that are waiting for I/O operations to complete to make way for
programs that can use the CPU more productively. An OS also provides convenient
abstractions (such as files rather than disk locations) which isolate application programmers
and users from the details of the underlying hardware.
BRIEF HISTORY
In the late 1960s, researchers from General Electric, MIT and Bell Labs launched a joint project
to develop an ambitious multi-user, multi-tasking OS for mainframe computers known as
MULTICS (Multiplexed Information and Computing System). MULTICS failed, but it did
inspire Ken Thompson, who was a researcher at Bell Labs, to have a go at writing a simpler
operating system himself. He wrote a simpler version of MULTICS on a PDP7 in assembler
and called his attempt UNICS (Uniplexed Information and Computing System). Because
memory and CPU power were at a premium in those days, UNICS (eventually shortened to
UNIX) used short commands to minimize the space needed to store them and the time needed
to decode them - hence the tradition of short.
The limitation of UNICS was not portable.In oder to overcome the limitation , Ken Thompson
started to work on the development of system using higher level language called B Language.
As B language did not yield expected results,Dennis ritchie developed higher level language
called C .Ken Thompson then teamed up with Dennis Ritchie, the author of the first C
compiler in 1973. They rewrote the UNIX kernel in C - this was a big step forwards in terms
of the system's portability - and released the fifth Edition of UNIX to universities in 1974.
UNIX ARCHITECTURE
It is loaded into memory when the system is booted and communicates directly with the
hardware. The kernel manages system memory, processes, decides priorities.
Shell: interface between Kernel and User. It functions as command interpreter i,e it receives
and interprets the command from user and interacts with the hardware. There is only one kernel
running on the system, there could be several shells in action- one for each user who is logged
in.
Files and Process: file is an array of bytes and it contain virtually anything. Unix considers
even the directories and devices as members of file system. The dominant file type is text and
behavior of system is mainly controlled by text files.
The second entity is the process, which is the name given to a file when it is executed as a
program. Process is simply a time image of an executable file.
1.1 System Calls: Though there are thousands of commands in the unix system, they all
use a handful of functions called system calls. User programs that need to access the hardware
use the services of the kernel, which performs the job on users behalf. These programs access
the kernel through a set of functions called system calls.
Ex: open()-- system call to access both file and device. Write()—system call to write a file.
FEATURES OF UNIX
Several features of UNIX have made it popular. Some of them are:
• Portable: UNIX can be installed on many hardware platforms. Its widespread use
can be traced to the decision to develop it using the C language. Because C programs
are easily moved from one hardware environment to another, it is relatively simple to
port it to different environments.
• Multiuser: The UNIX design allows multiple users to concurrently share hardware
and software
• Multitasking: UNIX allows a user to run more than one program at a time. In fact
more than one program can be running in the background while a user is working
foreground.
• Networking: While UNIX was developed to be an interactive, multiuser,
multitasking system, networking is also incorporated into the heart of the operating
system. Access to another system uses a standard communications protocol known as
Transmission Control Protocol/Internet Protocol (TCP/IP).
• Organized File System: UNIX has a very organized file and directory system that
allows users to organize and maintain files.
• Device Independence: UNIX treats input/output devices like ordinary files. Input or
output to a program can be from any device or file.The source or destination for file
input and output is easily controlled through a UNIX design feature called redirection.
• Utilities: UNIX provides a rich library of utilities that can be use to increase user
productivity.
• Services: UNIX also includes the support utilities for system administration and
control.
• Personal environment
• Timesharing environment: Many users connected to one computer
• Client/server environment Computing split between a central computer (server) and
users’ computers (clients)
Personal environment originally unix designed as a multiuser environment, many user user are
installed UNIX on their personal computers this tends to personal unix system environment
Timesharing environment
All of this work tends to keep the central computer busy so, user has to wait more
time for get done their work so, it is nonproductive because of slow response.
UNIX STRUCTURE
Kernel: is the heart of UNIX system. It contains two basic parts of the OS: process control and
resource management. All other components of the system call on the kernel to perform these
services for them.
Shell: interface between Kernel and User. It functions as command interpreter i,e it receives
and interprets the command from user and interacts with the hardware. There is only one kernel
running on the system, there could be several shells in action- one for each user who is logged
in. Shell has two major parts.
a. Interpreter: reads your commands and works with the kernel to execute them.
b. Shell Programming: is a programming capability that allows you to write a shell scripts.
A shell script is a file that contains the shell commands that perform a useful function. It is also
known as shell program.
• C shell: developed in the Berkeley by Bill joy, Its commands look like C statements.
• Korn shell: developed by David Korn, also of AT&T Labs is the newest and powerful.
Utilities: A utility is a standard Unix program that provides a support for users. Three common
utilities are text editors, search programs and sort programs.
Applications: are programs that are not a standard part of UNIX. Written by system
administrator’s professional programmers or users they provide an extended capability to the
system.
Dennis Ritchie’s decision to rewrite UNIX in C didn't quite make UNIX very portable. UNIX
fragmentation and absence of a single conforming standard adversely affected the development
of portable applications. First ,AT &T created the System V Interface Definition(SVID). Later,
X/Guide(XPG). Products conforming to this specification were branded UNIX95, UNIX98 or
UNIX03 depending on the version of the specification.
Yet another group of standards, the portable operating system interface for computer
environments(POSIX), were developed at the behest of the Institution of Electrical and
Electronics Engineers(IEEE). POSIX refers to operating systems in general, but was based on
UNIX. Two of the most cited standards from the POSIX family are known as POSIX.1 and
POSIX.2. POSIX.1 specifies the C application program interface the system calls. POSIX.2
deals with the shell and utilities.
In 2001, a joint initiative of X/Open and IEEE resulted in the unification of the two standards.
This is the single UNIX Specification, version 3(SUSV3). The “write once, adopt everywhere”
approach to this development means that once software has been developed on any POSIX
compliant UNIX system, it can be easily ported to another POSIX- compliant UNIX machine
with minimum modifications. We make reference to POSIX throughout this text, but these
references should be Interpreted to mean the SUSV3 as well.
Commands are entered at shell prompt.The components of the command line are:
• the verb;
• any options required by the command
• the command's arguments (if required).
Verb: is the command name. The command indicates what action is to be taken. This action
concept gives us the name verb for action .
option: modifies how the action is applied.
argument: provides additional information to the command.
Note: Options MUST come after the command and before any command arguments. Options
SHOULD NOT appear after the main argument(s). However, some options can have their
own arguments
if options are enclosed within the [] then options are not mandatory else it is compulsory
if arguments are enclosed within the [] then options are not mandatory else it is compulsory
UNDERSTANDING OF SOME BASIC COMMANDS SUCH AS echo, printf, ls, who,
date, passwd, cal.
THE DATE COMMAND:
date: displays the system date and time.If the system is local that is one in your own area-it is
the current time.If the system is remote, such as across the country the reply will contain the
time where the system is physically located.
The input for the date is the system itself.The date is actually maintained in the computer as a
part of OS.The date command sends its response to monitor.
$date
$date -u
$ cal
Output:
April 2016
Su Mo Tu We Th Fr
Sa 1 2
3456789
10 11 12 13 14 15 16
17 18 19 20 21 22 23
24 25 26 27 28 29 30
February 2015
Su Mo Tu We
Th Fr Sa 1 2 3
4567
8 9 10 11 12 13 14
15 16 17 18 19 20 21
22 23 24 25 26 27 28
• To display complete year calendar.
$ cal -y
who: THE Who is online Command
The who command prints information about all users who are currently logged in.
$who am i
Displays the same information, but only for the terminal session where the command was
issued, for example:
$who –u
Indicates how long it has been since there was any activity on the line. This is known as
Idle time. It also returns the process id for the user.
$who -uH
Displays "all" information, and headers above each column of data, for example:’
passwd syntax
passwd [OPTION] [USER]
$passwd
Running passwd with no options will change the password of the account running the
command. You will first be prompted to enter the account's current password:
The passwd command changes passwords for user accounts. A normal user can only change
the password for their own account, but the superuser can change the password for any account.
Passwd can also change or reset the account's validity period — how much time can pass before
the password expires and must be changed.
Before a normal user can change their own password, they must first enter their current
password for verification. (The superuser can bypass this step when changing another user's
password.)
After the current password has been verified, passwd checks to see if the user is allowed to
change their password at this time. If not, passwd refuses to continue, and exits.
Otherwise, the user is then prompted twice for a replacement password. Both entries must
match for passwd to continue.
If you specify the -e option, the following escape sequences are recognized:
Ex 3:
$echo -e “Here\bthe\bspaces\bare\bbackspaced”
Outputs the following text:
Output :Herthspacearbackspaced //one character before backslash is deleted.
man syntax
$man [option] command name
Options
-K,-- Search for text in all manual pages. This is a brute-force search, and is likely to take some
time; if global- you can, you should specify a section to reduce the number of pages that need to be
searched. aproposSearch terms may be simple strings (the default), or regular expressions if the --
regex option is used
used.
Section Numbers
The section numbers of the manual are listed below. While reading documentation, if you see
a com- mand name followed by a number in parentheses, the number refers to one of these
sections. For exam- ple, man is the documentation of man found in section number 1. Some
commands may have docu- mentation in more than one section, so the numbers after the
command name may direct you to the cor- rect section to find a specific type of information.
The section numbers, and the topics they cover, are as follows:
Section number Description
1 Executable programs or shell commands
2 System calls (functions provided by the kernel)
3 Library calls (functions within program libraries)
4 Special files (usually found in /dev)
5 File formats and conventions eg /etc/passwd
6 Games
7 Miscellaneous (including macro packages and conventions),
e.g. man, groff
8 System administration commands (usually only for root)
9 Kernel routines [Non standard]
searches the manual pages for a keyword or regular expression. Each manual page has a short
descrip- tion included with it. apropos searches these descriptions for instances of keyword.
$apropos find
aa_find_mountpoint (2) - find where the apparmor interface filesystem is mounted
chkdupexe (1) - find duplicate executables
ffs (3) - find first bit set in a word
ls COMMAND
The ls command lists all files in the directory that match the name. If name is left blank, it will
list all of the files in the directory.
Syntax
The syntax for the ls command is:
ls [options] [names]
Option Description
-a Displays all files.
-b Displays nonprinting characters in octal.
-c Displays files by file timestamp.
-C Displays files in a columnar format (default)
-d Displays only directories.
-f Interprets each name as a directory, not a file.
-F Flags filenames.
-g Displays the long format listing, but exclude the owner name.
-i Displays the inode for each file.
-l Displays the long format listing.
-L Displays the file or directory referenced by a symbolic link.
-m Displays the names as a comma-separated list.
-n Displays the long format listing, with GID and UID numbers.
-o Displays the long format listing, but excludes group name.
-p Displays directories with /
-q Displays all nonprinting characters as ?
-r Displays files in reverse order.
-R Displays subdirectories as well.
-t Displays newest files first. (based on timestamp)
-u Displays files by the file access time.
-x Displays files as rows across the screen.
-1 Displays each entry on a line.
To show long listing information about the file/directory.
$ls -l
a. Field 1:
▪ 1st Character – File Type: First character specifies the type of the file. In the example
above the hyphen (-) in the 1st character indicates that this is a normal file. Following are
the possible file type options in the 1st character of the ls -l output.
• Field Explanation
• – normal file
• d directory
• s socket file
• l link file
▪ 2nd to 9th character -- File Permissions: Next 9 character specifies the files permission.
Each 3 characters refers to the read, write, execute permissions for owner, group and other.
g. Field 7 – File name: The last field is the name of the file.
$ ls -l /etc
total 3344
-rw-r--r-- 1 root root 15276 Oct 5 2004 a2ps.cfg
-rw-r--r-- 1 root root 2562 Oct 5 2004 a2ps-site.cfg
drwxr-xr-x 4 root root 4096 Feb 2 2007 acpi
-rw-r--r-- 1 root root 48 Feb 8 2008 adjtime
drwxr-xr-x 4 root root 4096 Feb 2 2007 alchemist
$ ls -ld /etc
drwxr-xr-x 21 root root 4096 Jun 15 07:02 /etc
tThe above command first it displays the line count, word cont and byte or character
count along with this it also display the details of note file.
When you learn to redirect the output of these commands you may even like to group
them together within parentheses .
Example :
$(wc note ; ls -l note) > newlist
The combined output of the two commands is now sent to the file newlist. Whitespace
is provided here only for better readability. You might reduce a few keystrokes like
this
$wc note;ls -l note)>newlist
When a command line contains a semicolon, the shell understands that the command
on each side of it needs to be processed separately. The ; here is known as a
metacharacter, and you'll come across several metacharaters that have special
meaning to the shell.
A command line can overflow or be split into multiple lines
A command is often keyed in. though the terminal width is restricted to 80 characters, that
doesn't prevent you from entering a command, or a sequence of them, in one line even though
the total width may exceed 80 characters. The command simply overflows to the next line
though it is stil in a single logical line.
Sometimes, you'll find it necessary or desirable to split a long command line into multiple
lines. In that case, the shell issues a secondary prompt, usually >, to indicate to you that the
command line isn't complete. This is easily shown with the echo command:
▪ For example, when using the "cd" command, no process is created. The current
directory sim- ply gets changed on executing it.
External Command:
External commands are not built into the shell. These are executable present in a separate file.
When an external command has to be executed, a new process has to be spawned and the
command gets exe- cuted.
▪ For example, when you execute the "cat" command, which usually is at /usr/bin, the
executable /usr/bin/cat gets executed.
type command:
$ type cd
cd is a shell builtin
$ type cat
cat is /bin/cat
For the internal commands, the type command will clearly say its shell built-in, however for
the external commands, it gives the path of the command from where it is executed.
THE TYPE COMMAND: knowing the type of a command and locating it.
type - Display information about command type.
The type command is a shell built-in that displays the kind of command the shell will
execute, given a particular command name. It works like this: type command where
“command” is the name of the command you want to examine. Here are some examples:
$type type
Output: type is a shell built-in
$type ls
Output: ls is aliased to ‘ls –color=tty’
$type cp
Output: cp is /bin/cp
Here we see the results for three different commands. Notice that the one for ls (taken from a
Fedora system) and how the ls command is actually an alias for the ls command with the “--
color=tty” option added. Now we know why the output from ls is displayed in color!
Root: the system administrator’s login. The unix system provides a special login name for the
exclusive use of the administrator, it is called root. This account doesn’t need to be separately
created but comes with every system. Its password is generally set at the time of installation
of the system and has to be used on logging in
Login: root
Password: ******
The prompt of the root is # other users (non privileged user) either $ or %
Once you login as root, you are place in root’s home directory. Depending on the system, this
could be / or /root
Any user can acquire super user status with the su command if she knows the root password.
Example, the user GMIT becomes a super user in this way
$su
Password: ******
#pwd
/home/GMIT
Though the current directory does not change the # prompt indicates the GMIT now has
powers of a super user. To be in root’s home directory on superuser login, use su –l.
Creating a user’s Environment: users often rush to the administrator with the complaint
that a pro- gram has stopped running. The administrator first tries running it in a simulated
environment.su, when used with a -, recreates the user’s environment without taking the login
password route:
$su – GMIT
This sequence executes GMIT’s. profile and temporarily creates GMIT’s environment. Su
runs a separate sub shell, so this mode is terminated by hitting [Ctrl-d] or using exit.
UNIX FILES
UNIX system has thousands of files. If you write a program, you add one more file to the system. When
you compile it you add some more. Files grow rapidly, and if they are not organized properly, you will
find it difficult to locate them. So UNIX has a file system (UFS) to manage or organizes its own files in
directory.
b. Directory files
i. Contains no data, but keeps some details of the files and subdirectories that it contains.
ii. A directory file contains an entry for every file and sub directory that it houses. Each entry has
two components
• The filename
• A unique Identification number for the file or directory( called the inode number)
iii. A directory contains the filename but not the contents of file.
iv. When you create or remove a file the kernel automatically updates its corresponding
directory by adding or removing the enter i.e inode number associated with that file.
c. Device files
i. Used to represent a real physical device such as a printer, tape drive or terminal, used for
Input/Ouput (I/O) operations
ii. Unix considers any device attached to the system to be a file - including your terminal:
iii. By default, a command treats your terminal as the standard input file (stdin) from which to
read its input
iv. Your terminal is also treated as the standard output file (stdout) to which a command's output
is sent.
All files in UNIX are related to one another. The file system in unix is a collection of all ordinary,
directory and device files and organized in a hierarchical structure as shown in below fig.
The implicit feature of every UNIX file system is that there is a top which serves as reference point for
all files.This top is called root & is represented by a /(front slash). Root is actually a directory. The root
directory has a number of sub directories under it. These sub directories in turn have more sub directories
and others files under them.
For instance bin and usr are two directories directly under root, while a second bin and kumar are sub
directories under usr.
Every file apart from root must have a parent. Thus the home directory is the parent of kumar , while /
is the parent of home and grandparent of kumar. If you create a file login.sql under the kumar directory
,then kumar will be the parent of this file.
The first group contains the files that are made available during system installation
• /bin and /usr/bin: these are the directories where all the commonly used UNIX commands are
found.
• /sbin and /usr/sbin: If there’s a command that you can’t execute but the system administrator
can execute, it would be probably in one of these directories.
• /etc: this directory contains the configuration files of the system. You can change a very important
aspect of system functioning by editing a text file in this directory. Your login name and password
are stored in files /etc/passwd and etc/shadow
• /dev: This directory contains all device files. These files don’t occupy space on disk.there could
be more sub directories like pts, dsk and rdsk in this directory
• /lib and /usr/lib: Contains shared library files and sometimes other kernel-related files.
• /usr and /include: contains the standard header files used by C programs. The statement
#include<stdio.h> used in most C programs referes to the file stdio.h in this directory.
• /usr/share/man: this is where the man pages are stored. There are separate subdirectories
here(like man1,man2 etc) that contains the pages for each section. For instance, the man page of
ls can be found in /usr/share/man/man1
User also work with their own files, they write programs, send and receive mail and also create temporary
files. These files are available in the second group shown below
• /tmp: the directory where users are allowed to create temporary files. These files are wiped
away regularly by the system
• /var: The variable part of the file system. Contains all your print jobs and your outgoing and
incoming mail.
• /home:On many systems users are housed here.Kumar would have his home directory in
/home/kumar
HOME DIRECTORY : When log on to the system, UNIX automatically places you in a directory
called the home directory.
• It is created by the system when user account is opened.
• If you log in using the login name sharma , you will land up in a directory that could have the
pathname
/home/sharma
• The shell variable HOME known’s yours home directory
$echo $HOME
/home/sharma
You will be doing much of your work in your home directory and subdirectories.
• Home variable: it is also called environment variables. Environment variables are a set of
dynamic named values that can affect the way running processes will behave on a computer.
• Here $HOME is a environment variable it indicates the home directory of the current user: the
default argument for the cd built-in command.
PATH VARIABLE:
• The PATH environment variable is a colon-delimited list of directories that your shell searches
through when you enter a command.
• Program files (executables) are kept in many different places on the Unix system. Your path
tells the Unix shell where to look on the system when you request a particular program.
• To find out what your path is, at the Unix shell prompt echo $PATH
• Your path will look something like the following.
/usr2/username/bin:/usr/local/bin:/usr/bin:.
You will see your username in place of username. Using the above example path, if you enter the ls
command, your shell will look for the appropriate executable file in the following order: first, it would
look through the directory /usr2/username/bin, then /usr/local/bin, then /usr/bin, and finally the local
directory, indicated by the. (a period).
Command Function
cd Returns you to your login directory
cd ~ Also returns you to your login directory
cd / Takes you to the entire system’s root directory
cd /root Takes you to the home directory of the root or superuser,account
created at installation, you must be root user to access this directory.
cd /home Takes you to the home directory where user login directories are
usually stored
cd .. Moves you up one directory
cd ~otheruser Takes you to the otheruser’s login directory
cd /dir/subdirfoo Regardless of which directory you are in, the absolute path takes you
directly to subdirfoo, a subdirectory of dir.
Ex .1: Assume the current directory is /home/kumar/progs/data/text, using cd .. will move one level up
$pwd
/home/kumar/progs/data/text
$ cd ..
$pwd
/home/kumar/progs/data
ABSOLUTE PATHNAMES:
• If the first character of a pathname is / the files location must be determined with respect to root(/)
. Such a pathname is called absolute pathname.
cat /home/kumar
• When you have more than one / in a pathname for such / you have to descend one level in the file
system. Thus Kumar is one level below home and two levels below root.
• When you specify a file y using frontslashes to demarcate the various levels,you have a
mechanism of identifying a file uniquely.No two files in a UNIX system can have same absolute
pathnames.
• When you specify the date command, the system has to locate the file date from a list of
directories specified in the PATH variable and then execute it.
• However if you know the location of a command in prior, for example date is usually located in
/bin or /usr/bin . Use absolute pathname i,e precede its name with complete path
$/bin/date
For example if you need to execute program less residing in /usr/local/bin you need to enter
the absolute pathname
$/usr/local/bin/less
Basic syntax
$pwd [option]
Options Description
-L (logical) Use PWD from environment, even if it contains symbolic links
-P (physical) Avoid all symbolic links
–help Display this help and exit
–version Output version information and exit
If both ‘-L‘ and ‘-P‘ options are used, option ‘L‘ is taken into priority. If no option is specified at the
prompt, pwd will avoid all symlinks, i.e., take option ‘-P‘ into account.
Ex 2: When cd used without arguments: cd when used without arguments reverts to home directory
$pwd
/home/kumar/gmit
$cd
cd without argument will change directory from gmit to its home directory Kumar
$pwd
/home/kumar
Ex 3: If your present working directory is /home/Kumar and you need to switch to /bin directory
directly, use absolute pathname i.e /bin wd cd command
$pwd
/home/kumar
$cd /bin
$pwd
/bin
Department of CSE, ATMECE, Mysuru 32
Ex 2: To create three directories at a time, named patch, dbs, doc, pass directory names as argu- ments.
$mkdir patch dbs doc
• You cant remove a sub directory unless you are place in a directory which is hierarchically above
the one you have chosen to remove.
$ls
Output:08_packets.html
calendar
dept.lst
emp.lst
helpdir
uskdsk06
ls options:
• Output in multiple columns(-x):
$ls -x
08_packets.html calendar dept.lst emp.lst
helpdir progs usdsk07 usdsk07
If we specify two directories named helpdir and progs , the contents of the directory i,e filenames are
listed out.
• Recursive listing(-R)
The recursive option lists all sub-directories and files in a directory tree structure.
$ls -xR
08_packets.html calendar cptodos.sh dept.lst
emp.lst helpdir progs usdsk07
./helpdir
forms.hlp graphics.hlp
./progs
arrays.pl n2words.pl
cp chap01 unit1
if destination file i.e unit1 does not exist, first it will be created before copying.if not it will
be simply overwritten without any warning.
• Copying a file to another directory
ex: assume there is a file named chap01 and it has to be copied to progs directory
cp chap01 progs
output: chap01 is now copied to directory named progs with the same name chap01.
• Copying a file to another directory with different name
ex: assume there is a file named chap01 and it has to be copied to progs directory with chap01
file renamed as unit1
cp chap01 progs/unit1
output: chap01 is now copied to directory named progs with the same name unit1
cp options:
Interactive copying (-i): the -i option warns the user before overwriting the destination file.
Ex: $ cp -i chap01 unit1
cp: overwrite unit1(yes/no)? y
A y at this prompt will overwrite the file.
Copying directory structure(-R) : the -R command behaves recursively to copy an entire directory
structure say progs to newprogs.
Ex say progs directory contains three files kernel, bash, korn. To copy all three files under progs to
newprogs directory
$ cp -R progs newprogs
rm : deleting files
The rm command deletes one or more files.
Ex 1: The following command deletes three files chap01, chap02, chap03.
$ rm chap01 chap02 chap03
Ex 2: to delete files named chap01 and chap02 under progs directory
$ rm progs/chap01 progs/chap02
Ex 3: to remove all file
$ rm*
Recursive deletion(-r or -R) deletes all subdirectories and files recursively. Rm wont normally
remove directories but when used with -r or -R option it will.
$ rm -r *
Forcing removal: rm prompts for removal, if a file is write protected. The -f option overrides this minor
protection and forces removal.
$ rm -rf * /*(deletes everything in the current directory and below)
To rename a directory:
$ mv pts perdir
pts directory is renamed as perdir
$ od -b odfile
The -b option displays the octal values for each character.
000000 127 150 151 164 145 040 163 160 141 143 145 040 151 156 143 154
000000 165 144 145 163 040 141 040 011 012 124 150 145 040 007 040 143
Each line displays 16 bytes of data in octal , preceded by the offset in the file of the first byte in the
line.
The octal equivalent of characters are displayed ex for W- 127, i-151, \t (tab)-011, \n(newline)-012
^G(Bell character)- 007
MODULE 2
FILE ATTRIBUTES AND PERMISSIONS
The ls command with options
ls –l: LISTING FILE ATTRIBUTES
ls command is used to obtain a list of all filenames in the current directory. The output in UNIX lingo is
often referred to as the listing. Sometimes we combine this option with other options for displaying other
attributes, or ordering the list in a different sequence. ls look up the file’s inode to fetch its attributes. It
lists seven attributes of all files in the current directory and they are:
File type and Permissions
The file type and its permissions: The first column shows the type and permissions associated with each
file.The first character in this column is mostly a – which indicates that the file is an ordinary one. In
unix, file system has three types of permissions- read, write and execute.
Links: The second column indicates the number of links associated with the file. This is actually the
number of filenames maintained by the system of that file.
Ownership: The third column shows the owner of files. The owner has full authority to tamper with files
content and permissions. Similarly, you can create, modify or remove files in a directory if you are the
owner of the directory.
Group ownership: The fourth column represents the group owner of the file. When opening a user
account, the system admin also assigns the user to some group. The concept of a group of users also
owning a file has acquired importance today as group members often need to work on the same file.
File size: The fifth column shows the size of the file in bytes. The important thing to remember here is
that it only a character count of the file and not a measure of the disk space that it occupies.
Last modification time: The sixth, seventh and eighth columns indicate the last modification time of the
file, which is stored to the nearest second. A file is said to be modified only if its content have changed
in any way.If the file is less than a year old since its last modification time, the year won’t be displayed.
Filename: The last column displays the filename arranged in ASCII collating sequence.
For example, $ ls –l
total 72
-rw-r--r-- 1 kumar metal 19514 may 10 13:45 chap01
-rw-r--r-- 1 kumar metal 4174 may 10 15:01 chap02
-rw-rw-rw- 1 kumar metal 84 feb 12 12:30 dept.lst
-rw-r--r-- 1 kumar metal 9156 mar 12 1999 genie.sh
drwxr-xr-x 2 kumar metal 512 may 09 10:31 helpdir
drwxr-xr-x 2 kumar metal 512 may 09 09:57 progs
FILE PERMISSIONS
• UNIX has a simple and well defined system of assigning permissions to files.
• Lets issue the ls –l command once again to view the permissions of a few lines .
$ls -l chap02 dept.lst dateval.sh
-rwxr-xr-- 1 kumar metal 25000 May 10 19:21 chap02
-rwxr-xr-x 1 kumar metal 890 Jan 10 23:17 dept.lst
-rw-rw-rw- 1 kumar metal 84 Feb 18 12:20 dateval.sh
Third group(r--):
• has the write and execute bits absent.
• This set is applicable to others i,e those who are neither the owner nor group.
• This category is referred to as the world.
RELATIVE PERMISSIONS
chmod only changes the permissions specified in the command line and leaves the other permissions
unchanged.
Its syntax is:
chmod category operation permission filename(s)
chmod takes an expression as its argument which contains:
user category (user, group, others)
operation to be performed (assign or remove a permission)
type of permission (read, write, execute)
Ex 1:
$ls –l xstart
-rw-r--r-- 1 kumar metal 1906 sep 23:38 xstart
Here user is having the only read and execute permission .
Using relative file permission need to add the execute permission to user
chmod category operation(+,-) permission filename.
$chmod u + x xstart
$chmod u+x xstart
$ ls –l xstart
-rwxr--r-- 1 kumar metal 1906 sep 23:38 xstart
After executing the chmod command, the command assigns (+) execute (x) permission to the user (u),
other permissions remain unchanged.
Ex 2: To remove execute permission from all and assign read permission to group and others
$chmod a-x, go+r xstart /*to remove execute permission from all(a)ie user, group, others
/*to assign read permission to group and others (go+r)
ABSOLUTE PERMISSIONS
A string of three octal digits is used as an expression. The permission can be represented by one octal
digit for each category. For each category, we add octal digits. If we represent the permissions of each
category by one octal digit, this is how the permission can be represented:
• Read permission – 4 (octal 100)
• Write permission – 2 (octal 010)
• Execute permission – 1 (octal 001)
We have three categories and three permissions for each category, so three octal digits can describe a
file’s permissions completely. The most significant digit represents user and the least one represents
others. chmod can use this three-digit string as the expression.
Ex 2:
To assign read and write for user and remove write, execute permissions from group and others
• Here to assign rw- corresponds to digit 6
• Remove write , execute permissions is nothing but assigning only read option to group and oth-
ers
• Only read permission is r—corresponds to 4
$chmod 644 xstart
Ex 3:
To assign all permissions to the owner, read and write to group and only execute for others.
$chmod 761 xstart
Ex 4
To assign all permissions to all categories.
$chmod 777 xstart
----------
This is simply useless but still the user can delete this file
On the other hand,
chmod a+rwx xstart or
chmod 777 xstart
After Executing either of the one command it adds the all permission to all categories as shown
below
-rwxrwxrwx
The UNIX system by default, never allows this situation as you can never have a secure system. Hence,
directory permissions also play a very vital role here.
This makes all the files and subdirectories found in the shell_scripts directory, executable by all users.
DIRECTORY PERMISSIONS
• Directories also have their own permissions and the significance of these permissions differ
from those of ordinary files.
• The default permissions of a directory are,
rwxr-xr-x (755)
• A directory must never be writable by group and others
If a directory has write permission for group and others also, be assured that every user can remove every
file in the directory. As a rule, you must not make directories universally writable unless you have definite
reasons to do so.
The group owner of the file dept.lst is changed from metal to dba by issuing the command
$chgrp dba dept.lst
2. After a command is entered, the shell scans the command line for metacharacters and expands
abbreviations (like * in rm *) to recereate the simplified command line.
3. It then passes on the command line to the kernel for execution
4. The shell waits for the command to complete and normallty cant do any work while the command is
running.
5. After command execution is complete, the prompt reappers and shell returns to its waiting role to
start the next cycle. Yo can now enter the next command.
Department of CSE, ATMECE, Mysuru 49
WILD CARDS
The metacharacters taht are used to construct the generalized pattern for matching filenames belong to
the category called wild cards.
The * and ?
The metacharacter * is one of the characters of the shell wild card set. It matches any
number of characters including none.
For example : to match filenames chap chap01 chap02 chap03 chap04
$ ls chap*
Output : // the * matches all strings along with none
chap
chap01
chap02
chap03
chap04
chap02
chap03
chap04
To match filename with 3 character that does not begin with an Upper Case letter
$ ls [!A-Z]??
ESCAPING
Placing a \ immediately before a metacharacter turns off its special meaning.
For instance \* , matches * itself. Its special meaning of matching zero or more occurrencesof
character is turned off.
Ex 1:
$rm chap*
removes all the filenames starting with chap. Chap, chap01,chap02 and chap03 are removed.
Ex 2:
If there are files with names chap01, chap02,chap03.
To list the filenames starting with chap0
$ ls chap0[1-3]
Output:
chap01
chap02
chap03
Ex 3:
To match the file named as chap0[1-3]
$ ls chap0\[1-3\]
Output:
chap0[1-3]
Escaping the space: To remove the file My document.doc, which has space embedded,
$rm My\ document.doc
QUOTING:
This is the another way of turning off the meaning of metacharacter.
When a command argument is enclosed within quotes, the meaning of all enclosed special
characters are turned off
$rm 'chap*' // * metacharacter meaning is turned off
removes the filename with chap* /*name of the file itself is chap*.
STANDARD INPUT
This file is indeed special
The keyboard, the default source
a file using redirection with the < symbol
another program using the pipeline
• The input redirection operator is less than character (<).
• When you use wc without an argument , it prompts you to provide the input from standard input
keyboard
$ wc
Unix is a multiuser multitasking OS
[ctrl-d]
• When wc is used with argument. Filename is passed as an argument i,e wc takes the input from
the filename we have specified
$ wc < sample.txt /*wc command takes input from the file sample.txt
output
1 6 36 /* count of characters, words, lines of file sample.txt
STANDARD OUTPUT
• All commands displaying the output on the terminal actually write to the standard output file as
a stream of characters and not directly to the terminal as such.
• There are three possible destinations of this stream
The terminal, the default destination
A file using the redirection symbol > and >>
As input to another program using a pipeline
• There are two basic redirection operators for standard output.
1 6 37
STANDARD ERROR
When you enter an incorrect command or try to open a non existent file, certain diagnostic messages
show up on the screen. This is the standard error stream whose default destination is the terminal.
$cat filelist
-rwxr—r-- 1 gilberg staff 1234 oct file1
Cannot access file2: no such file or directory
Department of CSE, ATMECE, Mysuru 55
$cat stdout
-rwxr—r-- 1 gilberg staff 1234 oct file1
$cat stderr
Cannot access file2: no such file or directory
$who | wc -l
Output: 5 /* count of number of lines of who command
Here the output of who command has been passed directly as the input to wc command and
who is said to be piped to wc.
The output of who will be displayed on the monitor and at the same time it will be saved in a file
user.txt
COMMAND SUBSTITUTION
When a shell executes a command, the output is directed to standard output. Most of the time
the standard output is associated with the monitor.
There are times, however such as when we write complex commands or scripts that we need to
change the output to a string that we can store in another string or variable.
Command substitution provides the capability to convert the result of a command to a string.
The command substitution operator that converts the output of a command to a string is a
dollar sign and a set of parentheses .
To invoke the command substitution, we enclose the command in a set of parentheses preceded
by dollar sign($)
When we use this command substitution, the command is executed and output is created and
then converted to string of characters.
Ex: simple demonstration of command substitution.
$ echo “ The date and time are:date“
Output:
The date and time are :date
Using command substitution with date- using dollar sign along with the command enclosed in
parenthesis
$echo “The date and time are: $(date)“
Department of CSE, ATMECE, Mysuru 57
Output
The date and time are : Mon Oct 3 07:09:48 GMT 2016
The grep,egrep
$cat emp1.lst
$cat emp2.lst
Ex 2: To search the pattern director from 2 files i,e emp.lst and emp2.lst
Ignoring case: when you look for a name but are not sure of the case, use the –i option to
ignore case for pattern matching.
Deleting lines(-v): The –v option selects all lines except those containing the pattern
The lines containing the pattern director are deleted in the output.
Dislaying filenames(-l)
The –l option displays only the names of the files containing the pattern.
Here the pattern manager is searched in all files ending with .lst (*.lst)
Negating a class(^)
Regular expressions use the caret(^) to negate the character class, while the shell uses bang(!)
Ex:
[^a-zA-Z] matches a non-alphabetic character
The *(asterisk)
The * refers to the immediately preceding character.Here it indicates that the previous character can
occur many times or not at all.
The patttern g*
Matches none, g, gg, gg, ggg,.....
The dot ( .)
A . matches a single character where as the shell uses ? to indicate that.
Here the . matches single character. It list all files beginning with 10 followed by single character.
It displays lines with id 101,102,103,104,105.
7.1.1 Specifying pattern locations(^ and $)
The two regular expressions characters that match pattern at the beginning or end of line .
^(caret) Matching at the beginning of the line
$(dollar) Matching at the end of the line
5...$ matches all the lines ending with four digit number beginning with 5.
The + and ?
+ Matches one or more occurrences of the previous character
? Matches zero or one occurrences of the previous character.
The + symbol matches one or more occurrences of character c i,e c, cc, ccc
The occurrences of a character is not matched by +c , since there is no c in the occurrences.
The characters ( and ) lets you group patterns and use of | inside the parenthesis, you can frame
more compact pattern
SHELL PROGRAMMING
DEFINING VARIABLES:
• Variables are defined as follows –
• variable_name=variable_value
For example:
NAME="Sumitabha Das"
ACCESSING VARIABLES:
• To access the value stored in a variable, prefix its name with the dollar sign ( $) –
• For example, following script would access the value of defined variable NAME and would
print it on STDOUT –
#!/bin/sh
NAME=“Sumitabha Das“
echo $NAME
ENVIRONMENT VARIABLES –
An environment variable is a variable that is available to any child process of the shell. Some programs
need environment variables in order to function correctly. Usually a shell script defines only those
environment variables that are needed by the programs that it runs.
SHELL: points to the shell defined as default.
DISPLAY : Contains the idenifier for the display that X11 programs should use by default.
HOME: Indicates the home directory of the current user; the default argument for the cd built in
command
IFS: Indicates the Internal Field Separator that is used by the parser for word splitting after expansion.
PATH : Indicates search path for commands.It is a colon separated list of directories in which the shell
looks for commands.
PWD: Indicates the current working directory as set by the cd command.
RANDOM: Generates a random integer between 0 and 32767 each time it is referenced.
SHLVL: Increments by one each time an instance of bash is created.
UID: Expands to the numeric user ID of the current user initialized at shell prompt.
• Following is the sample example showing few environment variables −
$ echo $HOME
/root
]$ echo $DISPLAY
$ echo $TERM
xterm
$ echo $PATH
/usr/local/bin:/bin:/usr/bin:/home/amrood/bin:/usr/local/bin
$
SHELL SCRIPTS
When a group of commands have to be executed regularly they should be stored in a file and the file
itself executed as a shell script or shell program.
output:
$sh script.sh
Todays date : Mon Nov 7 10:03:42 IST 2016
Department of CSE, ATMECE, Mysuru 68
My shell: /bin/sh
• A single read statement can be used with one or more variables to let you enter multiple
arguments.
read pname flname
• The script asks for a pattern to be entered. Input the string director, which is assigned to the
variable pname. Next the script asks for the filename enter the string emp.lst which is assigned
to the variable flname.
• grep runs with these two variables as arguments
#!/bin/sh
#emp1.sh
#
echo "Enter the pattern to be searched : \c"
read pname
echo " Enter the file to be used : \c"
read flname
echo " Searching for $pname from file $flname"
grep "$pname" $flname
echo "Selected rows shown above"
Output:
$sh emp1.sh
Enter the pattern to be searched:director
Enter the file to be used: emp.lst
Searching for director from file emp.lst
101 sharma|director|production|12/03/70|7000
102|barun|director|marketing|11/06/67|7800
selected rows shown above
grep "$1" $2
echo "\n job over"
Output:
$ sh emp2.sh director emp.lst
Program: emp2.sh
The number of arguments specified is:2
The arguments are director emp.lst
101| sharma|director|production|12/03/70|7000
102|barun|director|marketing|11/06/67|7800
job over
Shell Significance
Parameter
$1, $2... Positional parameters representing command line arguments
$# Number of arguments specified in command line
$0 Name of executed command
$* Complete set of positional parameters as a single string
"$@" Each quoted string treated as separate argument
$? Exit status of last command
$$ PID of the current shell
$! PID of the last background job
Its through the exit command or function that every command returns an exit status to the caller.
Further a command is said to return true exit status if it executes successfully and false if its fails.
• THE PARAMETER $? :It stores the exit status of the last command. It has the value 0 if the
command succeeds and a non zero value if it fails. This parameter is set by exit’s argument If no
exit status is specified then $? is set to zero(true).
• Consider two files file1 which exist in current directory and file2 which does not exist
$ ls –l file1; echo $? /*file1 attributes are listed
Output :0 /*exit status $?=0, since cmd executed successfully
CONDITIONAL STATEMENTS:
The if CONDITIONAL
If command is successful If command is successful If command is successful
then then then
execute commands execute commands execute commands
else fi elif command is successful
execute commands then ..
fi else ..
fi
The if statement makes two way decision making depending on the fulfillment of a certain condition.
#!/bin/sh
a=10
b=20
if [ $a==$b ]
then
echo “a is equal to b”
elif [ $a –gt $b ]
then
echo “ a is greater than b”
elif [ $a -lt $b ]
then
echo “ a is lesser than b”
else
echo “ None of the conditions met”
fi
output:
a is lesser than b
read choice
case “$choice” in
1 ) ls –l ;;
2 ) ps –f ;;
3 ) date ;;
4 ) who ;;
5 ) exit ;;
* ) echo “invalid option”
esac
3. Todays date
4.Users of system
5. Quit
Enter your option : 3
Sun Nov 6 18:03:06 IST 2016
Matching multiple patterns:
• case can also specify same action for more than one pattern.
• For example the expression y|Y can be used to match y in both upper and lower case letters.
NUMERIC COMPARISION:
The numeric comparison operators used by test are
Operator Meaning
-eq Equal to
-ne Not equal to
-gt Greater than
-ge Greater than or equal to
-lt Less than
-le Less than or equal to
Numeric comparison in the shell is confined to integer values only , decimal values are simply
truncated.
$ x=5; y=7; z=7.2
$ test $x –eq $y ; echo $?
Output : 1
STRING COMPARISION
test can be used to compare strings with yet another set of operators.
Test True if
s1=s2 String s1=s2
s1!=s2 String s1 is not equal to s2
-n stg String stg is not a null string
-z stg String stg is null string
Stg String stg is assigned and not null
s1==s2 String s1= s2(Korn and bash only)
Example:
#!/bin/sh
a=”abc”
b=”efg”
if [ $a = $b ]
then
echo “a is equal to b”
else
echo “a is not equal to b”
fi
output:
a is not equal to b
FILE TESTS
test can be used to test the various file attributes like its type(file, directory or symbolic link) or its
permissions(read,write,execute)
$ls –l emp.lst
-rw-rw-rw- 1 kumar group 870 Sep 8 15:52 emp.lst
Ex:
$for file in chap20 chap21 chap22
do
cp $file {$file}.bak
echo $file copied to $file.bak
done
Output:
chap20 copied to chap20.bak
chap21 copied to chap21.bak
chap22 copied to chap22.bak
Ex:
$echo “\$1 is $1, \$2 is $2, \$3 is $3”
Output: $1 is 989, $2 is 878, $3 is 779
$echo “The $# arguments are $*”
Output: The 3 arguments are 989 878 779
$shift 2
$echo $1 $2 $3
Output: 09:04:30 IST 2016
• If the message is short you can have both the command and message in the same script.
mail sharma << MARK
Your program for printing the invoices has been executed
on `date`. The updated file is $flname
MARK
• The here document symbol(<<) followed by three lines of data and a delimiter (the string
MARK)
• The shell treats every line following the command and delimited by MARK as input to the
command.
• Sharma at the other end will see the three lines of message text with the date inserted by
command substitution and the evaluated filename.
trap ‘rm $$* ; eho “Program Interrupted” ; exit ’ HUP INT TERM
trap ‘cmd_list’ sig_list
• trap is a signal handler.
• Here it first removes all the files expanded from $$*, echoes a message and finally terminates the
script when the signals SIGHUP(1), SIGINT(2), SIGTERM(15) are sent to the shell process
running the script.
• When the interrupt key is pressed it sends the signal number 2.