File System BASIC DEFINITIONS A file is a collection of information stored on a computer system.

A directory is a special file that contains the names of other files and/or subdirectories. Files can be organized efficiently with the use of directories. A directory can be used to group related files and subdirectories together. A subdirectory is a directory within a directory. FILE ORGANIZATION Files can be organized into a hierarchy of directories. A typical UNIX file organization structure resembles an upside-down tree.

In this inverted-tree structure, any file can be accessed from anywhere in the tree. DEFINITIONS The home directory is a directory assigned to a user. Usually, all files created by the user are within this directory. The current working directory (.) is the directory in which you are currently located. After you login, your current working directory is your home directory. The parent directory (..) of a file is the directory which directly encloses the file. For example: charles is the parent directory of .login, history, calculator, and calculator.c The root directory (/) is the directory that encloses ALL files and directories; therefore, it has no parent directory.

PATHNAMES Because of this hierarchy of directories, it is sometimes necessary to specify the path to a file or directory. Example: /home/1/charles/history/week1.notes and history/week1.notes are both parameters to the week1.notes file Note that / is used to separate directory names. Absolute Pathname: A pathname that starts with / (the root). Since the path always starts at the root, it is correct regardless of what the current working directory is. Relative Pathname: A path that starts from the current working directory. Any pathname that does not start with / is taken to be a relative pathname. Example: If the current working directory is /home/1/charles then the absolute and relative pathnames to week1.notes respectively are /home/1/charles/history/week1.notes and history/week1.notes

cal - displays a calendar

cal [[ month ] year ] Cal displays a simple calendar. If arguments are not specified, the current month is displayed. A single parameter specifies the year (1 - 9999) to be displayed; note the year must be fully specified: ``cal 89 '' will not display a calendar for 1989. Two parameters denote the month (1 - 12) and year. If no parameters are specified, the current month's calendar is displayed.
Ex: Code:
# cal 3 2001 ______March 2001 Su Mo Tu We Th Fr ____________01 02 04 05 06 07 08 09 11 12 13 14 15 16 18 19 20 21 22 23 25 26 27 28 29 30

Sa 03 10 17 24 31

date - print or set the system date and time date Display the current time . Ex: $ date Sun oct 16 16:15:21 IST 2011 $ date +%m 08

$date +%h Aug $date +%h %m Aug 08 Options: d- The day of the month (1 to 31) y- The last two digits of the year. H,M and S The hour, minute and seconds respectively. D- The date in format mm/dd/yy T- The time in the format hh:mm:ss

who - show who is logged on who options H, --heading print line of column headings -u, --users list users logged in ex: $ who root kumar $ who am i kumar pts/10 Aug 1 08:56 (pc123.datastage.com) console sep pts/10 Aug 1 07:10 1 08:56 (:0) (pc123.datastage.com)

passwd Allows the user to change your account password. Your password should be at least four characters in length, up to a maximum of eight. It is also desirable to include numbers and mixed case characters within the password for additional security. Develop a habit of changing your password on a regular basis

cd path The change directory command moves the user from the current working directory to the directory specified. If path is defined as a simple name such as docs, then docs is located directly below the current directory and the user is moved into it. The command cd, by itself on a line, returns the user to the login directory. Examples of uses of cd are

cd docs : Move down to the directory named docs. cd .. : Move up to the parent directory of the current directory. cd ../data : Move up to the parent directory of the current directory, then down to the directory named data. cd /usr/local/bin: Move to the explicit directory location.

mkdir directory This command is used to create a new directory within the current directory.

pwd The pwd command (print working directory) allows you to determine the path name of the directory in which you are presently working.

rmdir directory The rmdir (remove directory) command is used to delete a directory. You can only delete an empty directory.

cp file1 file2

The cp (copy) command takes the contents of one file ( file1 ) and duplicates it to another file (file2 ). cp file1 file2 file3 progs cp file* progs cp i file1 file2 cp R progs newprogs this command behaves recursively to copy an entire directory structure progs to newprogs. The copying depends on the existence of the destination directory. If newprogs exists progs becomes a subdirectory under newprogs. If newprogs doesnt exist, cp creates it along with the associated subdirectories. Destination should be a folder. Copies all the files starts with file to progs

mv file1 file2 Move (mv) renames and moves files. If file2 already exists, it is deleted before file1 is renamed mv file1 file2 file3 progs The last one should be directory.

mv progs progs1 It depends on the destination directory exists or not.

rm filename The rm (remove) command is used to delete a file. rm file 1 file2 file 3 rm progs/file4 progs/file4 rm * rm i file 1 file2 file3

i is for interaction.

rm r * rm rf *

very dangerous and removes all the files in subdirectories also. very very dangerous because all the files will gone though we dont

have permissions to do that.

ls<CR> Lists the contents of the current directory. ls has many parameters.

ls -a Lists all files, including invisible files (files with a leading dot (.)). ls A --almost-all do not list implied . and .. ls -F Places a slash (/) after directory files and an asterisk (*) after executable files, and places an at-sign (@) after symbolic links.

ls -R Recursively lists all files, including those in sub-directories from the current sub-directory to all sub-directories below. ls -r, --reverse reverse order while sorting

ls -d, --directory list directory entries instead of contents ls -l Lists all visible files and some attributes. $ ls l -rw-r- -r- 1 kumar kumar dstage 1954 may 11 13:45 file1 dstage 512 june 15 15:45 progs1

drwxr-xr-x 2

chmod Allows the owner of a file or directory to change the access permissions. Use of pneumonics is preferred over the octal numbers as pneumonics are additive and subtractive, octal numbers are absolute and may change something that you didn't want to.

chmod go-rwx filename chmod 600 filename Makes a file private. chmod a+r filename chmod 644 filename Makes a file public in read mode. chmod go-rwx,u-wx+r filename chmod 400 filename Makes a file private and protects it from accidental change/deletion.

chmod u+x filename chmod 700 filename Makes a file executable by the owner. This file can then be executed by simply entering its name. chmod a+rx dirname chmod 755 dirname Makes a directory public in read mode. Note:

Directories must be "executable" in order to be searched.

Cat: It is used to display the contents of a file on the terminal. $cat dept.lst 01 | accounts | 6213 02 | progs | 5423

03 | marketing | 6521 04 | personnel | 2365 05 | sales $cat file1 file2 The contents of the second file are shown immediately after the first file without any header information. $cat n file1 The n option numbers lines. |1006

We can also use cat to create a file. $cat > foo This is the first which is been created so far. [ctrl-d] $_ This means we entered some text into the file. If we do $cat foo This is the first which is been created so far. Cat is used for both creating a file as well as to enter some input into file

wc : Counting lines, Words and Characters $ cat infile I am the wc command I count characters, words and lines With options I can also make a selective count $wc infile 3 20 103

A line is any group of characters not containing a new line. A word is a group of characters not containing a space, tab or newline. A Character is the smallest unit of information, and includes a space, tab and newline $ wc l infile 3 infile $ wc w infile 20 infile $ wc c infile 103 infile

$ wc file1 file2 file3 5 8 7 16 24 28 45 75 86 file1 file2 file3

cmp: Comparing two files

$ cmp file1 file2 file1 file2 differ: char 9, line1

The two files are compared byte by byte and the location of the first mismatch will be displayed on the screen. By default, cmp doesnt bother about the possible subsequent mismatches.

If the two files are identical then it wont displays any message, but simply returns the prompt.

Comm.: What is Common? It requires two sorted files and lists the differing entries in different columns. $cat file1 c.k. Shukla chanchal singhvi s.n. dasgupta sumit chakrobarty

$ cat file2 anil aggarwal barun sengupta c.k. shukla lalit chowdury s.n. dasgupta

Both files are sorted and have some differences. When you run the comm, it displays a three-columnar output.

$ comm. file1 file2 anil aggarwal barun sengupta c.k. shukla chanchal singhvi lalit chowdury s.n.dasgupta sumit chakrobarty

diff: Converting one file to another file

diff is the third command that can be used to display file differences. Unlike its fellow members cmp and comm, it also tells you which lines in file one have to be changed to make the two files identical. $ diff file1 file2 0a1,2 > anil aggarwal > barun sengupta 2c4 < chanchal singhvi ->lalit chowdury 4d5 < sumit chakrabarty Diff uses certain symbols and instructions to indicate the changes that are required to make two file identical.

head: Displaying the beginning of the file $ head n 3 dept.lst 01 | accounts | 6213 02 | progs | 5423

03 | marketing | 6521 It displays the first 3 lines of the file dept.lst

If we give only $ head emp.lst It displays first 10 lines of a file by default.

tail: Displaying the end of a file $ tail n 3 dept.lst 03 | marketing | 6521 04 | personnel | 2365 05 | sales |1006

It displays the last 3 lines of the file dept.lst

If we give only $tail dept.lst By default, it gives last 10 lines of the file.

cut : Slitting a file vertically $ cut c 5-10,13-16 dept.lst accounts 6213 progs 5423

marketing 6521 personnel 2365 sales 1006

If I use cut c -3,6-9,11- dept.lst then, It displays the first 3 columns and 6 to 9 columns an then 11th column to till last.

cut d | f 2,3 dept.lst accounts | 6213 progs | 5423

marketing | 6521 personnel | 2365 sales | 1006

If we want to know all the users who logged in then, $ who | cut d f1

Paste: Pasting Files $ paste emp1.lst emp2.lst It means displays the files side by side If we want to display the files side by side but with a delimiter then we need to specify the delimiter. $ paste d | emp1.lst emp2.lst

sort : Ordering a file It is used to sort the content of a file. $ sort dept.lst The contents of the file are sorted. If we want to sort the contents based on a column then we need to specify the column, as specified below. $sort t | -k 3 dept.lst 05 | sales |1006

04 | personnel | 2365 02 | progs | 5423

01 | accounts | 6213 03 | marketing | 6521

If we want to sort them in the reverse order then we need use the option -r $sort t | -r -k 3 dept.lst 03 | marketing | 6521 01 | accounts | 6213 02 | progs | 5423

04 | personnel | 2365 05 | sales |1006

If we want to sort the data based on two columns one after the other then we need to specify both the columns.

$ sort t | k 3,3 k 2,2 dept.lst

If we want to sort a numeric file then we need to give the option n . $sort n numfile

uniq: Locate Repeated and Non Repeated lines And it takes a sorted file as input.

$ cat dept2.lst 01 | accounts | 6213 02 | progs 02 | progs | 5423 | 5423

03 | marketing | 6521 03 | marketing | 6521 03 | marketing | 6521 04 | personnel | 2365 04 | personnel | 2365 04 | personnel | 2365 04 | personnel | 2365 05 | sales 05 | sales |1006 |1006

$uniq dept2.lst 01 | accounts | 6213 02 | progs | 5423

03 | marketing | 6521 04 | personnel | 2365 05 | sales |1006

Options: Selectig the nonrepeated lines (-u) $uniq u dept2.lst 01 | accounts | 6213 It gives only the non-repeated lines.

Selecting the Duplicate Lines (-d): $uniq d dept2.lst 02 | progs | 5423

03 | marketing | 6521 04 | personnel | 2365 05 | sales |1006

Counting Frequency of Occurrence (-c): $uniq c dept2.lst 1 01 | accounts | 6213 2 02 | progs | 5423

3 03 | marketing | 6521 4 04 | personnel | 2365 2 05 | sales |1006

tr : Translating Characters $ tr | / < dept.lst 01 / accounts / 6213 02 / progs / 5423

03 / marketing / 6521 04 / personnel / 2365 05 / sales /1006

Changing Case of Text: $ tr [a-z] [A-Z] < dept.lst 1 01 | ACCOUNTS | 6213 2 02 | PROGS | 5423

3 03 | MARKETING | 6521 4 04 | PERSONNEL | 2365 2 05 | SALES |1006

Deleting Characters: $ tr d | < dept.lst 01 accounts 02 progs 6213 5423

03 marketing 6521 04 personnel 2365 05 sales 1006

Suppress or Compressing Multiple Consecutive Characters: $ tr s < dept.lst 01 | accounts | 6213 02 | progs | 5423 03 | marketing | 6521 04 | personnel | 2365 05 | sales |1006 $ tr s < dept2.lst | cut d | f2 | sort | uniq c | tee cntfields 1 accounts 2 progs 3 marketing 4 personnel 2 sales $ cut d | f2 dept2.lst | sort | uniq c 1 accounts 2 progs 3 marketing 4 personnel 2 sales

grep : Searching For a Pattern

If we want to search for a particular pattern then we need to give the pattern and the file name.

If we want to know the log in details of a particular user then we need to give the command $ who | grep kumar If we want to search in more than one file then we need to give the filenames as follows.

If we are searching for a particular pattern which contains more than one word, then we need to give the pattern in quotes. $ grep jai sharma emp.lst

Ignoring Case (-i): If we want to make the command case insensitive then we need to give the i option. $ grep i agarwal emp.lst 3546 | sudhir Agarwal Deleting Lines (-v) : $ grep v director emp.lst > otherlist $ wc l otherlist 11 otherlist Displaying Line Numbers (-n): If we want to display the line number along with the lines then we need to use the option -n $ grep n marketing emp.lst 3:5678 | sumit chakrobarty | d.g.m . . 15:0110| v.k.agarwal | g.m. | marketing | 31/12/40 | 9000 | marketing | 19/04/43|6000 | executive | personnel | 06/07/47 | 7500

Counting Line Containing Pattern (-c): $ grep c director emp.lst 4

Displaying Files: $ grep l manager *.lst design.lst emp.lst emp1.lst emp2.lst Matching Multiple Patterns (-e): If we want to search for multiple patterns then we need to give the e option. $ grep e Agarwal -e aggarwal -e agrawal emp.lst

Taking Patterns from a File (-f): We can store the pattern in a file and then we can give it as the input to the command. $ grep f pattern.lst emp.lst

Specifying Pattern Locations (^ and $): ^ - For matching at the beginning of a line. $ - For matching at the end of a line.

$ grep ^2 emp.lst It will display the lines which dstarts with 2 . $ grep 7$ emp.lst It will display the lines where the last 4 characters line between 7000 and 7999

sed : The Stream Editor If we want to display the first 3 lines of a file then we should use

Generally, we will be using the p (print) command to display the lines. We should use n option along with p option to suppress the duplication of lines.

To print only the last line we should use

We can also select lines from anywhere.

Selecting multiple group of lines sed -n 1,2p 7,9p $p emp.lst Negative Action(!): If we dont want to print the lines from 3rd to last line then we need to use $ sed n 3,$!p emp.lst We can use multiple instructions in a single command as follows. The following command will display lines 1,2 and 7-9 and then the last line. sed n e 1,2p, -e 7,9p e $p emp.lst we can also put these file numbers in a file and we can pass this file as the pattern. $ cat instr.lst 1,2p 7,9p $p sed n f instr.lst emp.lst

We can use sed for the pattern matching also. $ sed n /director/p emp.lst It will display all the lines which contains the pattern director in them. We can also use like this. $ sed n /[aA]ggarwal/p emp.lst It will display the lines which contains either Aggarwal or aggarwal. $ sed n /50..$/p emp.lst Then it displays the lines which contains 50 at 6th and 7th characters from last.

Writing Selected Lines into a File(w): Sed n /director/w dlist emp.lst Then it will write the lines which contains the pattern director in them into the file dlist. $ sed n 1,500w foo1 501,$w foo2 foo.12 Then it will write the first 500 lines of the foo.12 file into foo1 and the remaining lines into foo2.

Substitution(s): $ sed s/|/:/ emp.lst Then 1st occurrence of | in every line will be replaced by : If we want to make this change to the entire line, wherever it appears in the line then we need to use the option g(global). $sed s/|/:/g emp.lst Then the character | will be replaced by : in the entire file wherever it appears. $sed 1,3s/|/:/g emp.lst Only for the first 3 lines.

Basic Shell Scripting:

Interaction with the user:

The output will be

awk: An Advance Filter: Simple awk Filtering: If we are searching for a pattern in a file the we can give the command as follows.

And it is equivalent to the following commands. awk /director/ emp.lst awk /director/ { print $0 }

Splitting A Line into Fields: If we want to display only few fields in a file, then we can use the following command by specifying the delimiter.

The above command prints only the fields 2nd, 3rd, 4th and 6th fields only.

We can also specify the from which line to which line we need to display by using the NR value. In the below example, It will display the NR value, 2nd ,3rd and 6th fields of a file.