Unix Notes
Unix Notes
User Information:
Each user on a Unix system has:
A startup shell: The command-line interface you interact with (e.g., bash , sh ).
Powerful Utilities: Comes with simple yet powerful tools for various tasks.
Unix 1
Terminals, Shells, and Basic Commands
(Making Sense of It)
What happens after you log in?
Once you successfully log in, Unix starts a shell — a program that lets you interact
with the operating system by typing commands.
What is a Shell?
A shell is essentially a command-line interpreter. It takes your commands,
interprets them, and tells the operating system what to do. There are different
types of shells, including:
csh (C Shell).
Types of Terminals:
1. Local Terminal:
You type commands, and they are processed by the machine you're
working on.
Example command:
bash
CopyEdit
Unix 2
ssh user@remote_host
Some commands are internal (built into the shell), while others are external
(separate programs).
Example:
bash
CopyEdit
man ls
This will show you everything you need to know about the ls command.
bash
CopyEdit
man -k <keyword>
Example:
Unix 3
bash
CopyEdit
man -k copy
Flags (Options):
Flags modify the behavior of commands and usually start with a (dash).
Example:
The
l flag makes ls display files in a detailed list format.
bash
CopyEdit
ls -l
bash
CopyEdit
ls -la
Unix 4
Common Commands and What They Do:
Understanding how shells and commands work is essential for interacting with
Unix. Whether you're running commands locally or over SSH, it's all about
communicating with the operating system via the shell.
Unix 5
sort Sorts lines in a file alphabetically or numerically. sort names.txt
Displays the first ( head ) or last ( tail ) lines of a head -5 file.txt / tail -5
head / tail
file. file.txt
Displays file contents. Often used to combine cat file1.txt file2.txt >
cat
files. combined.txt
bash
CopyEdit
sort students.txt
2. Removing Duplicates:
bash
CopyEdit
sort students.txt | uniq
Unix 6
The uniq command only removes consecutive duplicates, so we use sort to
order them first.
3. Finding Files:
bash
CopyEdit
find . -name "*.java"
This searches the current directory ( . ) and all its subdirectories for files
ending in .java .
bash
CopyEdit
grep "error" logfile.txt
Searches for the word "error" in logfile.txt and displays the matching lines.
bash
CopyEdit
wc -l file.txt
bash
CopyEdit
head -10 file.txt
Unix 7
tail -10 file.txt
bash
CopyEdit
touch notes.txt
bash
CopyEdit
less bigfile.txt
Opens bigfile.txt one screen at a time, allowing you to scroll back and forth.
Use q to quit.
Unix 8
Compressing Files & Archives (Making
Sense of It)
In Unix, you often need to archive files (group them into one file) and compress
files (reduce their size) to save space or prepare them for transfer.
tar -tvf <archive> Lists the contents of an archive. tar -tvf archive.tar.gz
tar -xvf <archive> Extracts files from an archive. tar -xvf archive.tar.gz
Unix 9
file1.txt , file2.txt , and the folder folder/ into an archive called myfiles.tar .
bash
CopyEdit
tar -cvf myfiles.tar file1.txt file2.txt folder/
bash
CopyEdit
tar -czvf myfiles.tar.gz file1.txt file2.txt folder/
bash
CopyEdit
tar -tvf myfiles.tar.gz
bash
CopyEdit
tar -xvf myfiles.tar.gz
Unix 10
Sometimes you only need to compress a single file.
bzip2 <file> Compresses a file more efficiently than gzip . bzip2 bigfile.txt
Using tar combined with gzip or bzip2 is standard practice for backing up or
sharing files in Unix.
Processes
What is a Process?
Program in Execution: It’s a running instance of a program (like a text editor, a
game, or a compiler).
Unix 11
Unique Identifier (PID): Every process has a unique Process ID (PID).
Own Address Space: It gets its own memory space to prevent interference
with other processes.
nice Command:
Sets the initial priority of a process when you start it.
A lower priority allows other processes to run first (i.e., you are being “nice” to
others).
Example:
This starts
myprogram with a lower priority of 10 .
bash
CopyEdit
nice -n 10 myprogram
renice Command:
Changes the priority of a process that’s already running.
Example:
Unix 12
This sets the priority of the process with PID
1234 to 5 (higher priority).
bash
CopyEdit
renice -5 -p 1234
Viewing Processes:
There are several ways to check what processes are running.
Example:
bash
CopyEdit
ps -al
Typical output:
Unix 13
objectivec
CopyEdit
F UID PID PPID C STIME TTY TIME CMD
4 1000 12345 1 0 10:00 pts/0 00:00:01 bash
4 1000 12346 12345 0 10:05 pts/0 00:00:02 vim
Press q to quit.
Knowing how to adjust process priorities can make sure critical tasks run
smoothly.
Commands like ps and top are essential for system monitoring and
troubleshooting.
Unix 14
ps - list of processes(lots of options. e.g. ps -al (a-all users, l = long listing))
It encrypts your connection, making it safe to use even over public networks.
bash
CopyEdit
ssh user@remote_host
Example:
Unix 15
bash
CopyEdit
ssh alice@192.168.1.10
This will connect you to the machine at IP address 192.168.1.10 as the user alice .
Windows:
Use software like Putty (a free SSH client) or the built-in Windows
Terminal (if OpenSSH is installed).
bash
CopyEdit
sftp user@remote_host
Example:
bash
CopyEdit
Unix 16
sftp alice@192.168.1.10
close Ends the sFTP session but keeps the terminal open. close
exit Ends the sFTP session and closes the terminal. exit
bash
CopyEdit
sftp alice@192.168.1.10
Password: *****
sftp> ls
Documents Pictures Music
sftp> cd Documents
sftp> put myfile.txt
Unix 17
Uploading myfile.txt to /home/alice/Documents/myfile.txt
sftp> exit
sFTP makes it easy to transfer files securely without worrying about data
being intercepted.
Knowing how to use these tools gives you control over systems even when
you’re not physically present.
Links:
Unix 18
Disk Usage: Use the du command to check disk usage.
cp A B : Copy file A to B .
Example: cp ../file2.txt . copies file2.txt from the parent directory into the
current directory.
Device Files
Device Files: In Unix, device files are special files that provide interfaces to
hardware devices.
File Types: You can check the type of a file with the ls -l command (the first
character indicates whether it’s a regular file, directory, or device file).
I/O Redirection
Unix 19
Standard Streams: Every process has three standard streams—stdin (input),
stdout (output), and stderr (error output).
Redirection Operators:
< : Redirect input from a file (e.g., sort < inputfile reads from inputfile ).
> : Redirect output to a file (e.g., sort > outputfile writes sorted data to
outputfile ).
>> : Append output to a file (e.g., cat file1 >> file2 appends file1 content to
file2 ).
<<: Allows reading multiple lines from stdin (e.g., sort <<EOF allows input
until EOF is typed).
Examples:
ls -l > file.txt : Lists the current directory’s files and writes the output to file.txt .
I/O Piping
Piping ( | ): Piping connects the output of one command to the input of
another command.
Example: cat myfile | sort | uniq | grep CSI > output sorts the content of myfile ,
removes duplicates, and then filters for the term CSI , saving the result in
output .
Redirection with Piping: You can combine redirection with piping. Example:
gzip -dc archive.tar.gz | tar -xvf - > /dev/null decompresses and extracts an archive,
Aliases
Creating Aliases: You can create aliases to make commands shorter or more
convenient.
Unix 20
Example: alias dir='ls -lF' makes dir behave like ls -lF in the KornShell (ksh).
User (owner)
2. Permission Types:
bash
CopyEdit
ls -l myfile.dat
diff
CopyEdit
Unix 21
-rwxr-xr-x 1 userid gid 10 Feb 15 09:00 myfile.dat
1 = Number of links
r (read) 100 4
w (write) 010 2
x (execute) 001 1
Example
Permission string:
CopyEdit
r-xr-xrwx
Binary representation:
Unix 22
CopyEdit
101 101 111
Octal calculation:
CopyEdit
4+1, 4+1, 4+2+1 = 557 (octal)
Changing Permissions
Using chmod :
Symbolic form:
bash
CopyEdit
chmod og+w myfile # Adds write permission to group & others
Octal form:
bash
CopyEdit
chmod 777 myfile # Full permission for everyone
bash
CopyEdit
chgrp groupname myfile
Unix 23
Usually, only root or the file owner can change the group.
Example:
bash
CopyEdit
umask 022 # Files will be created with permissions 644 (rw-r--r--)
The Shell
Job Control
Job control allows you to manage multiple processes running in the
background or foreground within a shell session.
Jobs:
bg: Move a job to the background (so it runs without blocking your
terminal).
Unix 24
jobs: List all jobs running in the background or suspended.
Customization
Environment Variables: These are variables that influence the behavior of the
shell and programs. Some common ones are:
Shell Initialization Files: These files are executed when you start a new shell
session and can customize your shell environment:
For tcsh or csh : setenv EDITOR emacs sets the default editor to emacs .
Default Shell: The default shell for a user is set by the root user, often in
/etc/passwd .
Metacharacters
Metacharacters are special characters in the shell that have a specific
meaning:
Unix 25
Escaping Special Characters: To prevent special characters from being
interpreted by the shell, you can escape them with a backslash ( \ ). For
example, \* , \; will treat and ; as literal characters, not as
metacharacters.
Scripts
Shell Scripts: A script is a file that contains a series of commands that the
shell can execute.
Purpose: Scripts help automate tasks, define new commands, and execute
repetitive actions. They can be written interactively or as standalone files.
Syntax: The syntax depends on the shell being used. For example, the
Bash shell syntax is different from the KornShell ( ksh ) or C shell ( csh ). Use
man bash to read more about Bash scripting.
#!/bin/bash
NAME=name*
Unix 26
echo "Too few files: " $COUNT
else
echo "OK -Number files: " $COUNTfi
#must chmod u+x script
Bash Shell
Script comments
a variable is defined in the “for” loop
if [expression]; then
command
elif
command
else
command
Unix 27
fi
example 2
#!/bin/bash
#
# Script: check for 3 files as
# command line args and display their content
# $1 is arg1; $*is list of all command line args;
# $#number args
if [ $# != 3 ]; then
echo "Incorrect file number: " $#
else
for f in $*; do
echo "File: " $f
echo " -contains: "
cat $f
done
fi
Development Tools
Text Editors
vi/vim, emacs, joe: These are popular text editors used in Unix-based
environments for editing source code and other files.
Unix 28
vi/vim: A powerful and widely-used text editor with modes for inserting
text and running commands.
emacs: A highly customizable and extensible text editor, often favored for
its advanced features.
Compilers
javac: The Java compiler that translates Java source code into bytecode.
JVM (Java Virtual Machine): A virtual machine that runs Java bytecode,
making Java programs platform-independent.
java: The command used to run Java programs by executing the bytecode
produced by javac .
Naive Approach: Manually recompiling all project files whenever one file is
changed is inefficient.
Pros: It’s tailored for Java and handles dependencies for inner classes and
packages very well.
Cons: Since it’s Java-based, it might not be available on all Unix systems
by default.
Unix 29
Make: A widely-used Unix tool for managing project dependencies and
automating builds. It uses a configuration file (Makefile) to define rules for
building targets and their dependencies.
Pros: It works well with C++ and large open-source projects, and it’s often
available on Unix systems as GNU Make (gmake).
Command: After editing source code, you just run make to ensure the
project is up to date.
Makefile Example:
makefile
CopyEdit
HelloWorld.class : HelloWorld.java
javac HelloWorld.java
Makefile Configuration: The Makefile contains rules that specify which files
are dependencies and how to update them when the source files change.
For example, if
Unix 30
is modified, the Makefile ensures that
HelloWorld.java HelloWorld.class is
recompiled by executing javac HelloWorld.java .
yaml
CopyEdit
target : dependency_list
target_action
Makefile Usage:
Makefiles can contain various rules and options, and even scripting
commands.
Rule Types: Makefiles can include various types of rules to handle different
tasks, such as cleaning up old files ( clean ), compiling, and linking.
Efficiency: If the Makefile is set up correctly, only the modified files will be
rebuilt, saving time in large projects.
Text Editors: Use editors like vi , vim , or emacs for writing code in a Unix
environment.
Unix 31
Variables
Makefiles allow you to define variables to simplify configuration and avoid hard-
coding paths and commands. For example, the variable JAVAC is used to store the
path to the Java compiler”
JAVAC=/user/bin/javac
This means, instead of writing /usr/bin/javac everywhere in Makefile, you can use
$(JAVAC) to refer to the Java compiler
Womble.class: Womble.java
$(JAVAC) $(JFLAGS) Womble.java
This tells Make to create Womble.class from Womble.java by using the Java
compiler stored in JAVAC and any flags stored in JFLAGS. If JFLAGS is set
somewhere else in the Makefile, it would pass those flags to the computer
This is a built in rule that tells Make how to convert .java files into .class files. The
$< is a special macro that refers to the first prerequisite(the .java file in this case).
So if you have a .java file, Make will use the rule to create a .class file
Unix 32
Using .SUFFIXES
If you want to exlicitily define which suffixes Make should use,you can set the
.SUFFIXES variable
This tells Make that it should consider .java and .class as valid file suffixes when
applying rules
String Substitution
Makefiles also allow you to do string substitution with variables. This is useful
when you have a list of files and you want to convert one type to another(like
converting .java files into .class)
This uses string submission to convert .java extensions into .class. The resulting
value of OBJECTS would be A.class B.class C.class
example
# Define variables
JAVAC = /usr/bin/javac
JFLAGS = -g
SOURCES = A.java B.java C.java
OBJECTS = $(SOURCES:.java=.class)
# Default rule
all: $(OBJECTS)
Unix 33
$(JAVAC) $(JFLAGS) $<
# Clean rule
clean:
rm -f $(OBJECTS)
the all taget compiles A.java B.java and C.java into their respective .class files
The rule .java.class defines how to convert a .java file into a .class file using
the javac compiler
No space after text on a line: After you type your rule or command, press
<RETURN> (enter) to move to the next line.
Tab for indenting rules: Rules inside a Makefile must be indented with a
tab, not spaces.
2. Rule Evaluation:
Example:
makefile
CopyEdit
clean:
Unix 34
rm *.class
This clean target will delete all .class files when you run make clean from the
command line.
3. Including Files:
You can include other files in a Makefile using the include statement. This
helps keep Makefiles organized and reusable.
makefile
CopyEdit
# Messy Makefile
JAVAC = javac
JFLAGS = -g
JARFILE = blah.jar
CLASSFILES = A.class B.class C.class
# Default rule
C.class: A.class B.class C.java
$(JAVAC) $(JFLAGS) C.java
B.class: B.java
$(JAVAC) $(JFLAGS) B.java
# Explicit rules
clean:
@rm -f $(CLASSFILES)
jar:
Unix 35
jar cvf $(JARFILE) $(CLASSFILES)
In this example:
The jar target creates a JAR file from the .class files.
makefile
CopyEdit
# Better Makefile
JAVAC = javac
JFLAGS = -g
SOURCES = A.java B.java C.java
In this example:
The SUFFIXES line defines how files with the .java extension should be treated
and compiled into .class files.
The clean target removes all .class files based on .java files in the SOURCES list.
Unix 36
Git: Version Control System
Why Git?
Git is a lightweight and powerful version control system that helps track
changes in the source code over time.
Maintains a code repository (repo): It stores all the code and logs, recording
all committed updates.
Works with multiple contributors: Multiple people can work on the same
project simultaneously, and Git helps manage changes from all team
members.
History and Rollback: You can go back in time, revert code to previous
versions, and track the history of changes.
Advantages:
2. Remote Repositories: Git allows you to store your code repository on remote
servers, so you can access it from anywhere on the Internet. (Though this may
not be required in your current course, it’s common in larger projects.)
3. Rewind Code: You can revert to any commit in the history if something goes
wrong.
4. Team Collaboration: Git allows multiple people to work on the same project at
the same time without conflict.
Source Control: Git is a standard tool in the software industry for managing
large and collaborative projects. It’s crucial to know how to use Git effectively
in the workplace.
Unix 37
CVS, Subversion, and Mercurial are alternatives to Git, though Git is the most
widely used and favored for its speed and flexibility.
3. Add: The action of adding new files or changes to existing files to the "staging
area". This prepares them for later committing to the repository.
You can also unstage changes if you decide not to commit them.
4. Commit: This action adds your staged changes to the repository, with a
commit message explaining the changes made. This makes a snapshot of your
project at that point.
bash
CopyEdit
git config --global user.name "Joe Blogs"
git config --global user.email joe@whatever
2. Create a repository:
bash
CopyEdit
mkdir Prac0
Unix 38
cd Prac0
git init
Use git status to check which files are modified and which are staged for
commit.
5. Optional:
Remove unwanted files: You can delete files or directories using git rm .
Merge branches when you are ready to bring changes from one branch to
another.
Git Structure:
Working Directory: The files and directories you are working on in your
project. These are your physical files.
Staging Area: An index that keeps track of changes made to files, staging
them for the next commit.
Unix 39
Staged changes are included in the snapshot, but unstaged changes are
not.
Git HEAD: This is an internal pointer to the last commit on your current branch.
When you switch branches with git checkout , the HEAD pointer shifts to the new
branch.
bash
CopyEdit
mkdir Prac0
cd Prac0
git init
bash
CopyEdit
git add myfile.java
bash
CopyEdit
git add *.java
Unix 40
git status will show the files you’ve modified, added, or deleted.
3. Commit changes:
bash
CopyEdit
git commit -m "Add initial version of myfile.java"
Use git log to see the history of commits, and git log --oneline for a one-line
summary of each commit.
bash
CopyEdit
git rm a.java
git commit -m "Remove a.java from repo"
bash
CopyEdit
git rm -r dir
git commit -m "Remove directory dir"
git commit : Commits the staged changes to the repository with a message.
Unix 41
git log : Shows the commit history.
git log --oneline : Displays the commit history in a more compact format (one line
per commit).
bash
CopyEdit
sudo apt-get install git
Unix 42