Linux File System
/boot-- contains file that is used by boot loader ( grub.cfg)
/root-- root user home directory, It is not same as "/"
/dev-- System devices (e.g. disk,cdrom,speakers,flashdrive,keyboard etc.
/etc-- configuration files
/bin->/usr/bin -- everyday user commands
/sbin->/usr/sbin -- system/filesystem commands
/opt-- Optional add-on applications ( Not part of OS apps)
/proc-- Running Processes ( only exist in memory)
/lib->usr/lib -- C programming library files needed by commands and apps
[ strace -e open pwd]
/tmp-- Directory for temporary files
/home-- Directory for user
/var-- System logs
/run--system daemons that start very early (eg.systemd and udev) to store temporary runtime files like PID files
/mnt-- To mount external filesystem ( Eg. NFS)
/media-- For cdrom mounts
Linux File Types
"-" --> Regularfile
d --> Directory
l -- Link
c -- special file or device file
s -- socket
p -- Named pipe
b -- block device
File System Paths
There are two paths to navigate to a filesystem
1. Absolute path
2. relative path
1 - An Absolute Path always begins with a "/". This indicates that the path starts at the root directory.
An Example of an absolute path is
cd /var/log/samba
2. A Relative path does not begin with "/" it identifies a location relative to your current position. An example of a relative path is
cd /var
cd log
cd samba
Creating files and directories
Creating files -- touch,cp,vi
creating folders -- mkdir
Copying Directories
1.Command to copy a directory
cp
2. To copy a directory on linux, you have to execute the "cp" command with the "-R" option for recursive and specify the source and destination directories to be copied
cp -R <source_folder> <destination_folder>
Find Files and Directories
Two Main commands are used to find files/directories
- find
eg: find / -name "file"
- locate
eg: locate filename
Difference Between find and locate
Locate: uses a prebuilt database, which should be regularly updated.
while find iterates over a filesystem to locate files. Thus, locate is much faster than find, but can be inaccurate if the database is not updated
To update locate databse run updatedb
once you create a file, you need to update the database
sudo updatedb
is the command to update the database with new files.
WildCards
A Wildcard is a character that can be used as a
substitute for any of a class of characters in a search
* - represents zero or more characters
? - represents a single character
[] - represents a range of characters
\ - as an escape character
^ - the beginning of the line
$ - the end of the line
Eg:
if you want to create multiple files
touch abcd{1..9}-xyz
if you want to remove/find files
ls -l abc*
if you want to find files with last words
ls -l *xyz
ls -l ?bcd* -- will find all the files which has bcd
ls -l [cd] -- will find all the files which has c and d
Soft and Hard Links
inode = pointer or number of a file on the hard disk
soft link = line will be removed if file is removed or renamed.
Eg ln -s
softlinks are only created at different location compare to original file
ln -s /home/devops/hulk
hardlink = deleting renaming or moving the original file will not affect the hard link
Eg: ln
ln /home/devops/hulk
to check inodes and permissions, groups, groups
ls -ltri
To add text for a empty file
echo "hulk is superhero" > hulk
To add text for a modified file
echo "123" >> hulk
File Permissions
Unix is a multi user system.
every file and directory in your account can be protected from or made accessible to other users by changing its access permissions
Every user has responsibiltiy for controlling access to their files.
Permissions for a file or directory maybe restricted to by types
There are 3 type of permissions
r - read
w - write
x - execute to run a program
each permission rwx can be controlled at three levels
u -- user
g -- group
o -- others
Examples
chmod u+x filename to add execute permissions
chmod u-x filename to remove execute permissions
Octal Permissions
Binary Decimal Permission Type
000 0 No Permission
001 1 Execute
010 2 Write
011 3 Write + Execute
100 4 Read
101 5 Read + Execute
110 6 Read + Write
111 7 Read + Write + Execute
rwx | r-x | r-x
111 101 101
7 5 5
chmod 755
other way
Permission Value
r 4
w 2
x 1
- 0
rwx|r-x|r--
421 401 400
7 5 4
chmod 754
File Ownership
There are 2 owners of a file or directory
user and group
command to change file ownership
chown changes the ownership of a file
chgrp changes the group ownership of a file
Recursive ownership change option
-R
Access Control List (ACL)
Which provides an additional, more flexible permission mechanism for FS.
It is designed to assist with UNIX file permissions.
ACL allows you to give permissions for any user or group to any disc resource
Basically, ACLs are used to make a flexible permission mechanism in Linux.
Commands to assign and remove ACL permissions are
setfacl and getfacl
Examples:
To Add permission for user
setfacl -m u:user:rwx /path/to/file
To Add permission for a group
setfacl -m g:group:rw /path/to/file
To allow all files or directories to inherit ACL entries
from the directory it is within
setfacl -dm "entry" /path/to/dir
To remove specific entry
setfacl -x u:user /path/to/file (for a specific user)
To remove all entries
setfacl -b path/to/file (For all users)
Adding Text to Files ( Redirects)
3 Simple ways to add text to a file
--> vi
--> Redirect command output > or >>
\> will replace the text on a file
\>> will add the text on a file
--> echo > or >>
Input or output redirects
There are 3 redirects in linux
standard input (stdin) and it has file descriptor number as 0
standard output(stdout) and it has file descriptor number as 1
standard error (stderror) and it has file descriptornumber as 2
output(stdout) - 1
by default when running a command its output goes to the terminal
the outpur of a command can be routed to a file using > symbol
eg: ls -l > listings
pwd > findpath
If using the same gile for additional output
or to append to the samefile then use >>
eg: ls -la >> listings
echo "Hello World" >> findpath
Input(stdin) - 0
-> Input is used when feeding file contents to a file
Error (stderr) -2
1.when a command is executed we use a keyboard and that is also considered (stdin -0)
2. That command output goes on the monitor and that output is studout -1
3. If the command produced any error on the screen then it is considered ( stderr -2)
we can redirecs to route errors from the screen
Eg: ls -l/root2> errorfile
telnet localhost 2>errorfile
Standard Output to a file (tee)
tee command is used to store and view the out of any command
Eg: echo "test devops" | tee devops
To add additional lines without deleting the existing texts use
echo " test devsecops" | tee -a devops
how to count words in the file
wc -a filename
Pipes
A pipe is used by the shell to connect the output of
one command directly to the input of another command.
eg: sudo /etc ls -ltr | more to get all the content in pages
File Maintenance Commands
.cp
.rm
.mv
.mkdir
.rmdir or rm -r
.chgrp
.chown
File Display Commands
Cat
more
less
eg: less filename
less filename press "J" to check line by line
less filename press "k" to go to previous line
head
tail
Filters/ Text Processors Commands
.cut --- it allows to cut the ouput
eg:
cut --version = check version
cut -c1 filename = list one character
cut -c1,2,4 = pick and chose character
cut -c1-3 filename = list range of characters
cut -c1-3,6-8 filename = list specific range of character
cut -b1-3 filename = list by byte size
cut -d: -f 6 /etc/passwd = list first 6th column separated by:
cut -d: -f 6-7 /etc/passwd = list first and 7th column separated by:
ls -l | cut -c2-4 = only print user permissions of files/dir
.awk -- it allows you to list by the columns
eg:
awk --version = check version
awk '{print $1}' file = List 1st field from a file
ls -l | awk '{print $1,$3}' = List 1 and 3rd field of ls -l output
ls -l | awk '{print $NF}' = List field of the output
awk '/word/ {print}' file = search for a specific word
awk -F" '{print $1}' /etc/passwd = Output only 1st field of /etc/passwd
echo "Hello Tom" | awk '{$2= "Adam" ; print $0}' = Replace word field words
cat file | awk '{$2="Adam";print $0}' = replace words field words
awk 'length ($0) >15 ' file = Get lines that have more than 15 byte size
ls -l | awk ' {if($9 == sienfeld") print $0;}' = Get the field matching seinfiled in /home/devops
ls -l | awk ' {print NF}' = Number of fields
grep and egrep -- if you want to search by keywords.
eg:
grep --version =check version
grep keyword file = search for a keyword from a file
grep -c keyword file = search for a keyword and count
grep -i Keyword file = search for a keyword ignore case-sensitive
grep -n keyword file = dispaly the matched lines and their line numbers
grep -v keyword file =- display everything except keyword
grep keyword file | awk '{print $1}' = search for a keyword and then only give the 1st field
ls -l | grep Desktop = search for a keyword and then give the 1st field
egrep -i "keyword|keyword2" file = search for 2 keywords
sort: sorts out the output in alphetical order
eg:
sort --version = check version
sort file = sorts file in alphetical order
sort -r file = sort in reverse alphetical order
sort -k2 file = sort by field number
uniq: it will not show any duplicates or repeated lines
eg:
uniq file = removes duplicates
sort file | uniq = always sort first before using uniq their line numbers
sort file | uniq -c = sort first then uniq and list count
sort file | uniq -d = sort firs then uniq and show repeated lines
wc: word command, waka command it will tell you how many words in a file
eg:
wc --version = check version
wc file = check file count, word count and byte count
wc -l file = get the number of lines in a file
wc -w file = get the number of words in a file
wc -c file = get the number of bytes in a file
ls -l | wc -l = Number of files
grep keyword | wc -l = Number of keyword lines
Compare Files
Diff (line by line)
cmp (byte by byte)
Compress and un-compress files
.tar
Keeps all the files in one folder with tar extension
eg:tar cvf filename.tar
gzip
Gzip compress the files which ends with tar extension
eg:tar xvf filename.tar.gz
gzip -d or gunzip
gzip -d are used to uncompress the file
eg: gzip -d filename.tar.gz
Truncate File Size
The Linux truncate command is often used to shrink or
extend the size of a file to the specified size
eg: truncate -s 10 filename
Combining and Splitting Files
Multiple files can be combined into one and one file can be split into multiple files
eg:cat file1 file2 file3 > file4
split file4
split will usually split large file into multiple files
eg: split -l 2 filename sep
Linux vs Windows Commands
command description windows Linux
Listing of a directory dir ls -l
Rename a file ren mv
copy a file copy cp
move file move mv
clear screen cls clear
delete file del rm
compare contents of files fc diff
search for a word in a file find grep
display command help command /? man command
display your location on FS chdir pwd
display the time time date
Linux File Editor
A text editor is a program which enables you to create and manipulate
data in a linux file
Vi -- Visual Editor
ed -- standard line editor
ex -- extended line editor
emacs -- a full screen editor
pico -- beginner's editor
vim -- advance version of vi
To replace words on the file through vi
:%s/oldname/newname/
sed Command
Replace a string in a file with a newstring
Find and Delete a line
Remove empty a lines
Remove the first or n lines in a file
To replace tabs with spaces
Show defined lines from a file
Substitute within vi editor
To replace with new word
Eg: sed -i 's/oldname/newname/g' filename
To remove the word
eg: sed -i 's/name//g' filename
To remove the word line completely
eg: sed -i '/name/d' filename
To remove empty space on the file
eg: sed -i '/^$/d' filename
To remove the firstline of the file
eg: sed -i '1d' filename
User Account Management
Commands
useradd
groupadd
userdel
groupdel
usermod
User record maintained three different files
/etc/passwd
/etc/group
/etc/shadoow
Example:
useradd -g groupname -s /bin/bash -c "user description" -m -d /home/username username
Cron Jobs
/etc/crontab
# Example of job definition:
# .---------------- minute (0 - 59)
# | .------------- hour (0 - 23)
# | | .---------- day of month (1 - 31)
# | | | .------- month (1 - 12) OR jan,feb,mar,apr ...
# | | | | .---- day of week (0 - 6) (Sunday=0 or 7) OR sun,mon,tue,wed,thu,fri,sat
# | | | | |
# * user-name command to be executed