Linux Command sed explained in this page with examples

Linux command Grep for searchn in files explained in this page

In this blog i will explain how to zip files and folder in a current folder using Linux Command zip

Linux Commad Find (serach , change ,modfy, criteria ,recursive, file size)

 

Database crash and recovery

1- delete  log files unnecessary in /var/log in order to create free space  that a case preventing mysqld to create mysql.sock file  (find big file size)

2- cd /var/lib/mysql then run " myisamchk *.MYI " to repair all databases

 

Empty trash large file by using mac terminal code 

sudo rm -rf ~/.Trash/*

Finds files larger than 50 MB in current directory 

 find . -type f -size +50000k -exec ls -lh {} \; | awk '{ print $9 ": " $5 }'

cd /your _project path ...

 

ifconfig : 

print the network interface
$ ifconfig | cut -c-10 | tr -d ' ' | tr -s '\n'  

grep ing ip address
$ ifconfig wlan0 | egrep -o "inet addr:[^ ]*" | grep -o "[0-9.]*"

192.168.0.82

host muslum21.com

nslookup muslum21.com

Scheduling 

* * * * * /bin/execute/this/script.sh

	* * * * *
minute (from 0 to 59) hour (from 0 to 23) day of month (from 1 to 31) month (from 1 to 12) day of week (from 0 to 6) (0=Sunday)	every minute of every hour of every day of the month of every month and every day in the week.

@reboot Run once, at startup @yearly Run once a year "0 0 1 1 *" @annually (same as @yearly) @monthly Run once a month "0 0 1 * *" @weekly Run once a week "0 0 * * 0" @daily Run once a day "0 0 * * *" @midnight (same as @daily) @hourly Run once an hour "0 * * * *" Leaving the rest of the fields empty, this would be valid: @daily /bin/execute/this/script.sh

• rsync :  advatage  over cp is rsync uses strong difference algorithm 
• dd :  clone harddrive
• version control  based git,apache subversion

• mksquashfs :  file system

• diff  : file difference

backup options :

• writing bash script
• exclude  file or files by cri,teria
• backup in remote host
• scheduling backup  at interval

Email regex pattern :[A-Za-z0-9._]+@[A-Za-z0-9.]+\.+[A-Za-z]{2,4} egrep -o  '[A-Za-z0-9._]+@[A-Za-z0-9.]+\.+[A-Za-z]{2,4}'  filename.txt URL regex pattern : http://[a-zA-Z0-9\-\.]+\.[a-zA-Z]{2,4}   egrep -o  ' http://[a-zA-Z0-9\-\.]+\.[a-zA-Z]{2,4}'    filename.txt

$ cd source_folder $ grep "any_function()" . -R -n

Commands cut: column in file  such as csv grep: grep with xargs sed :  using sed to perfom text replacement in a file. used with regular patern Using awk for advanced text processing  awk is a tool  designed for data stream ,it can also operate on column, rows.it supports built-in functions like arrays, functions. biggest advantage is flexibility. awk ' BEGIN{ print "start" } pattern { commands } END{ print "end" }  file The awk command works in the following manner: 1. Execute the statements in the BEGIN { commands } block. 2. Read one line from the file or stdin, and execute pattern { commands }. Repeat this step until the end of the file is reached. 3. When the end of the input stream is reached, execute the END { commands } block.

Creating  a file size  1Mb

$ dd if=/dev/zero of=junk.data bs=1M count=1
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.00767266 s, 137 MB/s



intersection and set differances  (A-B)  on text files

commands : sort  comm

$ comm A.txt B.txt -1 -3
-2 -3 removes the second and third columns

finding and deleting dublicate files  

by comparing files content example script exist in pdf or web  awk {}  used



working with file permission, ownership and the stiky bit

chmod   used to modify the permission

chown used to change the ownership of the file

-R option to apply cmd recursively

The sticky bit is an interesting type of permission applied to directories. By setting the sticky
bit, it restricts only the user owning it to delete the files even though group and others may
have sufficient permissions.
In order to set the sticky bit, +t is applied on a directory with chmod as follows:
$ chmod a+t directory_name



Making File immutable

when a file is immutable  that means any user cannot remove it

/etc/mtab

immutable a way of securing the file such as  /etc/resolv.conf

command :  # chattr +i file or -i 



Using loopback files

command : mount -o loop, dd,

$ dd  if=/dev/zero of=muslum21.img  bs=1G count=1
$mkfs.ext4 muslum21.img 
$file muslum21.img  ##  to test file information approval

then mount to any folder create a folder mkdir   muslum_folder
$ mount -o loop muslum21.img  muslum_folder

now your storage is ready.



Creating iso files and hybrid iso

# cat /dev/cdrom > image.iso  or  # dd if=/dev/cdrom of=image.iso

$ mkisofs -V "Label" -o image.iso source_dir/

hybrid iso is for usb storage devices   with  command  isohybrid  it can be



Finding differences between files and patching 

command: diff, diff -u,

patch  -naur  for applying patches.



head and tail command :  first 10 lines and last 10 lines .

You will probably want to use this on logfiles. The command to monitor the growth of the files
would be:
# tail -f /var/log/messages

ls ,cd , pwd , vi, gedit, cp,mv,rm,pico,touch

history

cat -n (-n line number)

>> or 2> save to file the output

script and scriptreplay  command  like videoplayer you can record the code uyou write  to terminal.

$ script -t 2> timing.log -a output.session
type commands;
…
..
exit

$ scriptreplay timing.log output.session

find  /home/muslum -name "*.php"     finds php files

-iname ignore case sensitive

-path search path

-regex :  $ find . -regex ".*\(\.py\|\.sh\)$"

using more condition:

new.txt some.jpg text.pdf
$ find . \( -name "*.txt" -o -name "*.pdf" \) -print
./text.pdf
./new.txt

find by  file ( access time - time, modification time -mtime , change time -ctime )  -ctime +7   means before  7 day ago 

search based on file size

-delete  : deleting matched files

serach based on permission (755) and ownership

$ find . -type f -name "*.php" !-perm 644 -print

$ find . -type f -user slynux -print

exec chown  username {}  changes file owner  change can be done by a mathes

serach  -prune  ignore matches

xargs

output  |  tr 'find text' 'replace text'

tr -d delete option

For example:
$ echo "Hello 123 world 456" | tr -d '0-9'

Hello world

tr -s

echo "ddddd 432  ddsa  dd"  | tr -s  'dd '
output :  "d432  dsa  d"

Checksum and Varification

$ md5deep -rl directory_path > directory.md5
# -r to enable recursive traversal
# -l for using relative path. By default it writes absolute file path in
output
Alternately, use a combination of find to calculate checksums recursively:
$ find directory_path -type f -print0 | xargs -0 md5sum >> directory.md5
To verify, use the following

Cryptographic tool and Hashes

cript ,  gpg,  base64, md5sum, sha1sum, opensslpasswd

Sorting unique and dublicates

commands: sort ,uniq

sorting file content  in alpha,numeric,reverse,merge sorted, and unique
sorting according to key and column

# Sort reverse by column1
$ sort -nrk 1 data.txt
4 linux 1000
3 bsd 1000
2 winxp 4000
1 mac 2000
# -nr means numeric and reverse

spliting file and data

you can spli5t files by matches and based on its content data, extension

split ,csplit

$ split [COMMAND_ARGS] PREFIX

$ csplit server.log /SERVER/ -n 2 -s {*} -f server -b "d.log" ; rm
server00.log
$ ls
server01.log server02.log server03.log server.log

based on extension

he name from name.extension can be easily extracted using the % operator. You can
extract the name from "sample.jpg" as follows:
file_jpg="sample.jpg"
name=${file_jpg%.*}
echo File name is: $name
The output is:
File name is: sample
The next task is to extract the extension of a file from its filename. The extension can be
extracted using the # operator as follows:
Extract .jpg from the filename stored in the variable file_jpg as follows:
extension=${file_jpg#*.}
echo Extension is: jpg
The output is:
Extension is: jpg

Rename 

 Renaming *.JPG to *.jpg:

$ rename *.JPG *.jpg

To replace space in the filenames with the "_" character:
$ rename 's/ /_/g' *
# 's/ /_/g' is the replacement part in the filename and * is the wildcard for the
target files. It can be *.txt or any other wildcard pattern.
To convert any filename of files from uppercase to lowercase and vice versa:
$ rename 'y/A-Z/a-z/' *
$ rename 'y/a-z/A-Z/' *
To recursively move all the .mp3 files to a given directory:
$ find path -type f -name "*.mp3" -exec mv {} target_dir \;
To recursively rename all the files by replacing space with the "_" character:
$ find path -type f -exec rename 's/ /_/g' {} \;

spell check and dictionary manipulation 

grep , look,  aspell list

$ look word filepath
Or alternately, use:
$ grep "^word" filepath

Automating interactive input

Making commands quicker by running parallel processes

yum -y update --skip-broken

the command  will skip broken packages with er. so there would not be error and continue working system. then you can try again same command  to update the packages without er. and updated system. bye bye..

login console as root apt-get install php5-curl that is all and check  you phpinfo

Hello linux ! Hello Webserver apache, nginx ! Hello python ! Hello Networking ! and Hello ...

Gd library installation  

1-  open terminal

2- login  as root

3-  in terminal console   " apt-get install php5-gd"

4- then [Y/n] appears on screen  press y  ,press ENTER

5- Finish