A Regular Expressions Approach To Sed And Awk

from ed and grep to sed and awk
-------------------------------------------------------

The unix utilities sed and awk are both based on pattern matching, i.e. regular expressions.

Historically, ed was the first utility on Unix to use regular expressions. It was a line editor and so to make edits, we had to specify both the line address (using the line number or any set of words in that line) and the action to be performed (typically print the line to screen, delete the line or substitute a set of words on that line.)

Almost every shell utility in Unix has the impression of ed. That is, they have the features of both a programming language and of a text editor.

In ed -
* Every command is of the type '/pattern/ {action}'
* A pattern is always a regular expression.
* An actions are typically print, delete or substitute.
* If a pattern is not specified, then the action is performed on the current line.

For example,
/pattern/d    # deletes the next line containing the regular expression "pattern"
g/pattern/d    # deletes all lines containing the regular expression "pattern"
p        # prints current line to screen

In fact, the name of the grep command is derived from the frequently used ed command g/re/p, which prints all lines with the regular expression "re."

In sed and awk, both the input file/stream and the command file/stream typically containg multiple lines.
The first line of the input is read in and all the commands are run against that line. As with ed, if a pattern is specified, then sed/awk will run the action only if the line matches the pattern. If no pattern is specified, then the action is invariably run.
This is what happens with each line of input till we reach end-of-file for the input file/stream.

References -

A Tutorial Introduction to the UNIX Text Editor
by Kernighan
Unix Seventh Edition Manual. (Volume 2A has the tutorial)
http://cm.bell-labs.com/7thEdMan/bswv7.html


sed & awk, 2nd Edition
Dale Dougherty
Arnold Robbins

Historical references -
awk - A Pattern Scanning and Processing Language (September 1, 1978)
SED - A Non-Interactive Text Editor (August 15, 1978) by Lee E. McMahon.

A Regular Expressions Approach To Sed And Awk

from ed and grep to sed and awk
-------------------------------------------------------

The unix utilities sed and awk are both based on pattern matching, i.e. regular expressions.

Historically, ed was the first utility on Unix to use regular expressions. It was a line editor and so to make edits, we had to specify both the line address (using the line number or any set of words in that line) and the action to be performed (typically print the line to screen, delete the line or substitute a set of words on that line.)

Almost every shell utility in Unix has the impression of ed. That is, they have the features of both a programming language and of a text editor.

In ed -
* Every command is of the type '/pattern/ {action}'
* A pattern is always a regular expression.
* An actions are typically print, delete or substitute.
* If a pattern is not specified, then the action is performed on the current line.

For example,
/pattern/d    # deletes the next line containing the regular expression "pattern"
g/pattern/d    # deletes all lines containing the regular expression "pattern"
p        # prints current line to screen

In fact, the name of the grep command is derived from the frequently used ed command g/re/p, which prints all lines with the regular expression "re."

In sed and awk, both the input file/stream and the command file/stream typically containg multiple lines.
The first line of the input is read in and all the commands are run against that line. As with ed, if a pattern is specified, then sed/awk will run the action only if the line matches the pattern. If no pattern is specified, then the action is invariably run.
This is what happens with each line of input till we reach end-of-file for the input file/stream.

References -

A Tutorial Introduction to the UNIX Text Editor
by Kernighan
Unix Seventh Edition Manual. (Volume 2A has the tutorial)
http://cm.bell-labs.com/7thEdMan/bswv7.html


sed & awk, 2nd Edition
Dale Dougherty
Arnold Robbins

Historical references -
awk - A Pattern Scanning and Processing Language (September 1, 1978)
SED - A Non-Interactive Text Editor (August 15, 1978) by Lee E. McMahon.

Simple Backup Script

Shell Script - Install RPM

#!/bin/bash 

# @(#) user1    Demonstrate rpm query.

uname -rv

bash --version

rpm --version

echo

P=${1?" must specify package name."}

rpm -qa "$P" > t1

my_size=$( wc -l < t1 )

echo " Size of report file is $my_size lines"

if [[ $( rpm -qa $P ) =~ ${P} ]]

# if [[ $( rpm -qa $P ) == *${P}* ]]

then

  echo " Package $P is installed."

else

  echo " Package $P not found."

fi

exit 0

Execute Shell Script in PHP

<?php
$output = shell_exec('ls -lart');
echo "<pre>$output</pre>";?>

Shell Script with Operator

#!/bin/bash
# Prompt for your Age...
echo "Please enter your age:"
read AGE
if [ "$AGE" -lt 20 ] || [ "$AGE" -ge 50 ]; then
echo "Sorry, you are out of the age range."
elif [ "$AGE" -ge 20 ] && [ "$AGE" -lt 30 ]; then
echo "You are in your 20s"
elif [ "$AGE" -ge 30 ] && [ "$AGE" -lt 40 ]; then
echo "You are in your 30s"
elif [ "$AGE" -ge 40 ] && [ "$AGE" -lt 50 ]; then
echo "You are in your 40s"
fi

Shell Login Script

#!/bin/bash

# This is program to login using password.

CORRECT_PASSWORD="geeks" #this is our password.

echo "Please enter correct password"

read PASSWORD

if [ "$PASSWORD" == "$CORRECT_PASSWORD" ]; then

echo "You have logged in Successfully"

else

echo "INCORRECT PASSWORD LOGIN ABORTED !"

fi

Shell Script Tar and Compress the backup

#!/bin/bash
TERM=linux
export TERM
NOWDATE=`date +%m%d%y` # Sets the date variable format for zipped file: MMddyy


# Mail Recipient
#
SENDTO="admin@opensourcegeeks.org"
export SENDTO
#


clear # clears terminal window
echo
echo "Hi, $USER!"
echo
echo "Beginning backup of files @ `date`"
echo
echo "Zipping directory structure..."
tar -cvzf $HOME/html/backups/dailyback/$NOWDATE.tar.gz $HOME/html/*
echo "Backup Complete!"
mailx -s "Backup Completed on $(NOWDATE}" ${SENDTO}

exit 0

Shell Script Copy all .C file from one location to Another

#!/bin/bash

file_list=`find ./abc/test -name "*.c"`
for file_n in $file_list
do
    cp $file_n $targetdir/$file_n
done

Introduction to Sed

Introduction to SED

Sed is String editor, Its used for editing lines in a file or a stream; Output re-directed to a new file or we can re-direct as a standard output.

Syntax: sed [options] 'command1' [files]

        sed [options] -e 'command1' [-e command2 ...] [files]

        sed [options] -f script [files]

Delete lines from 4 through 6 in file page.txt:

sed '4,6d' page.txt

Delete lines that contain "P" at the beginning of the line:

sed '/^P/d' page.txt

Translate capital R,T,O into small r,t,o:

sed 'y/CRO/cro/' page.txt

Delete empty lines:

sed '/^$/d' page.txt

Replace string C++ with Java for the first occurence on a line

sed 's/C++/Java/' page.txt

Remove pp string (replace with empty entry)for the first occurence on a line:

sed 's/pp//' page.txt

Remove ss string for all occurences on a line:

sed 's/pp//g' page.txt

Substitute a single space for any number of spaces wherever they occur on the

line:

sed 's/ */ /g' page.txt

Substitute underscore for any number of spaces wherever they occur on the

line:

sed 's/ */_/g' page.txt