awk

Awk is my favorite CLI program for parsing different input. In most cases I just use it to cut out different fields from some output as its superior to cut utility – it allows me to set custom, multi-char field separator instead of just single character in cut.

Blocks

{} – block executed once for each line of the input.

BEGIN{} – block executed  once at the start. useful to set some variables like custom field separator FS. You could set FS in the normal block {} also… but its faster if you don’t need to do it once for each line of the input.

END{} – block executed once in the end. useful to print out statistics if you count something.

Variables

FS – input field separator. default is any amount of whitespace – one or more spaces or tabs.

OFS – output field separator, defaults to space

RS – row or record separator. default is newline.
 
ORS – output record separator. set this to ” ” to remove newlines in the output.
 
NR – line/record  number of currently parsed input
 
NF – number of fields/columns on current line/record
 
$1,$2,$3….$NR  – references columns 1, 2, 3 up to the last one which can always referenced as $NR not depending on is it 10nth or 100th.
 
$0 – references entire line/record, all fields/columns and separators.
 

# awk print out last field using separator “.”

# echo "one.two.three" |awk 'BEGIN{FS="."}{print $NF;}'
three

# awk print out last but one field using separator “.”. This is also useful to remove file extension from list of files.

$ echo "one.two.three" |awk 'BEGIN{FS="."}{X=NF-1; print $X;}'
two
# remove file extension - quick and dirty solution. does not take into account files having zero or more than one . in the name.
£ find . -type f -printf '%f\n' |awk 'BEGIN{FS="."}{X=NF-1; print $X;}'

# awk – print fields 1-3 only from matching line using separator “:”. Simplified version using grep below.

# awk 'BEGIN{FS=":";}/root/{print $1 $2 $3;}'< /etc/passwd
rootx0
# grep root /etc/passwd| awk 'BEGIN{FS=":";}{print $1 $2 $3;}'
rootx0

# find all unique file extensions

# find /path/to/files/ -type f |awk 'BEGIN{FS=".";}{print $NF;}' |sort |uniq

 

to be continued…

If you found this useful, say thanks, click on some banners or donate, I can always use some beer money.