Category: unix

[problem]

Started using vi probably 13 years ago and ever so often, still learn new tricks with it! 🙂

Some of my favorites are: turning on control characters; mapping function keys; performing global search and replace; remapping columns.

[/problem]

[solution]

So to the basics, editing a file:


vi filename


[/solution]

[example]

Editing a file from a specific line number or pattern:


vi +lineno filename
vi +/patt/ filename


To turn on control characters, once in vi – do the following (exactly)


:set list


To turn it off again:


:set nolist


Global replace:


:%s/patt/rep/g


Map function keys:


:map #1 :w![ctrl v ctrl m]


that is type colon,map,#,1,space,colon,w,!,^M then hit enter. More on this later.

Remap columns:


:%s/(col1patt) (col2patt)/2 1/


To finish here is a cool but slightly bizarre one:


:g/^/m0


It reverses the lines of file. Effectively it says – global (all lines) with this pattern (a beginning line – so all lines), move to 0 – which is shorthand for the current line. So as vi applies the command to each line, it bubbles up to the top.

[/example]

[reference]

[tags]vim, regular expression, Unix Coding School[/tags]

• Linux Man Pages – vi command

[/reference]

[problem]

Ever needed to print out one column with awk, but do not know the field number?

[/problem]

[solution]

Here I’m running df and looking for a field with “Used” in it:

 df -k | head -1 | awk ' {       for(i=1; i< NF;i++) {          if($i ~ /Used/) {              print "# field "i          }       } } ' 

So stop counting columns and just use the code! 🙂

[/solution]

[example]

Now for a real life demo. 😉


[marcus@bree marcus]$ df -k | head -1
Filesystem 1K-blocks Used Available Use% Mounted on
[marcus@bree marcus]$ df -k | awk ' {
for(i=1;i < NF;i++) {
if($i ~ /Used/) { print "# field "i }
}
} '
# field 3


[/example]

[reference]

[tags], Unix Coding School[/tags]

• Linux Man Pages – gawk command

[/reference]

[problem]

Ever wanted to display a line number, with each matching record from awk. Useful as a way of counting the number of matching lines too.

[/problem]

[solution]

Here’s how with a demo which grabs network stats (netstat) off my linux box. It looks for connections to port 80 and in TIME_WAIT state – prints a line number and the matching line.

[/solution]

[example]


[marcus@bree marcus]$ netstat -an -t | awk ' /:80 / && /TIME_WAIT/ { print ++i,$0} '
0 tcp 0 0 x.x.x.x:80 x.x.x.x:44154 TIME_WAIT


If you wanted the line number, that matches your pattern – this would show that too:


[marcus@bree marcus]$ netstat -an -t | awk ' /:80 / && /TIME_WAIT/ { print "line:"NR,"count:"++i,$0} '
line:17 count:1 tcp 0 0 x.x.x.x:4298 x.x.x.x:80 TIME_WAIT


[/example]

[reference]

[tags]gawk, netstat, print line numbers, Unix Coding School[/tags]

• Linux Man Pages – gawk command
• Linux Man Pages – netstat command

[/reference]

[problem]

Need to send an email from UNIX command line, that contains HTML content?

[/problem]

[solution]

Most useful when generating statistics, etc – just spin round and spit out HTML table rows, through sendmail.

[/solution]

[example]

Here is the code:


/usr/lib/sendmail -t
To:[email protected]
From:[email protected]
Subject:This is a test
MIME-VERSION:1.0
Content-Type: multipart/mixed; boundary="gc0y0pkb9ex"

[/example]

[reference]

[tags], Unix Coding School[/tags]

• Linux Man Pages – crontab command
• Linux Man Pages – sendmail command

[/reference]

[problem]

You need to debug a shell script.

[/problem]

[solution]

Most shells (zsh, ksh, bash …) can either run with a minus x (-x) option or by specifying set -x.

If you have an existing shell, that you’d like to run in debug mode – just run it like this: bash -x scriptname. Or ksh -x for korn shell.

[/solution]

[example]

You can also edit the follow and put a minus x after the interpreter line, for example:

#!/bin/ksh -x

or

#!/bin/bash -x

Debugging on the command line can be performed, simply by typing set -x. Like this:


$ set -x
+ set -x
$ date
+ date
Tue Jun 13 18:59:44 WST 2006
$ echo "hello world"
+ echo 'hello world'
hello world
$ set +x
+ set +x
$ date
Tue Jun 13 18:59:57 WST 2006


[/example]

[reference]

[tags]Debug Shell Scripts, Unix Coding School[/tags]

• Linux Man Pages – set command
• Linux Man Pages – bash command
• Linux Man Pages – ksh command

[/reference]

[problem]

Sometimes you want to list all processes that have been initiated by cron.

[/problem]

[solution]

To do this you first need to know the Process ID (PID) of the cron process, then to list all children of that process.

Luckily Solaris has ptree and pgrep, so it is simple.

[/solution]

[example]


ptree `pgrep cron`


Looks like Linux also has pgrep and pstree:


pstree -p `pgrep cron`


[/example]

[reference]

[tags]Process commands, Unix Coding School[/tags]

• Linux Man Pages – pstree command
• Linux Man Pages – pgrep command

[/reference]

[problem]

You want to ascertain how long your script takes to run, displaying thousands of seconds and including CPU utilization.

[/problem]

[solution]

Just use the time command.

Really simple and an excellent way to see how long your scripts take. Plus showing the anticipated load on the system.

[/solution]

[example]

[/example]

[reference]

[tags], Unix Coding School[/tags]

• Linux Man Pages – time command
• Linux Man Pages – ps command

[/reference]

[problem]

You want to display processes, commands within the process tree.

[/problem]

[solution]

There are a number of ways to search the process tree.

Here are a few:

Show all processes for user apache.


ps -fu apache


Show all process on a given terminal (try command tty to see your own)


ps -ftpts/0


[/solution]

[example]

Show a given process


ps -fpPID


[/example]

[reference]

[tags], Unix Coding School[/tags]

• Linux Man Pages – ps command

[/reference]

[problem]

You want to create and use an UNIX FIFO.

[/problem]

[solution]

FIFO means first in, first out.

To create a FIFO (also called a named pipe), UNIX supplies the mknod command.

[/solution]

[example]

This is the code to generate a FIFO:


$ mknod myFIFO p


Then the resulting named pipe:


$ ls -ld myFIFO
prw-r--r-- 1 marcus adm 0 Jun 13 21:15 myFIFO


To use the FIFO, you need to have a process that is reading from it. This could be a bit of Perl, C code, etc.

A perfect way to demonstrate using FIFOs, is tail -f:


$ tail -f myFIFO


Then a test:


$ echo "
Hello there
This is a test of FIFOs
" > myFIFO


From the tail -f when now see following output:

Hello there
This is a test of FIFOs

[/example]

[reference]

– as always an excellent description on wikipedia

[/reference]

[reference]

[tags]FIFO, Named Pipes, mknod, Unix Coding School[/tags]

• Linux Man Pages – mknod command
• Linux Man Pages – tail command
• Linux Man Pages – base command

[/reference]

[problem]

You need to find a pattern in a file. Using regular expression.

[/problem]

[solution]

Normal command to display something in a file, is grep.


grep "pattern" filename


[/solution]

[example]

You can also display the line number it occured on (-n) and ignore case (-i):


grep -n -i "pattern" filename


Also you can match more than one pattern:


egrep "patternA|patternB" filename


To match the beginning of a line (^) or the end of line ($) – this will only match lines with this specific pattern on:


grep "^patternA$" filename


We can also use wildcards and ranges – this will match any line with PATTERN in it followed by any amount of numbers:


grep "^.*PATTERN[0-9]*$" filename


[/example]

[reference]

[tags]grep, regular expression, Unix Coding School[/tags]

• Linux Man Pages – grep command

[/reference]