Category : unix

Clearing an AIX Print Queue


You have jobs stuck in the print queue. You want to clear the queues and remove all jobs.


You have to stop all processes and remove the files from spooling directories.

Probably worth taking a backup of these files first. πŸ™‚ All at own risk.



stopsrc -cs qdaemon # bring down qdaemon and clear qdforks.
stopsrc -s lpd


rm /etc/qconfig.bin # compiled /etc/qconfig


rm /var/spool/lpd/qdir/* # queue requests; JDF


rm /var/spool/lpd/stat/* # info on status of devices


rm /var/spool/lpd/pio/msg* # error message files.


rm /var/spool/qdaemon # temp copies of queue jobs


rm /var/spool/lpd/* # temp copies of remote jobs


ps -ef | grep pio # kill any of these processes; look for piobe, pioformat, pioout


ps -ef | grep qd # kill any of these processes; look for qdforks.


startsrc -s qdaemon # restarting qdaemon
startsrc -s lpd # restarting lpd


lssrc -s qdaemon #look for active
ps -ef | grep qdaemon
lssrc -s lpd # look for active
ps -ef | grep lpd


ps -ef | grep srcmstr # if there are 2 running, kill -9 one of them with higher PID.


As a last resort – reboot


[tags], Unix Coding School[/tags]

DNS, Sendmail and Building Linux


Have been aware of DNS and sendmail for over 10 years, but never really dabbled.


Hit a lot of issues, mainly with dodgy clients – like outlook (ended up moving over completely to mozilla’s thunderbird email client!). Also with port forwarding and firewall, default routes, etc.

To set it all up these were my steps:

1) Build RH Linux server with fedora core 4 (5 is out now too I think).
2) Ensure BIND was installed. yum is your friend! πŸ™‚ Do yum –help as root.
3) Edit /etc/named.conf – add my zone file. Always make a bu: cp -i /etc/named.conf /etc/named.conf.$(date +%j)
4) Copied to # this domain is retired now. πŸ™‚
5) Modified to include my info: SOA (start of authority) – in my case I said as this then relates to the ip of broadband router. Bit backward in logic, specify your DNS server with a name registered in your DNS server! πŸ™‚ Then I created an A record and NS for zion; CNAME for www to zion and CNAME for mail; lastly an MX record for mail. I’ll happily supply my zone file if someone wants a peek. πŸ™‚
6) Register domain name with – $12.50 a year! πŸ™‚ Give it the ip of my DNS server – which is actually the broadband router, with port 53 forwarded to my RH Linux box. domaincentral have been great (I’ve now got 3 domains with them). – this one and (still setting this up).
7) Setup port forwarding on broadband router, then fire up named and it was all good. Setup up virtual hosts ( and in that domain. Now can setup any mail account or host in that domain, it is just beautiful! πŸ™‚
8) Installed/updated sendmail. Especially like watching sendmail’s incoming and outgoing messages. πŸ™‚
9) Edit /etc/mail/ – which is a macro config file that gets fed into m4 (macro language). Had to update m4, etc as kept breaking during conversion.
10) Edited various other files under /etc/mail – such as relay_domains (allow email from, etc); aliases …
12) Installed dovecot – which services POP and SMTP requests.
13) Created some accounts to handle mail – sales; enquiries, etc.
14) Modified my apache confs to handle and redirectmatch to new domain.



[tags]sendmail, DNS, POP, SMTP, Red Hat Linux, Unix Coding School[/tags]

AIX ODM removing device


Someone removes a tty, printer, etc by doing rm /dev/lp3.

Then you cannot recreate the device, as system returns “already defined at that location”.


You can try to recreate the device with mknod, but need the major/minor numbers.

Then do rmdev -l lp1 -d.

To get major, minor numbers you need to query the ODM.


odmget -q "value3=lp1" CuDvDr

Should return something like:



The major number is 15 and minor 2.

Therefore mknod /dev/lp1 c 15 2 creates the device again.

Then rmdev -l /dev/lp1 -d should remove it cleanly.

In extreme cases, where this still does not remove it, you may have to manipulate
the ODM directly. * Proceed with caution *


cd /etc/objrepos
cp -i CuDv CuDv.$(date +%j)
cp -i CuAt CuAt.$(date +%j)
cp -i CuDvDr CuDvDr.$(date +%j)

Remove ODM entries

odmdelete -q "name=lp1" CuAt
odmdelete -q "name=lp1" CuDv
odmdelete -q "value=lp1" CuDvDr
synclvodm rootvg # resync odm

Reboot if necessary.


[tags]AIX ODM, odmdelete, synclvodm, Unix Coding School[/tags]

who command 5 ways


You want to know who is on the system. Or which userid you are using.
What runlevel you are at – or when the system was booted.

All doable with who.


See the example and reference tabs, to see 5 ways to run who, showing different things about the system and users.


1. Basic

[[email protected] puterpet]$ who
marcus pts/1 Sep 19 16:07 (
marcus pts/3 Sep 22 17:46 (

2. With Heading and Idle time dot means currently active.

[[email protected] puterpet]$ who -Hu
marcus pts/1 Sep 19 16:07 ? 29984 (
marcus pts/3 Sep 22 17:46 . 29566 (

3. Show who I am in the current shell and where I came from.

[[email protected] puterpet]$ who am i
marcus pts/3 Sep 22 17:46 (

4. Show the current run level Shows boot time and last runlevel too.

[[email protected] puterpet]$ who -r
run-level 5 Sep 9 10:47 last=S

5. Show last boot time

[[email protected] puterpet]$ who -b
system boot Sep 9 10:47


[tags]who command, Unix Coding School[/tags]

List files 12 ways


You want to list files under UNIX! πŸ™‚ Here I show 12 different ways to do it.


See the example and reference tabs.


1. Basic

[[email protected] Perl_Bin]$ ls

2. Long

[[email protected] Perl_Bin]$ ls -l
total 68
-rwxr--r-- 1 marcus adm 335 Sep 12 2005
-rwx------ 1 marcus adm 151 Feb 5 2006

2. Hidden files

[[email protected] Perl_Bin]$ ls -la
total 76
drwxr-xr-x 2 marcus adm 4096 Aug 15 04:53 .
drwx------ 29 marcus adm 4096 Sep 23 11:47 ..
-rwxr--r-- 1 marcus adm 335 Sep 12 2005
-rwx------ 1 marcus adm 151 Feb 5 2006

3. Sort by last modified

[[email protected] Perl_Bin]$ ls -lt
total 76
-rwxr--r-- 1 marcus adm 286 Aug 15 04:53
-rw-r--r-- 1 marcus adm 210 Apr 30 08:07
-rwxr--r-- 1 marcus adm 3483 Feb 5 2006
-rwxr--r-- 1 marcus adm 335 Sep 12 2005

4. Sort by last modified – reverse

[[email protected] Perl_Bin]$ ls -ltr
total 76
-rwxr--r-- 1 marcus adm 335 Sep 12 2005
-rwxr--r-- 1 marcus adm 3483 Feb 5 2006
-rw-r--r-- 1 marcus adm 210 Apr 30 08:07
-rwxr--r-- 1 marcus adm 286 Aug 15 04:53

5. Show hidden characters in file name

[[email protected] Perl_Bin]$ ls -ld *test*
-rw-r--r-- 1 marcus adm 5 Sep 23 11:59 ???testfile1
[[email protected] Perl_Bin]$ ls -lb *test*
-rw-r--r-- 1 marcus adm 5 Sep 23 11:59 bbbtestfile1

6. Show a digest of directories With asterix for executable and slash for directory, etc.

[[email protected] Perl_Bin]$ ls -aF

7. Do a recursive listing

[[email protected] Perl_Bin]$ ls -lR
total 76
-rwxr--r-- 1 marcus adm 335 Sep 12 2005
-rw-r--r-- 1 marcus adm 5 Sep 23 11:59 ???testfile1

total 4
drwxr-xr-x 2 marcus adm 4096 Sep 23 12:01 test2

total 0
-rw-r--r-- 1 marcus adm 0 Sep 23 12:01 showit

8. List the directory not its contents.

[[email protected] marcus]$ ls -ld Perl_Bin
drwxr-xr-x 3 marcus adm 4096 Sep 23 12:01 Perl_Bin

9. Digest the listing

[[email protected] Perl_Bin]$ ls -m,,,,,,,,,,,,,,,,, test1, ???testfile1

10. Include the inode – useful for then doing find . -inum #### – to remove, etc

[[email protected] Perl_Bin]$ ls -li
total 76
647846 -rwxr--r-- 1 marcus adm 335 Sep 12 2005
648063 -rw-r--r-- 1 marcus adm 5 Sep 23 11:59 ???testfile1

11. Force ls to produce one column

[[email protected]d Perl_Bin]$ ls -1

12. Show file modified time in seconds Okay I cheated here – it is Perl! πŸ™‚ But hey it's my blog!! πŸ˜‰

[[email protected] Perl_Bin]$ perl -e
'print(localtime((stat($ARGV[0]))[9])."n");' mytestfile
Sat Sep 23 12:50:03 2006


[tags]Listing files, Unix Coding School[/tags]

Permissions extension setfacl


You want to allow access to files or directories, without giving exclusive use of a given group, user, etc.


Use setfacl. πŸ™‚ See the example and reference tabs.


setfacl -m user:USERID:rw- filename

getfacl filename

Can also do this: getfacl filename1 | setfacl -f – filename

Performing an ls -ld filename, will show a plus in the permission column.


[tags]setfacl, Unix Coding School[/tags]

Linux dircmp – directory compare


I wanted to run a dircmp with the -d option, to show all files that are different.

Plus display those differences.

Linux replied – command not found! πŸ™‚


Just diff the directories, with the -d flag. See the example and reference tabs for more info.


diff -d dir1 dir2


[tags]UNIX, Linux, Dircmp, Unix Coding School[/tags]

Redundant key strokes


Carrying on with my lazy steps, there are bits of code used with UNIX commands, which are completely redundant.


Save yourself some key strokes, remember these examples. Or come back to and check them out again. πŸ™‚


chmod a+r filename

The "a" is redundant – if you just do a "+r" – all is implied.

ls -l | awk ' $4 > 1000 { print $0 } '

In awk printing $0 is default action.

find . -type f -print

With find default action is to print.

echo a b c | tr '[abcdef]' '[ABCDEF]'

Use a range: echo a b c | tr '[a-z]' '[A-Z]'

In vi use ZZ instead of :wq πŸ™‚

Only 2 chars less (including the shift) – but also just your little finger, instead of a hand span. πŸ™‚


[tags]Redundant key strokes, UNIX shortcuts, Unix Coding School[/tags]

Restarting Linux services boot time


Every time you reboot your Linux Box – web server, samba server, etc is down.


Use chkconfig. See the example tab and then reference tab for man page.


Show all services

chkconfig --list

List just httpd

chkconfig --list httpd

Change restart settings so web server starts at run level 5.
(do a runlevel to see normal level – most likely 5).

chkconfig --levels 5 httpd on


[tags]chkconfig, Linux reboot, Linux services, Linux restarts, Unix Coding School[/tags]

Lazy tip – wrong password


You start typing your password in UNIX, then realize – hold on that ain’t right.

Sometimes your control c it – other times just hit enter and do it again.

There is a quicker way. πŸ™‚


When your sat at that password: prompt and lord only knows what you’ve typed. No problem – just do one control u and all is right with the world again. πŸ™‚

I only know from the billion odd times I’ve done it too! πŸ˜‰ That equates to many wasted hours, this lazy tip will save you – over time.


Nothing to see here, check out the solution tab.


[tags]UNIX Lazy tip, UNIX passwords, Unix Coding School[/tags]

YUM fails with 404


If you (like me) was well keen, to get your shiny new fedora core 5 system up and running. Then you run yum and think, hang on that ain’t right – 404. Linux is better than this!! πŸ™


Don’t bother searching the news groups, for posts that go nowhere (like I did for close on 2 hours – that I’ll never get back!! πŸ™‚ ), here is the answer – Do a search for yum – follow the prompts and you’re in.


Nothing to see here, check out the solution tab.


[tags]Linux YUM, YUM fails, YUM problems, Unix Coding School[/tags]

Linux iptables blocking IP address


You do a netstat -an and see a raft of connections to your apache host, coming in at 10 – 20 a second.

It shows ::ffff: ::ffff: in netstat output. These are IPV6 connections AFAIK.

Also you see ::1 – – [DD/MMM… in apache log. Restarting apache does nothing, the connections just reappear.


You need to block all connections from this IP address using iptables. See example tab.


cp -i /etc/sysconfig/iptabes /etc/sysconfig/iptables.$(date +%j).bu
vi /etc/sysconfig/iptables

Add a line like the one below:


Then restart iptables like this: /etc/init.d/iptables restart as root.

Finally check your rules like this: /etc/init.d/iptables status as root.


[tags]Linux Security, Linux iptables, Fedora Core iptables, Linux Firewall, Unix Coding School[/tags]

UNIX Lesson Six


You want to write elegant code, which integrates well (pipes) with other UNIX commands.


Learn these UNIX Laws, from the inventor of UNIX pipes.


No examples – see the reference tab.


1. Make each program do one thing well. To do a new job, build afresh rather than complicate old programs by adding new features.

2. Expect the output of every program to become the input to another, as yet unknown, program. Don’t clutter output with extraneous information. Avoid stringently columnar or binary input formats. Don’t insist on interactive input.

3. Design and build software, even operating systems, to be tried early, ideally within weeks. Don’t hesitate to throw away the clumsy parts and rebuild them.

4. Use tools in preference to unskilled help to lighten a programming task, even if you have to detour to build the tools and expect to throw some of them out after you’ve finished using them.

Doug McIlroy, the inventor of Unix pipes and one of the founders of the Unix tradition.

[tags]UNIX Law, UNIX pipes, Unix Coding School[/tags]

UNIX Lesson Five


Coding in UNIX is an art form, where certain philosophies prevail.


To really excel at UNIX coding and produce elegant, efficient, low maintenance code – you need to learn UNIX philosophy.


No examples – just see the reference tab.


1. Rule of Modularity: Write simple parts connected by clean interfaces.

2. Rule of Clarity: Clarity is better than cleverness.

3. Rule of Composition: Design programs to be connected to other programs.

4. Rule of Separation: Separate policy from mechanism; separate interfaces from engines.

5. Rule of Simplicity: Design for simplicity; add complexity only where you must.

6. Rule of Parsimony: Write a big program only when it is clear by demonstration that nothing else will do.

7. Rule of Transparency: Design for visibility to make inspection and debugging.easier.

8. Rule of Robustness: Robustness is the child of transparency and simplicity.

9. Rule of Representation: Fold knowledge into data so program logic can be stupid and robust.

10. Rule of Least Surprise: In interface design, always do the least surprising thing.

11. Rule of Silence: When a program has nothing surprising to say, it should say nothing.

12. Rule of Repair: When you must fail, fail noisily and as soon as possible.

13. Rule of Economy: Programmer time is expensive; conserve it in preference to machine time.

14. Rule of Generation: Avoid hand-hacking; write programs to write programs when you can.

15. Rule of Optimization: Prototype before polishing. Get it working before you optimize it.

16. Rule of Diversity: Distrust all claims for “one true way”.

17. Rule of Extensibility: Design for the future, because it will be here sooner than you think.

[tags]UNIX Laws, UNIX Philosophy, Unix Coding School[/tags]

UNIX Lesson One


You are a complete beginner and keen to learn UNIX!


Okay in the examples, I am going to start very basically.

From the outset, you need to understand some basic terms.

  • Firstly – what is UNIX. Unix is an operating system. Other operating systems including Windows, Apple Mac, Novell, etc. It is what turns the computer, from an over-sized paper weight – into a living, breathing machine. πŸ™‚

  • The operating system, talks to all the hardware. Definition of hardware is that if you drop it on your foot, it hurts. πŸ™‚ It talks to the CPU (brain chip) of the computer, the memory (RAM Chips) which is sort of like your desk space, a working area. The bigger your desk space the more you can work on at once. Why size does matter – how big is big. Well currently 256mb (256 megabyte) should be the minimum, ideally 1GB (1 gigabyte) should be plenty.

  • So how do we interface with UNIX? Through a terminal and shell generally. Yes you can use GUIs (Window) – but the real power of UNIX is in the shell.

  • What is a terminal and a shell for that matter? A terminal is just a program that runs, that allows you to connect to UNIX machines – including your own. If you have just installed your own Linux machine, or having to use one at work – you’ll generally start from a windows environment – not necessarily Microsofts either! πŸ™‚ If your are running Linux, generally you can right click your mouse on the background of your desktop and select terminal. From MS Windows there is a terminal program you can run, or ideally look up putty SSH client at the reference tab.

  • more to follow shortly …



[tags]Free Linux Lessons, Lesson One, Unix Coding School[/tags]

Compare command output with file


You want to run a command, save the output to a file – then compare the output at a later date with the file.

For example – you want to update your crontab: save a backup crontab -l > cronfile.$(date +%j), make your updates, then compare it with the backup file.


Use diff with the hypen to represent read from stdin.

This saves keeping multiple copies.


crontab -l > cronfile.$(date +%j)
crontab -e # make your diffs
crontab -l | diff - cronfile.$(date +%j)

Also you can use cmp, which just reports the line number, etc

crontab -l > cronfile.$(date +%j)
crontab -e # make your diffs
crontab -l | cmp - cronfile.$(date +%j)


[tags]crontab, diff, cmp, comparing command output with file, UNIX Coding School[/tags]

Simple Basic UNIX Permission Denied


You are quite happily editing away, then you come to save … oh dear, permission denied.

Either the file is set to readable, or it is owned by another user.

But you have write permission to the directory, either by user or group!


If it is just a simple case of the file being unwritable and you own it, if in vi following the first example.

If you are not the owner, write the file to /tmp. Then follow the second example.


Allow write permission to a file you are currently vi editing – but the file is not writable.

:!chmod u+w %

If you are not the owner, no need to worry, even without root we can change the owner if you have write access to the directory! πŸ™‚

mv -i your_filename /tmp/
cp -i /tmp/ your_filename

Obviously if not in vi, you'll need to break out to a shell using the mechanism in your editor. Also you may not have zsh, so ksh – bash, etc will do.

I tend to do all this on the command line, with a semi-colon between the mv and cp – so I can use command line expansion (the move has not happened yet!).

mv -i thefile[tab] /tmp/thefileXXXX;cp -i /tmp/thefileXXXX . [return]

Network sniffing on Linux


Just what the hell is your apache web server doing!! πŸ™‚

Seeing how clients are connecting to servers and what data is passing between, can prove invaluable in finding the solutions to issues.


Network trace it – showing the headers and content being sent and received.


tcpdump -vvv -A -XX host and port 80

Just replace with your client ip address.

tcpdump -c 50 port 80
tcpdump: listening on eth0
00:43:09.262942 > bree.http: S 2445348839:2445348839(0) win 16384
00:43:09.263069 bree.http > S 3557400364:3557400364(0) ack 2445348840 win 5840 (DF)
00:43:09.265882 > bree.http: . ack 1 win 17520
00:43:09.268591 > bree.http: P 1:493(492) ack 1 win 17520

On AIX you can ordinarly use iptrace or tcpdump.

On Solaris, we simply use snoop.
snoop -x0 src hostname

Where source is the client and hostname is the server. Other options are port portnum, etc. snoop –help or man snoop for more options.


[tags]Network, Tracing, tcpdump, Linux[/tags]

Removing links to missing files


You have many symbolic links that point to missing files.

I guess it is all just about good housekeeping! πŸ™‚ Well and the fact your application might break, depending on something being there, that is not.


This bit of shell under example tab, deals with them swiftly and sharply. πŸ™‚

Effectively it just does a find from the current directory, then loops around looking for missing files.


for na in `find . -type l`
ls -lLd $na 2>&1 | grep "No such file" |
while read junk delme junk
rm ${delme%:}


[tags]symbolic links, find, rm, while, UNIX Coding School[/tags]

compressed using zip gzip bzip2


You have been sent (or want to download) a zip file. But also want to verify the contents, before decompressing.


Under UNIX there are a number of ways to compress files, or deal with those that have already been compressed.

See the examples tab.


  • Firstly dealing with files ending with .gz.

    To view the contents of a gzipped file, just use gzip -l filename.gz

    To unzip the file, use gzip -d filename.gz or gunzip filename.gz

    To view the contents of the gzipped file, without decompressing it: gzip -c -d filename.gz.

    To create a gzip – gzip –best filename – or gzip reads from stdin, in the best of UNIX traditions. πŸ™‚ You can also pipe gzip output or redirect it.

  • Next unzip.

    To view the contents of a zipped file, just use zipinfo

    To unzip the file, use unzip

    To view the contents of the gzipped file, without decompressing it: unzip -c filename.gz.

    To create a zip – zip filename – or zip reads from stdin.

  • Lastly bzip.

    To decompress the file, use bunzip2 filename.bz2.

    To view the contents of the bzipped file, without decompressing it: bzcat filename.bz2.

    To create a zip – bzip2 filename – or zip reads from stdin.


[tags]gzip, zip, bzip2, UNIX Coding School[/tags]

Strip XML comments


You want to strip XML comments out of your XML file.

This should also work with HTML comments.

Very handy for spotting errors, especially in configuration files! πŸ™‚


In this example, I use pure shell commands to identify and remove comments.

Obviously it assumes comments are not embedded, within valid XML, but on their own line.

I’ve done some brief testing, appears to work fine so far.

Have n’t tried nested comments either. πŸ™‚



[ $# -ne 1 ] && { echo "Usage: $0 file.xml"; exit 1; }


[ ! -f $file ] && {
echo "$0: $file is not a file, exiting ..."; exit 1


cat $file | while read line
[[ $(echo "$line" | grep -c -- '<!--') -eq 1 ]]
&& { flag=1 }

[[ $(echo "$line" | grep -c -- '-->') -eq 1 ]]
&& { flag=2 }

[ $flag -eq 0 ] && { echo "$line" } ||
[ $flag -eq 2 ] && { flag=0 }


exit 0

Drop me a comment, with your experiences of it.


[tags]Strip XML Comments, UNIX Coding School[/tags]

catch unix process – ps


You want to catch a process, but they just shoot by too quickly.

Every time you run a ps, you cannot spot that specific command.

You need a capture script.


Run the code at the example and it will catch changing and new processes. To be more aggressive, just change “sleep 5” to “sleep 1”.



ps –fu $user > catchdiff

while :


ps -fu $user | diff - catchdiff | egrep -v 'catchdiff|-bash$|^---|^[0-9]'

sleep 5



[tags]ps, diff, egrep, while, process[/tags]

Remove content from gzips


You need to remove content from zip files.

Effectively unzip, remove the pattern and re-zip.


A collection of shell commands, to perform the unzip, swap and rezip.


* First to find zips with those patterns in (substitute pattA and B with your patterns, on each of the commands below):

for na in *gz
[[ $(gzip -c -d $na | egrep -ic 'pattA|pattB') -ne 0 ]] && {
echo $na
done > /tmp/filelist.out

* Then to test removing the offending lines:

for na in $(</tmp/filelist.out)
gzip -c -d $na | egrep -iv 'pattA|pattB' |
gzip --best > /tmp/replace.gz
echo going to mv /tmp/replace.gz $na
ls -ld /tmp/replace.gz $na


* Now to the actual move:

for na in $(</tmp/filelist.out)
gzip -c -d $na | egrep -iv 'pattA|pattB' |
gzip --best > /tmp/replace.gz
mv /tmp/replace.gz $na


* Then a retest:

for na in $(</tmp/filelist.out)
gzip -c -d $na | egrep -i 'pattA|pattB' |
sed "s/^/$na: /"



[tags]gzip, sed, Unix Coding School[/tags]

Cut and paste a zip file? You bet!


You have a shell open on two UNIX boxes, but scp, ftp, etc, etc is locked down?

Well they left Perl!!! πŸ™‚


I recognise this could be messy for massive files – but it works beautifully on binaries, images, etc.

I am demo’ing it here on tar gzipped output. It produces some errors, but still recreates the files.


  • First to prove the files with checksums (11171, 28913)

  • $ find test -type f -exec sum {} ; -ls
    11171 18
    294632 20 -rwxr--r-- 1 marcus marcus 17461 Feb 15 21:49 test/testA/mun-weir.jpg
    28913 19
    294633 20 -rw-r--r-- 1 marcus marcus 18505 Feb 15 21:49 test/testB/perth-swan-river.jpg

  • Now to generate the text for copying – using base64 encoding to convert binary to text

  • tar zcvf - test | perl -MMIME::Base64 -e 'while() {
    $str.=(encode_base64($_)); } print "$strn";'

    This produces a listing of the files/dirs and then starting from H4, the actual text version of the binary output. I have truncated this – as it gets quite big. You can redirect this into a file, if using cygwin – then just open it with wordpad.


  • Now to extract it
  • First we paste it into tar.uu – which could be anything or you can just do cat | perl and paste to stdin.

    [[email protected] ~]$ cat tar.uu | perl -MMIME::Base64 -e 'while() { $str.=(decode_base64($_)); } print "$strn";' | tar zxvf -

    gzip: stdin: decompression OK, trailing garbage ignored
    tar: Child returned status 2
    tar: Error exit delayed from previous errors

    As you can see produces a bit of garbage.

  • Prove the files are reproduced – with sums (11171, 28913)

  • [[email protected] ~]$ find test -type f -exec sum {} ; -ls
    28913 19
    1555 24 -rw-r--r-- 1 marcus adm 18505 Feb 15 21:49 test/testB/perth-swan-river.jpg
    11171 18
    1553 24 -rwxr--r-- 1 marcus adm 17461 Feb 15 21:49 test/testA/mun-weir.jpg

    And you are done. πŸ™‚


    [tags]cut and paste binary files, base64, Unix Coding School[/tags]

    Optical dump file – od


    Whenever you cat a file, it shows nothing!

    Or it destroys your terminal. πŸ™‚

    … or when you diff to files – the look identical – but it reports them as not.


    You probably have control characters in the file, or no newline at the end of the file.

    The example shows how to run an optical dump on a file.


    # optical dump of a file, useful for seeing control chars, etc and lines with no newline.

    od -c filename


    [tags]Optical dump, od, Unix Coding School[/tags]

    Find secrets – undocumented feature


    You want to find a pattern in a files, but have the file name and pattern printed.

    If you run find . -type f -exec grep -i “pattern” {} ; it will only show the pattern turning up – i.e not the file name as well.

    If you run find . -type f -exec grep -il “pattern” {} ; it will only show the file name turning up – i.e not the line containing the pattern as well! πŸ™‚


    Maybe you have seen this solution before, but AFAIK it is completely undocumented.

    I discovered it whilst trying to do this very thing and I just thought, hmmm … I wonder. πŸ™‚


    To force grep to print the line containing the pattern and the file name, we pass /dev/null as another argument. Find sees this as multiple arguments, so prints the file name and line containing pattern.

    • The normal basic find

    • $ find . -type f -exec grep -i author {} ;

      <ADDRESS CLASS="doc-author">The OpenLDAP Project <<A target=_blank HREF=""></A>></ADDRESS>
      <META name="Author" content="TechRock">
      <META HTTP-EQUIV="AUTHOR" content="TechRock">

    • find looking for just the file name

    • $ find . -type f -exec grep -il author {} ;


    • Undocumented feature, print file name and line content

    • $ find . -type f -exec grep -i author {} /dev/null ;

      ./cs/<ADDRESS CLASS="doc-author">The OpenLDAP Project <<A target=_blank HREF=""></A>></ADDRESS>
      ./<META name="Author" content="TechRock">
      ./<META HTTP-EQUIV="AUTHOR" content="TechRock">


    [tags]unix find, find undocumented features, find secrets, Unix Coding School[/tags]

    Synchronize directories between firewalled hosts


    You have an admin host (ADMIN), which can see two servers (hostA and hostB) behind firewalls.

    You are not allowed to scp/rdist, etc from hostA or B to ADMIN. Nor are you allowed to scp/rdist, etc between hostA and B.

    ADMIN can scp, etc though to both hosts. Now in this real life example, updates are made directly to hostA – but not hostB. So they need to be synchronized somehow, looking at the real files and performing a pull to ADMIN from hostA and then a push to hostB.

    But then we only want new files …


    In this script at the example, I pull from hostA and push to hostB. But only “look” for files that have updated, since last time I looked. πŸ™‚


    It starts off just setting variables, creating temporary directories on ADMIN host.

    Then at line 26 I obtain a list of files from hostA, that have been updated since last time I ran this script.

    At line 29 I spin around this list and copy the given file to the temp directory (line 34). Notice I use ${file##*/} this strips off the path – so I just get the file (i.e. bob.gif not /dirABCD/…./bob.gif).

    I then copy this file (line 36) back out to the full path on hostB. Build a test and check the file exists on hostB – lines 38 – 42. Then remove the temporary copy of the file (line 44).

    Spin back round for the next file at line 45. Lastly cleaning up.

    Whilst this is not as efficient as rdist (opening one ssh connection), it will only copy updated files. So not a tar of the whole directory, etc. Plus by pepper potting between hostA and hostB – there is a few seconds delay between each file update – so should n't overload either host.

    7:dtg=$(perl -M'English' -e 'print $BASETIME."n";')
    11:[ -d $tempdir ] && {
    12: echo "$0: $tempdir already exists - exiting ..."
    13: exit 1
    16:mkdir $tempdir || {
    17: echo "$0: mkdir $tempdir failed - exiting ..."
    18: exit 1
    21:cd $tempdir || {
    22: echo "$0: $tempdir not accessible - exiting ..."
    23: exit 1
    26:/bin/ssh hostA /bin/find $basedir -type f -a
    -newer $touchfile > $filelist 2> /dev/null
    27:#/bin/ssh hostA /bin/find $basedir -type f -a
    -mtime -1 > $filelist 2> /dev/null
    29:for file in $(<$filelist)
    31: echo "copying $file"
    32: echo "copying $file" >> $report
    34: /bin/scp hostA:$file . 2> /dev/null
    36: /bin/scp ${file##*/} hostB:$file 2> /dev/null
    38: check=$(/bin/ssh hostB /bin/ls -ld "${file}" 2>&1)
    40: [[ $(echo $check | grep -c "No such") -gt 0 ]] && {
    41: echo "$0:$dtg:$file failed" >> $report
    42: }
    44: /bin/rm ${file##*/}
    47:/bin/ssh hostA /bin/touch $touchfile 2> /dev/null
    49:/bin/rm $filelist
    51:cd /tmp
    53:rmdir $tempdir
    55:exit 0

    See reference to download this code.

    UNIX chat shell code


    As most UNIX bods know, with fairly recent audit lock downs – talkd has been disabled.

    But you want to chat remotely with someone, i.e. another techie bod! πŸ™‚


    Now where is the fun in that! πŸ™‚

    Don’t know about you, but quite often I’ve needed to talk to other people remotely and don’t want to hold a phone or use hands free.

    See the example.


    Here is a quick and dirty solution, that works with bash – should work with zsh too.

    I welcome any enhancements or suggestions, via the comments.


    trap 'kill $tailpid;echo "*$nick leaves chat*" >> ${chatdir}/discussion;
    rm ${chatdir}/.${nick};exit 0' 2


    [ -d $chatdir ] || {
    mkdir $chatdir || (echo "failed to mkdir $chatdir";exit 1)

    chmod 777 /var/tmp/unixchat

    while [ $(echo X${nick} | grep -c "^X$") -eq 1 ]
    echo -n "Nick: "
    read nick

    [ -f "${chatdir}/.${nick}" ] && {
    echo "$nick already exists, try another";nick=""


    touch ${chatdir}/.${nick}

    [ ! -f ${chatdir}/discussion ] && {
    touch ${chatdir}/discussion
    chmod 666 ${chatdir}/discussion

    echo "*$nick joins chat*" >> ${chatdir}/discussion

    tail -1000f ${chatdir}/discussion&

    echo ">> type quit to exit <<"

    while [ $(echo X${entry} | grep -c "^Xquit$") -ne 1 ]
    read entry
    echo "[${nick}@`date '+%d-%b %H:%M:%S'`] says: $entry" >> ${chatdir}/discussion&


    kill $tailpid; rm ${chatdir}/.${nick}

    echo "*$nick leaves chat*" >> ${chatdir}/discussion

    exit 0

    Here is a screen shot:

    UNIX chat code - simple solution written in shell

    extract specific line number from file using sed


    You want to extract a specific line number from a file.

    Or a range of numbers from a file.


    Once you have your line number – see grep, we can extract lines around the pattern.

    To do this sed (stream editor) can be used to print just desired lines – see the example which says don’t print all lines (-n); start at line 456 and finish at line 466 – print:


    This says don't print all lines (-n); start at line 456 and finish at line 466 – print:

    sed -n 456,466p filename

    Also with sed, we can say delete specific lines – in this case remove lines 5 to 10:

    sed 5,10d filename

    That's not all sed can accept patterns, as start/end identifiers:

    sed /start_pattern/,/end_pattern/d filename


    [tags]UNIX sed, UNIX, sed, UNIX Coding School[/tags]

    Debugging Sendmail


    Issues with sendmail, tacking on another domain name, etc.

    You want to manually test you mail host, from the command line.


    Whilst debugging sendmail issues, came across this great test.

    You can run this from the command line. See example.


    telnet mailrelay.domain 25

    .... answer from mail relay .....

    helo unix hostname
    mail from: [email protected] hostname
    rcpt to: email [email protected] out

    mail test from unix


    [tags]sendmail debugging, debugging sendmail, sendmail, debugging, UNIX Coding School[/tags]