Coding School

[problem]

You want to run a command, save the output to a file – then compare the output at a later date with the file.

For example – you want to update your crontab: save a backup crontab -l > cronfile.$(date +%j), make your updates, then compare it with the backup file.

[/problem]

[solution]

Use diff with the hypen to represent read from stdin.

This saves keeping multiple copies.

[/solution]

[example]


crontab -l > cronfile.$(date +%j)
crontab -e # make your diffs
crontab -l | diff - cronfile.$(date +%j)


Also you can use cmp, which just reports the line number, etc


crontab -l > cronfile.$(date +%j)
crontab -e # make your diffs
crontab -l | cmp - cronfile.$(date +%j)


[/example]

[reference]

[tags]crontab, diff, cmp, comparing command output with file, UNIX Coding School[/tags]

• Linux Man Pages – crontab command
• Linux Man Pages – diff command
• Linux Man Pages – cmp command

[/reference]

[problem]

You are quite happily editing away, then you come to save … oh dear, permission denied.

Either the file is set to readable, or it is owned by another user.

But you have write permission to the directory, either by user or group!

[/problem]

[solution]

If it is just a simple case of the file being unwritable and you own it, if in vi following the first example.

If you are not the owner, write the file to /tmp. Then follow the second example.

[/solution]

[example]

Allow write permission to a file you are currently vi editing – but the file is not writable.

:!chmod u+w %

If you are not the owner, no need to worry, even without root we can change the owner if you have write access to the directory! 🙂


:!zsh
mv -i your_filename /tmp/your_filename.mv
cp -i /tmp/your_filename.mv your_filename


Obviously if not in vi, you’ll need to break out to a shell using the mechanism in your editor. Also you may not have zsh, so ksh – bash, etc will do.

I tend to do all this on the command line, with a semi-colon between the mv and cp – so I can use command line expansion (the move has not happened yet!).

mv -i thefile[tab] /tmp/thefileXXXX;cp -i /tmp/thefileXXXX . [return]

[/example]

[reference]

[tags]Unix permission denied, chown owner without root, Unix Coding School[/tags]

• Linux Man Pages – vi command
• Linux Man Pages – chmod command
• Linux Man Pages – chown command
• Linux Man Pages – mv command
• Linux Man Pages – cp command

[/reference]

[problem]

Just what the hell is your apache web server doing!! 🙂

Seeing how clients are connecting to servers and what data is passing between, can prove invaluable in finding the solutions to issues.

[/problem]

[solution]

Network trace it – showing the headers and content being sent and received.

[/solution]

[example]


tcpdump -vvv -A -XX host 10.0.0.8 and port 80


Just replace 10.0.0.8 with your client ip address.

tcpdump -c 50 port 80
tcpdump: listening on eth0
00:43:09.262942 10.0.0.8.1288 > bree.http: S 2445348839:2445348839(0) win 16384
00:43:09.263069 bree.http > 10.0.0.8.1288: S 3557400364:3557400364(0) ack 2445348840 win 5840 (DF)
00:43:09.265882 10.0.0.8.1288 > bree.http: . ack 1 win 17520
00:43:09.268591 10.0.0.8.1288 > bree.http: P 1:493(492) ack 1 win 17520
On AIX you can ordinarly use iptrace or tcpdump.

On Solaris, we simply use snoop.
snoop -x0 src hostname

Where source is the client and hostname is the server. Other options are port portnum, etc. snoop –help or man snoop for more options.

[/example]

[reference]

[tags]Network, Tracing, tcpdump, Linux[/tags]

• Linux Man Page – tcpdump

[/reference]

[problem]

You have many symbolic links that point to missing files.

I guess it is all just about good housekeeping! 🙂 Well and the fact your application might break, depending on something being there, that is not.

[/problem]

[solution]

This bit of shell under example tab, deals with them swiftly and sharply. 🙂

Effectively it just does a find from the current directory, then loops around looking for missing files.

[/solution]

[example]


for na in `find . -type l`
do
ls -lLd $na 2>&1 | grep "No such file" |
while read junk delme junk
do
rm ${delme%:}
done
done


[/example]

[reference]

[tags]symbolic links, find, rm, while, UNIX Coding School[/tags]

• Linux Man Pages – find
• Linux Man Pages – while
• Linux Man Pages – rm

[/reference]

[problem]

You have been sent (or want to download) a zip file. But also want to verify the contents, before decompressing.

[/problem]

[solution]

Under UNIX there are a number of ways to compress files, or deal with those that have already been compressed.

See the examples tab.

[/solution]

[example]

• Firstly dealing with files ending with .gz.

  To view the contents of a gzipped file, just use gzip -l filename.gz

  To unzip the file, use gzip -d filename.gz or gunzip filename.gz

  To view the contents of the gzipped file, without decompressing it: gzip -c -d filename.gz.

  To create a gzip – gzip –best filename – or gzip reads from stdin, in the best of UNIX traditions. 🙂 You can also pipe gzip output or redirect it.

• Next unzip.

  To view the contents of a zipped file, just use zipinfo filename.zip

  To unzip the file, use unzip filename.zip.

  To view the contents of the gzipped file, without decompressing it: unzip -c filename.gz.

  To create a zip – zip filename – or zip reads from stdin.

• Lastly bzip.

  To decompress the file, use bunzip2 filename.bz2.

  To view the contents of the bzipped file, without decompressing it: bzcat filename.bz2.

  To create a zip – bzip2 filename – or zip reads from stdin.

[/solution]

[reference]

[tags]gzip, zip, bzip2, UNIX Coding School[/tags]

• Linux Man Page – gzip
• Linux Man Page – zip
• Linux Man Page – bzip2

[/reference]

[problem]

You want to strip XML comments out of your XML file.

This should also work with HTML comments.

Very handy for spotting errors, especially in configuration files! 🙂

[/problem]

[solution]

In this example, I use pure shell commands to identify and remove comments.

Obviously it assumes comments are not embedded, within valid XML, but on their own line.

I’ve done some brief testing, appears to work fine so far.

Have n’t tried nested comments either. 🙂

[/solution]

[example]


#!/bin/bash

[ $# -ne 1 ] && { echo "Usage: $0 file.xml"; exit 1; }

file=$1

[ ! -f $file ] && {
echo "$0: $file is not a file, exiting ..."; exit 1
}

flag=0

cat $file | while read line
do
[[ $(echo "$line" | grep -c -- '<!--') -eq 1 ]]
&& { flag=1 }

[[ $(echo "$line" | grep -c -- '-->') -eq 1 ]]
&& { flag=2 }

[ $flag -eq 0 ] && { echo "$line" } ||
[ $flag -eq 2 ] && { flag=0 }

done

exit 0


Drop me a comment, with your experiences of it.

[/example]

[reference]

[tags]Strip XML Comments, UNIX Coding School[/tags]

• Linux Man Pages – bash
• Linux Man Pages – cat command

[/reference]

[problem]

You want to catch a process, but they just shoot by too quickly.

Every time you run a ps, you cannot spot that specific command.

You need a capture script.

[/problem]

[solution]

Run the code at the example and it will catch changing and new processes. To be more aggressive, just change “sleep 5” to “sleep 1”.

[/solution]

[example]

user=”marcus”

ps –fu $user > catchdiff

while :

do

ps -fu $user | diff - catchdiff | egrep -v 'catchdiff|-bash$|^---|^[0-9]'

sleep 5

done

[/example]

[reference]

[tags]ps, diff, egrep, while, process[/tags]

• ps
• diff
• egrep

[/reference]

[problem]

You need to remove content from zip files.

Effectively unzip, remove the pattern and re-zip.

[/problem]

[solution]

A collection of shell commands, to perform the unzip, swap and rezip.

[/solution]

[example]

* First to find zips with those patterns in (substitute pattA and B with your patterns, on each of the commands below):


for na in *gz
do
[[ $(gzip -c -d $na | egrep -ic 'pattA|pattB') -ne 0 ]] && {
echo $na
}
done > /tmp/filelist.out


* Then to test removing the offending lines:


for na in $(</tmp/filelist.out)
do
gzip -c -d $na | egrep -iv 'pattA|pattB' |
gzip --best > /tmp/replace.gz
echo going to mv /tmp/replace.gz $na
ls -ld /tmp/replace.gz $na

done


* Now to the actual move:


for na in $(</tmp/filelist.out)
do
gzip -c -d $na | egrep -iv 'pattA|pattB' |
gzip --best > /tmp/replace.gz
mv /tmp/replace.gz $na

done


* Then a retest:


for na in $(</tmp/filelist.out)
do
gzip -c -d $na | egrep -i 'pattA|pattB' |
sed "s/^/$na: /"

done


[/example]

[reference]

[tags]gzip, sed, Unix Coding School[/tags]

• gzip
• sed

[/reference]