Category: unix

[problem]

1. You want to compare 2 streams, with byte offset and line numbers?
2. Pull off content with a byte offset?
3. Display lines before or after a pattern?
4. Send an email – upon job failure?

[/problem]

[solution]

The shell is a powerful beast! Anything is possible … with a little know-how. 🙂 See the examples.

[/solution]

[example]

1. md5sum TARBALL | awk '{print $1;}' | cmp - TARBALL.md5
   # cmp good for comparing output with file contents, or diffing files. Outputs byte offset and line number. md5sum creates a hash (checksum) of file.
2. dd bs=1 skip=106 if=infile
   # disk to disk copy, example outputs infile with offset of 106 bytes (block size of 1 byte, skip 106 bytes).
3. Ever needed to display a line before or after a pattern. Yep sure you can use Perl, but this is a bit simplier to work out.

   See the code here

   See a run through screen shot here

   Usage – ppgrep #1 #2 pattern file – where #1 is the number of lines prior to the pattern (could be 0) and #2 is number of lines post pattern.

4. Sending email upon UNIX cron failure

[/example]

[reference]

[tags]dd, cmp, md5sum, grep, sed, Unix Coding School[/tags]

• Linux Man Pages – dd command
• Linux Man Pages – cmp command
• Linux Man Pages – md5sum command
• Linux Man Pages – grep command
• Linux Man Pages – sed command
• Linux Man Pages – crontab command

[/reference]

[problem]

1. You want to generate a self-signed certificate?
2. Encrypt a message with triple des
3. View base 64 encoded data
4. Generate and use random data
5. Test ciphers
6. Monitor certificate expiry dates

[/problem]

[solution]

openssl to the rescue! 🙂 See examples.

[/solution]

[example]

1. openssl code to generate self-signed cert
2. encrypt with tripledes
3. openssl base64 [ -d ]
   # encrypted with base64.
4. Useful fortransfering control/binary content – like this:

   dd bs=1 count=512 if=/dev/urandom | openssl base64

   Also useful for reading mail

   openssl base64 -d /var/spool/mqueue/qfk3GAlBMn018552

5. Using openssl s_client to test ciphers supported, with automated script – then loop sites and email.
6. Using openssl s_client to extract expiry dates, with automated script – then loop sites and email.

[/example]

[reference]

[tags]openSSL, dd, UNIX, SSL Certificates, PKI, Encryption, Generate Random Data, Unix Coding School[/tags]

• [ Linux Man Pages – openSSL ]
• [ Linux Man Pages – dd ]

[/reference]

[problem]

You want to send an attachment, from UNIX command line or from within a shell script

[/problem]

[solution]

uuencode works a treat with Solaris. Not sure on RH Linux, etc.

[/solution]

[example]


cat filename | uuencode wat_u_want_attatch_2b_called.ext | mail -s "this is the subject and here is wat u want attach 2b called" email_addr


Simple as that, it will send the filename as an attachment to a blank email, with your subject.

Here is a demo:

cat index.php | uuencode index.php | mail -s "test uuencode" [email protected]

To do multiple attachments, simply cat files – then uuencode them into a temp file. Like this:


cat file1.doc | uuencode attach1.doc > /tmp/send1.doc

Repeat this for say file2 into attach2 and send2. Then to send them all do this:

cat /tmp/send[0-9].doc | mail -s "your subj" youraddr

[/example]

[reference]

[tags]uuencode, Unix Coding School[/tags]

• Linux Man Pages – uuencode command
• Linux Man Pages – mail command
• Linux Man Pages – cat command

[/reference]

[problem]

You want to make some backups with tar.

[/problem]

[solution]

tar is very functional and has many uses.

[/solution]

[example]

If you are lucky enough to have GNU tar (tar –version will return GNU if so), then you can compress and backup at the same time. 🙂

tar zcvf /tmp/filename.tgz .

This backups and zips the contents of the current directory, to filename.tgz in tmp directory.

If not – no drama. This will backup and pipe through content to gzip, which then zips it.

tar cvf - . | gzip --best > /tmp/filename.tgz

Recreate a directory:

mkdir new

tar -C old -cf - . | tar -C new -xvf

Restore over the network:

ssh remote_hostname "(tar -cf - remote_directory)" | tar -xvf -

For example:

ssh bree "(tar -cf - Perl_Bin)" | tar -xvf -

[/example]

[reference]

[tags]tar, backups, restores, Unix Coding School[/tags]

• Linux Man Pages – tar command
• Linux Man Pages – ssh command
• Linux Man Pages – gzip command

[/reference]

[problem]

You want to forward all traffic from a port on your box, to another host/port combination.

[/problem]

[solution]

The command in the example will port forward, from the local box – any traffic sent to port 8081, will be sent to port 8080 on www.example.com:

[/solution]

[example]

ssh -x -g -L 8081:www.example.com:8080 localhost

Don’t forget, if you need to open LINUX (FC4) firewall, do this (as root):

vi /etc/sysconfig/iptables # add following line then save, exit
-A RH-Firewall-1-INPUT -m state --state NEW -m tcp -p tcp --dport 8081 -j ACCEPT
/etc/init.d/iptables restart

[/example]

[reference]

[tags]Port forwarding, SSH, Unix Coding School[/tags]

• Linux Man Pages – crontab command
• Linux Man Pages – ssh command
• Linux Man Pages – vim command
• Linux Man Pages – iptables command

[/reference]

[problem]

You want to receive an email upon cron job failure.

[/problem]

[solution]

The example code can be wrapped around a normal cronjob, to send an email if the cron fails.

[/solution]

[example]

Add to beginning:

bash -c '( msg=`

Add to the end:

2>&1 > /tmp/logfail.log`; if [ ! -z "$msg" ] ; then echo "$msg" |

/bin/mail -s "`/bin/uname -n`:$LOGNAME:`/bin/date`: YOUR MESSAGE" YOUR_MAIL_LIST; fi )'

Demo

# # # # # bash -c '( msg=`YOUR COMMAND 2>&1 > /dev/null`; if [ ! -z "$msg" ] ; then echo "$msg" |/bin/mail -s "`/bin/uname -n`:$LOGNAME:`/bin/date`: YOUR MESSAGE" YOUR_MAIL_LIST; fi )'

Example of generating an email upon cron failure

Another example

[/example]

[reference]

[tags]bash, cron email, Unix Coding School[/tags]

• Linux Man Pages – crontab command
• Linux Man Pages – bash command
• Linux Man Pages – mail command
• Linux Man Pages – uname command
• Linux Man Pages – date command

[/reference]

[problem]

You want to display line lengths – then average line length of file.

[/problem]

[solution]

Snip using awk – see example.

[/solution]

[example]


awk ' { thislen=length($0); printf("%-5s %d\n", NR, thislen); totlen+=thislen}
END { printf("average: %d\n", totlen/NR); } ' filename

[/example]

[reference]

gawk, Unix Coding School

• Linux Man Pages – gawk command

[/reference]

[problem]

Had to find out how to allow a range with iptables recently, whilst setting up Samba and NFS.

[/problem]

[solution]

Always take a backup first, then vi /etc/sysconfig/iptables. In the example I am allowing access to all ports between 600 and 699 – for udp and tcp traffic.

[/solution]

[example]

# grep 600 /etc/sysconfig/iptables
-A RH-Firewall-1-INPUT -p tcp -m state --state NEW -m tcp --dport 600:699 -j ACCEPT
-A RH-Firewall-1-INPUT -p udp -m state --state NEW -m udp --dport 600:699 -j ACCEPT
Then run /etc/init.d/iptables restart

You can test if this connection is now viable, by using my Perl port testing code under Network connections – Perl Coding School

Other gotchas with samba and nfs, was fiddling with selinux and ensuring portmap was running, etc. Happy to explain further, feel free to post a comment with your questions.

[/example]

[reference]

[tags]iptables, Unix Coding School[/tags]

• Linux Man Pages – iptables command
• Linux Man Pages – vim command

[/reference]

[problem]

You want to encrypt some text, using the tripleDES Cipher.

[/problem]

[solution]

Openssl is a beautiful command, that performs many functions. See example.

[/solution]

[example]

Openssl tripleDES encrypt command:
openssl des3 -salt -in file_to_encrypt -pass pass:_your_password_

Place contents to encrypt in the file reference by file_to_encrypt

Replace your_password with your secret

Openssl will output to stdout – so best to capture like this:
myvar=$(the openssl command)
Openssl tripleDES decrypt command:
openssl des3 -d -salt -in file_to_encrypt -pass pass:_your_password_
Put cipher text to decrypt in filename supplied to -in

Replace _your_password_ with the secret

Outputs to stdout
See the full demo attached here for a blow by blow:

[ How to encrypt with openssl and tripleDES ]

[/example]

[reference]

[tags]openssl, encryption, tripleDES, passwords[/tags]

• OpenSSL Linux Man Page

[/reference]

[problem]

You want to generate a self-signed certificate, for use with a web server.

This will allow you to communicate with your web server over HTTPS, effectively encrypting your traffic.

[/problem]

[solution]

The very first time – you need to set up your own CA (certifying authority). Do this one time only!

[ See a run through screen shot here ] [ See the code here ]

Then create a certificate request – which can be sent off to Verisign, etc or self-signed.

[/solution]

[example]

To create a certificate request:

openssl req -new -days 730 -keyout keyna.pem -out keyna.pem

[ See a run through screen shot here ]

Self-sign the request – only do this if you are not sending off to a CA such as Verisign.

openssl ca -policy policy_anything -days 730 -out certna.pem -infiles keyna.pem

[ See a run through screen shot here ]

If you get ‘unable to write random state’, try this: look for command ssh-rand-helper usually under ssh install directories (or if ~/.ssh/.prng_seed exists – you can just link to it or copy it to $HOME).

ssh-rand-helper -b 1024 > $HOME/.prng_seed

To strip the password out, for restarts:
openssl rsa -in keyna.key -out keyna.key.unsecure

[/example]

[reference]

[tags]openssl, encryption, tripleDES, passwords, Unix Coding School[/tags]

• OpenSSL Linux Man Page
• Mark Foster’s cool openssl tips
• Paul Heinlein’s excellent and comprehensive openssl tips

[/reference]