By Mark Paget
[problem]
You want to display line lengths – then average line length of file.
[/problem]
[solution]
Snip using awk – see example.
[/solution]
[example]
awk ' { thislen=length($0); printf("%-5s %d\n", NR, thislen); totlen+=thislen}
END { printf("average: %d\n", totlen/NR); } ' filename
[/example]
[reference]
gawk, Unix Coding School
[/reference]
[problem]
Had to find out how to allow a range with iptables recently, whilst setting up Samba and NFS.
[/problem]
[solution]
Always take a backup first, then vi /etc/sysconfig/iptables. In the example I am allowing access to all ports between 600 and 699 – for udp and tcp traffic.
[/solution]
[example]
# grep 600 /etc/sysconfig/iptables
-A RH-Firewall-1-INPUT -p tcp -m state --state NEW -m tcp --dport 600:699 -j ACCEPT
-A RH-Firewall-1-INPUT -p udp -m state --state NEW -m udp --dport 600:699 -j ACCEPT
Then run /etc/init.d/iptables restart
You can test if this connection is now viable, by using my Perl port testing code under Network connections – Perl Coding School
Other gotchas with samba and nfs, was fiddling with selinux and ensuring portmap was running, etc. Happy to explain further, feel free to post a comment with your questions.
[/example]
[reference]
[tags]iptables, Unix Coding School[/tags]
[/reference]
[problem]
You want to encrypt some text, using the tripleDES Cipher.
[/problem]
[solution]
Openssl is a beautiful command, that performs many functions. See example.
[/solution]
[example]
Openssl tripleDES encrypt command:
openssl des3 -salt -in file_to_encrypt -pass pass:_your_password_
Place contents to encrypt in the file reference by file_to_encrypt
Replace your_password with your secret
Openssl will output to stdout – so best to capture like this:
myvar=$(the openssl command)
Openssl tripleDES decrypt command:
openssl des3 -d -salt -in file_to_encrypt -pass pass:_your_password_
Put cipher text to decrypt in filename supplied to -in
Replace _your_password_ with the secret
Outputs to stdout
See the full demo attached here for a blow by blow:
[ How to encrypt with openssl and tripleDES ]
[/example]
[reference]
[tags]openssl, encryption, tripleDES, passwords[/tags]
[/reference]
[problem]
You want to generate a self-signed certificate, for use with a web server.
This will allow you to communicate with your web server over HTTPS, effectively encrypting your traffic.
[/problem]
[solution]
The very first time – you need to set up your own CA (certifying authority). Do this one time only!
[ See a run through screen shot here ] [ See the code here ]
Then create a certificate request – which can be sent off to Verisign, etc or self-signed.
[/solution]
[example]
To create a certificate request:
openssl req -new -days 730 -keyout keyna.pem -out keyna.pem
[ See a run through screen shot here ]
Self-sign the request – only do this if you are not sending off to a CA such as Verisign.
openssl ca -policy policy_anything -days 730 -out certna.pem -infiles keyna.pem
[ See a run through screen shot here ]
If you get ‘unable to write random state’, try this: look for command ssh-rand-helper usually under ssh install directories (or if ~/.ssh/.prng_seed exists – you can just link to it or copy it to $HOME).
ssh-rand-helper -b 1024 > $HOME/.prng_seed
To strip the password out, for restarts:
openssl rsa -in keyna.key -out keyna.key.unsecure
[/example]
[reference]
[tags]openssl, encryption, tripleDES, passwords, Unix Coding School[/tags]
[/reference]
[problem]
You want to be sure that your web site is only serving 128 bits!
[/problem]
[solution]
Use openssl s_client – with specific ciphers – see the example.
[/solution]
[example]
The following script spins around 40, 56 and 64 bit ciphers – to check the lowest a site allows.
Ideally none of these ciphers should be allowed (check out www.modssl.org for apache config – look for ciphersuite).
openssl_cipher_check will produce the cipher details from the cipher codes returned in connection output.
If you need proxy support with s_client comment on this thread.
#!/bin/bash
[ $# -lt 1 ] && {
echo "$0: site:port"
echo "Usage: $0 www.example.com:443"
exit 0
}
fortyciphers='EXP-EDH-RSA-DES-CBC-SHA:EXP-EDH-DSS-DES-CBC-SHA:EXP-DES-CBC-SHA:EXP-RC2-CBC-MD5:EXP-RC2-CBC-MD5:EXP-RC4-MD5:EXP-RC4-MD5'
fiftysixciphers='EXP1024-DHE-DSS-DES-CBC-SHA:EXP1024-DES-CBC-SHA:EXP1024-RC2-CBC-MD5:EDH-RSA-DES-CBC-SHA:EDH-DSS-DES-CBC-SHA:DES-CBC-SHA:DES-CBC-MD5:EXP1024-DHE-DSS-RC4-SHA:EXP1024-RC4-SHA:EXP1024-RC4-MD5'
sixtyfourciphers='RC4-64-MD5'
grabCipher() {
cipher=$1
site=$2
echo "
GET / HTTP/1.0
EOT
" | openssl s_client -connect $site -cipher $cipher
}
site=$1
echo "$site"
echo $site | sed 's/./-/g'
for na in forty fiftysix sixtyfour
do
eval ciphers="$${na}ciphers"
thiscipher=`grabCipher $ciphers $site 2>&1 | awk ' /Cipher/ { print $NF } '`
[[ $(echo $thiscipher | grep -c "^$") -ne 1 ]] && {
openssl ciphers -v $thiscipher
exit 0
}
done
echo "No 40, 56 or 64 bit ciphers supported"
exit 0
[/example]
[reference]
[tags]Ciphers, 128 bits, openssl s_client, Unix Coding School[/tags]
[/reference]
[problem]
You want to automate checking expiry of SSL (HTTPS) Certificate expiry.
[/problem]
[solution]
The link to the script is displayed in examples. It connects to the given site and shows site expiry. This can then be automated into a database, using some of my PHP or Perl Scripts, which can also be used to then report on sites about to expire.
[/solution]
[example]
./openssl_cert_expiry_check www.example.com:443
Here is the code – but take note you may need the openssl client that supports proxying. Leave me a comment if you want this code.
#!/bin/bash
[ $# -ne 1 ] &echo -n "$1 - "
echo "
GET / HTTP/1.0
EOT
" | openssl s_client -connect $1 2>&1 |
sed -n '/-----BEGIN CERTIFICATE-----/,/-----END CERTIFICATE-----/p' |
openssl x509 -enddate |
awk -F= ' /notAfter/ { printf("Expires: %sn",$NF); } '
exit 0
[/example]
[reference]
[tags]openssl, SSL Certification Expiry Date Checker, Unix Coding School[/tags]
– Mark Foster’s cool openssl tips – Paul Heinlein’s excellent and comprehensive openssl tips[/reference]
[problem]
You want to compare two cacerts java certificate stores.
Say comparing development server against prod, to ensure they both have same certs stored in cacerts (sampled by java among others).
[/problem]
[solution]
This can come in very handy, if you need to compare 2 different projects or envs. See the example.
[/solution]
[example]
Code to generate list of certs, along with there alias, entry, owner and valid dates.
Start in the directory where your cacerts file resides.
../../../bin/keytool -list -v -keystore cacerts -storepass changeit | gawk ' {
if(/Alias name/) {
if(alias != /^$/) { printf("%s: [%s] %s [from: %s]n",alias,entry,owner,valid); }
alias=$NF;
}
if(/Entry type/) { entry=$NF; }
if(/Owner/) { owner=$2" "$3; }
if(/Valid/) { valid=$5"/"$4"/"$8; }
} END { printf("%s: [%s] %s [from: %s]n",alias,entry,owner,valid); }
' | sort
You may need to change gawk for nawk on Solaris, etc.
Dump this out into a file for each env. The run diff with something like this:
diff -s dump.cob dump.dev | egrep -v -- "^([0-9]|--)" | sort | grep -vn xxx
[/example]
[reference]
[tags]openssl, cacerts, java certifications, comparing java certs, Unix Coding School[/tags]
[/reference]
[problem]
You want to generate 512 bytes of random data, using UNIX dd (disk to disk) command.
[/problem]
[solution]
Using /dev/random blocks waiting for truly random data, whereas /dev/urandom just pumps out whatever is available.
You need to escape the control characters, or else it will trash your screen. cat -ve escapes these control characters for you.
See examples
[/solution]
[example]
dd bs=1 count=512 if=/dev/urandom | cat -ve
dd is a very powerful command, that can be used to read files, data, tapes, even disks, etc – varying the number of blocks with count and block size with bs.
if is the input device and of can be used to designate an output device.
dd bs=1 count=512 if=/dev/urandom | openssl base64
Also it can convert to upper and lower case, with conv=ucase, etc.
If you have an extremely large file, it can open at an offset – via the skip option, for example to skip the first 1k of the file and read 100 bytes:
dd if=access_log.techieblogs bs=1 skip=1024 count=1000
- - [04/Jun/2006:00:13:10 +0800] "GET /mysqldemo/run_q.php?database=mysql HTTP/1.1" 200 1364
[/example]
[reference]
[tags]dd, pseudo random data, cat, Unix Coding School[/tags]
[/reference]
[problem]
You want to loop around a number of times and perform an action.
[/problem]
[solution]
Being able to loop around a given number of times, fully utilizes the power of UNIX. See the example.
[/solution]
[example]
For example, spin around 100 times and print hello:
i=0; while [[ $i -lt 100 ]] ; do echo -n "hello"; ((i++)) ; done; echo ""
hellohellohellohellohellohellohellohellohellohello
hellohellohellohellohellohellohellohellohellohello
hellohellohellohellohellohellohellohellohellohello
hellohellohellohellohellohellohellohello...
Similarly the for command is a beauty too, spin around 5 times and kick off a loop 5 times – paste the result into 5 columns:
for i in 1 2 3 4 5; do for j in 1 2 3 4 5; do echo "$i:$j"; done; done | paste - - - - -
1:1 1:2 1:3 1:4 1:5
2:1 2:2 2:3 2:4 2:5
3:1 3:2 3:3 3:4 3:5
4:1 4:2 4:3 4:4 4:5
5:1 5:2 5:3 5:4 5:5
[/example]
[reference]
[tags], Unix Coding School[/tags]
[/reference]
[problem]
Ever wanted to produce columns of output, rather than your output spilling off the screen.
[/problem]
[solution]
Useful command for this is paste, see example for how it works – to split output into 3 columns.
[/solution]
[example]
ls | paste - - -
comments.php comments-popup.php fat.js
footer.php header.php index.php
screenshot.png sidebar.php style.css
As you’d expect with UNIX, that is not the end of this commands uses! 🙂
Create a comma seperated list:
ls | paste - - - -d,
comments.php,comments-popup.php,fat.js
footer.php,header.php,index.php
screenshot.png,sidebar.php,style.css
Then courtesy of the LINUX info command:
cat num2
1
2
$ cat let3
a
b
c
$ paste num2 let3
1 a
2 b
c
And:
$ paste -s num2 let3
1 2
a b c
[/example]
[reference]
[tags]paste, Unix Coding School[/tags]
[/reference]
![[ See a run through screen shot here ]](/content/generate_certifying_authority_cert.png){kind=link}
![[ See a run through screen shot here ]](/content/generate_two_year_cert_request.png){kind=link}
![[ See a run through screen shot here ]](/content/self_sign_two_year_cert.png){kind=link}