Category: unix

[problem]

You want to extract a specific line number from a file.

[/problem]

[solution]

Once you have your line number – see previous post on grep, we can extract lines around the pattern.

To do this sed (stream editor) can be used to print just desired lines – this says don’t print all lines (-n); start at line 456 and finish at line 466 – print:


sed -n 456,466p filename


[/solution]

[example]

Also with sed, we can say delete specific lines – in this case remove lines 5 to 10:


sed 5,10d filename


That’s not all sed can accept patterns, as start/end identifiers:


sed /start_pattern/,/end_pattern/d filename


[/example]

[reference]

[tags], Unix Coding School[/tags]

• Linux Man Pages – sed command

[/reference]

[problem]

Following on from the last post on removing lines with sed, how do we use sed to substitute output on the fly?

[/problem]

[solution]

This says substitute occurences of PATT with REPLACE global (or all occurences).


sed 's/PATT/REPLACE/g' filename


[/solution]

[example]

So to replace all occurences of Unix with UNIX:


sed 's/Unix/UNIX/g' filename


We can also replace patterns, with variable values, like this:


sed "s/PATT/${THEVAR}/g" filename


[/example]

[reference]

[tags]SED, Unix Coding School[/tags]

• Linux Man Pages – sed command

[/reference]

[problem]

Needing to view current activity of users on the system.

[/problem]

[solution]

Most flavours of UNIX provide a number of tools, to tell you who is logged in or when they last accessed the system.

[/solution]

[example]

A favorite of mine, is who -Hu – really good for seeing who is idle, quite literally! 🙂

Also you can just run last – although beware of rolled logs. Generally in /var/adm/wtmp* on Solaris and /var/log/wtmp* on Linux.

If you want to run last against an older file (by default it just uses wtmpx or wtmp) – just specify minus f – like this: last -f /var/log/wtmp.1

Additionally the finger command will quite often show last login time.

[/example]

[reference]

[tags]who, last, finger, Unix Coding School[/tags]

• Linux Man Pages – who command
• Linux Man Pages – last command
• Linux Man Pages – finger command

[/reference]

[problem]

You want to allow a user to run one command, as root user.

[/problem]

[solution]

To allow a user access to run stuff as root, just login as root and then add the following line to /etc/sudoers.


userid hostname=command


[/solution]

[example]

For example to allow user marcus access on bree, to restart, stop, start or check apache:


marcus bree=/etc/init.d/httpd


To allow all access on all boxes:


marcus (ALL)=(ALL)


[/example]

[reference]

[tags]sudo, switch user do, temporary UNIX authorisation, Unix Coding School[/tags]

• Linux Man Pages – sudo command

[/reference]

[problem]

Want to kill all processes, for a specific user.

[/problem]

[solution]

This command will send a -KILL signal, equivalent to minus 9:


pkill -KILL -U username


[/solution]

[example]

So to kill all processes owned by username marcus:


pkill -KILL -U marcus


If you issue kill -l – that is kill minus L, it will show all the symbol to numeric codes. Although you can just use numeric with pkill:


pkill -9 -U marcus


[/example]

[reference]

[tags]pkill, proc kill, Unix Coding School[/tags]

• Linux Man Pages – pkill command

[/reference]

[problem]

Sick of filtering through loads of logs, or just spotting real hits from the robots! 🙂

Seriously reduce your apache web logs, by filtering out images, style sheets and your own hits.

[/problem]

[solution]

Simple with Apache’s customlog and setenvif statements.

I’ve also included capturing the user-agent in a separate file, as well as the referer, which is brill for seeing which google searches brought traffic to you.

You can even still capture robot and own hits into a separate log, here is how below.

[/solution]

[example]

         SetEnvIf Request_URI ".(png|gif|jpg|js|css)" image-req         SetEnvIf Request_URI "favicon.ico" image-req         SetEnvIf Request_URI "/icons" image-req         SetEnvIf Request_URI "sitemap.xml.gz" image-req         SetEnvIf REMOTE_ADDR "127.0.0.1" image-req         SetEnvIf REMOTE_ADDR "127.0.0.1" home-req         SetEnvIf User-agent "(Googlebot|msnbot|Spider|crawl|slurp|Jeeves| Mediapartners|FeedBurner)" image-req         SetEnvIf User-agent "(Googlebot|msnbot|Spider|crawl|slurp|Jeeves| Mediapartners|FeedBurner)" bot-req      CustomLog logs/access_log.techieblogs "["%{Referer}i"]n %h %l %u %t "%r" %>s %b" env=!image-req     CustomLog logs/access_log.agents.techieblogs "%h ["%{Referer}i"] ["%{User-agent}i"]" env=!image-req     CustomLog logs/access_log.bots.techieblogs "["%{Referer}i"] %h %l %u %t "%r" %>s %b" env=bot-req     CustomLog logs/access_log.home.techieblogs "["%{Referer}i"] n %h %l %u %t "%r" %>s %b" env=home-req 

[/example]

[reference]

[tags]Apache Logging, Unix Coding School[/tags]

• Linux Man Pages – httpd command

[/reference]

[problem]

You want to background a process, or leave it running whilst shutting the terminal.

[/problem]

[solution]

To start a UNIX process is the background, simply append the command with an ampersand, like this below.

[/solution]

[example]


sleep 30&;


If you have already started a process and want to start in the background, just hit control Z. Then type bg to background (and continue running) the command.

To display jobs in the background, runs command jobs

To bring back into the foreground, run fg

It is always a good idea to redirect output, for backgrounded jobs. Also if you are going to exit the shell, need to prefix the command with nohup. Or if it is already running, background the job and run disown

[/example]

[reference]

[tags]UNIX Batch Mode, UNIX Background Process, Unix Coding School[/tags]

• Linux Man Pages – jobs command

[/reference]

[problem]

Performing debugging and you need to view the output, of more than one file at once.

[/problem]

[solution]

A simple way to keep an eye on a file is with tail.


tail -f logfile


You can background this, by appending an ampersand:


tail -f logfile&


[/solution]

[example]

But if you want to watch multiple files, how do you know which one received the update.

Sed to the rescue:


tail -f logfilea | sed 's/^/logfilea: /'&
tail -f logfileb | sed 's/^/logfileb: /'&


[/example]

[reference]

[tags], Unix Coding School[/tags]

• Linux Man Pages – tail command

[/reference]

[problem]

Are there a combinations of key strokes you constantly enter?

[/problem]

[solution]

Maybe you are a Java programming! 🙂 Or PHP/Perl – either way if you are using vi a lot, these tips could be worth your pc’s weight in gold. 😉

[/solution]

[example]

To create a one time map (of any key), just run this within vi:


:map #1 :!echo hey mapped f1^M


You need to hold down control and hit m to get the return, so this will run itself (no enter required).

To see current maps, just type :map

To make it permanent, just add same syntax to $HOME/.exrc

I think another good tip, is to map f1 to cat the exrc – like this:


:map #1 :!cat $HOME/.exrc^M


Then map #2, 3, 4 …

[/example]

[reference]

[tags]vi, macros, Unix Coding School[/tags]

• Linux Man Pages – vi command
• Linux Man Pages – cat command
• Linux Man Pages – bash command

[/reference]

[problem]

You need to substitute columns, whilst editing a file with vi.

[/problem]

[solution]

Okay, bit more complex with this one.

To substitute columns, we bracket off the pattern – like this (pattern)

Then we simply use an escaped number to substitute.

[/solution]

[example]

Here is a demo, whilst editing a file with vi:


:%s/(.*) (.*)/2 1/


This says match all lines (%), substitute (s). Match anything up to a space, store in column one. Match anything else, store in column two. Now swop column two with column one.

Here is a run through – unchanged:


a z
b y
c x
d w
e v
f u
g t
h s
i r
j q
k p
l o
m n


The substitute:


:%s/(.*) (.*)/2 1/


The result:


z a
y b
x c
w d
v e
u f
t g
s h
r i
q j
p k
o l
n m


[/example]

[reference]

[tags]UNIX, vi, regular expression, Unix Coding School[/tags]

• Linux Man Pages – vi command

[/reference]