Month: December 2007

[problem]

Sick of filtering through loads of logs, or just spotting real hits from the robots! 🙂

Seriously reduce your apache web logs, by filtering out images, style sheets and your own hits.

[/problem]

[solution]

Simple with Apache’s customlog and setenvif statements.

I’ve also included capturing the user-agent in a separate file, as well as the referer, which is brill for seeing which google searches brought traffic to you.

You can even still capture robot and own hits into a separate log, here is how below.

[/solution]

[example]

         SetEnvIf Request_URI ".(png|gif|jpg|js|css)" image-req         SetEnvIf Request_URI "favicon.ico" image-req         SetEnvIf Request_URI "/icons" image-req         SetEnvIf Request_URI "sitemap.xml.gz" image-req         SetEnvIf REMOTE_ADDR "127.0.0.1" image-req         SetEnvIf REMOTE_ADDR "127.0.0.1" home-req         SetEnvIf User-agent "(Googlebot|msnbot|Spider|crawl|slurp|Jeeves| Mediapartners|FeedBurner)" image-req         SetEnvIf User-agent "(Googlebot|msnbot|Spider|crawl|slurp|Jeeves| Mediapartners|FeedBurner)" bot-req      CustomLog logs/access_log.techieblogs "["%{Referer}i"]n %h %l %u %t "%r" %>s %b" env=!image-req     CustomLog logs/access_log.agents.techieblogs "%h ["%{Referer}i"] ["%{User-agent}i"]" env=!image-req     CustomLog logs/access_log.bots.techieblogs "["%{Referer}i"] %h %l %u %t "%r" %>s %b" env=bot-req     CustomLog logs/access_log.home.techieblogs "["%{Referer}i"] n %h %l %u %t "%r" %>s %b" env=home-req 

[/example]

[reference]

[tags]Apache Logging, Unix Coding School[/tags]

• Linux Man Pages – httpd command

[/reference]

[problem]

You want to background a process, or leave it running whilst shutting the terminal.

[/problem]

[solution]

To start a UNIX process is the background, simply append the command with an ampersand, like this below.

[/solution]

[example]


sleep 30&;


If you have already started a process and want to start in the background, just hit control Z. Then type bg to background (and continue running) the command.

To display jobs in the background, runs command jobs

To bring back into the foreground, run fg

It is always a good idea to redirect output, for backgrounded jobs. Also if you are going to exit the shell, need to prefix the command with nohup. Or if it is already running, background the job and run disown

[/example]

[reference]

[tags]UNIX Batch Mode, UNIX Background Process, Unix Coding School[/tags]

• Linux Man Pages – jobs command

[/reference]

[problem]

Performing debugging and you need to view the output, of more than one file at once.

[/problem]

[solution]

A simple way to keep an eye on a file is with tail.


tail -f logfile


You can background this, by appending an ampersand:


tail -f logfile&


[/solution]

[example]

But if you want to watch multiple files, how do you know which one received the update.

Sed to the rescue:


tail -f logfilea | sed 's/^/logfilea: /'&
tail -f logfileb | sed 's/^/logfileb: /'&


[/example]

[reference]

[tags], Unix Coding School[/tags]

• Linux Man Pages – tail command

[/reference]

[problem]

Are there a combinations of key strokes you constantly enter?

[/problem]

[solution]

Maybe you are a Java programming! 🙂 Or PHP/Perl – either way if you are using vi a lot, these tips could be worth your pc’s weight in gold. 😉

[/solution]

[example]

To create a one time map (of any key), just run this within vi:


:map #1 :!echo hey mapped f1^M


You need to hold down control and hit m to get the return, so this will run itself (no enter required).

To see current maps, just type :map

To make it permanent, just add same syntax to $HOME/.exrc

I think another good tip, is to map f1 to cat the exrc – like this:


:map #1 :!cat $HOME/.exrc^M


Then map #2, 3, 4 …

[/example]

[reference]

[tags]vi, macros, Unix Coding School[/tags]

• Linux Man Pages – vi command
• Linux Man Pages – cat command
• Linux Man Pages – bash command

[/reference]

[problem]

You need to substitute columns, whilst editing a file with vi.

[/problem]

[solution]

Okay, bit more complex with this one.

To substitute columns, we bracket off the pattern – like this (pattern)

Then we simply use an escaped number to substitute.

[/solution]

[example]

Here is a demo, whilst editing a file with vi:


:%s/(.*) (.*)/2 1/


This says match all lines (%), substitute (s). Match anything up to a space, store in column one. Match anything else, store in column two. Now swop column two with column one.

Here is a run through – unchanged:


a z
b y
c x
d w
e v
f u
g t
h s
i r
j q
k p
l o
m n


The substitute:


:%s/(.*) (.*)/2 1/


The result:


z a
y b
x c
w d
v e
u f
t g
s h
r i
q j
p k
o l
n m


[/example]

[reference]

[tags]UNIX, vi, regular expression, Unix Coding School[/tags]

• Linux Man Pages – vi command

[/reference]

[problem]

Exiting a file in vi.

[/problem]

[solution]

There are quite a few ways to exit a file in vi. Here are some of my favorites.

[/solution]

[example]

The standard: :wq

Force, in the case of read-only: :wq!

Abort updates: :q!

Reset the file without writing: :e!

Quick save and exit: 😡

Lazy one, without using a colon: ZZ

Going to another file: n filename

Go back to the first file, when you’v edited with wildcard or list of files: :rew

[/example]

[reference]

[tags]vi, Unix Coding School[/tags]

• Linux Man Pages – vi command

[/reference]