Category: perl

[problem]

You want to convert epoch into readable date and time.

Maybe you have a log file, which is pumping out lines prefixed with the epoch?

[/problem]

[solution]

We use the perl function localtime to convert epoch into a date and time.

At the example tab is the code to convert that epoch, into a recognizable date and time.

localtime, can also return an array – take a look at the reference tag for more info.

[/solution]

[example]

We quite simply supply the epoch to localtime and print it.


$ perl -e 'print localtime(1145980815)."n";'
Wed Apr 26 00:00:15 2006


[/example]

[reference]

[tags]Perl Date handling, Perl Coding School[/tags]

• Perl Docs – localtime

[/reference]

[problem]

During my first encounters with Perl many years ago, I asked how can I get a specific column.

It was so easy in AWK! 😉

[/problem]

[solution]

Well here it is in Perl – see example tab.

BTW you can always write an awkscript and run it through a2p – very good for learning Perl! 🙂

[/solution]

[example]

Show column 1:

perl -ane 'print $F[0]."n";'

Show column 2:

perl -ane 'print $F[1]."n";'

Show last column:

perl -ane 'print $F[$#F]."n";'

Show last but one column:

perl -ane 'print $F[($#F-1)]."n";'

So you just run your program, or cat your file, etc and pipe it through this code to get specific column.

[/example]

[reference]

[tags]Perl column handling, Awk to Perl, a2p, Perl Coding School[/tags]

• Perl Docs
• Perl Docs – Awk to Perl

[/reference]

[problem]

You want to listen on a port, maybe to test firewalls! 🙂

Excellent introduction to client server software.

[/problem]

[solution]

Another use bit of socket programming, courtesy of Perl! 🙂

Extremely useful bit of Perl, which can be used with the other post on this site, to confirm network connectivity (port scanner).

This code (see example tab) will listen on a port (although be careful it is over 1024, unless you are running as root/admin).

[/solution]

[example]


perl -MIO::Socket -e '$srv=IO::Socket::INET->
new(LocalPort=>$ARGV[0],Type=>SOCK_STREAM,Reuse=>1,Listen=>5)

or die "Failed trying to listen on $ARGV[0]n";while($cl=$srv->accept()) { while() { print }

} close($socket);' port

Can also use it as a cheap chat service! 🙂 Everything gets echo’d through – so just telnet host port and type away! 🙂

Also see my port scan code at the reference tab.

[/example]

[reference]

[tags]Perl Socket Programming, Network Programming, Perl Port Listen, Perl Client Server demo, Perl Coding School[/tags]

• Perl Docs – Socket Programming
• Perl Port Scanner

[/reference]

[problem]

You want to test a network port, on a remote system – over TCP/IP.

Maybe this is a new setup, or you want confirmation it is working.

Perhaps the firewall rules have just been changed! 🙂

[/problem]

[solution]

A nice small bit of Perl code that I’ve used thousands of times!

In fact I’m running it in most of my production environments,
as a check that the a process is not only running – but also responding.

Yep, you could just use telnet – but some systems have that taken off for security.

Additionally it is n’t as easy to program telnet – I know, I know – you can with expect. 🙂

[/solution]

[example]

Here is just a one liner, but you can easily incorporate this into a script.

perl -MIO::Socket -e '$socket=IO::Socket::INET->
new(Proto=>tcp,PeerAddr=>$ARGV[0],PeerPort=>$ARGV[1]);
if($@) { print "Failed: $@n" } else { print "Succeedn"; }' host port

[/example]

[reference]

[tags]Perl, Network, TCPIP, Port Scanner, Perl Coding School[/tags]

• Perl Docs – socket

[/reference]

[problem]

You want to capture the current epoch. Maybe to use in a log file, or as a filename.

Or maybe you want to calculate the date yesterday.

[/problem]

[solution]

This piece of code is very useful for performing date calculations. You can obtain the current epoch (time in seconds since Jan 1 1970), then add 3600 for 1 hour – or 86400 for 24 hours hence.

[/solution]

[example]

So use in a UNIX variable like this:

epoch=perl -M'English' -e 'print $BASETIME."n";'

To work out 24 hours ago, just subtract 86400.

perl -M'English' -e 'print(($BASETIME-86400)."n");'

Then to see the date yesterday:


$ perl -M'English' -e 'print(($BASETIME-86400)."n");'
1180746252
$ perl -M'English' -e 'print(localtime(1180746252)."n");'
Sat Jun 2 09:04:12 2007


[/example]

[reference]

[tags]Perl, epoch, date manipulation, Perl Coding School[/tags]

• Perl Docs – predefined variables
• Perl Docs – localtime

[/reference]

[recommended]

[/recommended]

[problem]

I wanted to upload a delimited field to mysql db, but hit the problem that the file contained nearly 3000 rows and no schema on the required size of each field.

Therefore I needed to traverse the file and calculate the length of each field. Then at the end, print the largest field found for each column.

[/problem]

[solution]

Perl to the rescue! Pretty easy in Perl, after scratching my head attempting with awk. 🙂

Anyway the code is under the example tab. I’m using the pipe symbol as a delimiter “|” – so just substitute this with your delimiter, cat your file and pipe it through this script.

In the example I show how you can manipulate the file, to produce pipe delimited fields too.

[/solution]

[example]

This is how to run the script. Basically you just need to pipe your output through find largest.

cat yourfile | ./find_largest.pl

Here is the code.

 #!/usr/bin/perl
 
@highest=();
 
while( < STDIN >  ) {
 
    @thisline=split(/|/);
 
    for($i=0;$i<=$#thisline;$i++) {
 
        $thislength=length($thisline[$i]);
 
        if($thislength > $highest[$i]) { $highest[$i]=$thislength; }
 
    }
 
}
 
print(join("|",@highest)."n");
 
exit(0);
 
__END__
 
 
 
So for example, if I want to find the largest fields in one of my web logs - for crunching into a db:
 
 
First off I only want lines starting with a space, then a number.
 
Next I need to replace all multiple spaces between fields, with 1 space.
 
Then I replace the spaces between each field, with a pipe.
 
Lastly I pump it through find_largest.pl - which gives me the largest sized field
 
 

 [marcus@bree]/var/log/httpd% grep "^ [0-9]" access_log.tools
 | sed -e 's/  / /g' -e 's/ /|/g' |
 ~/Perl_Bin/find_largest.pl
 |14|1|1|21|6|5|103|9|3|5
 
 
[/example]
 
[reference]
 
[tags]Perl, Data Analysis, Perl Coding School[/tags]
 
 
Perl Doc - storable
 
Linux Man Page - sed
 
Linux Man Page - grep
 
 
[/reference]
 	

[problem]

You have a script which runs too long and you want to time it out, after a given number
of seconds.

[/problem]

[solution]

Useful bit of code to time-out a section of your Perl script, via the alarm function.

See the example tab.

[/solution]

[example]


#!/usr/bin/perl

eval {

local %SIG;
$SIG{ALRM}=
sub{ die “timeout reached, after 20 seconds\n”; };
alarm 20;
print “sleeping for 60 secondsn”;
sleep 60; # This is where to put your code, between the alarms
alarm 0;
};

alarm 0;

if($@) { print “Error: $@\n”; }

exit(0);




__END__


[/example]

[reference]

[tags]Perl timeout, Perl Coding School[/tags]

[/reference]

[problem]

How to use Perl to connect to MS Outlook. Then descend through given folders and save items to disk, as text files.

[/problem]

[solution]

I wrote this some time ago, to traverse predefined outlook mail folders, saving items with given subject to text.

Requires WIN32:OLE perl module (which comes with activeperl by default).

Hardcoded is the upload and uploaded mail folders. Also a subject that contains – pattern: upload.

I used one outlook rule to move items from a specific source, with this subject into upload.

[/solution]

[example]

Here is the code – if you have any dramas with it, leave a comment.


#!perl

use Win32::OLE qw(in with);
use Win32::OLE::Const 'Microsoft Outlook';

# Connect to a running version of Outlook
eval { $Outlook =
   Win32::OLE->GetActiveObject('Outlook.Application')
};

die "Outlook not installed" if $@;

# If that fails start up Outlook
unless(defined $Outlook) {
   $Outlook =
     Win32::OLE->new('Outlook.Application', 'Quit')
   or die "Opps, cannot start Outlook";
}

# This appears to return a ref to the object
$namespace = $Outlook->GetNamespace('MAPI');
$thisFolder=$namespace->Folders("Mailbox - mailboxna")->
   Folders('upload');
$toFolder=$namespace->Folders("Mailbox - mailboxna")->
   Folders('uploaded');

# Workaround to be able to extract key/value pairs
%thisHash=%{$thisFolder};
$name=$thisHash{'Name'};

# This is the number of items in designated folder
$count=$thisHash{'Items'}{'Count'};
open(LOGFH,">> ol_save_to_text.log")
   or die("cannot open log filen");

# Drop out if there are no mail items in this folder
if($count > 0) {

   print LOGFH "Count: $count for $namen";

   $filename='yourname';
   open(FH,"> $filename")
   or die ("cannot open $filenamen");

   for($i=1;$i<=$count;$i++) {
     print LOGFH "Count: $countn";

     $oItems=$thisFolder->Items(1);

     %thisItem=%{$oItems};
     $subject=$thisItem{'Subject'};

     if($subject =~ /pattern: upload/) {
       print LOGFH "$i: $subjectn";
       $body=$thisItem{'Body'};
       print FH "$body";
       $oItems->Move($toFolder);
     } else {
       $nonitem+=1;
     }

     $oItems->Move($toFolder);
   }

} else { print LOGFH "No Files to Processn"; }

close(LOGFH);
1;


[/example]

[reference]

Microsoft Outlook Keyboard Shortcuts – Courtesy of RNIB

[tags]MS Outlook OLE Perl, MS Outlook, OLE, Perl win32, cygwin, Perl, Perl Coding School[/tags]

• Perl Docs Win32

[/reference]

[problem]

You want to use libcurl, driven through Perl.

[/problem]

[solution]

Beautiful for parse HTML and either extracted (screen scraping) content or performing actions based on results.

See the examples tab for this simple script, demonstrating the libcurl API for Perl.

[/solution]

[example]


#!/usr/bin/perl

$url="http://perl.coding-school.com/"; # set your url here
$|++;

use Curl::easy;
# Init the curl session

my $curl= Curl::easy::init() or die "curl init failed!n err: $!n";

sub body_callback {
   my ($chunk,$context)=@_;
   push @{$context}, $chunk;
   return length($chunk);
}

Curl::easy::setopt
   ($curl, CURLOPT_PROXY, $proxy) if($proxy);
Curl::easy::setopt
   ($curl, CURLOPT_PROXYPORT, $proxyport) if($proxyport);
Curl::easy::setopt
   ($curl, CURLOPT_SSL_VERIFYHOST, 0);
Curl::easy::setopt
   ($curl, CURLOPT_SSL_VERIFYPEER, 0);
Curl::easy::setopt
   ($curl, CURLOPT_URL, $url);
Curl::easy::setopt
   ($curl, CURLOPT_WRITEFUNCTION, &body_callback);

my @body;

Curl::easy::setopt
   ($curl, CURLOPT_FILE, @body);
Curl::easy::setopt
   ($curl, CURLOPT_ERRORBUFFER, "errbuf");
if (Curl::easy::perform
   ($curl) != 0) { print "Failed : $errbufn"; };
Curl::easy::cleanup($curl);

# Separate each line into one element in array
@lines=();

foreach (@body) { push(@lines,split('n', $_, 9999)); }

foreach (@lines) {
   # just to demonstrate it works!
   if(/icons/) { print("$_n"); }
}

exit(0);

Here is a demo screen shot of this code using perl and libcurl.

[/example]

[reference]

[tags]Perl libcurl demo, Perl libcurl, Perl, libcurl, curl, Perl Coding School[/tags]

• libcurl perl API
• Perl Docs – foreach function

[/reference]

[problem]

You want to generate a SHA digest for a given string.

[/problem]

[solution]

Perl has great encryption and digesting algorithms.

In the example tab we use the Digest Perl module.

[/solution]

[example]


perl -M'Digest::SHA1 qw(sha1_hex)'
-e '$dig=sha1_hex("my big test"); print "$dign";'


419e6139a21f51a3f2ea1a783cfe536a0dada873

[/example]

[reference]

[tags]Perl SHA Digest, Perl, SHA, Encryption, Perl Coding School[/tags]

• Perl Docs Digest

[/reference]