Showing posts from 2009

Perl Foundations Event Mailing List

Go, subscribe and be happy with Perl events.

Good to see the Perl Foundation getting more active.


Mouse and Moose in Perl - Walking in a winter wonderland... kinda

I like Moose but I work heavily in non-persistent applications. Sadly this makes Moose's start up expense too much.

But recently I revisited Mouse (thanks to a friend). The last time I looked at it, I misread the documentation and it sounds like it was no longer being supported by the original developer. But not anymore, w00t!

The one part I really like about Mouse (and Moose) is the declarative nature of creating accessors (and therefore attributes). I really like this because it reduces the number of simple accessor tests that you need to write (saving time, tendium and kittens).

I used to use Class::Accessor but I always ran into inheritance and validation issues (which I used Params::Validate but this caused lots of custom magic for inheritance).

I think the Mouse/Moose syntax is significantly cleaner.

Compare Class::Accessor:package User; use strict; use warnings; use Carp; use base qw( Class::Accessor ); __PACKAGE__->mk_accessors( qw( id username password first_name …

Setting Java Socket Timeouts

Oddly, Java's Socket library doesn't allow you to set the socket timeout directly in the constructor. Its important to note that there is no default so the socket will last forever (ish).

Note: The socket timeouts below apply to different situations: setSoTimeout applies to read timeouts. connect applies to connection establishment. At least that's how I read it.

I found two options, setting the option on the socket after creating one simplest (for me at least):
Socket s; try { s = new Socket( ip, port ); s.setSoTimeout( secs * 1000 ); // converts secs to ms // do other stuff like get input stream and read it } catch ( SocketException se ) { // catch timeout or other socket related error } catch ( Exception e ) { // any other exception } The other option is creating a Socket object without any parameters then calling the connect method with timeout.

Here is a rough prototype since it seemed like pain to me but I was curious about the code:
Socket s = new Socket; try {…

A journey through whitespace cleanup - Perl5 and split

Running through some older Perl5 code, I found someone using grep with a split to cleanup white space on some incoming data.

Found, this seemed confusing to me with lots of extra complexity:
my @line = grep { s/\s*$// } split /\|/, $_;

First Pass:
my @line = split /\|\s*/, $_;

But my guess is that all wrapping white space should be removed:
my @line = split /\s*\|\s*/, $_;

Even better (and more readable), use the Perl magic context:
my @line = split /\s*\|\s*/;

I would love to make that a cleaner regex but that will have to be another day...

Super Sweet CSV parsing with Perl

Today, everyday, every other day I need to pull in CSV files and write others out. After awhile, I was ready for a better solution. Lucky for me, the Perl community is amazing.

For some amazing parsing options look no further than Text::CSV_XS, it provides a super sweet parsing interface. No talking just pure Perl code: use strict;
use warnings;

use Text::CSV_XS;

my $csv = Text::CSV_XS->new( {eol => "\n"} ); # eol has nothing to do with parsing wait until later :)
my $input = shift || die "Missing filename\n";

open my $fh, '<', $input
or die "Failed to open $input: $!\n";

# assume no file header
while ( my $columns = $csv->getline($fh) ) {
print $columns->[0], $columns->[1], "\n"; # or whatever you want to do

close $csv;

That's for parsing. You can do some other super cool stuff too, with bind_columns. Lets' assume the file has the format of 'first name, last name, email address'.

Here the parsing loop updat…

unzip zip file with password

Never remember, its as simple as it seems it should be:unzip -P [password]
Or for older versions of info-zip:
unzip -[password]

YAHOO.util.Event.onDOMReady reminder

The callback argument signature wasn't obvious (or is it wrong?) on the YUI documentation page or very readable at the YUI API site.

The callback has the following format:
function init_for_ondomready(label, args, obj) {
}label - string of "onDOMReady"args - empty array (from source doesn't look like its populated today)obj - object passed to onDOMReady eventExample:
YAHOO.util.Event.onDOMReady(init_for_ondomready, thingy);
label = 'onDOMReady'args = []obj = thingyTherefore you need to include a holder for the empty args array or process the arguments array manually.

Grabbing inputs from the DOM with my favorite javascript libraries

Grabbing input elements with my favorite JavaScript libraries

WhatPrototypeYUIAll inputs$$('input')YAHOO.util.Selector.query('input');
All checkboxes$$('input[type=checkbox]')Sweet with .eachYAHOO.util.Selector.query('input[type=checkbox]:checked');
All inputs in table with class .data_display$$('.data_display input')YAHOO.util.Selector.query('.data_display input');

JBuilder 8 losing Libraries

After not running JBuilder 8 for a bit, when I tried to rebuild an applet at $work, I received a very strange set of missing class errors. These classes were right there in the Jar libraries... I swear :)

Well after a few hours of looking around I found that I need to remove the whole 'output path' option (under Run -> Configurations -> Path tab). Then it magically worked.

Go Figure.

I know JBuilder8 is ancient but one applet builds in it and nobody wants to fund it moving to Eclipse...

Recommended way to build URLs in Template Toolkit

On the mailing list, Andy Wardey recommended using the URL plugin as the best way to build URLs in Template Toolkit (TT). Since this is very helpful but sometimes hard to dig out of the TT documentation, I wanted to make a quick note here.

The URL plugin will make sure to properly HTML and URI encode your link(s).

Here my quick example

In your template:
[% USE profile = URL( '/profile', show_picture = 1 ) %]

<a href="[% profile( slayer = 'buffy' ) %]">Buffy's Profile</a>
<a href="[% profile( vampire = 'spike' ) %]">Spike's Profile</a>
This will render validly encoded html like:
<a href="/profile?show_picture=1&amp;slayer=buffy">Buffy's Profile</a>
<a href="/profile?show_picture=1&amp;vampire=spike">Spike's Profile</a>
Notice the & not a '&' which makes the html validation much happier.

This is much easier than trying to do it yourself with filte…

perltidy and block labels

perltidy has its own mind at times. Most of the time, its mind is better than my hands reformatting code.

But I've been writing lots of test code recently and found that I use blocks and labels, along with nested blocks with labels. Stuff that looks like:
my $shared_db_thingy = create_db_thingy( %params );

# create user tests
skip 'DB Thingy not created', $test_count
unless $shared_db_thingy;

lives_ok { $shared_db_thingy->create_user( %user_parameters ) } 'create_user';

# retrieve user tests
}Frustratingly, perltidy likes to tidy this into:
}I had to do some real digging into perltidy to find the -ola that outdents labels.

Setting the -nola fixes it. And doesn't having any (seen) side effects.


vim productive note

When writing tests for Perl objects (or modules), I like to store the name of the module in a named buffer/register in vim.

I usually visually highlight the object(or module) name, then put it in the "n" buffer.
Yank into n: "ny
Paste out of n: "np
I guess the m or n buffer might make sense.

If you forget like I do what buffer your using you can dump them in vim with:

So it was...

Building vim on os x 10.5

I like to build my own version of vim on systems. This usually avoids any problems later when I want to add something new in to it or upgrade to a newer version.

In the past older versions of OS X (10.4 and 10.3 ) I could just build vim straight but on 10.5 I had a strange issue of it starting up X11 when I ran it in my terminal window.

It seems that either vim or OS X became smarter when it came to talking between process using X Windows communication protocol. (XSMP)

Just adding a --without-x to the configure statement didn't help. It required disabling the XSMP protocol.

Here is the configure line:
./configure --without-x --disable-xsmp
Much better, now I just need to figure out the best terminal font for my 24" iMac. :)

3 ways to disable auto typing in SOAP::Lite client

Often SOAP::Lite will try to autotype a parameter in a request and it does it wrongly (or correctly) causing the SOAP server to fail badly (for so many reasons).

SOAP::Lite provides three ways to disable this typing. Two of the ways work at the client object level and the other does it at the SOAP::Data object level.
Client Level The following options disable autotyping for all data items in a request:
autotype method. After creating a SOAP::Lite client you can disable autotyping by $soapclient->autotype(0).
Example: my $soap = SOAP::Lite->uri($xmlns)->proxy($proxy)
->on_action( sub {"$action?type=$reqname&ns=$xmlns"} );
Overriding the typelookup method. You can override the typelookup to run undef/false/etc.
sub typelookup { return undef; }
Data objectSOAP::Data object provide a type method. This allows manually typing of the object. When this is set to an empty string no type is set nor is it autotyped by serializer. Note: undef doe…

Quick YUI Panel styling sans Javascript

In doing some application mockups, I found a quick way to skin the YUI's Panel widget without having to create Javascript objects to do it for me.

For each panel you need to just add the class yui-panel.

Here is a markup example ( ripped out all but the body stuff out):
<body class="yui-skin-sam">
<div id="doc2" class="yui-t5">
<div id="hd">Header Stuff</div>
<div id="bd">
<div id="yui-main">
<div id="panel1" class="yui-b yui-panel">
<div class="hd">Panel Header</div>
<div class="bd">Here is stuff for the body of the panel</div>
<div class="ft">Panel Footer</div>
<div id="panel2" class="yui-b yui-panel">
<div class="hd">2…

Easiest way to get SQLite to put in timestamps

After doing some more digging (must have been blind before), I found that SQLite supports a default timestamp at insert.

Here is an example of a create statement using the default with SQLiteCURRENT_TIMESTAMP keyword.
create table user (
id integer primary key,
username text not null,
password text not null,
first_name text,
last_name text,
created text default CURRENT_TIMESTAMP
That is much easier that my previous attempt. Not sure how I missed it. But it doesn't solve the missing now() function for updates. ( did find a language solution for that in Perl but that is for another post). In that case, it will still require the use of datetime('now').

Reference: SQLite CREATE TABLE reference.

2 ways to get SQLite to put dates into columns

2 ways to get SQLite to put dates into a column.
insert into mytable values( null, datetime('now') );insert into mytable values( null, strftime('%s', 'now'));The first one inserts a row somewhat like this:
1|2009-03-10 18:47:46

The second inserts an unix timestamp:

It might be best to use that unix timestamp with an integer column type for dates since SQLite doesn't support a datetime one.

It makes comparisons and ordering much easier:
select * from dt where lu > strftime('%s', '2009-03-10');

Output of:
5|1236711519But the formatting is pretty ugly. :-/

Hey what about formatting it within the select with the SQLite datetime function:
select id, datetime(lu, 'unixepoch') lu from dt order by lu;
6|2009-03-10 00:00:00
2|2009-03-10 18:56:51
3|2009-03-10 18:58:36
4|2009-03-10 18:58:38
5|2009-03-10 18:58:39
Better but having to add that to each select is kind of a pain. I like adding…

Adding http request/response logging to SOAP::Lite

Sometimes I just want to save the xml versions of the SOAP requests and responses (generally to share with someone else).

The SOAP::Lite perldoc page points to using SOAP::Trace. It has lots of good stuff but its a bit heavy. And in my case, I'm trying to figure out how to log response and requests over http so the following will hopefully help simplify my next search. :)

here is my approach, first the code (all good things start in code :) :
package SOAP::HTTP::Logging;

use warnings;
use strict;

use Exporter;

our @EXPORT = qw(

our $logfile;
our $logboth = 0;
our $log_request = 0;
our $log_response = 0;

# setup global options
sub enable_logging { $log_request = $log_response = 1; }
sub disable_logging { $log_request = $log_response = 0; }
sub soap_log_file { $logfile = shift if @_; return $logfile; }

# logging sub
sub log_message {
my $in = shift; # SOAP::Lite pushes in the object its called on

# only log configured and enabled

How to confuse ssh

I'm sure there are lots of ways to confuse ssh, just like there lots of ways to confuse me. But this is the one i found.

At work we run Solaris with lots of old tools. I end up building lots of new stuff for my self and then having to work around the old ones. When I worked in a Irix shop, I had to do the same thing but for different reasons.

One of the oddities that I run into is that our login shells are often csh. I like bash or ksh in a pinch. Well this means that I had to hack together a .cshrc that checks for bash and then execs it, leaving me with a perfect world of bash.

It looks something like this:
if ( $_ == "/bin/which" || $_ == "/usr/bin/which" ) then
set which="true"

set BASH = "$PWD/bin/bash"

if ( -x $BASH && $which != "true" ) then
exec $BASH

set BASH = "/usr/bin/bash"

if ( -x $BASH && $which != "true" ) then
exec $BASH
Well ssh does not like this for executin…

Javascript and NaN

I always seem to forget but you have to use the function isNaN to test javascript variable. I always want to write it wrong with value == NaN...

Here is an example of it (not a great one but a general idea of what's going on):
var num = Number(v);

if ( isNaN(num)) console.debug("v is not a number");
Such a strange thing that num isn't null or that it doesn't throw an exception. The crazy typing stuff inside of languages :)

Template Toolkit virtual methods with less mess

Template Toolkit has a very cool feature called Virtual Methods. It provide methods on native Perl5 data types.

An example is this (In TT syntax):
[% listy = [ 47, -47, 42, -55, 4, -24 ] %]
[% listy.size %]
[% listy.join(', ') %]
Often its called autoboxing. Well TT allows you to define your own. The details are available in the TT VMethod Manual. In that page there is a TODO about a method called define_vmethod.

Since I like to use TT from CGI::Application and I don't like to actually see its guts. I choose to find a way to add a vmethod without the unpretty $Template::Stash::LIST_OPS->{} syntax. I think that is quite unfriendly to TT noobs like most of my coworkers.

I found that define_vmethod is available off the TT Context object. This is available with $tt->context or in my case off plugin TT object in CGI App $self->tt_obj->context.

The parameters are context->define_vmethod( $type, $name, \&implementation )
$type - data type, 'scalar', 'list…

vim - insert current filename

When writing comments in source code or for some other reason than I can think of right now, I like to include the current filename into current vim session.

Enter insert modeuse following key sequence: ctrl-r %POOF! You have the filename (it does include the path).

In vim % is the current filename. It is very useful when writing mappings or running commands from the : prompt.

Frozen Perl 2009

Frozen Perl 2009 is right around the corner, well its on Sat Feb 6. Be there or be cubed (ice cube)...

Not really goodbye to Google Notebook

I really like Google Notebook but it seems that Google doesn't see it as having many more features to add. Fair enough, they aren't killing it, just not adding to it. Maybe that is called feature complete.

Today at work I was wishing for an Open Source version of it that I could install locally for priority work stuff. Maybe its time to start creating one... with all that free time :)

Google Notebook Blog - Statement about Development Stopage

Weird make/gmake error

When gmake'ing tools, I started to get strange errors like:
/bin/bash: complete: command not found
/bin/bash: /home/users/lcarmich/etc/bash_completion: line 220: syntax error near unexpected token `(('
What I forget is that I build my own newer bash version (3.x) to be able to use the shell completions (and many other features) and the system default is 2.x. Don't ask about that. But gmake grabs the shell first in your path (older bash in my case).

I found that you can grab different version parts from bash in the BASH_VERSINFO array. Well I only really care about major version and this is available in variable BASH_VERSINFO or array element 0 BASH_VERSINFO[0]

I updated my bashrc to check version then load completions:
if [ -r $BASH_COMPLETION -a $BASH_VERSINFO -ge 3 ]; then
For future reference you can get array elements in bash with ${VARNAME[INDEX]}. For example:
mnsdev11 (netcat-0.7.1)$ echo $BASH_VERSION
mnsdev11 (netcat-0.7.1)$ echo $BASH_V…

Exobrain Note: Benchmark CGIs under Apache::Registry and without

Exobrain Note
Need to benchmark some of our internal CGIs under Apache::Registry and without to see what the performance gain might be.

I think building a client for the apps with WWW::Mechanize and WWW::Mechanize::Timed would be enough to evaluate the change.

SOAP::Lite and Test::MockModule

IntroAs part of a project to convert an existing SOAP client library over to a new version of calls, I found that I needed a way to test potential faults and new data formats without requiring live calls. After I built it for the new stuff, I used it to test existing calls and errors as well. This code was running live for several years before I needed to change it for the new version, this inspired me to be very careful and invest in testing. On a side note, I found plenty of bugs just creating the test suite in the existing code that had been there for years.
Creating the Test SuiteTo create the test suite, I needed to pretend to get SOAP server responses. I turned to Test::MockModule to step in and provide hooks to return them. I found that mocking these responses were pretty straightforward once I figured out how to use the SOAP::Lite deserializer. (like most things, its easy once you know how)

The following outlines what I did to create the suite, well at least the process of usin…

vim makes me pasty when pasting

For a while its bugged me that when I paste into a vim session with all nice options turned on, it formats the crap out of it. At some point setting set noautoindent stopped fixing this problem.

Thank goodness that I found a solution, it probably exists for years but with vim sometimes knowing the right verbage is the key to a finding how to do it. (its probably been there since version 5.0)

Here is how it works for me:
pre paste - :set paste
paste in code/snippet/sql/guinea pig
:set nopasteMan that makes my life so much easier...

Switch between windows in any application on OS X

To switch between windows in any application on OS X use: cmd - `.
I started to think this was only in Safari but it works with any multi-window OS X app.

A new must use module - Test::Exception

I write quite a few unit tests that have methods or subs that throw exceptions (like most good code should... this rule has numerous good exceptions! hahaha, sorry couldn't resist.). A few months ago I ran across Test::Exception.

In the past I would often write unit tests for these cases like:
my $err;
my $obj;

eval { $obj = MyObject->new; };

## constructor expected to fail
$err = $@;
ok $err, "constructor - missing parameters";

## or for methods that aren't suppose to throw
eval { $obj = MyObject->new( server => 'localhost' ) };

## check constructor
$err = $@;
ok !$err, "constructor";
isa_ok $obj, 'MyObject';
This construct became very tiring, verbose and distracts from what is really going on. (Yuck!)

With Test::Exception, I can convert these annoying eval blocks into nicely contain lines.

Example from above:
use Test::Exception;

my $obj;

throws_ok { $obj = MyObject->new; } qr/Error Message to match/,
'constructor - missing parameters&#…

Vim, Tidy and XML - how to make them work together

I use vim as my main IDE. I found that I often have to muck and view XML. Often this XML is from debugging logs or output that don't add whitespace (and why would they).

The following is my setup for a small vim filetype mapping to automatically clean up xml using HTML tidy. Another great tool.
How To First, You'll need $HOME/.vim and $HOME/.vim/ftplugin directories.

Create them, if they don't exist:
mkdir -p $HOME/.vim/ftplugin
Second, create a new ftplugin for xml.
vim $HOME/.vim/ftplugin/xml.vim
Third, add mapping for tidy (I use F3)
" tidy for xml -q makes it quiet
map <F3> :%! tidy -q -i -xml % <CR>Now when you edit a file with a type of xml, you can press F3 your xml will be tidy'd. This does require that you save the xml file somewhere first (you'll need a valid filename).
I've only used this on UN*Xs OSs
I use vim 7.x, it will probably work on 6
I enable filetype support in my .vimrc with command filetype plugin indent on