All posts by squish

dnstraverse

ramblings

I’m pleased to announce the first beta of dnstraverse, a Ruby Gem API and associated command line program that performs a similar job to my (Perl based) dnscheck service.

Many people have asked for the source to dnscheck over the years but it was written as a prototype and regrettably I’d be very embarrassed if I released it. So I’ve re-written it from the ground up in OO Ruby and the gem is the result.

Conceptually dnstraverse does the same thing as a DNS resolver, with the major difference that it doesn’t just pick the fastest answer it picks all the answers. It can then calculate the %age probabilities of which answer will be returned. It’s a great way of debugging DNS problems such as lame server delegation and intermediate server connectivity issues. dnstraverse is more standards compliant than dnscheck and returns errors in a more meaningful way.

Please give dnstraverse a go and let me know what you think. It’s beta, so there may be issues. If so, drop me a note and I’ll be more than happy to help you out.

On an aside note, I love Ruby as a language. I’m not sure I can ever go back to Perl again. Unfortunately, Perl still wins hands down for support modules, documentation and system administration.

Talking to Sharepoint Lists with Perl

ramblings

I’ve recently done some work to talk to Sharepoint with Perl and thought I would share my experiences. I couldn’t find any example code out there in the wild for doing this, so I had to figure a lot of this out by trial and error. It’s actually quite simple once you’ve got it set up. I hope this helps someone.

This code shows you how to connect via the Web Services interface with NTLM authentication (i.e. standard Windows authentication) to manipulate Lists, but you could do almost anything.

You will need:

  • SOAP::Lite for talking to Sharepoint Web Services interface
  • LWP::Authen::Ntlm to enable LWP to talk NTLM
  • Authen::NTLM which LWP::Authen::Ntlm uses for the NTLM authentication

Some information sources that you’ll find useful:

  • MSDN Sharepoint Web Services documentation
  • Go to /_vti_bin/lists.asmx on your Sharepoint server for a lot of useful information
  • Go to /_vti_bin/lists.asmx?WSDL for the WSDL definitions (if all else fails)

So here’s how to get started:

use LWP::UserAgent;
use LWP::Debug;
use SOAP::Lite on_action => sub { "$_[0]$_[1]"; };
import SOAP::Data 'name', 'value';
our $sp_endpoint = 'http://sp.example.com/sites/mysite/_vti_bin/lists.asmx';
our $sp_domain = 'sp.example.com:80';
our $sp_username = 'DOMAINusername';
our $sp_password = 'xyz';

The SOAP::Lite module needs to be told how to construct the SOAPAction header properly for Sharepoint. The on_action does just this, and means you’ll end up with a SOAPAction appending the URL and the method name together without anything in between (stops the default # that Sharepoint doesn’t want).

if ($debug) {
    LWP::Debug::level('+');
    SOAP::Lite->import(+trace => 'all');
}

Use the above code to turn on debugging if you get errors.

my @ua_args = (keep_alive => 1);
my @credentials = ($sp_domain, "", $sp_username, $sp_password);
my $schema_ua = LWP::UserAgent->new(@ua_args);
$schema_ua->credentials(@credentials);
$soap = SOAP::Lite->proxy($sp_endpoint, @ua_args, credentials => @credentials);
$soap->schema->useragent($schema_ua);
$soap->uri("http://schemas.microsoft.com/sharepoint/soap/");

This complete mess is the necessary steps to get SOAP::Lite to use a properly configured LWP UserAgent to do NTLM authentication. SOAP::Lite uses two UserAgents, one for the main SOAP calls and one for the Schema fetching. Although you don’t need to fetch a schema, I’ve included the proper set up above in case you want to call $soap->service(“$sp_endpoint?WSDL”); for some reason.

$lists = $soap->GetListCollection();
quit(1, $lists->faultstring()) if defined $lists->fault();

That’s all you need to do to get a list of all the lists on your Sharepoint site. And we can print them out:

my @result = $lists->dataof('//GetListCollectionResult/Lists/List');
foreach my $data (@result) {
    my $attr = $data->attr;
    foreach my $a qw/Title Description DefaultViewUrl Name ID WebId ItemCount/ {
        printf "%-16s %sn", $a, $attr->{$a};
    }
    print "n";
}

Or if you need to find a particular list to do operations on it, search for it in the results by looking up the Title with something like this:

sub lists_getid
{
    my $title = shift;
    my @result = $lists->dataof('//GetListCollectionResult/Lists/List');
    foreach my $data (@result) {
        my $attr = $data->attr;
        return $attr->{ID} if ($attr->{Title} eq $title);
    }
    return undef;
}

And here’s another useful subroutine to get all the items in a list:

sub lists_getitems
{
    my $listid = shift;
    my $in_listName = name('listName' => $listid);
    my $in_viewName = name('viewName' => '');
    my $in_rowLimit = name('rowLimit' => 99999);
    my $call = $soap->GetListItems($in_listName, $in_viewName, $in_rowLimit);
    quit(1, $call->faultstring()) if defined $call->fault();
    return $call->dataof('//GetListItemsResult/listitems/data/row');
}

That will use the default view. The 99999 is a hack to get all the items and stop the server “paging” the results. Putting this together you’d do something like this:

my $list_id = lists_getid('MyList');
print "List ID is: $list_idn";
my @items = lists_getitems($list_id);
foreach my $data (@items) {
    my $attr = $data->attr;
    # print Dumper($attr);
}

Here’s some code to add a new list item:

my $field_id = name('Field', 'New')->attr({ Name => 'ID'});
my $field_linktitle = name('Field', $title)->attr({ Name => 'Title'});
my $field_something = name('Field', $something_else)->attr({ Name => 'Something_x0020_Else'});
my $method = name('Method', [$field_id, $field_linktitle, $field_something])->attr({ ID => "anything", Cmd => 'New'});
my $batch = name('Batch', $method);
my $in_listName = name('listName' => $list_id);
my $in_updates = name('updates' => $batch);
my $call = $soap->UpdateListItems($in_listName, $in_updates);
quit(1, $call->faultstring()) if defined $call->fault();

The content for Name=”ID” must be “New”. Where it says “anything” it really can be anything, it’s just an identifier for responses. You can also see that spaces are encoded as _x0020_.

my $field_id = name('Field', $sp_id)->attr({ Name => 'ID'});
my $field_something = name('Field', $something_else)->attr({ Name => 'Something_x0020_Else'});
my $method = name('Method', [$field_id, $field_appname])->attr({ ID => $jira_name, Cmd => 'Update'});

The above is for modifying an item. In this case the $sp_id must be set appropriately from the “id” attribute of a list item you previously fetched.

I hope that helps someone. Perhaps one day someone can put the effort in to writing a module to do all this.

James

userextra-goes-wp2.0

ramblings

Lots of people kept asking for usermeta/userextra for WordPress 2.0. I’ve now updated both of them.

The only snag was that for people using usermeta for WP 1.5, if they upgrade without reading the upgrade guide, it will completely break their system. Moral of the story – always take backups of your database before upgrading.

Not that I did that. I upgraded without taking a backup. Stupid me. But thankfully nobody users my web site 😉

Oh yeah, in other news, I’ve modified the CSS of this site to work with the latest Firefox. I don’t think the old CSS was wrong, but Firefox hated it.

usermeta, userextra

ramblings

Y’know, it really has been a long while since I’ve started to program anything from scratch, i.e. a new open source project. However, I’m pleased to say I’ve written a couple of WordPress plug-ins, albeit in PHP which makes me want to throw up.

You can check them out over at wp-plugins.org: Usermeta which adds an API so that arbitrary meta data can be associated to users (a feature they’ve added to 1.6 and I’ve effectively back-ported), and Userextra which extends user profiles to include admin-defined attributes, and provides for category access controls with user-level granularity.

I’m particularly pleased with this as it will allow me to construct a community site for my local development based on WordPress (which is fantastic) and allow each set of residents to be given a private category in which to talk. This all started when I went to the 4th Annual Garden Party at the end of August and met the members of our resident’s association. You could see their eyes light up when they heard I was in web-based IT… 🙂 So anyway, they invite me out for a chinese and one thing leads to another and I’m writing plug-ins…