Archive

Author Archives: Ross

I had the opportunity to attend and present at the excellent ELAG conference last week in Bratislava, Slovakia.  The event was advertised as being somewhat of a European Code4Lib, but in reality, the format seemed to me to be more in line with Access, which in my mind is a plus.

Being the ugly American that I am, I made a series of provocative statements both in my presentation and in the Twitter “back channel” (or whatever they call hash tagging an event) about vendors, library standards, and a seeming disdain for both.  I feel like I should probably clarify my position here a bit, since Twitter is a terrible medium for in-depth communication and I didn’t go into much detail in my presentation (outside of saying vendor development teams were populated by scalliwags and ne’er-do-wells from previous gigs in finance, communications and publishing).

Here was my point I was angling towards in my presentation:  your Z39.50 implementation is never going to get any better than it was in 2001.  Outside of critical bug fixes, I would wager the Z39.50 implementation has not even been touched since it was introduced, never mind improved.  The reason for this is my above “joke” about the development teams being staffed by people that do not have a library background.  They are literally just ignoring the Z-server and praying that nothing breaks in unit and regression testing.  There are only a handful of people that understand how Z39.50 works and they are all employed by IndexData.  For everybody else, it’s just voodoo that was there when they got here, but is a requirement for each patch and release.

Thing is, even as hardware gets faster, and ILSes (theoretically) get more sophisticated, the Z-server just gets worse.  You would think that if this is the most common and consistent mechanism to get data out of ILSes that we would have seen some improvement in implementations as the need for better interoperability increases, but this is just not a reality that I have witnessed.  With the last two ILSes that I primarily worked with (Voyager and Unicorn) I would routinely, accidentally, completely bring down due to trying to use the Z39.50 server as a data source in applications.  For the Umlaut, I had to export the Voyager bib database into an external Zebra index to prevent the ILS from crashing multiple times a day just to look up incoming OpenURL requests.  Let me note that a vast majority of these lookups were just ISSN or ISBN.  Unsurprisingly, the Zebra index held up with no problems.  It’s still working, in fact.

Talis uses Zebra for Alto.  It’s probably the main reason we can check off “SRU Support” in an RFP when practically nobody else can.  But, again, this means the Z/SRU-server is sort of “outside” the development plan, delegated to IndexData.  Our SRU servers technically aren’t even conformant to the spec, since we don’t serve explain documents.  I’m not sure anybody at Talis even was aware of this until I pointed it out last year.

All of this is not intended to demonize vendors (really!) or bite the hand that feeds me.  It’s also not intended to denigrate library standards.  I’m merely trying to be pragmatic and, more importantly, I’m hoping we can make library development a less frustrating and backwards exercise for all parties (even the cads and scalliwags).

My point is that initiatives like the DLF ILS-DI, on paper, make a lot of sense.  I completely understand why they chose to implement their model using a handful of library standards (OAI-PMH, SRU).  The standards are there, why not use them?  The problem is in the reality of the situation.  If the specification “requires” SRU for search, how many vendors do you think will just slap Yaz Proxy in front of their existing (shaky, flaky) Z39.50 server and call it a day?  The OAI-PMH provider should be pretty trivial, but I would not expect any company to provide anything innovative with regards to sets or different metadata formats.

As long as libraries are not going to be writing the software they use themselves, they need to reconcile the fact that suppliers of their software is more than likely not going to be written by librarians or library technologists.  If this is the case, what’s the better alternative?  Clinging to half-assed implementations of our incredibly niche standards?  Or figuring out what technologies are developing outside of the library realm that could be used to deliver our data and services?  Is there really, honestly, no way we could figure out how to use OpenSearch to do the things we expect SRU to do?

I realize I have an axe to grind here, but this isn’t really about Jangle.

I have seen OpenURL bandied about as a “solution” to problems outside of its current primary use of “retrieving context based services from scholarly citations” (I know this is not what OpenURL’s sole use case is, but it’s all it’s being used for.  Period).  The most recent example of this was in a workshop (that I didn’t participate in) at ELAG about how libraries could share social data, such as tagging, reviews, etc. in order to create the economies of scale needed to make these concepts work satisfactorily.  Since they needed a way to “identify” things in their collection (books, journals, articles, maps, etc.) somebody had the (understandable, re: DLF) idea to use OpenURL as the identifier mechanism.

I realize that I have been accused of being “allergic” to OpenURL, but in general, my advice is that if you have a problem and you think OpenURL is the answer to said problem there’s actually probably a simpler and better answer to this if you approach it from outside of a library POV.

The drawbacks of Z39.88 for this scenario are numerous, but I didn’t go into details with my criticisms in Twitter.  Here are a few reasons why I would recommend away from OpenURL for this (and they are not exclusive to this potential application):

  1. OpenURL context objects are not identifiers.  They are a means to describe a resource, not identify it.  A context object may contain an identifier in its description.  Use that, scrap the rest of it.
  2. Because a context object is a description and not an identifier, it would have to be parsed to try to figure out what exactly it is describing.  This is incredibly expensive, error prone and more sophisticated than necessary.
  3. It was not entirely clear how the context objects would be used in this scenario.  Would they just be embedded in, say, an XML document as a clue as to what is being tagged or reviewed?  Or would the consuming service actually be an OpenURL resolver that took these context objects and returned some sort of response?  If it’s the former, what would the base URI be?  If it’s the latter… well, there’s a lot there, but let’s start simple, what sort of response would it return?
  4. There is no current infrastructure defined in OpenURL for these sorts of requests.  While there are metadata formats that could handle journals, articles, books, etc., it seems as though this would just scratch the surface of what would need context objects (music, maps, archival collections, films, etc.).  There are no ‘service types’ defined for this kind of usage (tags, reviews, etc.). The process for adding metadata formats or community profiles is not nimble, which would make it prohibitively difficult to add new functionality when the need arises.
  5. Such an initiative would have to expect to interoperate with non-library sources.  Libraries, even banding together, are not going to have the scale or attraction of LibraryThing, Freebase, IMDB, Amazon, etc.  It is not unreasonable to say that an expectation that any of these services would really adopt OpenURL to share data is naive and a waste of time and energy.
  6. There’s already a way to share this data, called SIOC.  What we should be working towards, rather than pursuing OpenURL, is designing a URI structure for these sorts of resources in a service like this.  Hell, I could even be talked into info URIs over OpenURLs for this.

We could further isolate ourselves by insisting on using our standards.  Navel gaze, keep the data consistent and standard.  To me, however, it makes more sense to figure out how to bridge this gap.  After all, the real prize here is to be able to augment our highly structured metadata with the messy, unstructured web.  A web that isn’t going to fiddle around with OpenURL.  Or Z39.50.  Or NCIP.  I have a feeling the same is ultimately true with our vendors.

There comes a point that we have to ask if our relentless commitment to library-specific standards (in cases when there are viable alternatives) is actually causing more harm than help.

While what I’m posting here might be incredibly obvious to anyone that understands unicode or Ruby better than me, it was new to me and might be new to you, so I’ll share.

Since Ed already let the cat out of the bag about LCSubjects.org, I can explain the backstory here.  At lcsh.info, Ed made the entire dataset available as N-Triples, so just before he yanked the site, I grabbed the data and have been holding onto it since.  I wrote a simple little N-Triples parser in Ruby to rewrite some of the data before I loaded it into the platform store I have.  My first pass at this was really buggy, I wasn’t parsing N-Triple literals well at all and was leaving out quoted text within the literal and whatnot.  I also, inadvertantly, was completely ignoring the escaped unicode within the literals and sending them verbatim.

N-Triples escapes unicode the same way Python string literals do (or at least this is how I’ve understood it), so 7⁰03ʹ43ʺN 151⁰56ʹ25ʺE is serialized into nt like: 7\\u207003\\u02B943\\u02BAN 151\\u207056\\u02B925\\u02BAE.  Try as I might, I could not figure out how to turn that back into unicode.

Jonathan Rochkind recommended that I look at the Ruby JSON library for some guidance, since JSON also encodes this way.  With that, I took a peek in JSON::Pure::Parser and modified parse_string for my needs.  So, if you have escaped unicode strings like this, and want them to be unicode, here’s a simple class to handle it.

$KCODE = 'u'
require 'strscan'
require 'iconv'
require 'jcode'
class UTF8Parser < StringScanner
  STRING = /(([\x0-\x1f]|[\\\/bfnrt]|\\u[0-9a-fA-F]{4}|[\x20-\xff])*)/nx
  UNPARSED = Object.new
  UNESCAPE_MAP = Hash.new { |h, k| h[k] = k.chr }
  UNESCAPE_MAP.update({
    ?"  => '"',
    ?\\ => '\\',
    ?/  => '/',
    ?b  => "\b",
    ?f  => "\f",
    ?n  => "\n",
    ?r  => "\r",
    ?t  => "\t",
    ?u  => nil,
  })
  UTF16toUTF8 = Iconv.new('utf-8', 'utf-16be')
  def initialize(str)
    super(str)
    @string = str
  end
  def parse_string
    if scan(STRING)
      return '' if self[1].empty?
      string = self[1].gsub(%r((?:\\[\\bfnrt"/]|(?:\\u(?:[A-Fa-f\d]{4}))+|\\[\x20-\xff]))n) do |c|
        if u = UNESCAPE_MAP[$&[1]]
          u
        else # \uXXXX
          bytes = ''
          i = 0
          while c[6 * i] == ?\\ && c[6 * i + 1] == ?u
            bytes << c[6 * i + 2, 2].to_i(16) << c[6 * i + 4, 2].to_i(16)
            i += 1
          end
          UTF16toUTF8.iconv(bytes)
        end
      end
      if string.respond_to?(:force_encoding)
        string.force_encoding(Encoding::UTF_8)
      end
      string
    else
      UNPARSED
    end
  rescue Iconv::Failure => e
    raise GeneratorError, "Caught #{e.class}: #{e}"
  end
end

Jonathan Rochkind recently started a thread on the Code4lib mailing list asking how to register an info URI for SuDocs.  Ray Denenberg responded with an explanation of the process.  I won’t get into my opinions of info URIs or the merits of either side of the ensuing debate that spun out from this thread, but my takeaway was that Jonathan wasn’t really looking for an info URI, anyway.

What Jonathan wanted was:

  • A generic URI model to define SuDocs
  • For this model to be maintained and hosted by somebody other than him
  • If possible, the URIs be resolvable to something that made sense for the supplied SuDoc

I think these are reasonable desires.

I also thought that there were existing structures out there that could meet his requirements without going through the “start up costs” of registering an info URI.  Also, info URIs are not of the web, so after going through the work of creating a ‘standard’, you cannot actually use it directly to figure out what the SuDoc is referring to.

SuDocs (like all other aspects of Government Documents) are arcane, niche and not understood by anyone other than GovDoc librarians, who seem to be rare.  That being said, there is a pretty convenient web presence implicit in SuDoc — in order for a SuDoc to exist, it needs to appear in the GPO’s catalog.  Since anything that appears in the GPO’s catalog can be seen on the web, we have a basis for a dereferenceable URI structure.

The GPO uses Ex Libris’ Aleph and whatever Aleph’s out of the box web OPAC is for their catalog.  Last week, I was Googling for some information about SRU and Aleph and it led me to this page about constructing CCL queries into Aleph (note, please disregard almost everything written on this page about SRU, CQL, etc., as it’s almost completely b.s.).  Figuring there must some way to search on SuDocs, I tried a couple of combinations of things, until I found this page in the GPO catalog.  Ok, so the index for SuDocs is called “GVD”.

This gives us URLs like:  http://catalog.gpo.gov/F/?func=find-c&ccl_term=GVD%3DE%202.11/3:EL%202

Now, this could work, but it’s incredibly awkward.  It’s also extremely fragile since it’s software (in this case, Aleph) dependent, and if it was to break, requires the GPO to redirect us to the right place.

This is, of course, exactly what PURLs were designed to do.  I had never actually set up a PURL and almost didn’t for this, since the purl.org service said that it wasn’t working properly so it would be disabled for another week.  However, all the links were there, so I forged ahead.  I was in the process of setting up a regular PURL, when I ran across partial redirects.  I figured something like this had to exist for PURLs that were used for RDF vocabularies and the like, but wasn’t aware of how they work.

Anyway, they’re extremly simple.  Basically you set up a base URL (http://purl.org/NET/foo/) and anything requested past that base URL (e.g. http://purl.org/NET/foo/bar) will be redirected to the PURL endpoint verbatim.

So, I set up a partial redirect PURL at the base:  http://purl.org/NET/sudoc/

The expectation that it would be followed by a properly URL escaped SuDoc:  E 2.11/3:EL 2 becomes http://purl.org/NET/sudoc/E%202.11/3:EL%202 which then tacks that SuDoc onto http://catalog.gpo.gov/F/?func=find-c&ccl_term=GVD%3D and redirects you to http://catalog.gpo.gov/F/?func=find-c&ccl_term=GVD%3DE%202.11/3:EL%202.

What you have then is a unique identifier for a SuDoc that resolves to a human readable representation of what the URI stands for.  If the GPO changes OPACs or Ex Libris changes Aleph’s URL scheme or the GPO comes up with a better representation of SuDoc, it doesn’t matter as long as the actual SuDoc class number can be used to redirect the user to the new location.

Obviously, there’s an expectation here that PURLs remain indefinitely and that purl.org is never lost to a third party that repurposes it for other uses.  However, there are major parts of the web that rely on purl.org, so there are a lot of people that would fight to not see this happen.

Basically, I think these are the sorts of simple solutions that I feel we should be using to solve these sorts of problems on the web.  We are no longer the center of the information universe and it’s time that we accepted that and begin to use the tools that the rest of the world is using to solve the same problems that everybody else is dealing with.

How many other ‘identifiers’ could be mint persistent, dereferenceable URIs this way?  I look forward to finding out.

By the way, there are currently three possible points of failure for this URI scheme:  purl.org, GPO and me.  I would prefer not to be a single point of failure, so if you would like to be added as a maintainer to this PURL, please let me know and I would be happy to add you.

Let me start this by saying this is not a criticism or a rant against any of the technologies I am about to mention.  The problems I am having are pretty specific and the fact that I am intentionally trying to use “off the shelf” commodity projects to accomplish my goal complicates things.  I realize when I tweet things like this, it’s not helpful (because there is zero context).

I’ve been in a bit of a rut this week.  Things were going ok on Monday, when I got the Jangle connector for Reserves Direct working, announced and started generating some conversation around how to model course reserves (and their ilk) in Jangle.  However, this left me without anything specific to work on.  I have a general, hand-wavy, project that I am supposed to be working on to provide a simple, general digital asset management framework that can either work on the Platform or with a local repository like Fedora, depending on the institution’s local needs or policies.  More on this in some other post.  The short of it is, I need to gather some general requirements before I can begin something like this in earnest, which led me to revive an old project.

When Jangle first started, about 15 months ago, Elliot and I felt we needed what we called “the Base Connector“.  The idea here was that there were always going to be situations where a developer doesn’t have direct, live access to a system’s data, and they would need a surrogate external database to work with.  The Base Connector was an attempt to provide an out of the box application that could simulate the basics of an ILS and be populated with the sorts of data you would get from commandline ‘report’ type applications.  The sort of thing you can cron and write out to a file on your ILS server.  Updates in catalog records.  Changes in user status.  Transactions.  That sort of thing.

After the amount of interest at Code4lib in Janglifying III Millenium, I decided to revisit the concept of the Base Connector.  Millenium’s (and to an extent, Unicorn’s, and there are no doubt others) lack of consistent database access, makes it a good candidate for this duplicate database.  I was hoping to take a somewhat different approach to this problem than Elliot and I had originally tried, however.  I was hoping to be able to come up with something:

  1. More generically “Jangle” and less domain specific to ILSes
  2. Easy to install
  3. Customizable with a simple interface
  4. Something, preferably, that could be taken mostly “off the shelf”, where the only real “coding” I had to do was to get the library data in and the connector API out.  I was hoping all “data model” and “management” stuff could be taken care of already.

In my mind, I was picturing using a regular CMS for this, although it needed to be able to support customized fields for resources.

Here is the rough scenario I am picturing.  Let’s say you have an ILS that you don’t have a lot of access to.  For your external ‘repository’, you’ll need it to be able to accomodate a few things.

  • Resources will need not just a title, an identifier and the MARC record, but also have fields for ISBN, ISSN, OCLC number, etc.  They’ll also need some sort of relationship to the Items and Collections they’re associated with.
  • Actors could be simple system user accounts, but they’ll need first names and last names and whatnot.
  • Collections, I assume, can probably be contrived via tags and whatnot.
  • The data loading would probably need to be able to be done remotely via some commandline line scripting.

I decided to try three different CMSes to try to accomplish this:  Drupal, Plone and Daisy.  I’ll go through each and where I ran into a snag for each.  I want to reiterate here that I know next to nothing about any of these.  My problems are probably not shortcomings of the project themselves, but more due to my own ignorance.  If you see possible solutions to my issues (or know of other possible projects that fit my need even better) please let me know.  This is a cry for help, not a review.

Drupal

One of the reasons I targeted Drupal is that it’s easy to get running, can run on cheap shared hosting, has quite a bit of traction in libraries and has CCK.  I actually got the farthest with Drupal in this capacity.  With CCK, I was able to, in the native Drupal interface, build content types for Resources and Items.  For Actors, I had just planned on using regular user accounts (since then I could probably piggyback off of the contributed authentication modules and whatnot).  Collections would be derived from Taxonomies.

Where things went wrong:

My desire is to decouple the ‘data load’ aspect of the process from the ‘bag of holding’ itself.  What I’m saying is that I would prefer that the MARC/borrower/item status/etc. load not be required to be built in Drupal module, but, instead, be able to be written in whatever language the developer is comfortable with and a simple way of getting that data into the bag of holding.

There are only two ways that I can see to use an external program to get data into Drupal:

  1. Use the XMLRPC interface
  2. Simulate the content creation forms with a scripted HTTP client.

I’m not above number two, but I would prefer not to if there’s a better way available.  The problem is that I can find almost zero documentation on the XMLRPC service.  What ‘calls’ are available?  How do I create a Resource content type?  How do I relate that to a user or an Item?  I have no idea where to look.  I don’t actually even know if the fields I created will be searchable (which was the whole point of making them).

Drupal seems promising for this, but I don’t know where to go from here.

Plone

I really thought Plone was going to be a winner.  It’s completely self-contained (framework, webserver and database all rolled into one installer) and based on an OODB.  Being Python based, I feel I can fall back on Python to build the scripts to actually do the dirty work of massaging and loading the data.  The downside to Plone (and I have looked eye-to-eye with this downside before) is that it and Zope are total voodoo.

It didn’t take me long to run into a brick wall with Plone.  I installed version 3.2.1 thanks to the handy OSX installer and got it up and running.

And then I couldn’t figure out what to do next.  I think I want Archetypes.  I followed the (outdated)  instructions to install it.  I see Archetypes stuff in the Zope control panel.  However, I never see anything in Plone.  I Google.  Nothing.  Feeling that it must be there and I’m just missing something I follow this tutorial to start building new content types.  I build a new content type.  It doesn’t show up in the Plone quick installer.  Nothing in the logs.  I Google.  Nothing.

Nothing frustrates me more than software making me feel like total dumbass.

I am at the point where I think Plone might be up to the task, but I don’t have the interest, time or energy to make it work.  At the end of the day, I’m still not entirely sure that it would meet my basic criteria of the ‘content type’ being editable within the native web framework anyway.  I also have no idea if my plan of loading the data via an external Python (or, even better, Ruby) script is remotely feasible.

Plone got the brunt of my disgruntled tweeting.  This is mainly due my frustration at seeing how well Plone would fit it my vision and being able to get absolutely nowhere towards realizing that goal.

Daisy

What, you’ve never heard of it?  I have a history with Daisy, and I know, without a doubt, it could serve my needs.  The problem with Daisy is that it has a lot of working parts.  To do what I want, you need both the data repository and the wiki running, as well as MySQL.  On top of that, some external web app would need to actually do the Jangle stuff (and, this would most likely be Ruby/Sinatra) interacting with the HTTP API.  This is a lot of running daemons.  A lot of daemons that might not be running at any given time which would break everything.  Daisy is a lot of things, but it’s not ‘self-contained’.

This is not a criticism.  If I was running a CMS, this would be ok.  When I was developing the Communicat, this was ok.  Those are commitments.  Projects that you think, “ok, I’m going to need to invest some thought and energy into this”.

The bag of holding is a stop-gap.  “I need to use this until I can figure out a real way to accomplish what I need”.  Maybe it’s the ultimate production service.  That’s fine, but it needs to scale down as far as it scales up.  I literally want something that somebody can easily get running anywhere, quickly and start Jangling.

If anybody has any recommendations on how I can easily get up and running with any of the above projects, please let me know.

Alternately, if anybody knows something else, a simple, remotely accessible dynamic, searchable data store, definitely enlighten me!  I realize the irony of this plea, given who I work for, but the idea here is for something not cloud based, since I would like for the user to be able to load in their sensitive patron data without having to submit it to some third party service.  There’s also the fact that there’s no front end that I can just ‘plug in’ to manage the data.

If I can’t get anything off the shelf working, I think I’ll be reduced to writing something simple in Merb or Sinatra with CouchDB or Solr or something.  I was really hoping to have to avoid doing this, though.

This Thursday, February 19th, from 89PM EST, I’ll be a guest DJ on our local NPR affiliate, WUTC. One of their promotions during their membership drive is to be a DJ for an hour, so Selena got me that for my birthday.

It’s easily one of the most exciting birthday presents I’ve ever gotten.

My playlist isn’t going to make history or anything, but I think it’s a pretty good cross-section of songs I really like that fit into the general formatting of the station. I had to make a few compromises on selections for time considerations (an hour goes by fast) and the songs aren’t necessarily a compilation of my favorite tunes ever. Most aren’t even my favorite by the artist in particular, but, for one reason or another (length, tone or language) made the final cut.

It’s been kind of a fun (and sometimes agonizing) study in curation.

Anyway, if you’re interested in hearing me on the radio, tune into 88.1 on Thursday, or listen live on the net (Windows Media).

Today in the #Code4Lib channel:

[1:36pm] jrochkind_lunch: woohoo! NYU is moving forward with Umlaut.
[1:40pm] edsu: nyu++
[1:41pm] mjgiarlo: jrochkind++
[1:42pm] rsinger: awesome
[1:43pm] rsinger: jrochkind++
[1:43pm] jtgorman: jrochkind++
[1:43pm] charper joined the chat room.
[1:43pm] rsinger: more_suckers++
[1:43pm] rsinger: oops
[1:43pm] rsinger: i mean…
[1:43pm] rsinger: nyu++
[1:43pm] charper left the chat room. (Client Quit)

I am not a programmer.

Since I first began writing code, my approach to learning a new language has been to take something that does the sort of thing I am looking for and start fiddling, seeing the results of the fiddling (most likely through some sort of error message) and refiddle until I start seeing the outcome I was looking for.  In mechanical terms, I open the hood, get out my biggest wrench and start whacking at things until the noises stop.  Or, in this case, start.

The arc of languages I primarily worked in at any given time is a pretty good reflection of this approach:  Perl, TCL, PHP, then Ruby with a small foray into Python.  All dynamic, all extremely whackable.  Where whacking doesn’t work, Google (or, previously, Yahoo or, previously, Alta Vista) generally does.  Cut, paste and resume whacking.

The same philosophy applies when it comes to developing new projects.  I know, basically, what I want on the other side, but I have absolutely no idea what it will take to get there.  Generally this means I’ll pick up the nearest tool on hand (usually a red colored wrench) and start whacking until I see what I want.  That the red wrench isn’t the right tool for the job isn’t the point, since I’m only looking for the destination, not the best route there (since I have no idea how to get there in the first place).  The more comfortable I get with a tool, the more likely I am to nest with it, since the detour of finding (and learning how to use) another tool slows me down from reaching the goal.

The perfect example of this was WAG the Dog.  PHP was a ridiculous language to try to use for it, but ping, ping, ping, ping, it worked!

So it stands to reason that I’ve never really taken to Java.  Java is not whacking.  Java is slowly, verbosely and deliberately constructing the specific parts you need to accomplish your goal.  Java is a toolbox full of parts and pieces I do not know the names of, what they do or how they would even do anything, much less the job I’m trying to accomplish.  Java is to my career what a Masters in Mechanical Engineering is to a wrench.  I don’t use Java because I don’t even know the questions to ask to get started in the right direction.

The irony is that when I was hired by Talis, I was ‘assigned’ (that’s a stronger term than really applies) to an entirely Java-based project, Keystone.  To this day, some 15 months later, I have contributed exactly 0.0 lines of code towards Keystone.

I am not a programmer.

However, I am a tinkerer.

In an effort to eat our own dogfood, I had begun to write a Jangle connector for our library management system, Alto.  Alto is built on Sybase and we already had a RESTful SOA interface, the aforementioned Keystone.  It would have been logical for me, were I a real programmer, to take Keystone, add some new models and routes and call it a connector.

But that’s not how I roll.

Initially, I took to using the JangleR Ruby framework to build the Alto connector, since all it would require is to query the appropriate tables and ping, ping, ping, ping things until JRuby and Warbler could give me a .war file.

Sybase, however, does not take well to whacking.  Especially from Ruby.  ActiveRecord-JDBC didn’t work.  Not sure if it was our particular schema or JDBC setup or just ActiveRecord, but no dice.  I couldn’t get Ribs to work at all, which is just as well, probably.  Finally, I had success just using java.sql objects directly in JRuby, but, since I really didn’t know what I was doing, I started worrying about connection pooling and leaving connections open and whatnot.  No need to show off that I have no idea by gobbling up all the resources on some customer’s Alto server.

At one point, on a lark, I decided to try out Grails, Groovy‘s web framework inspired by Rails, to see if I could have more luck interacting with Sybase.  My rationale was, “Keystone uses Hibernate, GORM (Grails’ ORM) uses Hibernate, maybe it will work for me!”.  And, it did.

So here I am, one week into using Groovy.  Just like I used Rails as an introduction to Ruby, Grails serves that purpose with Groovy pretty well.  I can’t say I love the language, but that’s purely my bias; anything that isn’t Ruby, well, isn’t Ruby.  I am certainly doing some thing inefficiently since I am learning the language as I go along.  The fact that there just isn’t much documentation (and the existing documentation isn’t all that great) doesn’t help.

For example, none of my GORM associations work.  I have no idea why.  It could very well be the legacy Sybase schema, or I might be doing something wrong in my Domain Model class.  I don’t have any idea and I don’t have any idea where to look either for an appropriate error or for a fix.  It’s not a huge issue, though, and so far I’ve just worked around it by writing methods that do roughly what I would have needed the associations to do.  Ping, ping, ping.

I also cannot make Services work the way they show in the documentation.  My controllers can’t find them when I do it like the docs, my domain models can’t find them when I do it like the doc…  But it’s no big deal.  I set my methods to be static, call the class directly, and everything works fine.  I’m not doing anything remotely sophisticated with them, so I can keep my hacks for now.

Being able to dynamically load Java classes, iterate over things with a call like foo.each { bar = it.baz } is pretty nice.  I am really amazed at what Groovy offers when it comes to working with Java classes, it’s like being able to audit those M.E. Master’s classes.  I am learning a considerable amount about Java by being able to whack away at objects and classes within the Groovy shell.

I’m not sure that Groovy was really intended for people like me, however.  All of the documentation and even some of the syntax seem to have the expectation that you are a Java developer looking for something dynamic.  It reminds me of a Perl developer going to PHP.  They are syntactically and functionally similar.  In many ways, a Perl developer will find PHP an easier language to use to get a simple, running web application.  And they always have the fallback of Perl, if they need it.  A Python developer that has to use PHP will probably curse a lot.  Groovy seems to have the same sort of relationship to Java.  A Java developer would probably immediately feel comfortable and find it amazingly easy to get up and running.  A Ruby developer (well, this Ruby developer) finds it a little more alien.

Groovy doesn’t have a tremendous amount of native language support for specific tasks, relying instead on the vast amount Java libraries out there to do, basically, anything.  This makes perfect sense and I don’t fault Groovy in the slightest for this choice, but relying on Java for functionality means factories and stream buffers and all the other things Java consists of.  Java developers would feel home.  I find it needs some getting used to.

Also needing to declare your variables.

And I’m sure I’m not really using closures to their fullest potential.

Overall, it’s nice to have this new addition to my toolbox.  Groovy is definitely whackable and development for the Jangle connector has been remarkably fast.  I expect the Aspire (née List) and Prism teams to have something to try out by the end of the month.  And for basically being Java, that ain’t bad.

When and if I rewrite the Alto connector, I’ll probably opt for GroovyRestlet over Grails, but I definitely couldn’t have gotten where I have at this point without the convention and community of Grails.  It’s a really good starting point.

Of course, none of this would have been necessary if it wasn’t for Sybase.  Consider this day one of my campaign to eventually migrate Talis Alto from Sybase to PostgreSQLPing, ping, ping.

III??I7$$$$$77II??++??+==+++I777III?I7777$7II7I???++==?I77IIII?++++??+::~=+++?I7
?+=~+?+++777IIII???II?++?III$$77+==+I777$I??I77II????+=+I777IIIIIIIII?===?III7$$
?+=~+?=~:=+?7777?++I7I++I7II$7II???777777777IIII7777I?+++??I7777777I?+==+7$$$$$$
?+==?I+~:~~~?I7I+==+?I++?77777$Z$$I?++????II$$$$7IIIII??+++?I777III77I++?$$$$$77
+++=?I=::~::~+?I===???++?$$Z$$I???+===++++++++?7$Z$I++?IIIII7$$$I????????I7$$777
?I77II==~==+?++?++++++?I$Z$7+==+++===~~====~==++?I$Z$7+?I$$77$ZZ7???++?III777III
$$$777?II??I7II???+=+?$Z$?+===~~~~=~~~~~~~~~~~~===+7$Z7?~+I77$ZZ$III??7$7??II??I
777777II77IIII??????7$7I?==~~~~~~~~~::~~~~~~~~~:~~=+?I7$I???I7$Z7777II$$7IIIIIII
I?I$$7II7$$$$77I???77I?+=~~~~~~::~~::::::::~~~~~~~~~=+?I7$I?++?I77$7II777?IIIIII
III7IIIII7$Z$777II7III=~~~~~~~:::::::::::::::::::~~~~=?II$$I??++??7777777???III7
III?????I7$$$777IIIII?=~~~~~:::::::,:::::::::::::::~~~+??77$?+++????III??++=?II7
7777II???7$ZZ$$7III?+===~~~~:::::::::::::::::::::::~~~~++I7$7??I7II?+?III+==I777
$$$$77?+==?$$77$III+====~~~~:::::,,,,,,,,,,,:::::::~~~~=+I7$$77$7IIIII77II??I777
$$$$77?+=~+I7I7$7I?=~===~~~~::::,,,,,,,,,,,,,:::::::~~~==+I7Z$77777777$$$77$$$ZZ
$$$$$7777III?I$7I?+=====~~~~::::,,,,,,,,,,,,,:::::::~~~===?7ZZ$7777$$$$$7777$$Z$
Z$$$$$77III?I7Z$I++======~~~~::::,,,,,,,,,,::::::::~~~~===+?$Z$7777I?+++=======+
ZZZOOZ$7II77$$$$I++===+==~~~~~::::,,,,,:::::::::::~~~~====++$ZZ$$77?+==+++==~:::
OOZZOO$$$$$ZZ$$$I?+==++=====~~~~~:::::::::::::::~~~~~~====++$ZOZ$7I++++???+=~:::
OOZZZZ$$$ZZZZ$$$I?+++++=====~~~~:::::::::::::::~~~~~~~====+?7ZZZ??++??I??+=~::::
OOOZZZ$$ZZZZZZZZ7I?+++=+=====~~~::::::::::::::::~~~~~~=+==+?7$$Z++??II??+?????II
Z$$Z$7II7ZZ$7ZOO$7I?I?+++=====~~::,,,:::::::::::~~~~~===+++?$ZOOI???II?+==++??II
$$Z$7IIIIIII$$OOZ$7+??+++======~::,:::::::::::::~~~~~~==+++?ZOZO?++++++=~====+??
7$$I??III7I78O88OZ$???+++=~~~~~~~::~::::::::~~~~~~~~~===++++ZOZZ?++?++++++++++?I
$77?+?II7$$ZD8OOZZ$????++=====~==~~~~~~~~~=====+?I??I?++++++$ZZO::?????++???IIII
$77III7$7ZI~~$OOOZ7????IIIII777$$$7++++==++?I7ZZOOZ7I777I++=7Z87:IO+???????II???
77IIII$II$7O:~IO8O7?+?+7Z$$77$OOOOOZZZI?+?7$ZOOO$I$777$$7+==IZZ+I$$=????????????
?I7$77777$+O$+=+OO7?++I$Z$ZOOOZZZZZ8OO?=~=+I8888888OO8I?+I++$$$?I?+=??IIIIIII???
I77777777I~I$O$?8OO?+=++77ZOZ7777I$$I$II=7$Z+=++====+?+==II+?$$I~:~=IIIIIIIIIIII
77777II7$7~==I$$O8$Z8O$?=++?++?????~~?I=~~+$?~~~=+++==~~~I==?7OZ==:+IIIII7IIIIII
7$777III$7~~=+IOZ8ZI?+~?~~~~=+++=~~=?$+=~~=?I~~~~~~~~~====~=I7Z7==:+IIIIIIIIIIII
7$77III777+~=++77OO$7?==?+==~~~~~===II+=~~=++??=~~~~~~+?=+++7$Z~~::+????IIIIIIII
77$7IIII7$$=:~:,?ZOZZ7II?I+~::~~~~=++++=~~==~~=+=======++==+II$~::=+++???IIIIIII
7777IIIII7OI=,~,+Z87$Z?=++??I77I?+===++=~~~=++++==~~~:~~==++?I$?=+:$??++????????
II7$7777IZ8O=?~:?O87I??++===~~~~=+?++?+=::~==~~+$$7+====++???IZI::=ZI???????????
III$$7I77DDD=~~:~O87III?+++===+I7??=~~+=::~===~=++?$I?+++IIII7Z7==ZD$I?????+++++
III7$77$8888Z?=~~$8Z77II?+++?7$7==?++++=~:~=+?7$+==IZOIIII7I7$O$+?DMO7I?????++++
III77$Z8$.,:7I+=~788Z7777II7OOI?++IDNDZ7+++78DD8?+?$$7OZ77777ZOZ$DNDNZ??I???????
77I7I78Z~…I7I?+7OOOZ$ZZ$$OZZ$$7I$ODNNN8O8DDDOZ$$$$ZOOZII77$ZOOZNMDDNI?I???????
III7$8+…..:?7$ZZO88OZZZ7$OOZOZZZ$OO8NNNNNND8ZZZ$$ZZOOZII7$$ZO8$8ND8D777IIIIIII
777777 ……,=IZOOD8OZZZ7$OO888OZ$ZOOODNNN8ZZ$$ZZ88O8O$I7$$Z888I$8DNDZ77777IIII
III??I:….  .,=7$Z8O888O7IZOOOODNDOOO$Z8OOOZZZ$OO$?7$$II7$$O8OO?I$8D8ZIIIIIIIII
7$$~:..,,..  ..,?I7O88O88O$$OZ$7+==?++=+===::~~~==+?7Z$$$ZOO8DNMZ7,,~87I?????++=
7OZ ………….:+8DDDDD8OOO8Z$II+++=~~~:~:~~=+==+I7OOZZ888DNND777..7O7III77III
ZZ?…………..,INNDDNDDDD88D8$7I++++==~~=+++=+??IZOOOO88DNMMD$7I7,~D$I$$$7777
$I,.. . … .. .=ONMMMNNNNDD8888$7I77?++++++++?I$7I$ZOO8DDDDDNMNZ$7I$=D87$Z$$$$7
$+,.. …. …,?NDDMMMNNNNNNDDOZO$$7$$7$7777$$$$II7ZZOO8DDDDNDN8OZ$II7Z8Z7$$ZZZZ
?,.,……..+ZD88NNNMMN8DNNNNDOO$$$$777$$$7$77I??I7$ZO8DDNDNNNO$OZ$7I778OII7$$$$
=……..,=ZOZ8NMMZNMMM88NNDN88OZZ$7I?==++???==~+?7$ZO8DD88MDM8$$$$ZZ77O8III7$77
,… …:OZ$NMMMN8$NMMM88DDNNNN8ZZZ7II+?+=?=+==+77$OO8NNNODM8OMN777$7$7$8IIIIII7
……?Z$Z88ZZ$ZDZ$8NMM88DDDNNNN8OZ$77?++?++++??77Z88DN8O8NMO$NMD7II777$O7IIIIII
….,I7OZ$77NNNNMO$ODMN8888DDDNM88OZZZII?+?7??7$ZZO8DNOOODNMZ$OMMMO7IIZZZ7?IIIII
….I8NZ7II8MMMMM8$O8MN88888DDDDNND888ZZ$$$Z$$ZO888ND8OOODNN$$Z8MMM7777$$I??I?II
 ..,7O777IZNMMMMMNZZDMD888888DDDDDDNN88888D8D88DNDD8OOOOODND$ZOOMMMNZ7777IIIIIII
…?7ZII77NNDDNN8O8$$7O8OOO88888DDDDDDNNNNNNNNNNDD8OOOOOODDZ$8OONO8MMOII7I?IIIII
,..O8Z777$OOOOO8OZDO$$ZOOOOO88888DDNNDNNNNNNNDDD888OOOZZZ87$O8ZZ8$$NMM77777IIII7
+:~$$$$$$OODNMNZDZO8DO$7$7$ZOOO8888DDDNNNNNNDD8888OOZZZZZ$$ZDZZZD$$$DMD$777II777
~..+ZZZZODNNDZ7$DZZ88O8Z$I+IZOOO8888DDDDNNNDDDD888OZZZZZ$ZDDZZZ8D$IZ8DMMO77$7777
8$:+OZO8DN8Z7$$8OZZZZOZ8D8Z$I$ZOOOO88DDDDDDN8888OOOZZ$$Z8N8OZO8N77II7$NMM7$$7777
ODDO8DNNNZ$$7Z8MZZZZZZOOOO8DO$$$ZOOOOO$$$Z77ZZ$$$ZZO8DNNDOO8NNNDIIIII7$8NMZ77777
O8OOZO8Z7Z$$I$88ZOOOOOOOOO88DND8ZZZZO8888OI78NNNNNNNNNDDDNNNNNDOII7II77$ZMMN7777
$OZO8O$7$$$7I$DDZO88O88888DDDDDDNMNNNDDNNN88NNNNNNNDDDMNMNNDOZO$IIII777778NMDZ7I
8OOZZZ7$$$$$IONNOZO88DDDDDNDMMMMNDDDDDNNNMN8MNNNDDDMMMNNNOZZZOO7IIIIII77$ZONNMN8
                                                                 GlassGiant.com

6:41 AM: Che pooped.

6:47 AM: We realize that Mollie, our Cocker Spaniel, apparently has slept all night in our bed with a “Klingon”.  Remove Klingon.

6:53 AM: Mollie, searching for bits of cat food crumbs that might have fallen on the floor, bumps into and knocks over a full-length mirror, shattering it into tiny bits.  Unfazed, she continues to walk over the glass in search of cat food crumbs.  I scream at Mollie to get out, Che cries.  Hug.

7:01 AM:  Ava, our 100 lb. Rottweiler, loses her footing going down the stairs and tumbles all the way down (17 stairs), taking out Mollie halfway down.  Both crash headlong into the child gate across the bottom of the stairs.  Both shaken, but apparently unhurt.

7:02 AM: Dogs outside.

7:06 AM: Che eats yogurt and oatmeal while watching Curious George.  A calm descends on the house.

7:09 AM: I clean up a million pieces of broken mirror.  Curse Mollie.

7:27 AM: Che and I brush our teeth, put on our shoes.

7:32 AM: Feed the dogs.

7:41 AM: Out the door, in the car, and on the way to the Montessori School.

8:21 AM: Come home to find Ava has eaten a stuffed turkey decoration my mother had sewn.  Batting and little styrofoam beads all over sunroom day bed.

8:39 AM: Shower.

9:01 AM: Talis, North America opens for another day of business.