For Ian Davis‘ birthday, Danny Ayers sent out an email asking people to make some previously unavailable datasets accessible as linked data as Ian’s present. It was a pretty neat idea. One that I wish I had thought of.
Given that Ian is my boss (prior to about a month ago, Ian was just nebulously “above me” somewhere in the Talis hierarchy, but I now report to him directly) one could cynically make the claim that by providing Ian a ‘linked data gift’ that I would just be currying favor by being a kiss-ass. You could make that claim, sure, but evidently you are not aware of how I hurt the company.
Anyway, as my contribution, I decided to take the data dumps from LibraryThing that Tim Spalding pretty graciously makes available [whoa, in the time that I first started this post until now, the data has gone AWOL, I suppose I did this just in time]. The data isn’t always very current and not all of the files are terribly useful (the tags one, for example, doesn’t offer much since the tags aren’t associated with anything — it’s just words and their counts), but it’s data and between ThingISBN and the WikipediaCitations I thought it would be worth it.
I wanted to take a very pragmatic approach to this: no triple store, no search, no rdf libraries, minimal interface. Mostly this was inspired by Ed Summers‘ work with the Library of Congress Authorities, but, also, if Tim (or, whoever at LibraryThing) saw that making LibraryThing linked data was as easy as a few template tweaks (as opposed to a major change in their development stack) this exercise was much more likely to actually make its way into LibraryThing.
What I ended up with (the first pass released before the end of Ian’s birthday, I might add) was LODThing: a very simple application written in Ruby’s Sinatra framework, DataMapper and SQLite. The entire application is less than 230 lines of Ruby (including the web app and data loader) plus 2 HAML templates and 2 builder templates for the HTML/RDFa and RDF/XML, respectively. The SQL database has three tables, including the join table. This is really simple stuff. The only real reason it took a couple days to create was trying to get the data loaded into SQLite from these huge XML files. Nokogiri is fast (well, Ruby fast), but a 200 MB XML file is pretty big. It was nice to get acquainted with Nokogiri’s pull parser, though.
There are a few things to take away from this exercise.
- When data is freely available, it’s really quite simple to reconstitute it into linked data without any need to depart from your traditional technology stack. There is nothing even remotely semantic-webby about LODThing except its output.
- We now have an interesting set of URIs and relationships to start to express and model FRBR relationships.
- The Wikipedia citations data is extremely useful and could certainly be fleshed out more. One could imagine querying DBpedia or Freebase on these concepts and identifying if the Wikipedia article is actually referring to the work itself and use that. Right now LODThing makes no claims about the relationships except that it’s a reference from Wikipedia.
LODThing isn’t really intended for human consumption, so there’s no real “default way in”. The easiest way to use it is to make a URI from an ISBN:
If you know the LibraryThing ‘work ID’, you can get in that way, too:
Also, you can all of these resources as RDF/XML by replacing the .html with .rdf.
So, Tim, you wrote on the LT API page that you would love to see what people are doing with your data, here you go. It would be even more awesome if it made it’s way back into LT — after all, it would alleviate some of the need for you to have a special API for this stuff.
Also, special thanks to Toby Inkster for providing a ton of help in getting this to resemble something that a linked data aware agent would actually want and finally turning the httpRange-14 light bulb on over my head. He also immediately linked to it from his Amazon API LODifiier, which is sort of cool, too.
I’ll be happy to throw the sources into a github repository if anybody’s interested in them.