Jabberwocky

Tokyo Cabinet plugin for Rails

Posted in open source, rails, ruby by elisehuard on July 1, 2009

I made a Rails plugin as a kind of adapter for Tokyo Cabinet. Tokyo Cabinet, apart from having a funny name, is a database engine from (duh) Japan, allowing a few different data storage paradigms.

I got to know it through this post of Ilya Grigorik where he basically says it’s blazingly fast. And when he says it’s fast, I tend to believe him.

So what did I use it for ? I needed to do a multiple shortest path calculation, using the Floyd-Warshall algorithm. I figured storing intermediate results in a Tokyo B+Tree, and table database (for random-length paths) is certainly faster than storing it in your garden variety relational database. It’s definitely slower than doing results in memory … until it isn’t. Storing everything in memory is not a good idea if you want your calculation to be scalable to any size of graph.

I wrote this small plugin tokyo_cabinet4r, which has no greater ambition than to make the use of Tokyo Cabinet in Rails easier, and to a certain extent similar to the use of ActiveRecord (using the Ruby API). If you have any suggestion at all, or you want to use it and a feature’s lacking, let me know. I’ll probably fine-tune and fiddle with it in the next few days, amongst other things.

Advertisements

6 Responses

Subscribe to comments with RSS.

  1. Nicolas Trangez said, on July 1, 2009 at 11:52 pm

    Although I really like TC and the surrounding projects (we studied it quite a lot), when performing graph-manipulations, did you consider using Neo4J (http://neo4j.org)? It’s not Ruby code but JRuby is out there… (Disclaimer: I never used Neo4J myself, only read some articles/docs about it)

    You might be interested in some reading about Pregel (http://googleresearch.blogspot.com/2009/06/large-scale-graph-computing-at-google.html) as well.

  2. John Mettraux said, on July 2, 2009 at 6:04 am

    Hi Elise,

    you could use rufus-tokyo (http://jmettraux.wordpress.com/2009/02/27/rufus-tokyo-018-rufusedo/) and get a the same interface whether going FFI or Hirabayashi-Ruby (what you did). Cabinet and Tyrant.

    Keep up the great work, meilleures salutations,

  3. elisehuard said, on July 2, 2009 at 7:16 am

    Thanks, very interesting ! I’ll look into it.

  4. Nicolas Trangez said, on July 2, 2009 at 9:30 am

    *snif* Where did my comment go?

  5. elisehuard said, on July 2, 2009 at 9:38 am

    there it is 🙂 sorry, i missed it.
    Nicolas: no, i didn’t really look into using Neo4J, but thanks for the pointer. The project is in Ruby, not JRuby, but it may be useful for a next project.

  6. Nicolas Trangez said, on July 2, 2009 at 11:45 am

    Np 😉 Enjoy exploiting graphs, they’re extremely cool/useful/underestimated beasts 🙂


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: