Ruby support via Thrift


#1

Anyone out there using MapD with Ruby (via Thrift)? I generated the Ruby classes from the mapd.thrift IDL file (thrift --gen rb mapd.thrift) and found that the “mapd_types.rb” file it generated has types with circular references, and Ruby just can’t handle it.

2.3.3 :002 > require './app/models/map_d/mapd_types'
NameError: uninitialized constant TDatum
Did you mean?  TDatumVal
	from app/models/map_d/mapd_types.rb:80:in `<class:TDatumVal>'

If I change the order so that TDatum is defined first, we run into a similar problem where TDatumVal is now undefined. There may be other circular references in this file, but this is the first one I tripped on.

Is this a solved problem, or are we breaking new ground here?


#2

This is something we’ve hit in the Python world as well. The issue is that we currently represent arrays as recursive types, which some languages don’t like.

For Python I have a script at https://github.com/mapd/mapd-core/blob/master/SampleCode/fix_recursive_structs.py which circumvents the issue by delaying the initializing a few things until after all the types have been defined.

Alternatively you can simply remove array support. Not desirable, but it will at least let you continue without further hacks:

Modify mapd.thrift to remove these two lines:
4: list<TDatum> arr_val from TDatumVal (~line 43)
4: list<TColumn> arr_col from TColumnData (~line 81)

and then regenerate the ruby bindings with thrift -gen ...


#3

Thanks for the quick reply. Are there plans to change how arrays are represented to avoid this problem, or are you sort of stuck with doing it this way?

I ended up exploiting Ruby’s feature of re-opening classes to simply define “skeleton” constants before where they were being referenced, and then let the real definition come later. For example:

# Hack to avoid circular reference
class TDatum
end

class TDatumVal
  ....
  ARR_VAL => {:type => ::Thrift::Types::LIST, :name => 'arr_val', :element => {:type => ::Thrift::Types::STRUCT, :class => ::TDatum}}
  ....
end

class TDatum
  ....real definition....
end

Thanks again.


#4

One other question related to this. The MapD service in mapd.thrift has this:

  TSessionId connect(1: string user, 2: string passwd, 3: string dbname) throws (1: TMapDException e)

How do I specify the hostname and port to connect to given this API definition? Is that embedded in dbname somehow?


#5

Ah, nevermind, I see that I have to create the thrift transport and protocol objects which define this stuff.

Thrift newbie. :slight_smile: