How to use mapd in golang


#1

how to use mapd in go programming language
can you give some example program for simple queries in go

i want to create a api its input will be a sql query then it should hit mapd database and then return query result

can someone help me


#2

Hi nagulan,

We don’t currently have a native Go database/sql-compatible driver, but you can generate Thrift bindings and call them directly.

First let’s generate those bindings. Most Linux distros still have old versions of Thrift in their package, so I’ll use the Thrift Docker image to stay up-to-date & cross-platform. Be sure to run this in the directory containing mapd.thrift and completion_hints.thrift.

docker run \
  -v $PWD:/data \
  -u $(id -u):$(id -g) \
  thrift:0.11.0 \
  thrift \
    -o /data \
    -r \
    --gen go:package_prefix=nagulan/gomapd/gen-go/ \
    /data/mapd.thrift

Pay attention to the package_prefix=nagulan/gomapd/gen-go/, that’s where we’ll put the generated bindings in your GOPATH

The generated bindings are placed in a directory named gen-go. Move these to the package_prefix (mentioned above) in your GOPATH:

export GOPATH=$HOME/go
mkdir -p $GOPATH/src/nagulan/gomapd
mv gen-go $GOPATH/src/nagulan/gomapd/

Let’s change to that directory and use dep to manage our dependencies:

cd $GOPATH/src/nagulan/gomapd
go get -u github.com/golang/dep/cmd/dep
dep init

With a bit of luck dep will see the generated bindings in gen-go and automatically figure out which version of Thrift it needs. (In our case we used the thrift:0.11.0 Docker image, for Thrift 0.11.0).

Finally, time to write some code. This connects to mapd_server using the Thrift TCP/binary protocol, calls connect, gets the server version with get_version, then executes a query using sql_execute.

package main

import (
        "context"
        "log"
        "net"
        "os"

        "git.apache.org/thrift.git/lib/go/thrift"

        "nagulan/gomapd/gen-go/mapd"
)

func main() {
        host := "localhost"
        port := "9091"

        trans, err := thrift.NewTSocket(net.JoinHostPort(host, port))
        if err != nil {
                log.Fatalln(os.Stderr, "error resolving address:", err)
        }
        defer trans.Close()
        protocolFactory := thrift.NewTBinaryProtocolFactoryDefault()
        iprot := protocolFactory.GetProtocol(trans)
        oprot := protocolFactory.GetProtocol(trans)
        client := mapd.NewMapDClient(thrift.NewTStandardClient(iprot, oprot))
        if err := trans.Open(); err != nil {
                log.Fatalln(os.Stderr, "Error opening socket to ", host, ":", port, " ", err)
        }

        sid, err := client.Connect(context.Background(), "mapd", "HyperInteractive", "mapd")
        if err != nil {
                log.Fatalln("Error connecting to server:", err)
        }
        log.Println("Session ID:", sid)

        version, err := client.GetVersion(context.Background())
        if err != nil {
                log.Fatalln("Error getting version:", err)
        }
        log.Println("Server version:", version)

        query := "SELECT count(*) FROM flights_2008_10k;"
        results, err := client.SqlExecute(context.Background(), sid, query, true, "", -1, -1)
        if err != nil {
                log.Fatalln("Error getting version:", err)
        }
        log.Println("Results")
        log.Println(results)
}
$ go run mapd_example.go
2018/07/17 00:42:03 Session ID: X9M9BJMqu4WzIuWJ6jvKg6bTF0Z0IFaT
2018/07/17 00:42:03 Server version: 4.0.0-20180620-520eb8b
2018/07/17 00:42:03 Results
2018/07/17 00:42:03 TQueryResult_({RowSet:TRowSet({RowDesc:[TColumnType({ColName:EXPR$0 ColType:TTypeInfo({Type:INT Nullable:true IsArray:false Encoding:NONE Precision:0 Scale:0 CompParam:0}) IsReservedKeyword:false SrcName: IsSystem:false IsPhysical:false})] Rows:[] Columns:[TColumn({Data:TColumnData({IntCol:[10000] RealCol:[] StrCol:[] ArrCol:[]}) Nulls:[false]})] IsColumnar:true}) ExecutionTimeMs:19 TotalTimeMs:20 Nonce:})

Parsing the resultset is an exercise for the reader. :slight_smile: Things are a bit nested, but you can untangle them by looking at how they’re defined in Thrift at: https://github.com/mapd/mapd-core/blob/a62ff0c82075b3c1afdc506489017f9e5652e474/mapd.thrift#L144-L149

It might also be helpful to look through how our Thrift bindings are used in pymapd: https://github.com/mapd/pymapd/tree/master/pymapd


#3

thank you for your time andrew but i need some i have doubts
i want to use this ‘mapd://mapd:HyperInteractive@deepbox.thickdata.io:9091/mapd?’‘protocol=binary’ url to connect to database in single line

and while using client.SqlExecute( ) i was supposed to sent only query
but in above you sent has so many parameters to sent to database
is there any way to use this in simple way like in posgres
that is
connStr := “postgresql://root:login123@thickset-dev-postgres.cu9mypqoftvu.us-
eas1.rds.amazonaws.com
db, err := sql.Open(“postgres”, connStr)
query := c.Param(“query”)
rows, err := db.Query(query)

suppose if there is no option to use in this way you can reply me


#4

Unfortunately we don’t have a database/sql driver right now, which is what would give you that much simplified usage.

As for client.SqlExecute(), most of the parameters can be ignored/defaults used, at least initially. The method is defined at https://github.com/mapd/mapd-core/blob/a62ff0c82075b3c1afdc506489017f9e5652e474/mapd.thrift#L497 :

TQueryResult sql_execute(
  1: TSessionId session,
  2: string query,
  3: bool column_format,
  4: string nonce,
  5: i32 first_n = -1,
  6: i32 at_most_n = -1
) throws (1: TMapDException e)

where:

  • session is the session ID obtained from connect()
  • query is your SQL query
  • column_format determines if the results are returned column-wise (true) or row-wise (false). (Struct of Arrays vs Array of Structs)
  • nonce might be useful for tracking results from multiple queries at once
  • first_n limits the resultset to the first n results, useful in case you forget to add a LIMIT in your SQL
  • at_most_n throws an exception if your query would return too many results.

#5

@ andrew
i got an error while pulling from docker
[FAILURE:arguments:1] Could not open input file with realpath: /data/mapd.thrift


#6

What directory are you running in? It needs to be run in the directory containing mapd.thrift and completion_hints.thrift, either a clone of https://github.com/mapd/mapd-core or, if you’re running Community Edition, the extracted version of the tarball or /opt/mapd if installed from yum/apt.