Mapd jdbc driver along with spark jdbc writes duplicate record


#1

Below is the code i am using in a spark job to write data to mapd core data base. It is inserting multiple duplicate records. Please help me in debugging this issue.

df.write().format(“jdbc”).mode(SaveMode.Overwrite)
.option(“url”, “jdbc:mapd:localhost:9091:mapd”).option(“driver”, “com.mapd.jdbc.MapDDriver”)
.option(“dbtable”, “table”).option(“user”, “mapd”).option(“password”, “*****”)
.save();


#2

@Debadarsini thanks for finding this! It will be fixed in our next release.


#3

Thanks @easy for quick reply.


#4

As JDBC driver is buggy right now. What client interface can be integrated from java code?


#5

At least do we have any eta on when is this going to be fixed?


#6

Hi @Debadarsini,

This is already fixed in master (commit here), and will be in our next release which is targeted for the end of this month.

If you need the fix earlier you could consider building the open source code, not sure if that is an option.

Regards


#7

Hi @darwin ,

I am having the same issue with the JDBC driver. I tried to click on the commit you have posted above to use, but it took me to the github 404 page not found.

Could you repost the commit please?

Thanks,
Abraham


#8

@abeduplaa unfortunately, that repo is private, but the code will be publicly released quite soon.


#9

Apologies, linked to the wrong repo. Post edited!


#10

Hey This fix is generating lot of logs which is filling up disk. Is there any way to change log settings.