TSocket::write_partial() send() <Host: localhost Port: 9091>Broken pipe


#1

I am using the Pentaho 8.1 ETL tool to copy data from mapd 4.0 to postgres 10.4. I can get everything to work as long as the table in mapd is small.

But if I try to copy an absolutely identical table (only difference is that it has more rows - about 100 million). I cannot copy the table out of mapd. I get the error message:

TSocket::write_partial() send() <Host: localhost Port: 9091>Broken pipe

Also, if I do a simple “select count(*) from my_table” I also get the same error message - but, it does actually eventually return the actual number of rows (about 100 million).

I find the various hard coded limits in mapd frustrating. I don’t care how long it takes the query to return - I just want to do a simple 100 million row copy from mapd to postgres.

Are there configuration options you can set in mapd to cause it to allow large queries to run to completion even if the amount of time they take is large?


#2

You can use the pseudo column rowid to run the query in batches.

Select * from table avere rowid between [start] and [start]+100000

I Guess you can place this query on a uncached lookup and call It with a self generated driving table.


#3

You might find these replies to be useful on \cpu mode and --allow-cpu-retry flag

You might want to provide info on your table schema and gpu RAM + host RAM