Failed to get a high quality cardinality estimation


#1

Hi,

When we try to do a group by on the following 4 columns on a table with 500M rows we get this error:

Failed to get a high quality cardinality estimation

col1 TEXT NOT NULL ENCODING DICT(8) - 91
col2 INTEGER NOT NULL ENCODING FIXED(8) - 19
col3 TEXT NOT NULL ENCODING DICT(8) - 15
col4 INTEGER NOT NULL - 6463010

The number at the end of each column is the number of distinct values each column has. What does this error mean? Is there a way to work around it?

Thanks!


#2

Hi @malv,

Thanks for reporting this. Can you tell us what version of MapD Core you are running on?

Regards


#3

We just installed it using the MapD AWS AMI from the Amazon Marketplace two days ago, I assume that must be the latest version but I am not sure.

How do we check the version?


#4

Found it:
MapD Server Version: 3.3.1-20171108-32e7bcc


#5

I recently saw the same behavior on a 1B+ row table. When the number of columns in the GROUP BY statement was reduced to 2 columns the query finished.

Not sure why, but we hit this:

E0105 20:29:54.444176 16366 MapDHandler.cpp:3260] Exception: Failed to run the cardinality estimation query: Query couldn’t keep the entire working set of columns in GPU memory

We have watchdog turned off.


#6

Hi @malv,

Thank you for the info. There may be a pathological case at work here. We’ll investigate.

Regards


#7

#8