Columnar data representation


#1

Hi.

I’m trying to benchmark mapd-core data transferring and query execution and one of thing that I want to tune is execution option ‘output_columnar_hint’. I changed it to enable state but it seems nothing changed for most queries I tried. But some queries (for example, picked from test case, “SELECT dd AS key1, COUNT(*) AS value1 FROM test GROUP BY key1 HAVING key1 IS NOT NULL ORDER BY key1, value1 DESC LIMIT 12”) aren’t work (F1108 06:26:04.447276 12206 GroupByAndAggregate.cpp:2879] Check failed: !outputColumnar()).

Second question is about ResultSet data representation. By default mapd-core places query result data into row-oriented data arrays, but it intends convert it to columnar representation between subqueries because query expects columnar data as input. Why you are using row-oriented output rather than columnar? Probably it will be changed in future?

Thank you


#2

Question is still actual


#3

Hi,

Your questions probably need to be clarified and some more specific examples of what you changed and why and what you expect before we can get to real details.

There are options to return query results in row or columnar format, by default we return in columnar format. Somehow I feel this is not the level you are looking at and this is not answering your question.

I would suggest one question at at time and give some more context around where and what you are looking for and looking at. Perhaps github issues may be a better place for this discussion if you want to get into the nity grity of the code.

regards