How to solve "Query couldn't keep the entire working set of columns in GPU memory"?


#1

Hi,

I am trying mapd at RHEL5.3. However it prompts “Query couldn’t keep the entire working set of columns in GPU memory”. I am using 4 x 16G Nvidia cards.

Does it mean I can only process 64G (16G x 4) dataset? I don’t think so and probably I missed some parameters.
Please help to indicate.

Thanks,

  • Simon

[root@minsky-sh-01 mapd-core]# nvidia-smi
Wed Jun 21 12:09:04 2017
±----------------------------------------------------------------------------+
| NVIDIA-SMI 361.119 Driver Version: 361.119 |
|-------------------------------±---------------------±---------------------+
| GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |
|===============================+======================+======================|
| 0 Tesla P100-SXM2… Off | 0002:01:00.0 Off | 0 |
| N/A 39C P0 37W / 300W | 10816MiB / 16280MiB | 0% Default |
±------------------------------±---------------------±---------------------+
| 1 Tesla P100-SXM2… Off | 0003:01:00.0 Off | 0 |
| N/A 41C P0 39W / 300W | 10816MiB / 16280MiB | 0% Default |
±------------------------------±---------------------±---------------------+
| 2 Tesla P100-SXM2… Off | 0006:01:00.0 Off | 0 |
| N/A 38C P0 37W / 300W | 10816MiB / 16280MiB | 0% Default |
±------------------------------±---------------------±---------------------+
| 3 Tesla P100-SXM2… Off | 0007:01:00.0 Off | 0 |
| N/A 36C P0 39W / 300W | 10816MiB / 16280MiB | 0% Default |
±------------------------------±---------------------±---------------------+

±----------------------------------------------------------------------------+
| Processes: GPU Memory |
| GPU PID Type Process name Usage |
|=============================================================================|
| 0 141297 C ./bin/mapd_server 8479MiB |
| 0 163290 C ./bin/mapd_server 2335MiB |
| 1 141297 C ./bin/mapd_server 8479MiB |
| 1 163290 C ./bin/mapd_server 2335MiB |
| 2 141297 C ./bin/mapd_server 8479MiB |
| 2 163290 C ./bin/mapd_server 2335MiB |
| 3 141297 C ./bin/mapd_server 8479MiB |
| 3 163290 C ./bin/mapd_server 2335MiB |
±----------------------------------------------------------------------------+

[root@minsky-sh-01 mapd-core]# mapdql --port 19091 -p HyperInteractive < /tmp/tt.sql
User mapd connected to database mapd
Exception: Query couldn’t keep the entire working set of columns in GPU memory
User mapd disconnected from database mapd
[root@minsky-sh-01 mapd-core]# cat /tmp/tt.sql
select count(*) h8_30_to_9
from store_sales, household_demographics , time_dim, store
where ss_sold_time_sk = time_dim.t_time_sk
and ss_hdemo_sk = household_demographics.hd_demo_sk
and ss_store_sk = s_store_sk
and time_dim.t_hour = 8
and time_dim.t_minute >= 30
and ((household_demographics.hd_dep_count = 3 and household_demographics.hd_vehicle_count<=3+2) or
(household_demographics.hd_dep_count = 0 and household_demographics.hd_vehicle_count<=0+2) or
(household_demographics.hd_dep_count = 1 and household_demographics.hd_vehicle_count<=1+2))
and store.s_store_name = ‘ese’
;

[root@minsky-sh-01 mapd-core]# echo “select count() from store_sales;" | mapdql --port 19091 -p HyperInteractive
User mapd connected to database mapd
EXPR$0
575995629
User mapd disconnected from database mapd
[root@minsky-sh-01 mapd-core]# echo "select count(
) from household_demographics;” | mapdql --port 19091 -p HyperInteractive
User mapd connected to database mapd
EXPR$0
7199
User mapd disconnected from database mapd
[root@minsky-sh-01 mapd-core]# echo “select count() from time_dim;" | mapdql --port 19091 -p HyperInteractive
User mapd connected to database mapd
EXPR$0
86399
User mapd disconnected from database mapd
[root@minsky-sh-01 mapd-core]# echo "select count(
) from store;” | mapdql --port 19091 -p HyperInteractive
User mapd connected to database mapd
EXPR$0
211
User mapd disconnected from database mapd


Exception: Query would require a scan without a limit on table(s):
#2

Hi Simon,

There are flags to allow paging into GPU memory or for CPU retry as described here: Is there a memory replacement mechanism for mapd?.

However it seems your problem here is that you have two mapd servers running, both contending for memory. (Visible in the nvidia-smi report). Please try killing your servers and running the system with only one server running.


#3

Thanks Darwin.

Basically I want to check how much performance boost mapd can gain with GPU instead of CPU:

I noticed the first option is to turn off watch dog.
“The first option is to turn the watch dog off which will allow the query to run in stages on the GPU. MapD will then orchestrate the transfer of data through the various layers of our data abstraction to move the data onto the GPU for execution.”

Does it imply following code change:
[root@minsky-sh-01 mapd-core]# git diff MapDServer.cpp
diff --git a/MapDServer.cpp b/MapDServer.cpp
index b4b57a4…7c7bbc1 100644
— a/MapDServer.cpp
+++ b/MapDServer.cpp
@@ -234,7 +234,7 @@ int main(int argc, char** argv) {
LdapMetadata ldapMetadata;
MapDParameters mapd_parameters;
bool enable_rendering = false;

  • bool enable_watchdog = true;
  • bool enable_watchdog = false;
    bool enable_dynamic_watchdog = false;
    unsigned dynamic_watchdog_time_limit = 10000;

However with the compiled mapd, the query is not successful.
[root@minsky-sh-01 mapd-core]# mapdql mapd -u mapd -p HyperInteractive --port 19091 < /tmp/tt.sql
User mapd connected to database mapd
Thrift: Wed Jun 21 13:24:11 2017 TSocket::write_partial() send() <Host: localhost Port: 19091>Broken pipe
Thrift: Wed Jun 21 13:24:11 2017 TSocket::open() connect() <Host: localhost Port: 19091>Connection refused
Thrift: Wed Jun 21 13:24:11 2017 TSocket::open() connect() <Host: localhost Port: 19091>Connection refused
Thrift error: connect() failed: Connection refused
Thrift: Wed Jun 21 13:24:11 2017 TSocket::open() connect() <Host: localhost Port: 19091>Connection refused
Thrift: Wed Jun 21 13:24:11 2017 TSocket::open() connect() <Host: localhost Port: 19091>Connection refused
Thrift error: connect() failed: Connection refused

Is there any other parameter I should tried?

I have to share the server with someone else so there is 2 mapd server running simultaneously.

Thanks,

  • Simon

#4

I reversed the “enable_watchdog” change and killed another mapd instance.

Now there is only 1 mapd instance.

mapdql> \memory_summary
MapD Server Memory Usage
CPU RAM IN BUFFER USE : 0.00 MB
GPU VRAM USAGE (in MB’s)
GPU MAX ALLOC IN-USE FREE
0 16152.81 0.00 0.00 16152.81
1 16152.81 0.00 0.00 16152.81
2 16152.81 0.00 0.00 16152.81
3 16152.81 0.00 0.00 16152.81

mapdql> \timing
mapdql> select count(*) h8_30_to_9
…> from store_sales, household_demographics , time_dim, store
…> where ss_sold_time_sk = time_dim.t_time_sk
…> and ss_hdemo_sk = household_demographics.hd_demo_sk
…> and ss_store_sk = s_store_sk
…> and time_dim.t_hour = 8
…> and time_dim.t_minute >= 30
…> and ((household_demographics.hd_dep_count = 3 and household_demographics.hd_vehicle_count<=3+2) or
…> (household_demographics.hd_dep_count = 0 and household_demographics.hd_vehicle_count<=0+2) or
…> (household_demographics.hd_dep_count = 1 and household_demographics.hd_vehicle_count<=1+2))
…> and store.s_store_name = ‘ese’
…> ;
Exception: Query couldn’t keep the entire working set of columns in GPU memory

Besides fallback to CPU mode, is mapd possible to run with my current dataset on GPU?

  • Simon

#5

Simon,

You will have to share a little more details

How are you starting MapD, when you say you reversed the enable_watchdog change, do you mean you changed the parameter. Please share the command you are starting mapd with or the mapd.conf file you are running.

You should be able to simply set enable-watchdog = false in your config file or as a parameter to starting mapd_server and have the query execute step wise on GPU.

It would also help if you shared a clean mapd_server.INFO log to help us see what you have configured.

regards


#6

Hi,
Thanks for the help.

“I reversed the enabled_watchdog change”, I mean I reversed following code change which is mentioned in comment 3rd:
[root@minsky-sh-01 mapd-core]# git diff MapDServer.cpp
diff --git a/MapDServer.cpp b/MapDServer.cpp
index b4b57a4…7c7bbc1 100644
— a/MapDServer.cpp
+++ b/MapDServer.cpp
@@ -234,7 +234,7 @@ int main(int argc, char** argv) {
LdapMetadata ldapMetadata;
MapDParameters mapd_parameters;
bool enable_rendering = false;

  • bool enable_watchdog = true;
  • bool enable_watchdog = false;
    bool enable_dynamic_watchdog = false;
    unsigned dynamic_watchdog_time_limit = 10000;

So we can forget about the above.

I should also note that I had following change to bypass a scan limit checking:
[root@minsky-sh-01 mapd-core]# git diff QueryEngine/Execute.h
diff --git a/QueryEngine/Execute.h b/QueryEngine/Execute.h
index 651fd1c…9af6478 100644
— a/QueryEngine/Execute.h
+++ b/QueryEngine/Execute.h
@@ -368,7 +368,7 @@ class Executor {
void interrupt();
void resetInterrupt();

  • static const size_t high_scan_limit{10000000};
  • static const size_t high_scan_limit{10000000000};

private:
void clearMetaInfoCache();

With the above change, I start mapd server with following command line:
[root@minsky-sh-01 mapd-core]# ./startmapd --enable-watchdog=false

And perform following test:(It take a long time):
[root@minsky-sh-01 mapd-core]# mapdql mapd -u mapd -p HyperInteractive --port 19091 < /tmp/tt.sql
User mapd connected to database mapd
Thrift: Thu Jun 22 17:21:39 2017 TSocket::write_partial() send() <Host: localhost Port: 19091>Broken pipe
Thrift: Thu Jun 22 17:21:39 2017 TSocket::open() connect() <Host: localhost Port: 19091>Connection refused
Thrift: Thu Jun 22 17:21:39 2017 TSocket::open() connect() <Host: localhost Port: 19091>Connection refused
Thrift error: connect() failed: Connection refused
Thrift: Thu Jun 22 17:21:39 2017 TSocket::open() connect() <Host: localhost Port: 19091>Connection refused
Thrift: Thu Jun 22 17:21:39 2017 TSocket::open() connect() <Host: localhost Port: 19091>Connection refused
Thrift error: connect() failed: Connection refused

— The log -------
[root@minsky-sh-01 mapd_log]# cat mapd_server.INFO
Log file created at: 2017/06/22 17:09:04
Running on machine: minsky-sh-01
Log line format: [IWEF]mmdd hh:mm:ss.uuuuuu threadid file:line] msg
I0622 17:09:04.255259 69708 MapDServer.cpp:493] MapD started with data directory at '/home/simon/mapd-core/MYDB’
I0622 17:09:04.255587 69708 MapDServer.cpp:500] Watchdog is set to 0
I0622 17:09:04.255600 69708 MapDServer.cpp:522] cuda block size 0
I0622 17:09:04.255609 69708 MapDServer.cpp:523] cuda grid size 0
I0622 17:09:04.255616 69708 MapDServer.cpp:524] calcite JVM max memory 1024
I0622 17:09:04.266677 69708 MapDHandler.cpp:244] MapD Server 3.1.1dev-20170622-8083204
I0622 17:09:04.583073 69708 CudaMgr.cpp:127] Using 4 Gpus.
I0622 17:09:04.583441 69708 DataMgr.cpp:120] cpuSlabSize is 4096M
I0622 17:09:04.583483 69708 DataMgr.cpp:122] reserved GPU memory is 128M includes render buffer allocation
I0622 17:09:04.583499 69708 DataMgr.cpp:132] gpuSlabSize is 2048M
I0622 17:09:04.583509 69708 DataMgr.cpp:132] gpuSlabSize is 2048M
I0622 17:09:04.583518 69708 DataMgr.cpp:132] gpuSlabSize is 2048M
I0622 17:09:04.583528 69708 DataMgr.cpp:132] gpuSlabSize is 2048M
I0622 17:09:04.583639 69708 FileMgr.cpp:116] Read table metadata, Epoch is 0 for table data at '/home/simon/mapd-core/MYDB/mapd_data/table_0_0/'
I0622 17:09:04.583772 69708 Calcite.cpp:133] Creating Calcite Handler, Calcite Port is -1 base data dir is /home/simon/mapd-core/MYDB
I0622 17:09:04.583781 69708 Calcite.cpp:42] Creating Calcite Server local as JNI instance, jar expected in /home/simon/mapd-core/build/bin
I0622 17:09:04.736917 69708 MapDHandler.cpp:290] Started in GPU mode
I0622 17:09:52.867317 69762 MapDHandler.cpp:437] User mapd connected to database mapd
I0622 17:09:52.867825 69762 MapDHandler.cpp:2432] select count() h8_30_to_9 from store_sales, household_demographics , time_dim, store where ss_sold_time_sk = time_dim.t_time_sk and ss_hdemo_sk = household_demographics.hd_demo_sk and ss_store_sk = s_store_sk and time_dim.t_hour = 8 and time_dim.t_minute >= 30 and ((household_demographics.hd_dep_count = 3 and household_demographics.hd_vehicle_count<=3+2) or (household_demographics.hd_dep_count = 0 and household_demographics.hd_vehicle_count<=0+2) or (household_demographics.hd_dep_count = 1 and household_demographics.hd_vehicle_count<=1+2)) and store.s_store_name = ‘ese’ ;
I0622 17:09:52.868764 69762 Calcite.cpp:196] User mapd catalog mapd sql 'select count(
) h8_30_to_9 from store_sales, household_demographics , time_dim, store where ss_sold_time_sk = time_dim.t_time_sk and ss_hdemo_sk = household_demographics.hd_demo_sk and ss_store_sk = s_store_sk and time_dim.t_hour = 8 and time_dim.t_minute >= 30 and ((household_demographics.hd_dep_count = 3 and household_demographics.hd_vehicle_count<=3+2) or (household_demographics.hd_dep_count = 0 and household_demographics.hd_vehicle_count<=0+2) or (household_demographics.hd_dep_count = 1 and household_demographics.hd_vehicle_count<=1+2)) and store.s_store_name = ‘ese’ ;'
I0622 17:09:53.578639 69762 Calcite.cpp:221] Time marshalling in JNI 0 (ms), Time in Java Calcite 710 (ms)
I0622 17:09:53.579694 69762 FileMgr.cpp:116] Read table metadata, Epoch is 808 for table data at '/home/simon/mapd-core/MYDB/mapd_data/table_1_27/'
I0622 17:09:54.834256 69762 FileMgr.cpp:116] Read table metadata, Epoch is 1 for table data at '/home/simon/mapd-core/MYDB/mapd_data/table_1_18/'
I0622 17:09:54.844177 69762 FileMgr.cpp:116] Read table metadata, Epoch is 1 for table data at '/home/simon/mapd-core/MYDB/mapd_data/table_1_9/'
I0622 17:09:54.854181 69762 FileMgr.cpp:116] Read table metadata, Epoch is 1 for table data at '/home/simon/mapd-core/MYDB/mapd_data/table_1_13/'
I0622 17:09:55.114876 69762 BufferMgr.cpp:266] ALLOCATION slab of 4194304 pages (2147483648B) created in 1 ms GPU_MGR:0
I0622 17:09:55.114941 69762 BufferMgr.cpp:266] ALLOCATION slab of 8388608 pages (4294967296B) created in 0 ms CPU_MGR:0
I0622 17:09:55.116724 69762 BufferMgr.cpp:266] ALLOCATION slab of 4194304 pages (2147483648B) created in 1 ms GPU_MGR:1
I0622 17:09:55.118228 69762 BufferMgr.cpp:266] ALLOCATION slab of 4194304 pages (2147483648B) created in 1 ms GPU_MGR:2
I0622 17:09:55.119644 69762 BufferMgr.cpp:266] ALLOCATION slab of 4194304 pages (2147483648B) created in 1 ms GPU_MGR:3
I0622 17:09:55.653095 69762 RelAlgExecutor.cpp:1495] Query ran out of GPU memory, punt to CPU
I0622 17:18:20.745025 71315 BufferMgr.cpp:266] ALLOCATION slab of 8388608 pages (4294967296B) created in 0 ms CPU_MGR:0
I0622 17:18:37.874302 71315 BufferMgr.cpp:266] ALLOCATION slab of 4194304 pages (2147483648B) created in 1 ms GPU_MGR:0
I0622 17:19:21.585608 69762 RelAlgExecutor.cpp:1495] Query ran out of GPU memory, punt to CPU

It is the only one mapd instance running at my machine.

  • Simon

#7

I am reading following:


“What happens if the dataset cannot fit in GPU memory?

MapD treats the combined memory (VRAM) of all the GPUs on a server as its lowest level cache, i.e. it tries to keep compressed versions of hot partitions of hot columns in GPU memory whenever possible. This approach can easily scale to working sets in the terabytes on a single node. For those cases when the working set cannot entirely fit in GPU memory, MapD caches a greater subset of the data in CPU memory, which although slower is typically significantly larger than available GPU memory. It will then either stream data from CPU to GPU or execute queries simultaneously in CPU and GPU. Although this method is slower than if the needed data fits entirely in GPU RAM, we have still found MapD to be faster than other in-memory solutions in such situations due to the highly optimized nature of the database.”

  • But looks it is not the case in my testing?

Simon


#8

Simon,

There appears to be a crash happening somewhere here.

Would it be possible for you to update your repo to the latest repo and retry your run.

Is there any way we could get access to your schema and data. Are you running on a power machine? If we could get your data we could try it on our power machine and see if we can reproduce your issue.

I have concerns about compatibilty with the older OS as normally we run on more recent releases of redhat 7.

regards


#9

Hi,

Thanks for the response.
I upgraded the git repo to the latest version and the problem is still here.

Yes. I am running it on a powerpc machine.

When I compiled the mapd, I disabled folly with following command (I have some problem when installing folly):
[root@minsky-sh-01 build]# cmake -DCMAKE_BUILD_TYPE=debug -DENABLE_FOLLY=off …
I don’t know whether it is an issue. Is it a must to deploy FOLLY?

For the data sheet, it is generated by tpc-ds data generate tool. I am using 2.4.0.
http://www.tpc.org/TPC_Documents_Current_Versions/download_programs/tools-download-request.asp?bm_type=TPC-DS&bm_vers=2.4.0&mode=CURRENT-ONLY
The data was generated by:

  • dsdgen -scale 250 -terminate n -dir <DATA_DIR>
    Then create table by tpcds.sql installed at tpc-ds directory.
    And import the data generated.

Thanks,

  • Simon

#10

By the way, I attached a gdb to mapd_server, and looks it is terminated by SIGKILL:

[Thread 0x3bfffe28e620 (LWP 52348) exited]
[Thread 0x3fff743accd0 (LWP 52292) exited]

Program terminated with signal SIGKILL, Killed.
The program no longer exists.
(gdb)
The program is not being run.

Thanks,

  • Simon

#11

hi dwayneberry,

Is there any way I can debug it? I am using traditional gdb but looks no useful info can be got.

Thanks,

  • Simon

#12

Hi

Simon I have created the dataset and will load into a x86 machine here, to see if I can reproduce your behavior. If i cannot I will move onto a power machine. From there I can give you more direction of what you could look into.

regards


#13

Great. Thanks for the help. dwayneberry.

  • Simon

#14

Hi dwayneberry,

Are you able to reproduce my issue?
Any other suggestion will also be appreciated.

Thanks,

  • Simon

#15

Hi,

Sorry for the long delay in response, been waiting for releases to land.

I was not able to reproduce your issue, but I beleive the root issue in your case appeared to be bad optimization of the query, it basically was trying to do a cross product join across all tables before trying to apply the filters, Which in turn tried to generate a crazy large intermediate result which was potentially the issue you were seeing.

We had been working on that and a number of other issues in the join space and many of those fixes have landed with v3.2.1.

When I run your query locally on a 4 GPU machine on same dataset I currently get the following result

mapdql> select
..> count(*) h8_30_to_9
..> from store_sales,household_demographics,time_dim,store
..> where ss_sold_time_sk = time_dim.t_time_sk
..> and ss_hdemo_sk = household_demographics.hd_demo_sk
..> and ss_store_sk = s_store_sk
..> and time_dim.t_hour = 8
..> and time_dim.t_minute >= 30
..> and
..> (
..>    (
..>       household_demographics.hd_dep_count = 3
..>       and household_demographics.hd_vehicle_count<=3+2
..>    )
..>    or
..>    (
..>       household_demographics.hd_dep_count = 0
..>       and household_demographics.hd_vehicle_count<=0+2
..>    )
..>    or
..>    (
..>       household_demographics.hd_dep_count = 1
..>       and household_demographics.hd_vehicle_count<=1+2
..>    )
..> )
..> and store.s_store_name = 'ese';
h8_30_to_9
442484
1 rows returned.
Execution time: 100 ms, Total time: 100 ms

i have not had an opportunity to try this new release on power architecture with this data so would be interested if you could give it a try.

I did have to remove some of the not null flags on the create tables to make sure the join keys were identical types between the tables. The definitions need to be identical in MapD currently. I can share my create and load scripts if you are interested.

On a performance note, if you run the query with explicit joins it will perform 20% faster currently as we still have more work to do on the optimizer path

..> count(*) h8_30_to_9
..> from store_sales
..> join household_demographics on ss_hdemo_sk = household_demographics.hd_demo_sk
..> join time_dim on ss_sold_time_sk = time_dim.t_time_sk
..> join store on ss_store_sk = s_store_sk
..> where 
..> time_dim.t_hour = 8
..> and time_dim.t_minute >= 30
..> and
..> (
..>    (
..>       household_demographics.hd_dep_count = 3
..>       and household_demographics.hd_vehicle_count<=3+2
..>    )
..>    or
..>    (
..>       household_demographics.hd_dep_count = 0
..>       and household_demographics.hd_vehicle_count<=0+2
..>    )
..>    or
..>    (
..>       household_demographics.hd_dep_count = 1
..>       and household_demographics.hd_vehicle_count<=1+2
..>    )
..> )
..> and store.s_store_name = 'ese';
h8_30_to_9
442484
1 rows returned.
Execution time: 79 ms, Total time: 79 ms

regards


#16

Hi,

I moved the data over to a power machine (two Telsa P100’s) and ran the same queries on the same data without issue.

I was using the official 3.2.1 power release distribution for my test.

GPU1961 mapd $ nvidia-smi                                                                                              
Thu Aug 31 11:26:23 2017       
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 375.51                 Driver Version: 375.51                    |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|===============================+======================+======================|
|   0  Tesla P100-SXM2...  On   | 0002:01:00.0     Off |                  Off |
| N/A   30C    P0    40W / 300W |   6442MiB / 16276MiB |      0%      Default |
+-------------------------------+----------------------+----------------------+
|   1  Tesla P100-SXM2...  On   | 000A:01:00.0     Off |                  Off |
| N/A   32C    P0    40W / 300W |   6442MiB / 16276MiB |      0%      Default |
+-------------------------------+----------------------+----------------------+
                                                                               
+-----------------------------------------------------------------------------+
| Processes:                                                       GPU Memory |
|  GPU       PID  Type  Process name                               Usage      |
|=============================================================================|
|    0      2218    G   /usr/lib/xorg/Xorg                               7MiB |
|    0     16962    C   ./bin/mapd_server                             6433MiB |
|    1      2218    G   /usr/lib/xorg/Xorg                               7MiB |
|    1     16962    C   ./bin/mapd_server                             6433MiB |
+-----------------------------------------------------------------------------+
GPU1961 mapd $ uname -a                                                                                                
Linux GPU1961 4.4.0-87-generic #110-Ubuntu SMP Tue Jul 18 12:53:44 UTC 2017 ppc64le ppc64le ppc64le GNU/Linux
GPU1961 mapd $ time /media/1TBSATA/michael/mapd-ee-3.2.1-20170826-327e7ee-Linux-ppc64le/bin/mapdql -p HyperInteractive 
User mapd connected to database mapd
mapdql> \timing
mapdql> select
..> count(*) h8_30_to_9
..> from store_sales
..> join household_demographics on ss_hdemo_sk = household_demographics.hd_demo_sk
..> join time_dim on ss_sold_time_sk = time_dim.t_time_sk
..> join store on ss_store_sk = s_store_sk
..> where 
..> time_dim.t_hour = 8
..> and time_dim.t_minute >= 30
..> and
..> (
..>    (
..>       household_demographics.hd_dep_count = 3
..>       and household_demographics.hd_vehicle_count<=3+2
..>    )
..>    or
..>    (
..>       household_demographics.hd_dep_count = 0
..>       and household_demographics.hd_vehicle_count<=0+2
..>    )
..>    or
..>    (
..>       household_demographics.hd_dep_count = 1
..>       and household_demographics.hd_vehicle_count<=1+2
..>    )
..> )
..> and store.s_store_name = 'ese';
h8_30_to_9
442484
1 rows returned.
Execution time: 109 ms, Total time: 110 ms
mapdql> 

If you could retry with a recent build would be great. Hopefully we can call this closed.

Regards


#17

Hi,

Just to finish the loop here I downloaded and built the mapd-core latest version on the power machine and reloaded the data etc, to confirm there where no issues going that route.

It all appears to be fine and behaving as expected

GPU1961 mapd $ nvidia-smi
Thu Aug 31 13:14:15 2017       
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 375.51                 Driver Version: 375.51                    |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|===============================+======================+======================|
|   0  Tesla P100-SXM2...  On   | 0002:01:00.0     Off |                  Off |
| N/A   30C    P0    40W / 300W |   6442MiB / 16276MiB |      0%      Default |
+-------------------------------+----------------------+----------------------+
|   1  Tesla P100-SXM2...  On   | 000A:01:00.0     Off |                  Off |
| N/A   32C    P0    40W / 300W |   6442MiB / 16276MiB |      0%      Default |
+-------------------------------+----------------------+----------------------+
                                                                               
+-----------------------------------------------------------------------------+
| Processes:                                                       GPU Memory |
|  GPU       PID  Type  Process name                               Usage      |
|=============================================================================|
|    0      2218    G   /usr/lib/xorg/Xorg                               7MiB |
|    0     79717    C   ./bin/mapd_server                             6433MiB |
|    1      2218    G   /usr/lib/xorg/Xorg                               7MiB |
|    1     79717    C   ./bin/mapd_server                             6433MiB |
+-----------------------------------------------------------------------------+
GPU1961 mapd $ uname -a
Linux GPU1961 4.4.0-87-generic #110-Ubuntu SMP Tue Jul 18 12:53:44 UTC 2017 ppc64le ppc64le ppc64le GNU/Linux
GPU1961 mapd $ time /media/1TBSATA/michael/mapd-core/build/bin/mapdql -p HyperInteractive 
User mapd connected to database mapd
mapdql> \version
MapD Server Version: 3.2.2dev-20170831-66c9d53
mapdql> \timing
mapdql> select
..> count(*) h8_30_to_9
..> from store_sales,household_demographics,time_dim,store
..> where ss_sold_time_sk = time_dim.t_time_sk
..> and ss_hdemo_sk = household_demographics.hd_demo_sk
..> and ss_store_sk = s_store_sk
..> and time_dim.t_hour = 8
..> and time_dim.t_minute >= 30
..> and
..> (
..>    (
..>       household_demographics.hd_dep_count = 3
..>       and household_demographics.hd_vehicle_count<=3+2
..>    )
..>    or
..>    (
..>       household_demographics.hd_dep_count = 0
..>       and household_demographics.hd_vehicle_count<=0+2
..>    )
..>    or
..>    (
..>       household_demographics.hd_dep_count = 1
..>       and household_demographics.hd_vehicle_count<=1+2
..>    )
..> )
..> and store.s_store_name = 'ese';
h8_30_to_9
442484
1 rows returned.
Execution time: 156 ms, Total time: 156 ms
mapdql> 

I will mark this post as solved for now.

@Simon If you are still having issues please let us know

regards