Toy benchmark on AWS cloud - GPU usage?


#1

First of all: thank you for open sourcing mapd! I have this toy benchmark for a super-simple aggregation and join of no-bigdata and I just tried out mapd (the AWS marketplace community edition on a p2.xlarge). Mapd performs extremely well, though I don’t think it’s using the GPU (based on nvidia-smi readings). Here it is what I’m doing:


and here it is a short description of the toy benchmark:

Any thoughts?
Thanks,
-Szilard


#2

Szilard,

Thanks for taking the time to give MapD a look.

What does the output of nvidia-smi look like?
You should see something like this when mapd is running

nvidia-smi
Mon May  8 22:27:49 2017       
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 375.51                 Driver Version: 375.51                    |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|===============================+======================+======================|
|   0  Tesla K80           On   | 0000:00:08.0     Off |                  Off |
| N/A   36C    P8    27W / 149W |  10276MiB / 12205MiB |      0%      Default |
+-------------------------------+----------------------+----------------------+
                                                                               
+-----------------------------------------------------------------------------+
| Processes:                                                       GPU Memory |
|  GPU       PID  Type  Process name                               Usage      |
|=============================================================================|
|    0     12294  C+G   /home/mapd/prod/mapd/bin/mapd_server         10270MiB |
+-----------------------------------------------------------------------------+

this is showing the mapd_server is running on the GPU.

Are you starting MapD manually or via systemd?

If you look in the data directory you specified when starting mapd, you will see a directory called mapd_log In that directory there is a link mapd_server.INFO which will tell you a lot of info about the running server. Please attach that to this report with the info from nvidia-smi

Thanks


#3

I also suspect you may not be running on gpu.

I ran your steps on my laptop (1 x 1060 GPU) and these are the times I got

User mapd connected to database mapd
mapdql> create table d(x int, y float);
mapdql> create table dm(x int);
mapdql> copy dm from '/tmp/dm.csv' with (header = 'false');
Result
Loaded: 1000000 recs, Rejected: 0 recs in 0.086000 secs
mapdql> copy d from '/tmp/d.csv' with (header = 'false');
Result
Loaded: 100000000 recs, Rejected: 0 recs in 12.831000 secs
mapdql> select x, avg(y) as ym 
..> from d 
..> group by x
..> order by ym desc 
..> limit 5;
x|ym
...
mapdql> \timing
mapdql> select x, avg(y) as ym from d group by x order by ym desc limit 5;

Execution time: 464 ms, Total time: 465 ms

Execution time: 467 ms, Total time: 467 ms

mapdql> select count(*) as cnt 
..> from d
..> inner join dm on d.x = dm.x;

Execution time: 141 ms, Total time: 142 ms

Execution time: 67 ms, Total time: 67 ms

Execution time: 69 ms, Total time: 69 ms 

#4

Szilard,

I have checked the code on a kepler machine (k80machine like you are running on AWS) and I notice the IR code is targeting the CPU for the AVG(float) query.

My laptop is pascal so didn’t see the CPU behaviour.

if you run EXPLAIN <stmt> you will see the IR ode generated for the query. In your case I suspect it is going to say :

explain select x, avg(y) as ym from d group by x order by ym desc limit 5;
Explanation
IR for the CPU:
===============

....

We have recently changed this so more operations will run on Float on kepler architecture.

I will get that applied to this edition as soon as possible.

Sorry for the confuson.


#5

Hi Michael:

Thanks for help. I already shut down the ec2 instance and for some reason I cannot start one now, but last night I played a bit more and figured out some of the things you mentioned. I’ll look more into it when I’m able to start a new ec2 instance, but from my memory from last night:

Yes, I’ve seen mapd in the nvidia-smi output just the way you have it. Except when I was running the aggregation query that runs for ~1sec I did not see GPU usage going anywhere above 0%.

I’ve seen later on from the query plans (explain) that for aggregation it was using the CPU, but for the join query it was using the GPU. So, my guess was that the optimizer decides which to use CPU/GPU, right? Anyway even on the CPU the result is great (compare to the other tools in my toy benchmark), and your run on the GPU (460ms) is even better.

Overall the results with mapd are really impressive and I’m really happy you guys open sourced. And thanks for help with the above comments.

-Szilard