MapD GPU RAM and IOPS


#1

I have an instance p2.xlarge on AWS with IOPS volume of 256GB.

My tables are of following schema.

CREATE TABLE table1 (
	created BIGINT, 
	hash TEXT ENCODING DICT(32), 
	id TEXT ENCODING DICT(32), 
	ix TEXT ENCODING DICT(32), 
	ts DATE)

And

CREATE TABLE table2 ( 
	id TEXT ENCODING DICT(32), 
	sg1 TEXT ENCODING DICT(32), 
	sg2 TEXT ENCODING DICT(32), 
	sg3 TEXT ENCODING DICT(32), 
	sg4 TEXT ENCODING DICT(32))

My table sizes are as under:

table1: 768,414,160 records
table2: 836,257 records

My query is:

SELECT table1.ts AS C_DAY,
       table2.sg1,
       count(1)
FROM table1
LEFT JOIN table2 ON table1.id=table2.id
WHERE table1.created >= 1501372800000
  AND table1.created <= 1501545600000
GROUP BY table1.ts,
         table2.sg1
ORDER BY table1.ts;

It gives me following error:

Exception: Query couldn't keep the entire working set of columns in GPU memory.

My Question is:

  1. Can MapD use CPU and GPU RAM both to perform the query?
  2. How to set optimal value of IOPS?

#2

Hi @desainakitb,

Thanks for trying out MapD.

While we have functionality to run the query on the CPU as described elsewhere (Is there a memory replacement mechanism for mapd?), in general to get GPU acceleration for the query and dataset size you are describing you will need an instance with more GPUs and consequently more VRAM (i.e. the p2.8xlarge or p2.16xlarge).

I’m not quite sure of the optimal IOPS value (faster is likely better in general, although a) our storage manager is specifically constructed to read data in large blocks for maximum throughput and b) we cache the hot data in memory so it only needs to be read from disk once). Perhaps someone more knowledgeable on IOPS on AWS can chime in.


#3

making the flag --enable-watchdog=false worked for now.

So, If my data is 15TB then for real-time zero latency queries, I will require 15TB of GPU RAM? Am I understanding it correctly? (May be distributed but 15TB really???)


#4

No the amount of gpu ram depends from queries you are running. On GPUs ram are needed only columns involved on group by/filters/calculations/join conditions and the amount also depend from encoding you are using; on system ram everythings else needed for final projection plus dictionary tables and of course what needed by gpu (they transit on system ram, they are transferred on gpu’s pool)

If you can provide tables, number of rows and possibly cardinalities of columns me or other users would provide an estimate of amount of ram needed by cpu/gpu pool


#5

@aznable Thank you. I shall work on getting required data which you asked for and post here asap.

By the time, today while working I came across following issues:

While inserting the data into MapD I had performed query of which I had listed above.

I came across the results which were completely inconsistent. (I had run the query while data was getting inserted)

I run the same query again after few mins (still the data was getting inserted) (this time the results were consistent)

In an another senario, once the data is cached in GPU memory and/or CPU memory, it is retriving the data stored in memory, it is not considering updated data), I mean to say, althought the table was updated with new data, the results which query generated were the same.

Questions:

  1. Do we have to programatically uncache the data from GPU memory?
  2. Why the result is inconsistent?

Thanks,
Ankit


#6

INHi,

let’s start from memory occupation of your query

ts 8 bytes (you can use a fixed encoding if the range of 1901-2038 is enough for you occupying just 4 bytes)
sg1 4 bytes (you can use a different dictionary encoding if the distinct vales are less than 32768 or 127 for 2 or 1 bytes)
created 8 bytes
id of both tables 4 bytes

so for table1 (8+8+4)*768414160 15GB
table2 (4+4)*836257 6MB

so now you are using 16GB of GPU/CPU RAM, in your specific case there aren’t differences between pools.

Using a date/timestamp fixed encoding 32 the ram consumption would be (8+4+4)*768414160 12GB.

About inconsistent results i tried querying a table while using copy command to populate it and i cant see any problem; how are you feeding the tables?

this is a batch of query launched while the copy command was running

mapdql> select avg(arrdelay),count() from flights where UniqueCarrier=‘UA’;
7.332720|5681724
1 rows returned.
Execution time: 30 ms, Total time: 33 ms
mapdql> select avg(arrdelay),count(
) from flights where UniqueCarrier=‘UA’;
7.468432|5929224
1 rows returned.
Execution time: 50 ms, Total time: 51 ms
mapdql> select avg(arrdelay),count() from flights where UniqueCarrier=‘UA’;
7.475837|5938506
1 rows returned.
Execution time: 25 ms, Total time: 25 ms
mapdql> select avg(arrdelay),count(
) from flights where UniqueCarrier=‘UA’;
7.717962|6831777
1 rows returned.
Execution time: 92 ms, Total time: 92 ms
mapdql> select avg(arrdelay),count(*) from flights where UniqueCarrier=‘UA’;
7.778210|6973394
1 rows returned.
Execution time: 39 ms, Total time: 40 ms

Also inserting data with insert command from a session and querying from another i havent any issues, the results are consistent


#7

As @aznable suggested, I changed table schema as follow.

CREATE TABLE table1 (
created BIGINT,
hash TEXT ENCODING DICT(32),
id TEXT ENCODING DICT(32),
ix TEXT ENCODING DICT(32),
ts DATE ENCODING FIXED(32))

and

CREATE TABLE table2 (
id TEXT ENCODING DICT(32),
sg1 TEXT ENCODING DICT(8),
sg2 TEXT ENCODING DICT(8),
sg3 TEXT ENCODING DICT(8),
sg4 TEXT ENCODING DICT(8))

This new schema saved a lot of disk space as well as, now I am able to accommodate more data in GPU/CPU ram. (Thank you so much for that!!!)

I am using jdbc, spark to insert data in MapD.

About query results, I found a bug in my inserted values.

In place of 'null', I was using null. As well as, my join condition was incorrect. (this solved another problem too that is mentioned below.

mapdql> select * from table2;
Thrift: Wed Aug 23 06:26:25 2017 TSocket::write_partial() send() <Host: localhost Port: 9091>Broken pipe
Thrift: Wed Aug 23 06:26:25 2017 TSocket::open() connect() <Host: localhost Port: 9091>Connection refused
Thrift error: connect() failed: Connection refused

Saying that, I came across another observation which I would like to mention here, while my query is running htop shows 100% CPU utilization. is that normal?

And how to identify that CUDA cores are being utilized or not? is there any utility similar to htop to visually or statistically check utilization of GPU? nvidia-smi only shows memory usage right?

Thanks,
Ankit


#8

yes you can use nvidia-smi to track also the GPU ulitilization for the whole card

nvidia-smi -lms 100 (this is the syntax where you can specify the loop interval in milliseconds; in this case every 100 ms)

i dont think to have the cpu running at 100% is normal, but maybe the query is falling back to CPU path because of insufficient GPU memory.

i don’t know exactly how much memory is provided with you aws instance (a full k80 board would have 2 GPUs ID and 24GB Ram, but i guess amazon is providing only a GPU and 12 GB but i can wrong).

you can check how much memory is used from mapdql with the switch \memory_summary and for a more detailed use of gpu memory you can use \memory_gpu.

if you want to make sure to be able to run you query on gpu, you could create another set of tables loading a subset of records in table1, or you could try to define the created field on table one with a text encoding dict (32) but i dont know exactly how this would affect performances (never tried by a such high cardinality dictionary encoding).

anyway keep trying and experimenting with mapd, in my experience is the fastest database (tried oracle in-memory, exasol and others in-memory) and the gap in performance wise is very big (50-100 times with GPU 30 times with CPU)


#9

nvidia-smi -lms 1000 works fine to find GPU utilization.

AWS instance with one GPU is equipped with 12G of GPU ram and 60G of CPU ram and 4 cores of CPU.

Yes, you are right my 20% data is stored in GPU ram and 80% in CPU ram, probabaly thats the reason why I see 100% CPU usage. (I had made watchdog false to over come out of memory issue, as i know my data can’t fit in GPU memory only)

I will give your thought a try today and will update you asap.

Thanks for the suggestions.

Just a last question before we wind up, here making watchdog off works for using GPU and CPU memory both, will it also look in to disk for the data? or it will just give out of memory error when data can’t be accommodated in GPU and CPU ram? (although I gonna try this today by inserting more data, but still if someone can tell me in advance, it will save my time of experimenting)

Thanks,
Ankit


#10

hi,

i dont know what happens if the system run out of memory (i guess you have to fill the swap space too); maybe with the parameter enable-watchdog = false the server stream the data needed for the query from disk, but the performance would decrease dramatically.
The bandwidth of disk subsystem is vastly lower than the one of system memory and even lower than gpu one; in the system i am using right now are:

disk 0.6GB/sec system 20GB/sec gpu 484GB/sec (the B/W between CPU and GPU is 15GB/sec); of course we are talking about peak performances and we aren’t taking in account the overheads of the software loading and unloading buffers on his internal cache pools located in system and GPU Ram.

now a small benchmark with a system under heavy load

mapdql> \clear_gpu
MapD Server GPU memory Cleared
mapdql> \clear_cpu
MapD Server CPU memory Cleared
mapdql> select count(DepTime) from flights;
121232840
1 rows returned.
Execution time: 927 ms, Total time: 927 ms --> time taken to load data from disk, populate system and GPU pools and execution of statement

mapdql> \clear_gpu
MapD Server GPU memory Cleared
mapdql> select count(DepTime) from flights;
121232840
1 rows returned.
Execution time: 227 ms, Total time: 227 ms --> time to populate the GPU pool previsously cleared and execution of statement
mapdql> select count(DepTime) from flights;
121232840
1 rows returned.
Execution time: 14 ms, Total time: 15 ms --> execution of stament; just look how is big the performance hit for repopulate the data on GPU
mapdql> \cpu
mapdql> select count(DepTime) from flights;
121232840
1 rows returned.
Execution time: 54 ms, Total time: 55 ms --> time of execution on CPU mode (on simple statement the difference isnt so big)

so even the watchdog would allow a streaming of data from disk the performance would be sub-optimal because the software is designed to exploit the maximum bandwidth of memory subsystems.

i will take some test asap anyway, and i will post the findings on this thread

and take in account the overhead of software