Immerse not showing data points when zooming in


#1

I have installed MapD 3.6.1 (on a p2.8xlarge instance), and imported a flavour of the New York Taxi data.

When I add a pointmap to the dashboard it appears to work. However when I zoom in to New York all the data points vanish. You can carry on zooming in to street level, but the data points do not show themselves on the pointmap. The total number of records changes as you zoom in and out of the map, so the underlying data looks like it must be there. The graphical representation of any points do not re-appear until you zoom right out (to state level). It should be working just like the MapD taxi demo, so I am a bit confused why it is not working.

Does anybody know what might be wrong?


#2

Hi @ArnoldJ,

Thanks for your interest in MapD. This could be an issue of point autosizing? In the pointmap editor can you uncheck autosizing and bump up to the point size to see if you can see the points?

There is a known bug that we just fixed (should be in the next release) where on a new dashboard, point autosizing doesn’t work correctly in the initial pointmap create screen (it should work fine after clicking “Apply” and adding that chart to the dashboard). Another case in which auto-sizing may not lead to the desired behavior is when many points are stacked on top of each other.

Regards


#3

Thanks for the reply. I have tried making the amendments you suggested, but it hasn’t resolved the issue. It looks pretty strange.

Here is a screenshot with the point size turned up


#4

Which data type have you used for Lon/lat fields? The points are there but they looks grouped (an approximation problem?)

Could you provide us a sample of rows of Lon/lat fields extracted with a query in mapdql or immerse;
you can extract the query immerse is using in your dashboard from mapd_server.INFO located on your data dir in subdirectory mapd_log

the query would look like

SELECT conv_4326_900913_x(pickup_longitude) as x, conv_4326_900913_y(pickup_latitude) as y, yellow_tripdata.rowid FROM yellow_tripdata WHERE ((pickup_longitude >= -74.03701895157319 AND pickup_longitude <= -74.007578650705) AND (pickup_latitude >= 40.767513793266346 AND pickup_latitude <= 40.78000494831389)) LIMIT 2000000

the output would look like this

mapdql> SELECT pickup_longitude, pickup_latitude, yellow_tripdata.rowid FROM yellow_tripdata WHERE ((pickup_longitude >= -74.03701895157319 AND pickup_longitude <= -74.007578650705) AND (pickup_latitude >= 40.767513793266346 AND pickup_latitude <= 40.78000494831389)) LIMIT 10;
-74.01001|40.77298|16716
-74.03293|40.77361|21055
-74.01001|40.77298|6173
-74.01316|40.77258|298410
-74.02173|40.77242|361866
-74.03295|40.76809|302314
-74.02896|40.77246|406971
-74.0145|40.76849|527997
-74.0121|40.77644|372598
-74.00935|40.76863|364972
10 rows returned.
Execution time: 64 ms, Total time: 65 ms

#5

Thanks a lot for your help.

It looks like I got the wrong precision setup on the decimal field used to store the lat/lon.

Is there a preferred data type for this in MapD? float, real or shall I just put it in something like Decimal(9,6)?


#6

i used decimal in mapd for things that need predictable results, like money amounts because they are non-native types on GPUs and they costs 8 bytes of storage; single precision float costs just 4 bytes.

anyway decimal(9,6) would give enoug precision; this is my result using decimal(9,6) on lon/lat and this is my result on columnbus ave in NY

so if you need more precision you can use decimal, but performace/storage wise float is preferable


#7

Hi @ArnoldJ,

I would suggest float or double for rendering, with double obviously being much more precise but requiring 8 bytes versus 4 bytes per field. We will soon have geo types that can be compressed such that you will get very high precision while only needing 4 bytes per field.

Regards