MapD Import and Render GeoJSON Shapes


#1

Hello,

How do I generate my own polygon data onto a map using polygon shapes (GeoJSON, etc.)?

When I import the my GeoJSON file, and I attempt to create a CHOROPLETH with the data, the shapes do not render on the map. Am I missing something? Or am I supposed to select another CHART type?

Thank You


Working with polys
#2

Do you guys have a guideline to check if a GeoJSON file is valid?


Mapd shapefiles as layers on immerse
#3

Hi

Please take a look at this thread for some information about checking your file

Regards


#4

Admittedly, I do have repetitive vertices in my GeoJSON data. However, I’m not receiving that error “import_geo_table” when I import the data. It seems to import the data successfully. How come I’m not seeing this error when I import the data?

Thanks!


#5

you have to put ypour json file on $MAPD_HOME/frontend/geojson and adding an entry on geoconfig.json; something like that

“province_light.json”: {
“label”: “IT L_PROV”,
“isTopo”: false,
“topoKey”: “”,
“keys”: [“DEN_CMPRO”,“SIGLA”]
},

keys have to be matched from a filed on your data

the json you have added on file could be select on choropleth chart for visualization


#6

Hello,

Thanks so much for replying. I have a few questions.

  1. So when you say $MAPD_HOME, are you referring to the $HOME of the CENTOS host?

  2. I’m still confused about what “keys” is supposed to contain. Is “keys” a list of properties available with the GeoJSON? Which field does it have to be matched to?

  3. And is this “geoconfig.json” supposed to be an already existing file in the host? Or do I manually create it myself?

Thanks!


Recognising GEO and lat and long variables
#7

for $MAPD_HOME is mean the folders where mapd is installed e.g.

/opt/mapd/mapd-ce-3.1.3-20170727-edbb780-Linux-x86_64-render

  1. no in json file you have just properties; using us-countries.json as an example

{“type”:“Feature”,“id”:“01”,“properties”:{“abbr”:“AL”,“name”:“Alabama”},“geometry”:{“type”:“Polygon”,“coordinates”
[a set o polygons]
}
{“type”:“Feature”,“id”:“02”,“properties”:{“abbr”:“AK”,“name”:“Alaska”},“geometry”:{“type”:“Polygon”,“coordinates”
[a set o polygons]
}
and so on…

the keys are defined in the geoconfig.json the file e.g.
“us-states.json”: { // this has to match with the name of your json file
"label": “US State”,
“isTopo”: false,
“topoKey”: “”,
“keys”: [“name”, “abbr”]
}
in this expample the keys are the name and abbr properties defined in us-countries.json

so the dimension you are using have to match with the content of name or abbr.

  1. the geoconfig.json already exists, so you just have to update if you nadd more json files with doifferent maps and it’s located in $MAPD_HOME/frontend/geojeson (e.g. /opt/mapd/mapd-ce-3.1.3-20170727-edbb780-Linux-x86_64-render/frontend/geojson).
    the json files containing the maps have to be placed in the same directory

#8

Thanks so much for your explanation. I have a few more questions.

  1. Is the process that you outlined for GeoJSON going to be different for loading KML files, SHP files, etc.?

  2. And in the documentation, it states that you are able to load Shape Data (GeoJSON, KML, SHP, etc) through the Immerse UI, but so far, that doesn’t seem to be the case. Why is there a discrepancy between the documentation and the process you outlined for me? Is what stated in the Immerse Documentation meant for something else?

Thanks!


#9

Hello,

I have tried following your instructions, but the shapes don’t render. I chose a table to join on a column in the GeoJSON, but nothing is showing up. Then I tried using an existing us-states.json and copying that to a new JSON file to see if that would work. But that hasn’t worked as well. Are there any logs I peruse to see what went wrong?

Thanks


#10

Hi,

@ccsn79 Is it possible for you to share your geojson file so we can do a walk through with you to get this working?

regards.


#11

Sure, thanks!

So I was able to get one file working.

Working:

  1. states-simplified-working.json (https://pastebin.com/0cXq30sX)

GeoConfigs:
“states-simplified-working.json”: {
“label”: “Block Groups”,
“isTopo”: false,
“topoKey”: “”,
“keys”: [“abbr”]
}


#12

And these are the ones not working:

Not Working:

  1. arkansas-shapes-not-working.json (https://pastebin.com/trzgp8z2)
  2. block-group-shapes-not-working.json (https://pastebin.com/KBEYGJeB)

GeoConfigs:
“arkansas-shapes-not-working.json”: {
“label”: “Block Groups”,
“isTopo”: false,
“topoKey”: “”,
“keys”: [“st_abbrev”]
},
“block-group-shapes-not-working.json”: {
“label”: “Block Groups”,
“isTopo”: false,
“topoKey”: “”,
“keys”: [“st_abbrev”]
}


#13

Hi,

Just to try to work through some of the confusion here.

The details @aznable has mentioned above, where a geojson file is placed into the $MAPD_HOME/frontend/geojson and the geoconfig.json is edited, is to add a custom geojson file to be used by immerse Choropleth. These polygons are rendered by the frontend are are not loaded into the MapD Core database. These Immerse Choropleths geojson’s do not need a render enabled backed as all the image “work” is done in the client.

Extending these Immerse Choropleth Geojson’s is not something that is currently documented.

What is documented here

https://www.mapd.com/docs/latest/immerse-user-guide/importing-shapes/?highlight=importing%20shapes

Is the process of loading a shape file into the MapD Core database, which can then be used by the Render Engine to do server-side based render as you see in Mapd-Charting here:

https://mapd.github.io/mapd-charting/example/exampleMultiLayerMap.html

Regards


#14

What’s the difference between Mapd-Charting and Vega?


#15

yes, the geojson used in this way is transmitted over the network and rendered client side.

the documentation in my opinion is correct, it outlines how to load GeoJSON, KML, SHP into the mapd database trhu immerse, isnt stating you can use that data in immerse for rendering a choropleth map; if you take a look to the docs referring the choropleth object is obvious it’s a server side things and you dont need to load data into database.

this geo data loaded into database can be used into queries and in mapd-charting with objects supporting server side rendering geo data stored into database.


#16

Hello,

@aznable
@dwayneberry

I have another question.

I keep on getting this error: “import_geo_table failed: failed to triangulate polygon Feature: 37” when I load GeoJSON data using the Immerse UI.

Why is this the case?

Thanks


#17

Hello,

@aznable
@dwayneberry

I also received another error: “alloc”.

Why would this be the case?

Thanks


#18

Never such an error, but I loaded geo data just to assist another user a month ago, but he got another errors; duplicate vertex or a thing like that.

Now I am in the middle of Mediterranean Sea and I can try anithinf

Ottieni Outlook per iOShttps://aka.ms/o0ukef


#19

Hi

@ccsn79 we use the gdal library to do the processing of the input shapefile. It sounds like there is something in your file gdal does not like.

Please supply more detailed information for reproducing your issue. Include input files and logfiles.

Regards


#20

Yes @aznable is correct.

To be clear, MapD supports two methods of rendering choropleths. There is client-side rendering, which is the only thing that MapD Immerse currently supports. To add new geometries you must add geojson/topojson and the appropriate config entries to the Immerse folder as discussed by @aznable. The method is currently not documented but is relatively straightforward.

The other method is server-side rendering, which is supported by a MapD Charting API. For server-side rendering you must upload the geometries in shapefile, geojson, or kml to the server either via mapdql or through the Immerse importer. You can see various examples of this on the MapD Charting examples page. This method requires that the server support server-side rendering (i.e. has an Nvidia GPU and is set up appropriately).

The nice thing about having the data server-side is that it will allow us to do cool things like give the user the option to geocode lon/lat pairs on insert (i.e find the zipcode for a geocoded tweet on import and then allow joining to the relevant zipcode geometries).

We will likely replace client-side choropleth rendering with server-side rendering in the next months, since it allows for rendering of significantly more granular geometries than client-side rendering (i.e. being able to render zip codes and census tracts interactively). Its possible we will keep frontend rendering around as a fall back for users who don’t have server-side rendering available.

We realize its a bit confusing now, the product is just in a state of transition from one method of rendering to the other.

EDIT: I realized this is somewhat duplicative of @dwayneberry’s response here. Will leave it up in case it provides some additional color on our thinking.