Unable to create a ScatterPlot


#1

Hi there,

I started very recently playing with mapd+mapd-connector+mapd-charting and I am hitting a wall when trying to make scatter plots to work, maybe someone has solved issue similar to this one? In essence, I am trying to reproduce a simple X vs Y scatter in MapD Immerse with ReactJS.

I started from the MapD Tweeter example (shows how to use xplot+map with React) and MapD Charting example (shows how to use xplot), here are the 2 files in particular that creates the xplot:

I eventually got to an almost working piece with that:

function(chartId, params) {
    const w = 400;
    const h = 400;

    const crossfilter = getCf()
    const connection = getmapDConnection();
    const parent = document.getElementById(chartId)

    var pointMapDim = crossfilter.dimension(null).projectOn([params.fieldX + " as x", params.fieldY + " as y"]);

    // hardcoded boundaries for now, I encounter another problem getting the extent dynamically, another topic of discussion :)
    var xDim = crossfilter.dimension(params.fieldX).filter([150, 600]); 
    var yDim = crossfilter.dimension(params.fieldY).filter([1000, 2000]);

    pointLayer = dc
        .rasterLayer("points")
        .dimension(pointMapDim)
        .group(pointMapDim)
        .cap(500000)
        .sampling(true)
        .xAttr("x")
        .yAttr("y")
        .xDim(xDim)
        .yDim(yDim)

    pointMapChart = dc
        .rasterChart(parent, false)
        .con(connection)
        .height(h)
        .width(w)
        .pushLayer("points", pointLayer)
        .renderHorizontalGridLines(true)
        .renderVerticalGridLines(true)
        .xAxisLabel('X Axis')
        .yAxisLabel('Y Axis')
        .init()

        return Promise.resolve(pointMapChart)
}

Here is what happens:

  • checking the mapd log: the proper SQL request is made
  • checking the Network request: the POST request “render_vega” is valid too, and the response it a base64 image showing the scatter (after decoding the base64 of course)
    => I deduce that connection is working great and server-side rendering is producing the expected chart

However nothing shows up in the interface (I trippled-checked: the chartId is correct, the

element exists). I’m forcing to redraw and/or rerender the chart and the components: no luck. I am wondering if this is a problem with React or with MapD: I create other types of charts (bar charts, line charts, etc.) which are client-side charts: everything works beautifully. e.g.:
function(chartId) {
    const w = 400;
    const h = 400;

    var colorScheme = ["#22A7F0", "#3ad6cd", "#d4e666"]
    const crossfilter = getCf()
    var rowChartDimension = crossfilter.dimension("dest_state");
    var rowChartGroup = rowChartDimension.group().reduceCount();
    dcBarChart = dc.rowChart('#' + chartId)
                    .height(h/1.5)
                    .width(w/2)
                    .elasticX(true)
                    .cap(20)
                    .othersGrouper(false)
                    .ordinalColors(colorScheme)
                    .measureLabelsOn(true)
                    .dimension(rowChartDimension)
                    .group(rowChartGroup)
                    .autoScroll(true);
    return Promise.resolve(dcBarChart);

Here is more to it:

  • if I clone and build the tweetmap example, I have the same problem: all the charts show up except the scatter (and I see the “render_vega” POST request returning a valid scatter as base64 image too)
  • I admit that I’m getting a bit of frustrated with documentation and mapd-charting versions: code examples don’t seem to match the documentation nor the library code => I can’t find which version to use with the docs and/or the example. I end up guessing by trial and error the versions of mapd-connector/mapd-charting/mapd-crossfilter each time I get errors related to functions that don’t exist (for instance I cannot run example4.js out of the box, setState() is undefined, etc.). Am I missing something obvious or can someone share where I should be looking at to get the functions documentation?

The fact that MapD Immerse works tells me that it should be possible :slight_smile:
Any clue is really appreciated. thanks a lot.
cheers


#2

making some progress here and I can now make the mapd-tweet example to work:

  • using mapd-connector v2 (not v3)
  • using current master version of mapd-charting (not v1.x release)
  • changing the mapd-tweet rasterChart to .crossfilter(crossfilter).setState().xDim(xDim).yDim(yDim) rather than .dimension(...).group(...).cap(...)...

Let’s see if I can replicate that in my simple xplot app.

I am still a bit puzzled around where to get the information, for instance I cannot find the documentation (i’m looking here: https://mapd.github.io/mapd-charting/docs/) for setState(). It looks like the JSON to input is like the ones describes in the vega documentation (https://www.mapd.com/docs/latest/mapd-core-guide/vegaTutorials/) however when I look at the mapd-charting example of setState() there are a few discrepancies in the format.

IMHO mapd is an excellent technology and having an ecosystem of mapd-charting + mapd-connector + the ability to make server-side rendering is an extremely valuable differentiating factor over other software (I read that support for geospatial data is in the pipe, so that would be another big plus). If there were a bit more comprehensive doc for the mapd-connector+mapd-charting (release notes, API changes between versions, function description for all public functions) that would make adoption a lot easier (and if it exists and I missed it, apologies in advance, I’ll be happy to be corrected).

thanks


#3

Latest update: got it to work in my app by tweaking the format of setState(). From what I understand, whatever parameters sent through setState() will be translated into the well documented Vega JSON input format before being sent to mapd.
Is there a description of the setState() input format: I’m mostly guessing from the code examples and trial and errors, which is quite painful :slight_smile:
thank you!


#4

Hey @claude - sorry for the slow reply! I’m looking internally for the right person to help you out. I’ll make sure someone jumps in here with you this weekend.

Thanks for your patience and your kind words about MapD! I’m really glad you’ve found it useful.