Transactions, Consistency


Hi there,

I’m looking to evaluate MapD for a finance client. A couple of their key requirements are:

  • transactions, to give consistent views of data sets
  • updates

I see support for updates/upserts is planned (Update / Upsert support). Any news on when that is likely to be?

For transactions, in some use cases we would need clients to query consistent snapshots isolated from data being ingested, then make the data set visible atomically. Transactions wouldn’t have to be supported explicitly, as we can insert data with logical timestamps or transaction sequence numbers, and perform queries bounded by those timestamps which I imagine would be lightning fast. But we’d need consistency guarantees on when data becomes visible. Is a row consistently, durably, visible across a cluster once a JDBC insert statement has completed, or a Kafka consumer has processed a record?



@alex I’m going to get one of our technical salespeople to answer this question here, so that everyone can benefit from the answer, and so that you can continue an evaluation discussion with someone that can get you more resources.



I know you’re already in touch with someone from sales, so I hope that works out. I wanted to post this information here as well so that the community can benefit.

During bulk insert operations (COPY FROM) the isolations semantics are a dirty read. The data is visible and queryable as it’s being ingested.

On completion, if a successful response is returned, the JDBC insert statement is consistent and durable and visible across a cluster.

When consuming from a kafka queue, we tie the commit to the database and the update of the topic offset by the consumer into a single step so a failure in the consumer should not cause a sync issue with what is in the database. The size of each batch for sync is an option on the KafkaInserter utility.

Is isolating a dataset during ingestion a critical or must have?
How much data are they inserting and how often?

As for the upsert/update support, we’re always planning and delivering new features. Keep following the blog regarding announcements for our next few releases.

Hopes this helps!


Hi Eric,
Thanks for confirming. Consistent reads is a must-have. With those consistency guarantees I think we can effectively build transactions and snapshots on top. So this sounds promising.