I’m benchmarking ingestion throughput and am utilizing multiple simultaneous StreamInsert instances to insert data into a MapD database running on Amazon p2.8xlarge. I’m getting great benchmarks with up to 8 simultaneous producers but as soon as I add any additional producers the 9th/10th/etc… hang until one of the first 8 producer completes. I thought maybe this was due to having 8 gpus on a p2.8xlarge so I spun up a p2.16xlarge thinking 16 gpus might give me a different result but it had the same limitation. Given this, is there a way to tweak the configuration of MapD core in such a way that I could have more than 8 simultaneous StreamInsert producers or is there some other recommended approach? Thanks in advance for your assistance, Adam M.
What you are seeing is the limitation of number of concurrent connections.
Discussion here Unable to connect after a few hours
For older versions you can increase the limit, in new version (upcoming 3.3.0) the limitation has been removed.
Thanks for the quick response.
I updated the mapd.conf.in file:
sudo vi /opt/mapd/systemd/mapd.conf.in
threadpool-size = 30
and added the threadpool-size entry into the file.
Then stopped and restarted mapd & mapd-web processes:
sudo systemctl stop mapd_server
sudo systemctl stop mapd_web_server
sudo systemctl start mapd_server
sudo systemctl start mapd_web_server
but it did not seem to have any change in behavior as my 9th producer still hangs until another one finishes. Do I have the property right, did I edit the wrong file or did I miss something else in the stop/start process?
Thanks again for the help on this,
I also just tried:
tthreadpool-size = 30
with an extra ‘t’ in front since I thought that might not have been a typo in the other thread you referenced along with a stop/start but that did not resolve the issue either.
I guess you have to disable backend rendering to have more than 8 concurrent sessions
I think the issue here is that a different config file is being used. The
mapd.conf.in file is the input template for the config file, which by default will get placed at
/var/lib/mapd/mapd.conf. Since you’re using systemd, you might be able to determine the path to your config file by running:
systemctl status mapd_server | grep config
That being said, I am in the process of finishing the v3.3.0 builds, which remove this restriction. They should be up in the next hour or so.
Thank you Andrew for getting the v3.3.0 builds up on the community AMI with this restriction removed. I spun up a new instance earlier this morning and verified that I no longer have this constraint with v3.3.0. Thanks again!