Mac os Sierra(2015 MacBook Pro 10.12 with Nvidia GPU) successfully working


#1

For those interested, I got mapd-core to compile and build on my mac (10.12.6) with the discrete GPU - NVIDIA GeForce GT 750M 2048 MB (mid 2015 MBP with Nvidia graphics).

Some notes

  1. The pre-requisite is to first get CUDA functional WITHOUT MapD, and test the ability to compile and run the sample programs from the CUDA installer. That way, you know the toolchain works. It may not work for much longer (since the nvcc already warns about Maxwell generation devices being deprecated)

  2. The OS X deps build script in the github repo is useful but stomps all over the SW environment so I ended up going back and installing the deps by hand using Homebrew. I’ll document this later.

  3. The arrow feature (one I was interested in) is broken (may be my fault) - I pointed to the libs/include in my anaconda install of pyarrow but the arrow 0.5 release seems out of whack since the build failed. So i went back and dropped the -DARROW_CONVERTER_ON flag and rebuilt (since i was more interested in getting it working)

  4. The bigger issue is when i ran the sanity tests - it hangs with this error showing up multiple times. I’ll attach the log shortly.

“3: [ RUN ] Select.Strings
3: unknown file: Failure
3: C++ exception with description “FailedToCreateFirstSlab” thrown in the test body.
3: [ FAILED ] Select.Strings (1509 ms)”

  1. I was able to run startmapd and actually load the smaller sample dataset, and get Immerse working as well :slight_smile:. I’m not sure how I can verify if it is using the GPU - is there a simple way to check?

Will poke around more and document more details shortly

Venkat


MacOS homebrew doesn't contain CUDA
#2

Hi

@niviksha Glad to see you playing with MapD

I believe @andrew is about to publish some more recent docs for building on Mac, so you two should compare notes.

When MapD starts up it will write a log mapd_server.INFO which contains information about if it is running in cpu or gpu mode. Review that log and you should be able to tell. Additionally from mapdql you can run a \memory_summary command after you have run a few queries and see the usage on your GPU. If no gpu shows up in the report, no gpu in use.

regards


#3

Hey, cool to see this. I have that same Mac. I’d be delighted to see this working before it goes the graveyard.


#4

@dwayneberry So it does use GPU memory and i found this out the hard way :wink:
i found this out from the log file this morning, when i got this error in the log - “E0817 11:52:35.749225 16551936 MapDHandler.cpp:2024] Exception: Query couldn’t keep the entire working set of columns in GPU memory”

@billmaimone
The first step before any of the installation is to get the CUDA Toolchain on macOS working properly. Not just for MapD, but for other tools I use (TensorFlow and Pytorch), this is the biggest cause of pain during installation or use (Nvidia drivers out of whack with the Xcode/clang toolchain).

The Nvidia page for OS X has this http://docs.nvidia.com/cuda/cuda-installation-guide-mac-os-x/index.html#axzz4MhqEjPgP

The key is to use Xcode 8.2 (but not later) and CUDA 8.0, as the compatibility matrix says. So, assuming you’r in OS X Sierra (10.12)

  1. Update Xcode to 8.2, check the clang version in terminal

  2. Download and install CUDA for OS X, using the fat .dmg installer (the network installer never worked for me).

  3. Key step - update the path and library path (in the install guide)

  4. Even more key step for me - Install the CUDA samples, and go into any one example directory in the CUDA install folder and build it. This allowed me to verify that the CUDA toolchain was working before anything else.

  5. Next, clone the github repo and set up the dependencies. I used the scripts/macosx.sh script, but ran into issues because brew installs its own LLVM toolchain and the script clobbered a few things for me by changing .bash_profile the first time, so I added all the brew deps in that script manually, and skipped the LLVM ones the second time. I think it’s still useful to make it scripted, but it would be useful to have prompts in the script.

  6. I tried adding arrow, but that failed because it seems the library versions used by the GPU data frame and the current version of arrow are out of whack (the compile failed because function call signatures didnt match). I’ll try and dig deeper. So i did a straight compile, which worked (this is surprisingly quick considering the torture I went through with PyTorch and Tensorflow)


Support for Mac OS X
#5

Hi, I also went through this (somewhat painful) experience of compiling to run on OSX but in the end went hybrid to where the server will run on a GPU Linux-based cloud. There are still a few items (map layouts) that will not be reported correctly if the server is running on OSX, I think OSX is fine for Immerse portion (which I wish you can do standalone), I was using gfxCardStatus to monitor GPU usage (https://gfx.io/) and I had to disable SIP to run NVIDIA CUDA which I didn’t really want to do.

In general even though the GPU hardware itself might seem to be there I don’t think Apple wants you to have easy access to it and for MapD to support only a handful(?) of models w/ this chipset doesn’t seem as straight forward as say a more complete setup like AWS, etc. Just food for thought.


#6

Hi - these are valid points. This was more to see if I could just get the software functional, not to use it as a serious platform, especially given the age of the HW (2015 Maxwell generation GPUs).

To be clear, the painful part of the experience for me was not MapD compile itself, but the CUDA/MacOSX interaction. That said, there is a large community of Mac OS X users who use it as their primary laptop for development etc - the immediacy of running it on a laptop is terrific, and with eGPUs available for OS X, it provides an entry point for individual enthusiasts to try out MapD without the operational complexity of a hybrid set up (managing a cloud subscription so the costs don’t go out of hand etc).


#7

Thank you all for these useful posts.
I finally compiled MapD on my macOS 10.12 and it was a combination of @niviksha solutions and my self.
I think one of the most important key points needs to be noticed is switching b/w Integrated/Discrete graphics at System Preferences => Energy Saver. You need to play with the auto switch check box during compilation/first-time-running processes!
By the way it would be my pleasure to help other people who want to run MapD on their MBPs.

Thanks and good luck.

P.S. My MBP is 15-inch Mid 2014, NVIDIA GeForce GT 750M 2048 MB.


#8

I know someone asked this question before, but I can not find the Q&A:
How can I make sure that I’m executing the query on GPU not CPU?


#9

Hi @manili!

Is this the thread you’re looking for?

Patricia


#10

This may be what you’re looking for - Mac os Sierra(2015 MacBook Pro 10.12 with Nvidia GPU) successfully working


#11

OMG!
I should just look at some posts up!
That was it, thank you.
So I am running on GPU without any problem :slight_smile: .


#12

@niviksha Re: Arrow: the compile issues are due to us currently using an older version (0.4.1), though there is an ongoing PR to clean up the code a bit and target the new APIs in 0.6.0 and 0.7.0.

To compile Arrow this older version yourself, you can do something like:

VERS=0.4.1
curl -O https://github.com/apache/arrow/archive/apache-arrow-$VERS.tar.gz
tar xvzf apache-arrow-$VERS.tar.gz
mkdir -p arrow-apache-arrow-$VERS/cpp/build
pushd arrow-apache-arrow-$VERS/cpp/build
cmake \
    -DCMAKE_BUILD_TYPE=Release \
    -DARROW_BUILD_SHARED=on \
    -DARROW_BUILD_STATIC=off \
    -DARROW_BOOST_USE_SHARED=on \
    -DARROW_JEMALLOC_USE_SHARED=on ..
make -j
make install

Support for Mac OS X
#13

I would be intersted in details on compiling mapd-core for mac.
My fails are always due to llvm. Anyone have tips on including llvm libs for header paths in macOS Sierra?


#14

I found that using the brew llvm tended to cause all sorts of errors, especially if you’re using the macOS dependencies script on mapd-core github. If you’re starting with a ‘clean’ llvm (that ships with Xcode), that seems to work fine. so one thing to do might be to try uninstalling the brew-installed llvm toolchain and seeing if that helps.


#15

Thanks niviksha
I think the issue is my library paths. Even removing brew’s llvm, I get an the error:

fatal error: ‘llvm/IR/Function.h’ file not found

At least with brew’s llvm I knew where those headers were.


#16

Check out https://github.com/mapd/mapd-core/issues/4 if that may be of help


#17

@Niviksha Thanks so much for your help here. Turns out, it was LLVM 5.0.
I went down to llvm 4.0, adjusted the deps-macos script and built the long, slow way (single thread)

macOS Sierra, 10.12.6, macbook Pro 15" 2017


Can't build from sources (MacOS Sierra 10.12.6)
#18

Yes LLVM seems to change w/ nearly every release but the build scripts are locked at using @llvm only and not a particular version.

Despite all of this I would still love to see Pointmap(s) working in OSX eventually, especially for testing a low case of samples locally before adjusting the “# OF POINTS” on GPU server.