GPU computing in Matlab

February, 2012

We're in the process of putting together a group order for many more licenses for Matlab's Parallel Computing Toolbox (so far we have just two.) This is useful to get Matlab making much more effective use of multi-core CPUs, including systems with dual CPUs as well as CPUs with 2, 4 or more cores. One Matlab session using one PCT license can initiate up to twelve parallel "worker" processes to handle concurrent tasks. We're seeing more and more multi-core systems in mainstream PC configurations.

One particular feature of PCT is its support for direct access to CUDA-capable GPUs. Many nVidia display adapters (video cards) include CUDA capability to one degree or other. This page explains what this enables:

Matlab GPU computing via PCT

The GPU must meet a minimum requirement of Compute Capability v.1.3 or higher. Most current nVidia cards that are CUDA-capable now offer CC v.2.1. Note that not every nVidia card is CUDA-capable even today; the lowest-priced cards omit this, and lower-mid price range cards may include some CUDA support, but with fewer GPU multiprocessors (and thus fewer total GPU cores - there are 8 cores per MP).

I gathered info on the GPUs in any nVidia cards in our Tier 1 linux workstations. Unfortunately the great majority of these are a bit older and most only offer CC v. 1.1, so they can't be accessed by PCT at least via the easiest route. It looks to me like there are other things you can do with building CUDA kernels (e.g. in C, compiling with the 'nvcc' compiler) and then calling those kernels from Matlab. I haven't gone into the details of those methods to see whether / how Matlab would support that. I also have surveyed only a small proportion of Tier 1 Windows PCs so far, and I have no way to check what  GPUs may be in any Tier 2 and Tier 3 PCs. That's up to you; here's how to check.

Testing if your system has a CUDA-enabled GPU

You can get the make and model of your video card without opening the case as follows:

• In Windows, open Device Manager (right-click on "My Computer", choose "manage") then look at the Display Adapters section. It will list make and model.
• In Linux, at a shell prompt, type: lspci | grep VGA

You can also ask Matlab to tell you if it has detected a CUDA-capable GPU in your PC. For this to work, you need up-to-date nVidia video drivers installed. The commands are:

gpuDeviceCount

{should return 0, 1 or 2 if you have a dual-GPU setup!}

gpuDevice

This displays a block of info about your GPU, including what ComputeCapability ver.# it has.

Here is Matlab's page on Identifying and Selecting a GPU in Matlab

If your system doesn't have a GPU
  ... or yours GPU's CC version is too old for Matlab PCT

If you want to take advantage of CUDA from Matlab, the best route (once our big order of more PCT licenses arrives) would likely be to buy a newer nVidia card, taking care to select one with enough GPU cores for what you want it to do.

Overall 'you get what you pay for'; card prices tend to work out to about $0.40 to $0.85 per GPU core. While there are a very few under-$80 cards with CC 2.1 (and at 50-65W, no PCI-power cable needed, see below), these have only 6 or 12 GPU MPs (48 or 96 cores). You need to spend at least $90-100 to get a mid-range CUDA GPU unit with 24 or 30 GPU MPs (192 or 240 cores; e.g. GeForce GTX 260, 275, 280, 285) and you'll need PCI-power cabling to supply their power draws of 200+W.

POWER DRAW
One thing to watch out for is that the more powerful CUDA-enabled cards are both larger and have much higher power draw than a typical video card. So you need to ensure first that there is room in your PC case for the card, positioned in the correct type of PCI slot (typically PCI-express x16). Next, you need to see if your PC's power supply has enough spare power to drive such a large card, and whether it has the required PCI-power connectors. These are either 6- or 8-pin plugs with yellow and black wires (for 12v and ground). Different GPU cards require either one or two 6-pin PCI-power connectors, or on the largest cards, a 6-pin plus an 8-pin or two 8-pin ones. The spec calls for 6-pin plugs to deliver 75W, and 8-pin plugs 150W. Only very specialized and high-output power supplies have any 8-pin connectors, and small to mid-size power supplies often don't even offer any 6-pin ones.

You also need to consider cooling and airflow for these. Perhaps plan to include a good GPU card in your next new system purchase, and have your PC supplier ensure the case and power supply are up to the heat and power loads.

Which boards?
Here is a link to a list of all of nVidia's CUDA-capable GTX video cards. (I haven't listed the Quadro or Telsa boards.) Each name links to the info page for that model, where you can find the specs including # of cores, Compute Capability level, power draw and power connector requirements.

I've compiled a summary of that info for several models currently for sale plus a few of the ones we have in place. I don't think this blog software does tables, so here is a table of nVidia GPU cards with CCC >= v.1.3, showing number of cores, power draw and PCI-power connector counts, and links back to nVidia's product page for each item.