Fail to Use deviceQuery from CUDA in Ubuntu

Question

The Current OS is Ubuntu 14.04.02 LTS. I have followed http://docs.nvidia.com/cuda/cuda-getting-started-guide-for-linux/#ubuntu-installation to install CUDA. However, when I use deviceQuery at the post-installations action, it shows the following message:

cudaGetDeviceCount returned 38
-> no CUDA-capable device is detected
Result = FAIL

The installation guide says it may be some problem for /dev/nvidia*, so I run /dev$ ls -l nvidia*:

crw-rw-rw- 1 root root 195,   0  6月 25 20:58 nvidia0
crw-rw-rw- 1 root root 195, 255  6月 25 20:58 nvidiactl

Also nvidia-smi -a:

Failed to initialize NVML: GPU access blocked by the operating system

Now I do not know what is wrong. Can anyone give me a suggestion?

Answer 1

My graphic cards: 1. Tesla K40 2. Quadro FX580

I am also having the same problem, which took me hours to figure out. My solution is that you may have to lower your CUDA from 7.0 to 6.5 or even more to 5.5.

What happens is that for some unknown reasons your graphic cards (one of them) only work well with Nvidia driver version below 346, e.g., 340 version. However, for CUDA 7.0/6.5, you need 346. This problem contradicts itself. I have encountered this scenario and have infinite log-in loop problem (basically, cannot login to my computer).

What you do is:

1. if you cannot login in the GUI, press Ctrl+Alt+F1/F3 to go to command line.
2. In the terminal, login. And then type sudo apt-get install nvidia-cuda-toolkit. (This will install driver version 340, which helps you able to solve the infinite login screen problem <-- however this will not allow for you to use Cuda7.0, I believe).
3. Once you are able to login, cd /usr/local --> rm -rf cuda7.0.
4. sudo apt-get install cuda6.5 (they will force you to install version 346 again).
5. sudo apt-get install nvidia-340 (a bit lower level driver; while you are installing it, it will remove cuda6.5 and some other files, but not all)
6. nvidia-smi <-- you should be able to see two of your graphic cards now. However, you are missing cuda
7. nvcc <-- this should give you error and suggested solution. Something like "type sudo apt-get install nvidia-cuda-toolkit". Follow it and you should be able to install cuda5.5
8. Now, at least you can use cuda5.5 with one good graphic card and one bad graphic card.

Note: You may enter a problem where your gcc has problem. For example, they may not accept gcc4.8.2 but gcc4.8.4. Please double be aware of that.

Ya, sometime, it is frustrating to encounter this kind of contradicting problem where the driver does not support one card but not another and there are other dependencies. It takes a long time for me to figure this out. Hope this helps!

---

Update on 10/03/15.

So, I tried caffe with Cuda5.5. And it came to very weird problem, which I believe it is because of the cuda version is too low. So, I tried to update to 6.5 again. So, I successfully installed cuda6.5 following this:

1. pre-install 340.93 Nvidia driver.
2. download cuda6.5 https://developer.nvidia.com/cuda-toolkit-65
3. MAKE SURE you use gcc 4.4.7. gcc 4.8.4 won't work. Follow this to do so: https://askubuntu.com/questions/26498/choose-gcc-and-g-version
4. Install the .run file you downloaded for cuda. Make sure do not install driver 340.29 that is recommended by the installer.
5. All set after installation, do sudo nvidia-smi to make sure it runs. Reboot. Done.

This works on my computer ubuntu 14.04 with Tesla K40c and Quadro FX580.